UBS Europe chief Sergio P. Ermotti will take over immediately as interim chief executive until Gruebel's replacement is appointed.
Gruebel's departure caps 10 days of speculation over his future following the bank's announcement that a single London-based trader had evaded internal control systems and gambled away $2.3 billion.
The trader, 31-year-old Kweku Adoboli, was arrested Sept. 15 and charged with fraud and false accounting. A judge ordered him Thursday to be held in jail until a hearing next month.
The rogue trading loss is the third major scandal to hit Switzerland's biggest bank in recent years, and the second to come out of its investment unit. During the financial crisis, UBS was one of the European banks worst hit by failed investments in the subprime mortgage market. A year later, it became embroiled in an embarrassing U.S. tax evasion case.
UBS President Kaspar Villiger told reporters the board had tried to persuade Gruebel to stay until the bank's annual shareholder meeting next year, but the gravel-voiced German had wanted to send a strong signal about the trading loss immediately.
"He thinks that this act could maybe create a new basis for UBS to continue," said Villiger, who earlier noted that UBS wants to "turn this disaster into an opportunity."
Speculation has already begun over who is to succeed the 67-year-old Gruebel. Ermotti, a Swiss citizen who joined UBS from Italy's UniCredit, is considered a front-runner.
"Sergio is, of course, a strong candidate," said Villiger. He mentioned no other names.
Villiger said Gruebel, who was brought in more than two years ago to help revive the fortunes of the Zurich-based bank, had achieved "an impressive turnaround and strengthened UBS fundamentally."
Gruebel came out of retirement to lead UBS. A former trader, he steered the bank back to profit and resolved the U.S. tax evasion case, though not without blowing a hole in Switzerland's storied tradition of banking secrecy.
As head of UBS he also accepted new Swiss government rules that require the bank and its rival Credit Suisse to hold far greater capital reserves to prevent a possible collapse during a banking crisis.
"He steps down having helped make UBS one of the world's best capitalized banks," Villiger added.
Gruebel's resignation was inevitable, given the size of the loss and the bank's other recent risk-management failures, said Michael Robinson, a former U.S. Securities and Exchange Commission official who advises banks and companies on crisis communications.
"As soon as we found out that a single trader had lost what would surely have been their entire quarterly profit, [Gruebel] was just waiting to fall down and die," said Robinson, now a senior vice president at Levick Strategic Communications in Washington, D.C.
By responding quickly, Robinson said, UBS gave itself a chance to show it is serious about preventing similar problems in the future.
But the bank still must win over investors with substantive changes to its leadership and risk-management policies, he said. A similarly orderly transition wasn't likely at a scandal-plagued U.S. bank, given the scrutiny banks in the U.S. get from regulators, Congress and a recession-weary public.
"This is very orderly; it's the kind of thing you'd expect from a European institution as opposed to an American institution," Robinson said.
The UBS board, which met in Singapore this week, said it would seek to put in place measures to prevent a similar rogue trading loss from recurring.
The bank said it would hold on to its integrated strategy, but that the investment business would be simplified to ensure there was less risk, using less capital to produce more reliable returns. Asked whether investment bank chief Carsten Kengeter would keep his job, Villiger said: "I do not see any reasons to doubt [his] future."
Earlier this week, the Government of Singapore Investment Corp. - the largest UBS shareholder - said in a rare public rebuke that it was concerned about lapses at UBS and called on the bank to restore confidence.
The sovereign wealth fund, which owns about 6.4 percent of UBS, has suffered heavy losses on its investment as the bank's share price has more than halved since it became a shareholder.
In Washington, Tharman Shanmugaratnam, deputy prime minister of Singapore and minister of finance, refused to comment on UBS developments on Saturday, insisting on the separation between the government and the Government of Singapore Investment Corporation decisions. He said he was sure the GSIC officials were in touch with UBS officials and Swiss regulators. Tharman was in Washington, chairing the meeting of the International Monetary Fund's policy-setting committee.
AP Business Writer Daniel Wagner and AP Economics Writer Martin Crutsinger in Washington contributed to this report.