Carlson was also ejected.
The two wrestled to the ground as the benches and bullpens emptied and other scuffles broke out between home plate and the New York dugout.
Yankees manager Joe Girardi emerged with cut and a bruise on his head. Crew chief and third base umpire Derryl Cousins left the game after the brawl. Only three umpires finished the game.
Earlier on, Roy Halladay (15-9) stymied the Yankees and rookie Travis Snider hit two of Toronto's five homers.
In other news, Yankees lefthander Andy Pettitte will miss tonight's start against Toronto because of a tired pitching shoulder.
And, the Yankees are cutting the prices of some of their most expensive season tickets for 2010.
The price of the best field-level seats will drop from $325 to $250 or $235 as part of season plans, the team said late Tuesday night.
A total of 3,400 field-level seats will have lower prices. Those seats cost $375 for individual games this year. The Yankees did not announce individual game prices for 2010.
In other games:
* Daisuke Matsuzaka (2-5) returned from a 3-month stint on the disabled list and pitched six shutout innings to lead the Boston Red Sox to a 4-1 victory over the Los Angeles Angels.
Matsuzaka (2-5) hadn't pitched in the majors since being placed on the disabled list with right shoulder problems on June 21.
Boston's David Ortiz hit his record-breaking 270th homer as a designated hitter to surpass Frank Thomas atop the all-time DH home run list.
* At Baltimore, Matt Wieters homered and had a career-high five RBI to lead the Orioles over the Tampa Bay Rays, 10-5.
* At Detroit, Alberto Callaspo, David DeJesus and Miguel Olivo all hit three-run homers and the Kansas City Royals beat the Tigers, 11-1.
* At Minneapolis, Nick Punto hit two RBI singles and Orlando Cabrera homered to lead the Twins over the Cleveland Indians, 5-4.
* At Arlington, Texas, Rajai Davis had four RBI to lead the Oakland Athletics over the Rangers, 6-1.
* Minnesota Twins first baseman Justin Morneau has a stress fracture in his lower back and will miss the rest of the season. Morneau said surgery will not be necessary.
* The Chicago White Sox announced that Jake Peavy will start on Saturday at home against the Kansas City Royals. It will be Peavy's first start for the White Sox since they acquired the righthander from the San Diego Padres at the trading deadline, a deal that cost them four minor leaguers.
Peavy has not pitched since injuring his right ankle in June.