"You don't make massive, sweeping changes like this unless something is wrong," said Colin Gillis, an analyst at BGC Financial, pointing to Wednesday's reports of declining PC shipments around the world.
Worldwide shipments of personal computers fell 11 percent in the April-June period, according to data from research firms Gartner and IDC. Analysts have blamed a massive consumer migration to tablets and other mobile devices for the falloff. But many observers also believe Microsoft's Windows 8 operating system - which comes installed on most new PCs - has turned consumers off.
The company's new divisions include engineering, marketing and business development.
Ballmer stressed the company's focus on "one Microsoft" in his memo. He said Microsoft will move forward operating as a cohesive company rather than a "collection of divisional strategies."
"Although we will deliver multiple devices and services to execute and monetize the strategy, the single core strategy will drive us to set shared goals for everything we do. We will see our product line holistically, not as a set of islands," Ballmer wrote.
The shake-up is being driven by competitive pressures as two of Microsoft's once much-smaller rivals, Apple and Google, have emerged as the technology trendsetters. In a world that increasingly revolves around mobile devices and Internet services, Microsoft has been scrambling to adapt to the upheaval.
Microsoft shares rose 2.9 percent, or 99 cents, to close at $35.69 in early afternoon trading.