President Bill Clinton was impeached by the House over lying about an affair with White House intern Monica Lewinsky, although the Senate declined to remove him from office. President George W. Bush failed to get a big Social Security overhaul through Congress and was slammed for his handling of Hurricane Katrina and growing voter anxiety over the Iraq and Afghanistan wars.
From Inauguration Day, a second-term president's influence and power begin to ebb.
"It's called fatigue. People burn out," said Ross Baker, a political science professor at Rutgers University. "Typically, the top people are recruited for the first term. For the second term, you kind of go to the bench. It's a little less illustrious than the starting lineup. You're going to get more people perhaps a little less sure-footed. That's putting it, perhaps, mildly."
There's something of a political Continental Divide with second terms. At some point everybody's attention starts flowing to the next election.
Also, Obama sets out against a backdrop of fiscal showdowns that will come to a head in March. And some of his top second-term goals, such as immigration and tax-code overhaul, gun control, and climate-change legislation, come as grim budget realities cast a long shadow over what he can accomplish.
History is littered with troubled second terms.
Richard Nixon resigned in disgrace. Ronald Reagan's second term was marred by the Iran-contra guns-for-hostages scandal.
Even George Washington had an ugly second term. His backing of the Jay Treaty expanding trade ties with Revolutionary War foe Britain divided the nation. Many leaders - including future president Thomas Jefferson - challenged Washington. Jefferson called the treaty a "monument of folly." Angry crowds gathered outside Washington's house and talk simmered of impeachment.
Franklin D. Roosevelt was able to win four terms but had a tumultuous second one despite a 1936 reelection landslide. His effort to expand and pack the Supreme Court with ideological allies was soundly rebuffed by Congress.
But second terms don't have to be failures - and Obama won't necessarily struggle.
William Galston, a domestic-policy adviser in the second Clinton administration, said the notion of a second-term jinx or curse was an oversimplification because "a lot of presidents have trouble in their first terms" and don't get reelected. And second-term achievements - such as Clinton's - need to be weighed along with setbacks, he said.
Galston also suggested some things may be easier for Obama in his second term given the dynamics of his reelection victory - such as immigration and tax-code overhaul. He has already gotten Congress to raise taxes on the wealthiest Americans, something he couldn't do earlier.
Clinton's second term? "I would judge it as an incomplete success. And its incompleteness is largely his own fault," said Galston, now a senior fellow at the Brookings Institution.
In his book Presidential Power in Troubled Second Terms, Alfred Zacher concluded that only one president had a truly better second term than his first: James Madison, president from 1809-1817.
In Madison's first term, the still-new nation was drawn back into armed conflict with Britain in the War of 1812. But in Madison's second term, the United States scored a dramatic victory in the Battle of New Orleans and the war ended with the Treaty of Ghent. Madison's popularity surged.
Seven others had OK second terms despite setbacks, Zacher wrote, most recently Dwight Eisenhower, Reagan, and Clinton. Reagan, despite Iran-contra, oversaw a major 1986 simplification of the tax code and the unraveling of the Soviet Union. Clinton learned how to reach across the aisle to deal with Republicans on welfare overhaul and deficit reduction and left office with an annual budget surplus - an achievement no other president since Andrew Jackson can claim.