"I don't know," he answered. "As the saying goes, I have big heels to fill."
Kerry told his new agency's employees that he and President Obama needed their help to make America safer and the world more prosperous and peaceful.
And after noting the deaths of Ambassador Chris Stevens and three other Americans in the Sept. 11, 2012, terrorist attack in Benghazi, Libya, Kerry said the protection of American diplomats would be a top priority.
"I guarantee you that beginning this morning when I report for duty upstairs, everything I do will be focused on the security and safety of our people," he said.
Kerry, 69, is the son of a diplomat and member of the Senate Foreign Relations Committee for 28 years - chairman since 2009. He spoke of his childhood time in Germany and learning the virtues of freedom.
Kerry recounted how, as a 12-year-old in postwar Berlin of the 1950s, he could have caused a diplomatic incident by riding his bicycle around the destroyed and divided German capital, past the burned-out Reichstag and the Brandenburg gate and - using his first diplomatic passport - into the Soviet-controlled eastern part of the city.
"If the tabloids today knew I had done that, I can see the headlines that say, 'Kerry's Early Communist Connections!' " he joked, before describing more seriously how he explained to his irritated parents why he felt the need to cross the Iron Curtain.
"There were very few people," he said. "They were dressed in dark clothing. They kind of held their heads down. I noticed all this. There was no joy in those streets."
Kerry is likely to spend much of the week reaching out to foreign leaders. Over the weekend, he spoke to Israeli Prime Minister Benjamin Netanyahu and Palestinian President Mahmoud Abbas, as well as the foreign ministers of Japan, South Korea, Turkey, Canada, and Mexico.