"This town really got hammered. If there's one more storm, it's going under," said the fisherman, who gave his name only as Andy K, 59, of nearby Toms River.
Mantoloking lost nearly 60 homes and saw dozens more all but destroyed during Sandy, which divided the town when the ocean and the bay met on Route 35. The town, one of the wealthiest per capita in America, is not surrendering itself to the sea, however.
"You know what they say. 'Home is where the heart is,' and here, our home is vulnerable," said William deCamp, whose Bay Avenue home was "lucky" to have merely gotten a foot of water into its first floor.
On Sunday, sounds of cranes, bulldozers and table saws were everywhere, a soundtrack that played out the entire summer, residents said. Some of deCamp's neighbors were already raising their century-old homes, their foundations perched high above the ground on wooden blocks.
"Everybody's doing what they need to do," said resident Chris Nelson, Mantoloking's special counsel for the Sandy recovery. "There are lots of sounds of construction everywhere, building up and tearing down."
On Route 35, Clifford Lewis was pruning pine trees outside the three-story, cedar-shake house he's lived in for nearly three decades and lamenting the loss of the town's cherry trees for road construction. Lewis, 89, and wife Jackie, 83, rode out the storm at home, although they wouldn't do it again.
"We should have left," he said, laughing. "We got lucky."
Like the fisherman, most of Mantoloking is trying to think of the next storm and how to bolster itself against it. The answer, officials said, is a 4-mile steel wall that will run the entire width of Mantoloking's coastline. The $40 million wall will rise 16 feet above the ground and will be covered by sand, although the process has hit some snags.
Seven homeowners in Mantoloking have refused to sign easements to permit the town to construct the wall on portions of their oceanfront property.
"We've moved into legal proceedings to begin the eminent-domain process," Nelson said of the holdouts.
The wall is needed, deCamp said, if people still want to live on this small sliver of a town, but he worries about the long term, about sea-level rise and what, if anything, could ever stop the bay from flooding Mantoloking from the back end.
"The wall will buy us time, a lot of time," deCamp, president of the nonprofit Save Barnegat Bay, said. "I don't really hear any solutions to the problem of sea-level rise, though."
Mantoloking, with little long-term parking and no retail, has gotten used to post-Sandy tourism, deCamp said, visitors taking pictures of houses torn in two, or the tattered American flags waving from uninhabitable homes.
Not everyone welcomed the tourists, though.
"I understand it's hard to believe," a man told three women from nearby Bay Head who'd hiked up the sand to take pictures of his foundation, "but can you go look at someone else's ruined home?"
On Twitter: @JasonNark