"I'm not waiting anymore for these people to get their act together," Christie said, referring to town officials who have yet to spend their money. He is just enforcing a law that set a deadline of four years to use the money, he argues.
But mayors have cried foul and question how the state has determined what they owe. They say that the money, from fees paid to them by local developers, should stay in their towns and that the economy affected their ability to put it to use in time.
Some say Christie is just grabbing cash to support his tax-cut proposal. And they wonder whether the state agency to which they are supposed to send the money even exists.
"The man knocking on the front door of Town Hall who was driven there in an armored car? That's Chris Christie," said Adam Gordon of the nonprofit Fair Share Housing Center.
An Assembly hearing on the issue is set for Wednesday, and there already are court challenges. That means the New Jersey quandary - how and where to house the state's poor, working-class, and disabled - will remain unresolved.
For four decades, America's most densely populated state has grappled with the problem. There are simply too few cheap places to live: A quarter of New Jersey renters devote half of their pay to housing, when 30 percent is "affordable," according to the Anti-Poverty Network of New Jersey.
Can a town, through zoning, keep out those who can't afford its existing housing? If not, can the state intervene?
Those questions, touching on issues of jobs, race, class, sprawl, environmental preservation, and, of course, taxes, have wrought controversy and confusion.
Opponents of housing regulations say they have forced development, depressed property values, raised taxes, destroyed open space, and caused traffic.
But others say the state must ensure that apartments and cheap houses coexist with strip malls and office parks. Otherwise, the economy stagnates and the poor are ghettoized in dying cities far from jobs.
A 1970 decision by Mount Laurel officials to forbid construction of garden apartments for poor black residents forcibly displaced by suburban housing led to the state Supreme Court's 1975 "Mount Laurel decision." That resulted in the housing quotas towns now face.
The case, which set a national precedent, led to 60,000 new homes for low- to moderate-income New Jerseyans. Families of four with household income of up to $70,000 a year are eligible, as are low-income seniors and those with special needs.
In conjunction with that housing, 100,000 homes for the middle class have been built, experts say.
But the often-confusing process of creating the housing has provoked frustration.
Since his campaign, Christie has promised to "gut" the state agency that enforces court mandates on the housing each town must build. The Council on Affordable Housing (COAH) is wasteful and bureaucratic and its methods "arbitrary," he says. He wants more local control.
COAH is popular with neither party. Legislation to eliminate it passed the Democratic legislature in 2011, but Christie would not sign it.
Its rule that most towns keep 10 percent of their housing affordable is burdensome, he said. He wanted to exempt built-out towns from further obligations.
Last year, Christie used his executive power to eliminate COAH. The agency disappeared, but an appellate court ruled the move unconstitutional.
So, on July 24, Christie sent letters to 372 towns demanding they return their trust fund money by Aug. 13.
The League of Municipalities protested: Christie can't take the money without the COAH board's approval, they said, and that board has not met in more than a year.
Christie has pushed ahead anyway, even rejecting a Republican bill to extend the deadline. Community Affairs Commissioner Richard Constable III has said the seized money would go toward housing needs, from Section 8 vouchers to homeless shelters to housing for the disabled.
But Christie's budget says otherwise: Some of the money is earmarked for health programs, welfare, and prison initiatives that otherwise would be billed to the general fund.
"We're very concerned that this money is being taken out of towns' budgets . . . to backfill the governor's tax cut," said Staci Berger of the Housing and Community Development Network of New Jersey.
The administration denies that. All programs to benefit from the money will "support decent, affordable housing," a spokeswoman for Constable said. The state will match some of the money to rehab 88 homes for 300 people with developmental disabilities, she said.
Christie is unlikely to collect the full $141 million he wants. Some local officials say they will hold on to the money and prove to the state that they have plans for it.
Cherry Hill is holding on to $1.6 million because it says it has contracted with Camden County for repairs to its affordable housing. Gloucester Township is keeping $700,000 for land it has bought for senior housing. Pemberton Borough is using $313,000 to rehabilitate four apartments and build duplexes on an old school property.
"Trust me, I don't have any intention of writing a check that large and sending it to the state," Pemberton Mayor William Kochersperger said.
Mayors blame the state for not drafting regulations that would help them meet the deadline. The law says they must "commit" the money, and officials say they don't know why their efforts to do so have not been recognized.
"Tell us what the guidelines are, and we'll comply," said lawyer Jeffrey Surenian, whose firm represents 40 municipalities in COAH matters.
The depressed economy has required more time to use the money, local officials say. Developers can't get the rest of the funding they need to build.
Constable said the towns are making excuses. In an interview, he said that they had ample time and that Christie was following a law signed by his predecessor, a Democrat. "They should have been lobbying Jon Corzine, not Chris Christie," Constable said.
If a town's money is seized, its obligation to build affordable housing continues. But without money, mayors say, they must raise taxes to subsidize construction.
The state wants $247,000 from Evesham, which Mayor Randy Brown says recently approved a program to aid low-income homebuyers.
"We believe it meets COAH's needs. We have no intention of turning any money over," said Brown, a Republican and Christie supporter.
He acknowledged that Christie's deadline forced the town to act more quickly.
But the mayor, who is on the League of Municipalities' executive board, said he was "disappointed in the demanding of this money."
Affordable housing is "the most confusing aspect of my job" and the state has not cleared up the confusion, he said.
"COAH exists, COAH doesn't exist. . . . You need this number of houses in. You need to put this number of houses in," Brown said. "Which COAH - that we thought didn't exist - are we following?"
Contact Matt Katz at 609-217-8355, email@example.com, or follow on Twitter @mattkatz00. Read his blog, "Christie Chronicles," at www.philly.com/ChristieChronicles.