"We received complaints about this location for many years," Bengal said.
Bamont would never let agents inside the house, he said.
On Wednesday, an agent responded to another complaint and saw a dead cat in the foyer, Bengal said. That cleared the way for the search.
The house was flea-infested, with "feces everywhere," Bengal said.
"I think the lady is a rescue lady - has been for years - and this turned into a hoarding situation," he said.
Some of the cats appeared emaciated, he said.
Bengal said a younger man had been living at the house and was in jail after being arrested recently in a domestic incident.
Sources identified the man as Troy Robinson, 50, who has a lengthy criminal history. He has served time in prison for attempted rape, burglary, risking a catastrophe, and criminal trespassing. He also has been convicted of insurance fraud, stalking, terroristic threats, harassment, simple assault, false impersonations, theft, and intimidating a victim or witness.
Bengal said the man also might be charged with animal cruelty.
A man who appeared at the house after the cats were removed said he performed maintenance work there and helped Bamont clean litter boxes and shop for cat supplies. He described Bamont as well intentioned and caring.
He said Bamont had called him, crying and complaining that the authorities were taking her cats.
Elaine Stewart, who said she was a recent acquaintance of Bamont's, also showed up at the house after hearing about the raid.
Stewart said that she had been in the house in July and that conditions did not appear terrible.
"It was not great, but it was not anything that raised red flags," Stewart said.
She said that she had been calling Bamont recently to offer to help with the cats, but that Bamont did not return her calls.
A couple who live nearby who did not want to be named said they had three cats go missing over the years and believed Bamont may have taken them as rescues.
Bamont bought the brick house with black shutters and two-columned white portico in 1973 for $100,000, her daughter, Susan Sauer, said last year. The seller was Thomas Lawton, who bought the house from Princess Grace's mother three years earlier.
A historical marker was installed at the house a year ago.
As she walked her dog, Wilson, in front of the house, Blaire Kelleher said: "It's the Grace Kelly house. It's gorgeous. Now it's crazy inside?"