The home's owner, New York advertising executive Madonna Badger, and a male acquaintance escaped the blaze, which killed her parents, who were visiting for the holidays, and her three daughters, a 10-year-old and 7-year-old twins, police said.
The severely damaged $1.7 million house was torn down after the Buildings Department determined it was unsafe and ordered it razed, local fire Chief Antonio Conte said.
Neighbors said that they awoke shortly before 5 o'clock Christmas morning to the sound of screaming and rushed outside to help but could do nothing as flames devoured the large, turreted home.
The acquaintance who escaped the fire with Badger was a contractor working on the home, police said.
Johnson most recently worked as a Santa this year at the flagship store of Saks Fifth Avenue in Manhattan, a store spokeswoman said.
"Mr. Johnson was Saks Fifth Avenue's beloved Santa, and we are heartbroken about this terrible tragedy," spokeswoman Julia Bently said in a statement.
Holmes, who worked with Johnson for more than a decade at Brown-Forman, remembered his co-worker as a big man with white hair and a commanding presence.
"He was a man of not a lot of words, but when Lomer spoke or gave his opinion, it was always well thought out," Holmes said.
He said he was a bit surprised that the longtime security chief had become a department-store Santa, but added, "I could see Lomer doing something like that because Lomer had a passion for people."
During Johnson's long career with Brown-Forman, whose many brands include Jack Daniel's Tennessee Whiskey and Southern Comfort, he was responsible for security and safety at the company's headquarters and production plants.
Badger, an ad executive in the fashion industry, is the founder of New York-based Badger & Winters Group. She was treated at a hospital and was discharged by Sunday evening, a hospital supervisor said. The acquaintance also was hospitalized, but his condition wasn't released.
Property records show that Badger bought the five-bedroom, waterfront home for $1.7 million last year. The house was situated in Shippan Point, a wealthy neighborhood that juts into Long Island Sound.
The fire was Stamford's deadliest since a 1987 blaze that also killed five people, Conte said.