Up to 1,000 people were forced from their homes in the middle of the night in the town, which is about 155 miles east of Montreal and about 10 miles west of the Maine border.
The derailment caused several tanker rail cars to explode in the downtown core, a popular area known for its bars that is often bustling on summer weekend nights. Police said the first explosion tore through the town shortly after 1 a.m.
The fire spread to a number of homes in the lakeside town of 6,000, and witnesses said the flames shot up highter than the steeple on a nearby church.
Flames and billowing black smoke could be seen more than 12 hours after the derailment, which involved a 73-car train.
"When you see the center of your town almost destroyed, you'll understand that we're asking ourselves how we are going to get through this event," an emotional Mayor Colette Roy-Laroche told a televised news briefing.
The cause of the derailment was not immediately known.
Dozens of residents gathered hours after the explosion at the edge of a wide security perimeter and many feared the worst. About a half mile down the main street, flames danced around a railway tanker that sat near the road.
"On a beautiful evening like this with the bar, there were a lot of people there," said Bernard Demers, who owns a restaurant near the blast site. "It was a big explosion. It's a catastrophe. It's terrible for the population."
Demers, who fled his home, said the explosion was "like an atomic bomb. It was very hot. ... Everybody was afraid."
Resident Claude Bedard described the scene as dreadful. "The Metro store, Dollarama, everything that was there is gone," he said.
Environment Quebec spokesman Christian Blanchette said a mobile laboratory has been set up to monitor air quality.
The train, reportedly Maine-bound belongs to Montreal Maine & Atlantic.