Matt Read and Claude Giroux (winner) scored in the shootout for the Flyers.
Bryzgalov (27 saves) made his biggest save when he poke-checked the puck away from David Clarkson on the Devils' second shootout attempt. Patrik Elias fired wide on New Jersey's last try.
The Flyers ended a seven-game losing streak to the Devils, including four defeats in last year's playoffs.
"It was definitely a must-win," Max Talbot said.
The Devils had a pair of two-on-ones in the opening minutes of the third period. Defenseman Kimmo Timonen broke up one of them, and the Devils' Adam Henrique fired wide on the other.
New Jersey had a late power play because of a Brayden Schenn charging penalty with 1 minute, 42 seconds to go in regulation, but the Flyers escaped. Schenn went high on Peter Harrold with a hit near the boards and may get suspended.
Earlier in the week, Bryzgalov said the Flyers were "done" if they lost both games to the Devils. They were listless in a 5-2 loss in New Jersey on Wednesday, prompting Holmgren to tear into the team before Thursday's practice.
Winger Simon Gagne said Holmgren chastised the team for its listless play, "but at the same time, he still believes in this team. . . . It's definitely going to help us to play better."
Gagne added that "sometimes you need those little meetings to get the truth in your face a little bit to get to the next step."
Forward Danny Briere said Holmgren's talk helped.
"I think we played much harder than last game," he said after just the Flyers' second win in their last six games. "We won a lot more battles, we were a lot more aggressive than last game. But the big thing is, last game we took way too many chances that resulted in goals for them. We were more patient and played a smarter game."
Trailing by 1-0, New Jersey tied it as Clarkson ended a long scoring drought by redirecting Bryce Salvador's point shot past Bryzgalov with 13:51 left in the second period. It was Clarkson's first goal in 14 games.
The Flyers had taken a 1-0 lead when Talbot converted a slick backdoor pass from Gagne with 4:39 to play in the opening period.
The Flyers, with Bryzgalov leading the way, killed a five-on-three for 1:32 in the opening period. Their penalty kill was 5 for 5 in regulation.
"Just basics, hard work, and working for each other," defenseman Nick Grossmann said.
Defenseman Luke Schenn, who missed the loss in New Jersey because of the flu, returned and led the Flyers with nine hits.
New Jersey played most of the game without talented center Andrei Loktionov, who was injured in the first period.
The Devils outshot the Flyers, 28-22, and the hosts managed just 12 shots over the last 45 minutes. But Bryzgalov bailed them out.
"I just know the value of every game and every point," Bryzgalov said when asked if he felt pressure because of his statement about the importance of the home-and-home series with the Devils.
Contact Sam Carchidi at firstname.lastname@example.org. Follow on Twitter @BroadStBull.