Brady had struggled to win Mayor Nutter's support for the project, which he considered a "slam dunk" for two areas in constant need of city funding.
"Why isn't that a slam dunk that we shouldn't be pushing as hard as we can for that?" Brady asked after hearing the news. "Tell me what the bad side of this was? I don't understand it. I just don't understand it."
Alan Greenberger, the city's deputy mayor for economic development, said there are still four good applicants for the license. He added the city is hoping the state Gaming Control Board awards the license this summer.
Two of the pending applications are in Center City, the other two are in South Philly.
Another factor cited in Penn National's decision was market saturation.
"The market potential in Philadelphia is less today then when we first applied, as a result of the ongoing gaming saturation in the mid-Atlantic region, as well as continued softness in the economy," the new release quoted Wilmott as saying.
Brady, chairman of the city's Democratic Party, has been pushing the concept since November 2012. Two-thirds of the casino would have been owned by a nonprofit dedicated to funding schools and pensions while the other third would have been owned by Penn National.
The company, which already owns a majority share of a Pennsylvania casino, is prohibited by state law from owning more than a third of another casino license in the state.
Casino developer Steve Wynn also withdrew a casino license application for a project in Fishtown, in November.
The Gaming Control Board, due to meet July 9, has not scheduled a vote on the license.
Doug Harbach, a spokesman for the board, said a filing was received from Penn National and will be reviewed by the board's Office of Enforcement Counsel.
Harbach said "the board is continuing its discussions relative to the issuance of the second license in Philadelphia."
On Twitter: @ChrisBrennanDN