The store, smaller than a rowhouse, is largely the same as the day it opened - six wooden booths, five red-leather-topped counter stools under an antique tin ceiling and two large fans.
Everything looks the same, but the cherubic Groh senses something has changed. His business was down 10 percent in June and 15 percent last month.
When Groh, 50, announced his plan to rename the restaurant - he has worked there for 35 years and bought it from Sherman's widow in 1999 - there were two explosions. The first was in the media. The little steakshop never before received so much attention. The second was backlash from some longtime customers who hated the name-change, calling it a surrender to political-correctness. They have a point.
An online petition to keep the old name drew 10,000 signatures. That's a lot of names. Groh was condemned on social media.
In separate incidents in recent months, Groh's rechristened place had "Chink's" painted on the pavement and on his front window by dead-enders with too much time on their hands.
At the time of the name change, Groh was asked "why" and his answers usually flowed along the lines of: it was time, it is 2013, it was never his intent to hurt anyone's feelings.
Sitting in a back booth at Joe's, I look Groh in the eye and ask him if he really wanted to change the name.
"In all honesty, no," he says, reflecting on its history and it being the only place he's ever worked and the affection everyone had for Chink Sherman.
But, "I am Joe. It's 2013. It was time to do it."
He didn't expect the fury, the cursing, people vowing they would never come in again.
Some have abandoned him, some want to have it both ways. A little old lady, maybe 75, comes in with her son and daughter. She says to Groh, "You make me sick," orders a cheesesteak, sits down, eats it, then walks out telling him, "You still make me sick."
As always, 90 percent of his business comes from outside the neighborhood, a lot of New Jerseyans grab a few sandwiches as they head to the nearby Betsy Ross Bridge, but the recent decline in his business - after a small spike post-name change - "scared me," he says. He's giving some reluctant thought to moving, which would kill the 14 neighborhood jobs Joe's provides.
This address has been his home for 35 years - he even lived above the shop before moving to Cinnaminson, N.J. - but he is a businessman first. The shop is his legacy. His son Patrick, 24, is working there already.
If 10 percent of the do-gooders who cheered him for doing the right thing came in for an occasional steak or shake, he'd be fine.
Good deeds shouldn't carry a high price.
On Twitter: @StuBykofsky
Blog: ph.ly/BykoColumns: ph.ly/StuBykofsky