Gunshots in his native Liberty City, Fla., were daily alarms. He witnessed robberies. He saw his own home burglarized. Refuge was the football field, which isn't uncommon for NFL players. But even that didn't go as planned - he never played Division I-A football. He was kicked off the team at Fordham, took time off, and finished at small Washburn College in Topeka, Kan.
His path to a three-year, $17 million contract with the Eagles included a practice-squad stint, a reserve role, and eventually a starting spot on a Super Bowl champion.
"You can't let your circumstances dictate your life forever," the 28-year-old Williams said. "And I don't necessarily look at the bad as something constricting. There's always a way out."
Once that contract was signed, Williams' honeymoon was short-lived. The cornerback became one of Philadelphia's most controversial athletes even before playing a down for the Eagles. He missed optional workouts in the spring for his wedding, and then to watch his daughter's dance recital, get dental work, and finish building a home - sconces included.
He was critical of the Eagles defense after a preseason game in which he did not play, and had altercations during practice with the Patriots, a preseason game with the Panthers, and even with his own teammates.
Time with Williams helps you to understand the person, and how he's wired. He wants to be with his daughter because he didn't have that family structure at her age. A home is important to him because he didn't have a stable one. And that edge he plays with is a part of an ethos developed cleaning Frito-Lay trays at midnight in Kansas.
"It's just resilience, man," Williams said. "Being a fighter. Never giving up on my dream and never giving up on the opportunity to put my best foot forward regardless if I was a maintenance guy or I was just a guy who cleaned the streets."
Williams doesn't remember his parents' being together. His mother's schizophrenia would sometimes put her in mental institutions, and his father was forced to raise him and his brother, Ronald, who is one year younger.
"Every now and then, she'd go in and out of our lives dealing with the things she had to deal with," Williams said. "You didn't really have a mom there doing mom things or talk to, or really just pour your heart out if you had a bad day. You didn't have that person who was consoling. My dad was more the guy who was like, 'Get over it.' "
Williams' father, who is also named Cary, was once a standout high school athlete who involved his sons in sports to keep them out of trouble. He worked as a security guard at hotels and at the Miami Herald. He raised his sons until Williams became a teenager.
"It was an abusive situation," Williams said. "He was dealing with anger. In some instances, he didn't know how to handle two kids who were growing. We didn't know how to handle the situation."
There were bruises and cuts. They were concealed, but noticeable. Williams didn't know how to handle it. It ultimately became too much, and he wondered how it would end.
"Are we ever going to get out of this situation?" he thought. "Is this the life I'm going to live for the rest of my life? Is my dad going to continue to beat my ass like this?"
The elder Williams, who could not be reached for comment for this story, told the Baltimore Sun in 2012 that he did get physical with his sons, but he had a different version of how the situation unfolded and wished his sons were not taken away from him.
"There are people out there using drugs, using women, and you come take my kids?" he told the Sun. "I'm not proud of the way I had to discipline those boys. If I did it over again, I would do things differently. But I did what I had to do to keep them above ground."
Williams is careful about how his father is portrayed, because he still cares deeply about him. They maintain a relationship. He reconciled with his father in 2010. His father apologized and explained what life raising two kids on his own was like, Williams said.
"I told him I wasn't mad at him," said Williams, who added he "couldn't ask for a better grandfather" for his daughter.
His cousin, Calvin Golson, took Williams in after child services intervened. Golson became a father figure. Religion became a heavy influence. He spent more time around family.
Williams' father maintained a role, and Williams was a standout football player. But after he spurned an opportunity to play for North Carolina State while waiting for Miami to offer a scholarship that never came, Williams was left with few options. He played for Fordham before his attitude prompted his dismissal.
Back in Florida, exiled from football, Williams worked for DirecTV before the opportunity arose at Washburn. He developed into an NFL-caliber player, and was a seventh-round pick by the Tennessee Titans in 2008. Williams bounced between their roster and the practice squad before signing with the Ravens, and developed into a key player in Baltimore before signing with the Eagles.
After signing, Williams finally gave his fiancée the wedding she wanted. He missed a Birds practice for his daughter's dance recital, insisting that it was important for him to be with her. He said he wanted to finalize his new home because he wouldn't have time during the season.
"I take my responsibilities as a father very seriously," Williams said, "because the reason you have certain things going wrong in society is it starts at the home."
The Eagles had a walk-through last Saturday, two days after Williams fought Riley Cooper at practice. The scuffle was dismissed as two competitive players getting tangled up, but Williams didn't speak publicly the next two days.
At Saturday's practice, defensive coordinator Bill Davis noticed a particular focus and intensity from Williams. Davis called him a "fiery competitor," and said it jumped out at him during an otherwise innocuous walk-through.
"His footwork, his eyes, his hand placement, in a walk-through you would just feel that Cary was really amping up for this game and taking care of the details and the little things," Davis said. "He played the way he practiced, and he competes every down in practice like he did every down in that game."
Williams finished Monday's game with an interception and a sack, playing like the No. 1 cornerback the Eagles envisioned. It was a sterling performance for a player who had not previously endeared himself to his new team's fans.
Williams acknowledges that he can be "misunderstood." It happened in Baltimore, too, where he'd become so passionate that even his teammates would tease him about calming down. He doesn't complain about the way he's perceived in Philadelphia, and said he created the reputation.
Some of his issues aren't completely settled. The situation with his mother still weighs on him. There are lingering issues with different members of his family. It's more than a fan who pays for a ticket to watch him play would know.
"There's a lot of stuff that's still going on that I think about it and I try to release it every time on the field," Williams said. "It's just a lot of things I deal with on a daily basis that most people probably don't."
Williams then said all of the other players in the locker room have their own problems. Every player has his own story. This is Williams' story. And if the story is understood, the person might be understood, too.
"I'll continue to try to be a better person," Williams said.
"To be the best man I can possibly be - the best father, the best cornerback."
Contact Zach Berman at firstname.lastname@example.org. Follow on Twitter @ZBerm.