And what a life it has been.
Brown will start Saturday when Temple plays South Florida at Lincoln Financial Field in a Big East game that is also homecoming.
He grew up in an eight-bedroom, five-bathroom, three-story house that overlooks Lake Ashburton on Baltimore's west side. His father, Warren Brown, is a highly successful and wealthy lawyer who is known as the city's Johnnie Cochran.
But Matt Brown was on the fast track to joining the brutal Bloods street gang, so his father sent him off in 2005 to the prestigious Peddie School in Hightstown, N.J.
"Matt is a combination of the street and the Peddie School," Warren Brown said. "People don't really see the street, because they see the Peddie and so forth."
But in reality, Brown probably could've been incarcerated or dead by now if his father had not sent him to the Peddie School and forced him to spend summers with his younger sister, Candice, at Cushing Academy in Massachusetts.
These warring realities define, in no small part, the Matt Brown of today.
A complex package
Most of us know Brown as a diminutive playmaker. If he has a slice of daylight he is gone. But Brown is a bundle of serious complexity.
A likable guy, he befriends underprivileged youths who live near Temple's campus. "I want to give them guidance," he said.
Yet he can become confrontational and is quick to show his displeasure. That hasn't always kept him in good standing with Temple's former and current coaching staffs.
During his freshman season under coach Al Golden, Brown served a two-game suspension for complaining about his role in a 37-13 victory over Buffalo.
In the spring of 2011, Brown was suspended for getting into a verbal altercation with a teammate and an assistant coach.
"I think Matt just walks around with a chip," said coach Steve Addazio, who replaced Golden in December 2010. "He gets caught up sometimes in some nonsense. I think he's growing up and he's working at that.
"But at the same time, some of that gives him some of the great competitiveness. So it's a double-edged sword."
A former walk-on, Brown is on the watch list for the Paul Hornung Award, given annually to major college's most versatile player.
He is ranked sixth nationally with an average of 21.4 yards per punt return. Brown also averages 20.8 yards on kick returns.
On offense, he has 33 carries for a team-leading 202 yards (6.1 per carry). Brown had a 56-yard touchdown run against Villanova in the Mayor's Cup game.
He is tied for 10th for the most career 100-yard rushing performances (10) among active Football Bowl Subdivision players with South Carolina's Marcus Lattimore and Oklahoma State's Joseph Randle.
Brown's career highlight was a 226-yard, four-touchdown performance at Army as a sophomore.
Brown has had an impressive career, but there is a belief that his size (he is listed at 5-foot-5 and 165 pounds) will keep him out of the NFL.
"How can another person tell me what I am going to do and what I am not going to do?" he asked. "You are not going to determine my fate. You don't control what's going to happen to me.
"Ain't nobody going to take me out of my dream, because that's what I set out to do."
Brown has dealt with folks questioning his small stature since he started playing Pop Warner football for the Northwood Rams in Baltimore.
A renowned program, Northwood produced Baltimore Ravens wideout LaQuan Williams and former NFL players Antonio Freeman, Tommy Polley, and Vincent Fuller.
Northwood won back-to-back national championships in 2001 and 2002.
"If I would have given Matt 10 more carries, I would have went on to win another national championship," Northwood coach Herb Estep said of he the 2004 season ended, with the Rams losing in the second round of the East Regional. "I chose a bigger back over a bigger heart. I regret that every day."
There might be a few college coaches who regret not offering Brown a scholarship.
Without any offers, he played postgraduate football at Milford Academy in New Berlin, N.Y. During Brown's time at Milford, his stepmother, who had divorced Warren Brown, was killed by her boyfriend. Her death deeply affected the young man.
His only Division I opportunity after that prep season was an invitation to walk on at Temple.
After an impressive freshman campaign with the Owls in 2009, Brown was on full scholarship once school began in January 2010.
"What drives me on the football field is where I come from, my environment, my neighborhood, my circumstances," Brown said.
Saving a son
It can be argued that remaining true to his environment and neighborhood are the reason for his off-the-field circumstances. Many of Brown's cousins were involved with gangs.
"And they were influencing him," Warren Brown said. "It was really hanging out with his cousins as opposed to being attracted to the gang thing. He was just attracted to his cousins, his family."
Warren Brown tried a lot of different approaches to keep his son away from the Bloods.
He had Baltimore police officers talk to his son about the dangers of the lifestyle he was pursuing. Homicide detectives even told well-known Bloods "to stay away from Warren Brown's son."
"But the problem was getting Matt to stay away from them," Warren Brown said. "So it was kind of rough there for a minute."
Some of Brown's tattoos reflect that rough time.
Etched on his right forearm is the quote, "Some people keep their life and lose their pride. Some people keep their pride and lose their life."
He wanted that tattoo to remind him of a cousin, Meech Edmonds, who was shot to death three years ago.
Brown said he could have become a statistic himself if his father had not taken him out of that environment. "I still lose family members every day," he said. "Like my cousin, rest in peace."
Nowadays, Brown's biggest thrill comes from proving that he is an elite Division I player.
Football has become the focal point of his life. Brown is living out a dream. His father is, too.
"He's been playing ball since he was 6," Warren Brown said. "He's 23 now. I've never missed one of his games. I never missed even a play."
Warren Brown hopes his son, who is on pace to receive a communications degree next semester, continues to play at the next level: the NFL.
"I'm so concerned that I put it in God's hands," he said. "It's like a lifelong dream to him. I want him to get a taste of it. If he doesn't, he will be miserable."
Miserable enough to resume hanging around gangs?
"Oh, no," Warren Brown said. "He laughs at that. It is beneath him now. While disheartened, he would fill a void by working with children and coaching."
Contact Keith Pompey at 215-854-2939 or at email@example.com. Follow him @pompeysgridlock on Twitter. Read his blog at www.philly.com/OwlsInq.