"I just missed it," Brown said. "I knew I wanted to get back."
Before this season, Brown hadn't been on the sideline since being fired by the Charlotte Bobcats in December 2010. He hadn't coached in college since guiding Danny Manning and Kansas to the 1988 NCAA championship.
But he landed the job at SMU and the program is beginning to come to life.
Season-ticket sales are up 79 percent, according to the New York Times, and the university expects to complete a $40 million renovation of its on-campus facility, Moody Coliseum, by the end of 2013, shortly after the Mustangs are scheduled to move from Conference USA to the Big East.
Brown said his players are buying into the new regime and recruiting has gone well.
"They better get us this year," Brown told reporters after a recent game, "because we're going to be pretty good pretty quickly."
Since he left Kansas 24 years ago, the culture of college basketball, but especially recruiting, has changed dramatically. No argument from Brown, who was Dean Smith's freshman coach at North Carolina in the mid-1960s when he helped recruit Charlie Scott to become the school's first black scholarship athlete.
Brown recalls meeting with high school coaches, parents, and guidance counselors while recruiting a prospect. Now often it's AAU coaches and various other interested parties. "Then you were recruiting juniors, not seventh or eighth graders," he said.
Brown will tell you he's had lots of good years in the game. And plenty of former players he treasures.
Perhaps no player is more identified with him than Iverson, who once called Brown the greatest coach in the world.
"People are not sure who I am," Brown said, "but they know I coached Allen."
At SMU, Brown hopes to create more memories.
"We've been fooling 'em so far," he said. "We've got to get to next year, then we'll have some fun."