"Baylor fans have been blessed, the nation's been blessed, and he is a better person than a player," Drew said.
Baylor, wearing bright neon-green uniforms, never trailed and led by 18 points less than 8 minutes into the game.
Xavier played from behind all night, but never quit. The Musketeers cut the lead to just three, 71-68, with 22 seconds remaining.
Pierre Jackson had 16 and Perry Jones III had 14 for the Bears (30-7).
Tu Holloway led Xavier with 22 points.
Xavier was hurt by poor shooting, especially from beyond the arc. The Musketeers were 0-for-11 on 3-pointers before Justin Martin ended the drought with 2 minutes remaining. He added another 3 40 seconds later, and Holloway's 3 with 22 seconds remaining cut the lead the 71-68.
Holloway added another basket, but Baylor's Brady Heslip made four consecutive free throws in the final 17 seconds to protect the lead.
Heslip, coming off his career-high 27 points on nine three-pointers against Colorado, had only one trey against Xavier and finished with 11 points.
Baylor also won three NCAA tournament games in 2010 before losing to eventual national champion Duke.
Xavier (23-13) couldn't advance after making the Sweet 16 for the fourth time in five years.
Kenny Frease had 18 points and Mark Lyons had 16 for the Musketeers.
With Xavier stretching its defense and Baylor working the clock, Jackson sank a three-pointer with 2 minutes, 50 seconds remaining to give the Bears their biggest lead of the half at 65-54. Following a missed free throw by Lyons, Acy's basket pushed the lead to 13 points - too much for the Musketeers to overcome.
Heslip surprised the Musketeers by scoring on two drives to the basket in the opening minutes. Heslip added a three-pointer before a jumper by Quincy Miller stretched Baylor's lead to 22-4.
Xavier needed 10 minutes to reach double figures, but a flagrant one foul by Acy on Frease helped the Musketeers recover from their slow start.
Acy, who scored a combined 11 points in the Bears' first two wins in the tournament, knocked Frease to the floor when he hit the 7-foot center from behind with 5:08 remaining in the half. The officials reviewed a video replay before ruling on the severity of the foul.
Frease made both free throws and then scored three seconds later as Xavier retained possession. It was the start of 13 unanswered points as Xavier cut Baylor's lead from 33-16 to 33-29. Jackson ended the run with a three-pointer to give the Bears a 36-29 halftime lead.