The appeals court vacated Weaver's 14- to 24-year jail sentence and ordered Common Pleas Judge Stanley Kubacki to conduct a hearing to force the prosecutor to explain why he failed to accept any black people as jurors.
Kubacki had earlier rejected Weaver's claim of jury-selection discrimination.
Weaver was convicted of involuntary deviate sexual intercourse, possession of a controlled substance with intent to deliver and conspiracy.
Assistant District Attorney Richard Sax said Weaver and Eric Alston, 23, were inside their Cobbs Creek Parkway apartment when they attacked two women on Oct. 24, 1987. Alston is serving 22 1/2 to 45 years in jail for the crime.
Sax has denied any racial bias in jury selection. He argued that he was not required to say why he challenged prospective jurors because the trial judge didn't accuse him of discrimination.
The prosecutor also pointed out that Weaver's attorney exhausted his juror challenges to exclude whites, then asked for a mistrial on the basis "of discriminatory selection."
In addition, Sax said the case was not racial because both the victim and Weaver were black.
The Superior Court, however, said the trial judge was wrong not to find evidence of discrimination in jury selection.