PSSA-cheating reforms yield lower scores across Pa.

Posted: July 30, 2012

After authorities imposed unprecedented security measures on the 2012 statewide exams, test scores tumbled across Pennsylvania, The Inquirer has learned.

At some schools, Pennsylvania Secretary of Education Ronald Tomalis said, the drops are "noticeable" - 25 percent or more.

In some school systems, investigators have found evidence of outright doctoring of previous years' tests - and systemic fraud that took place across multiple grades and subjects.

In Philadelphia and elsewhere, some educators have already confessed to cheating, and investigators have found violations ranging from "overcoaching" to pausing a test to reteach material covered in the exam, according to people familiar with the investigations.

A "significant" number of administrators at 53 city public schools under investigation - some after admitting to cheating - have agreed to cooperate with investigators, sources said. The sources declined to name them.

In other parts of the state, including schools in Columbia and Beaver Counties, principals have already been disciplined for cheating on exams.

Statewide results of the exams, the Pennsylvania System of School Assessment, have not yet been made public. Their typical release date of late summer will probably be pushed back because of the scope of the investigation.

"I expect that it will have an impact on statewide scores," Tomalis said of the cheating investigations. Philadelphia student scores dropped in every grade, with double-digit drops in grades 3, 4, and 5, sources said. And the number of schools that made "Adequate Yearly Progress" under the federal No Child Left Behind law plummeted, the sources said, though AYP standards also got tougher this year.

While acknowledging the drops, a spokesman stressed that the school district has implemented new, stringent security measures. "While scores still have to be validated, the preliminary 2012 PSSA results are deeply disappointing. We are committed to understanding what caused the decrease," said Fernando Gallard, the spokesman.

State education officials put local schools on notice last year that they planned to intensively scrutinize the 2012 exam, paying special attention to schools suspected of cheating in the past.

In many of these schools, investigators found, erasure patterns reverted to normal, with fewer answers changed and more questions simply left blank. Often, scores in those schools fell dramatically, the sources said.

For the first time, before the 2012 exams were given out, school personnel were required to sign documents acknowledging that cheaters could be subject to criminal penalties.

In launching the inquiry last summer, the state identified 48 school districts or charter programs with apparently suspicious results in the statewide exam in 2009, 2010, or 2011.

Of those, 30 have now been officially cleared, including several local districts and charters.

Of the remainder, 10, including the Philadelphia School District, remain under investigation, with "extremely strong" data indicating that cheating took place.

Other schools in the region in this category include the city charters Imhotep, Philadelphia Electrical and Technical, Walter Palmer Leadership and Learning Partners; and Chester Community Charter.

In three districts, Berwick, Big Beaver Falls, and New Kensington-Arnold, probes have been closed, documents obtained by The Inquirer show. A principal in Berwick and a teacher in Big Beaver Falls have been suspended. The New Kensington-Arnold investigation did not result in any personnel action.

And five more districts remain under continued monitoring. There, the data point to improprieties, but investigators could not pinpoint who was to blame and have indicated they will be watching testing procedures closely.

Thousands of hours

Multiple agencies - from the state Department of Education and the Inspector General's Office to local districts - have spent thousands of hours over the last year investigating dozens of Pennsylvania schools for possible cheating. Original student test booklets have been unearthed from a warehouse in Minnesota.

The Inquirer began reporting about possible cheating in the Philadelphia School District last year, when teachers at Roosevelt Middle School in East Germantown said improprieties encouraged by administrators were responsible for huge jumps in 2009 test scores.

Later, teachers at Cayuga Elementary in Hunting Park, another school with impressive test-score jumps, told The Inquirer that they, too, witnessed cheating by administrators.

Soon after the Roosevelt teachers came forward, a long-buried 2009 forensic analysis of that year's state exams surfaced that showed possible cheating statewide.

In early 2011, Tomalis, new to the state's top education job, inquired what kind of forensic report had been done on state tests the prior year.

"The answer was none," Tomalis said in an interview.

He quietly ordered it reactivated. Last summer, after the revelation about the 2009 review, Tomalis announced that 2010 and 2011 forensic analyses were under way, as well as a statewide investigation in the 48 districts and charters.

In a lengthy interview in Harrisburg, Tomalis and Carolyn Dumaresq, deputy secretary for elementary and secondary education, walked Inquirer reporters through the sophisticated and multilayered techniques used to unearth cheating.

Districts that had been flagged were given the benefit of the doubt, invited to explain why their scores had jumped, why erasure patterns seemed abnormal.

On average, every student will erase a wrong answer and change it to the correct answer once on an exam, experts told state officials. But officials were conservative; they flagged only those schools where 10 percent of students had at least five wrong-to-right erasures.

Asked to explain their suspicious scores, some districts pointed to new curricula. Others talked about different teacher training or input errors.

Some stories checked out, and those schools were cleared. Others had no explanations or their stories didn't ring true - one blamed excessive erasures on the fact that the school had a higher number of immigrants enrolled. Investigators said that, in fact, English-language learners have no such tendency to erase more.

Handwriting experts were even brought in. Their analysis showed that in some cases, the same person had filled out test after test.

The experts even scrutinized the way test answer "bubbles" were filled in. They found that the handwriting changed in instances where results were erased - another sign someone apparently had altered results after the test was over.

In some cases, investigators also found wrong-to-right answers didn't match student work. A student working out a problem, for example, might write out 5 + 5 = 11 in his booklet. That answer would be bubbled in, then erased and changed to the correct answer with no "corollary calculation" in the booklet, Tomalis said.

"There's multiple layers of evidence pointing to things like that happening," he said.

Just to be sure, the state also looked at schools' erasures on science exams, which don't trigger sanctions against a school for poor performance.

Some schools' tests had suspicious erasure patterns in reading and math but perfectly normal ones in science.

As a result of this intensive statewide scrutiny, patterns emerged. In some cases, officials saw improprieties across grades. In others, they could pinpoint a single grade, or even one teacher.

Districts or charter officials with clear problems were summoned to Harrisburg and shown the data.

Officials were told: "You have a responsibility to fix this," Tomalis said.

Most leaders were cooperative, yet angry about what had happened in their schools. One leader was particularly "irate," the education secretary said, because he had been unfairly hounding fifth-grade teachers to bring up their scores.

The investigation in that school - which the sources declined to identify - revealed that fourth-grade scores had been inflated through cheating, leading to unrealistic expectations for fifth graders.

Where there was evidence of possible cheating over multiple years and across grades, state officials assigned members of the Inspector General's Office to investigate.

"It takes an awful lot of resources to go through each of the schools to try to isolate where the problem is," said Tomalis. "What we wanted to do was only focus our resources in those areas where we believe there was a very serious area of concern."

For the 10 districts and charters still under investigation, including Philadelphia and four area charters, it's not clear how much longer the probes will take.

"We look forward to wrapping up the investigations as soon as possible," Tomalis said, "but we will take the necessary time to conduct a comprehensive review."

'Direct ties'

In some Philadelphia schools under scrutiny, the evidence "is pointing in the direction of manipulation across grades, subject areas, across years - it's glaring," a source told The Inquirer earlier this year.

With strong evidence against them, some of the 53 district principals under the microscope have confessed to investigators, sources say.

"There have been confessions across the district in the affected schools," said a source with direct knowledge of the investigations. Officials "have been able to ascertain direct ties" between some administrators and suspicious test results.

Overall, "it's more at the assistant-principal level, but there were some cases at the principal level. And there were sporadic cases of classroom teachers" cheating, the source said, adding that in most cases, improper erasures happened after students turned in booklets.

Robert McGrogan, the head of the Commonwealth Association of School Administrators, the union that represents district principals and assistant principals, said that he could not discuss confessions specifically or cheating allegations generally.

"We are involved with several confidential matters related to the investigations that are ongoing," McGrogan said. Announcements will be made through the district and state Education Department, and McGrogan said he couldn't talk until then.

"This is a tough, tough, tough thing," said McGrogan.

Specific personnel actions are up to individual districts, but only the state can decide if an educator should lose his or her license for testing infractions.

Those decisions have not yet been made.

Criminal charges cannot be lodged for cheating on the PSSA tests administered before this year, and all the confessions apply to PSSAs from 2009, 2010, and 2011.

But under extraordinary security measures imposed on city schools this year, those responsible for administering the 2012 tests had to sign statements acknowledging that criminal penalties can be sought if wrongdoing is discovered.

Even without criminal penalties hanging over the 2009, 2010, and 2011 PSSAs, "there could be some very serious ramifications involved with some of those scores," Tomalis said.

The steroids era

Officials acknowledge that while most administrators and teachers followed the rules, those who cheated have blemished years of test gains.

"There truly was an artificial elevation of test scores," one source said.

Like the steroids era in baseball, some districts' test results from recent years could carry the equivalent of an asterisk.

It remains to be seen what ramifications that will have in Philadelphia, where officials have repeatedly boasted of multiple years of continued test-score growth. Former Superintendent Arlene C. Ackerman, who led the district during the years now being scrutinized, touted that test-score growth as a chief accomplishment of her administration.

Statewide, "I think the 2012 scores will be the first time that we can have strong confidence that these scores accurately reflect student achievement," said Tomalis, who likened it to hitting "the reset button."

Going forward, most of the changes put in place for the 2012 PSSAs will continue.

Beyond the statements test-givers now must sign acknowledging potential criminal liability for cheaters, there will continue to be stricter rules about proctoring.

Teachers in the districts and charters where the allegations were most serious - including Philadelphia - have been prohibited from administering their own students' exams. This year, there were state monitors inside classrooms and stringent new rules about who could have access to the tests.

"Part of the investigation," Tomalis said, "is to let people know that they are being observed."

In future years, tests will also be changed to make questions random - in other words, two students sitting next to each other won't have questions in the same order. That will make it difficult for one person to erase the answers of many students.

Cheating was the exception, not the rule, Tomalis said. "We have thousands of public schools where we see no problem," he said. "Overwhelmingly, the professionals in our education system are very serious about doing the right things as far as the assessments are concerned."

But, he said, cheating can always happen when no one is watching.

"The children are swept up in this," Tomalis said, "when the adults try to beat the system."


Investigations Of Cheating

Cheating investigations

The Pennsylvania Department of Education announced last year that it was investigating 48 districts or charters for possible cheating on state exams.

Here are the statuses of those investigations:

ONGOING

Allegheny County, Pittsburgh SD; Berks County, Reading SD; Dauphin County, Harrisburg SD; Delaware County, Chester Community Charter; Lackawanna County, Scranton SD; Luzerne County, Hazleton SD; Philadelphia County, Imhotep Charter; Philadelphia County, Philadelphia Electrical and Technical Charter;

Philadelphia County, Philadelphia SD;

Philadelphia County, Walter Palmer Leadership and Learning Partners Charter

CLOSED

Beaver County, Big Beaver Falls Area SD; Luzerne County, Berwick Area SD; Westmoreland County, New Kensington-Arnold Area SD

CONTINUED MONITORING

Lancaster County, Lancaster SD; Northampton County, Bethlehem Area SD; Pike County, Delaware Valley SD; Westmoreland County, Derry Area SD; Westmoreland County, Monessen City SD

CLEARED

Adams County, Lincoln Intermediate Unit 12; Allegheny County, Gateway SD; Beaver County, Ambridge Area SD; Beaver County, Pennsylvania Cyber Charter; Berks County, Governor Mifflin SD; Bucks County, Bristol Borough SD; Bucks County, Pennsbury SD; Delaware County, Wallingford-Swarthmore SD; Delaware County, William Penn SD; Fayette County, Connellsville Area SD; Fayette County, Uniontown Area SD; Franklin County, Chambersburg Area SD; Lackawanna County, Abington Heights SD; Lancaster County, Ephrata Area SD; Lebanon County, Lebanon SD; Monroe County, Pleasant Valley SD; Montgomery County, Cheltenham SD; Montgomery County, Spring-Ford Area SD; Northampton County, Easton Area SD; Philadelphia County, Alliance for Progress Charter; Philadelphia County, Charter High School for Architecture and Design; Philadelphia County, Maritime Academy Charter; Philadelphia County, Northwood Academy Charter; Philadelphia County, Wissahickon Charter; Schuylkill County, Minersville Area SD; Schuylkill County, North Schuylkill SD; Tioga County, Wellsboro Area SD; Westmoreland County, Belle Vernon Area SD; York County, Dallastown Area SD; York County, South Western SD


Contact Kristen Graham at 215-854-5146, kgraham@phillynews.com, or on Twitter @newskag. Read her blog, "Philly School Files," at www.philly.com/schoolfiles.

comments powered by Disqus
|
|
|
|
|