Fatigue kills

Video camera (right) records Nicholas Michael, a Rutgers Ph.D. candidate, demonstrating software that tracks eyelids and measures how fast they close, research under way at Rutgers.
Video camera (right) records Nicholas Michael, a Rutgers Ph.D. candidate, demonstrating software that tracks eyelids and measures how fast they close, research under way at Rutgers.

How to warn the sleepy: Scientists look to eyelids and voices to measure the alertness of pilots and drivers.

Posted: February 28, 2011

There was nothing wrong with the twin-engine plane before it slammed into a west Kansas field at 230 miles an hour, breaking apart in a fireball and killing the pilot and two passengers.

Nothing wrong with the pilot, either, as far as colleagues could tell. He had seemed active and alert before takeoff, eating snacks in the pilot lounge.

Yet he had been awake for nearly 21 hours and on duty for more than 14, leading federal investigators to conclude that fatigue was the probable cause of the 2004 crash.

Could someone have sounded the alarm before it was too late?

Fatigue is an epidemic in this country, and experts say it is both underrecognized and dangerous - accounting for more than 20 percent of transportation accidents, by some estimates. And while scientists can tell if a person is too tired to function well, there is no good way to evaluate anyone who can't stop what they're doing to take a test, such as pilots, truck drivers, surgeons, or astronauts.

So researchers at Pennsylvania State University are looking for clues in a characteristic readily accessible from people who communicate by radio: the human voice. They have gotten a recording of the airborne communications during the fatal Kansas flight from the National Transportation Safety Board, and in the lab, they have analyzed the voices of people who have stayed up all night.

At Rutgers University, meanwhile, computer scientists are using cameras and computer software to monitor another physical trait in tired people: the speed at which they open and close their eyelids when blinking.

Such techniques are desperately needed, said safety board member Mark R. Rosekind, who is himself a longtime sleep researcher.

"In most cases, we probably underestimate the role of fatigue," Rosekind said. "The reason for that is, we don't have a fatigue-alyzer."

The evidence is nevertheless troubling:

A study last year by the AAA Foundation for Traffic Safety estimated that one in six fatal crashes involves a drowsy driver. In a survey portion of the study, more than one in four motorists said that within the last month they had driven while being so sleepy that they had trouble keeping their eyes open.

Though various estimates have found fatigue to be a factor in more than 20 percent of vehicle accidents, it is usually not reported as a cause, Rosekind said.

And in a 1999 NASA survey that Rosekind helped conduct, 80 percent of regional airline flight crew members acknowledged having "nodded off" during a flight at some time.

The danger starts well before the person is asleep, and it goes beyond poor reaction time, said Cynthia M. LaJambe, a sleep researcher and psychologist at Penn State. Faced with a problem - declining altitude, say, or a flashing light on the dashboard - tired people start to make bad decisions.

Or they become unable to make decisions at all.

"You keep thinking of the same solution over and over, and you lose the creativity to think of . . . some alternative," LaJambe said. "This is the kind of thing that kills people, because they lose the ability to think their way out of dire situations."

No one will ever know just what happened to Brandon Bow, the Kansas pilot whose "air ambulance" plane crashed near Dodge City after he delivered a patient to Wichita.

But his voice is preserved in recordings of his conversations with air-traffic controllers - including one from 13 minutes before the accident and one from another flight six hours before.

LaJambe said Bow sounded "more fatigued" in the later recording, just before the crash. But she needs something better than that - some mix of vocal characteristics that could be analyzed by computer.

Last year LaJambe and colleagues, including investigator Malcolm Brenner from the safety board, presented findings from an initial study seeking to answer that question.

Thirteen people were kept awake for 36 hours, then asked to complete two tasks: counting quickly from 90 to 99, and reading aloud a short passage about the science and myths of rainbows.

The researchers then compared these samples with recordings of the participants going through the same exercises the day before.

There were several distinct differences in the sleepy speech, including a decline in one of the component frequencies that make up the vowel sound "eee," said Robert A. Prosek, a Penn State professor of communication sciences and disorders. More work is needed to establish a vocal "fingerprint" of fatigue, said the researchers, who presented the findings at a meeting of the Acoustical Society of America.

Closer to fruition, perhaps, is the eyelid-based work at Rutgers, led by computer science professor Dimitris N. Metaxas. The concept began in the early 1980s with Walter Wierwille, a Virginia Tech engineer who found a correlation between eyelid closures and the ability to stay in the correct lane on a driving simulator. He measured what percent of the time a person's eyes were at least 80 percent closed.

The concept was far more reliable than other drowsiness-detection methods that had been tried over the years, said David Dinges, a prominent University of Pennsylvania sleep researcher. He was enlisted by the federal government to compare the various approaches (not including voice analysis). He tested each of the methods - including one that measured how well drivers stayed in their lane - by seeing how well they correlated with reaction time, a known indicator of alertness.

"Eyelid closures just clobbered everything else," Dinges said. "They even were better than subjects' own ratings of their sleepiness."

But how to measure something like that in a real vehicle? Dinges and colleagues measured the reflection of infrared light off the retina, but it was cumbersome and subject to error if people turned their heads to either side.

Metaxas, who worked with Dinges at Penn before moving to Rutgers, is using computer vision. A simple webcam is combined with software that continuously tracks the eyelids, no matter the head's orientation, and measures how fast they close. At any given instant, it looks at the color of each pixel to determine what percent closed the eyes are.

When alert, a person needs about a tenth of a second to close the eyelid, Metaxas said. For a tired person, closure takes three times that long. Metaxas said he had already talked to auto manufacturers that were interested in installing such a system in cars, which he estimates would cost less than $40.

The software keeps track of a variety of facial features, not just eyelids, and thus also can be used to identify emotions, or lack thereof. So Metaxas also is exploring how this approach could be used to diagnose autism and mental illness. And NASA has enlisted him and Dinges to test the system as a way to gauge astronaut stress levels.

As for detecting fatigue, whether by voice, eyelids, or some other measure, manufacturers and regulators eventually would need to decide how to put such a thing to use. A dashboard warning light or bell could be used, but would vehicle operators take it seriously?

"Fatigue has been underestimated by our society the way we used to underestimate alcohol," said Brenner, the safety board investigator working with Penn State.

For pilots, an alert could be sent to a control tower.

Brandon Bow had no such warning.

"Cleared for the approach," he told an air-traffic controller at 2:44 a.m. on Feb. 17, 2004, the NTSB reported. "Have a good morning," the controller said.

"You too," Bow responded.

Thirteen minutes later, several miles past its destination, the plane hit the ground and disintegrated.

A witness said the engines sounded as if they were at full throttle before impact.

VIDEO: How Rutgers computers tell if you're tired.

Contact staff writer Tom Avril at 215-854-2430 or tavril@phillynews.com.

See SLEEPY on E2

comments powered by Disqus