Rescore needed for school tests

Student assessments are done so frequently that testing services cannot keep pace

A testing firm picked by the state Department of Education last year to replace a mistake-prone company will review all the tests it scored for Hawaii this year after finding possible scoring errors in at least 1,682 test booklets.

The Washington, D.C.-based American Institutes for Research says a scanning error by one of its contractors recorded answers in blank reading and math tests for last spring's Hawaii State Assessment.

A preliminary analysis showed the error might have affected at least 1.7 percent of nearly 98,000 booklets distributed. AIR will review all test booklets, starting with the 1,682 suspected errors, at no charge to the state.

The yearly state assessments are used to determine school-by-school compliance with federal requirements on student achievement under the No Child Left Behind law.

Isle public school students showed major gains in the test this year, nearly doubling the number of schools meeting progress benchmarks set by the federal government.

State education officials said that while serious, the errors should not affect the status of schools.

FULL STORY »

A new testing firm will review reading and math scores for all Hawaii public school students who took the latest Hawaii State Assessment to check how many exams might have been affected by a scanning glitch.

The American Institutes for Research, which administered the state's annual standardized tests for the first time this spring, blamed yesterday a scanning error by one of its contractors for recording answers in blank booklets.

A preliminary analysis showed the error might have affected at least 1,682, or 1.7 percent, of nearly 98,000 booklets distributed this spring.

"This doesn't typically happen," said Jon Cohen, vice president and director of the assessment program for AIR. "We really are sorry that this has happened, and we take it very seriously."

The error comes as a blow to AIR, which received a $27.3 million, three-year state contract in January 2006 to replace testing firm Harcourt Assessment Inc.

The latest glitch also touches on concerns that the testing industry still needs to grow to handle demands of the No Child Left Behind law since its passage six years ago. In 2004, Hawaii officials found more than 40 errors in the Harcourt tests, and a year later, exams were delivered late or to the wrong schools.

This time, the problem emerged in late July, when teachers noticed grades were given for students who were absent and did not take the test, education officials said. The scanner apparently logged multiple-choice answers even though bubbles in the test were unmarked, Cohen said.

While the errors are a concern, Superintended Pat Hamamoto said they are unlikely to shift the standing of 182 schools that met federal progress benchmarks this year.

An initial review of 310 third-grade booklets flagged for having possible mistakes found that 100 of them had valid results, said DOE spokesman Greg Knudsen.

Cohen said AIR will complete a review of the 1,682 suspect booklets by Nov. 1 and re-scan the remaining ones at no cost to the state by Dec. 15.

It is unclear how much the extra work could cost AIR. The state contract says it can fine the company $10,000 a day, starting from when errors are discovered. The penalty rises to $20,000 after 12 days, Hamamoto said.

Nationally, there have been concerns that the battery of standardized assessments required under NCLB has strained the testing industry.

"The demand grew quickly, and capacity is growing to meet that demand," Cohen said, "but, you know, a growth phase in anything is always a bit of a shaky time."

On its Web site, AIR's contractor, MetriTech Inc., boasts about having 20 states as clients and an "extensive experience in the custom development of high-stakes assessments, end-of-course tests, and benchmark tests for a broad spectrum of educational and government agencies." Calls seeking comment from MetriTech after business hours were not returned.

To ensure accuracy in future tests, AIR will use Pearson Educational Measurement to scan tests beginning this fall, according to the Education Department. Pearson administers the National Assessment of Educational Progress, also known as the nation's report card for fourth- and eighth-graders.