DOE blasted over giddy test report

The content had been changed to subjects known to students

Quick math test:

What's one apple minus one orange?

Wait. Apples and oranges are different units and cannot be subtracted, added or compared.

Yet that is just what the Department of Education did earlier this year when it touted student test scores as a dramatic improvement over previous years.

DOE officials now concede they should have been more cautious in comparing the results of this spring's Hawaii State Assessment with past performance. This recent test was changed to better reflect material actually taught to students. And students did much better. But that does not necessarily indicate much progress.

FULL STORY »

Top state Department of Education officials have accepted blame for celebrating gains in annual state testing without emphasizing that results were based on new standards that made year-by-year comparisons misleading.

But education officials continued to defend the version of this spring's Hawaii State Assessment, saying it was the first time in six years that the exam quizzed public school students only on subjects to which they had been exposed.

"I didn't do a good job of articulating the pros and the cons or the differences in how well you could do the comparison," said Robert McClelland, director of the DOE's systems accountability office.

Sixty percent of students tested scored "proficient" in reading, up from 47 percent last year. In the math section, 38 percent of students were proficient, compared with 27 percent in the previous year. Those gains pushed up the number of schools that met federal progress benchmarks to 182 from 100 a year ago.

But critics questioned whether test problems were "dumbed down" to help schools avoid sanctions under the No Child Left Behind Law, which requires schools to meet steadily rising proficiency percentages, culminating with all students being able to read and solve math problems at their grade level by 2014.

School board members said they got e-mails and calls from teachers and parents skeptical of the comparison of scores going back to the 2001-02 school year, when different standards were applied.

"Why was the department willing to compare apples and oranges?" asked board member John Penebacker. "You made a concerted effort to show a relationship between two different things, and it caused a great amount of confusion to a lot of people, including myself. Was there a reason why you folks did that when you couldn't validate the comparison?"

Schools Superintendent Pat Hamamoto said the department showed the upward trend of scores dating back six years because that is the base line used by the federal government to determine whether schools are meeting so-called Adequate Yearly Progress.

"So, for us, while it is in one sense a base line because it is a new test, in the eyes of the feds, it's a continuation of what we have been doing," she said. "That's the gap in the communication."

The new, grade-specific topics for reading and math replaced previous content that tested students on materials they had not yet been taught, McClelland said. He said those standards -- or the skills students should know at their grade level -- were set in February by a team of 100 teachers, school administrators, community members, parents and University of Hawaii faculty.

While changing standards makes it hard to compare annual results, the NCLB law requires states to refine their tests when they feel it is needed to make sure they paint a true picture of student progress, McClelland said.

"At no point in time has there been any attempt to rig the standard setting or to dumb down the standards or the test," the Education Department wrote in a report to the board.

McClelland said officials are waiting for this month's release of an annual test administered by the National Assessment of Educational Progress for fourth- and eighth-graders nationwide to check whether scores from the Hawaii students match or are similar to the new state test.

He said the gap between the Hawaii State Assessment and NAEP results have been small in past years, indicating the state's standards are comparable nationally.