Milwaukee JS Editorial on State Standards

State sets low test standards

Skills needed for students' proficient ratings vary across U.S., study says

By ALAN J. BORSUK
aborsuk@journalsentinel.com

Posted: Oct. 3, 2007

The bar for labeling a student proficient in reading and math is set lower in Wisconsin than in almost any other state among 26 in a study released Wednesday.

The study found that "cut scores" - the line between proficient and not proficient - vary widely among the 26 states, casting doubt on the question of what it means when a state says a certain percentage of its students are doing well. Those percentages are central to the way the federal No Child Left Behind education law works.

The law's accountability system, which focuses on things such as whether a school or district is making "adequate yearly progress," is driven largely by how many students meet the standards a state sets for proficiency in reading and math. The goal is that all students, with a handful of exceptions, be proficient by 2014.

"Five years into implementation of the No Child Left Behind Act, there is no common understanding of what 'proficiency' means. . . . This suggests that the goal of achieving '100 percent proficiency' has no coherent meaning, either," says a summary of the study, issued by the Washington, D.C.-based Thomas B. Fordham Institute.

To illustrate the differences among the states, the study's authors gave an example in which a fourth-grader in Wisconsin would be regarded as proficient if the child could correctly answer a fairly simple question involving cats and dogs, while a child in Massachusetts would not be proficient if he or she couldn't answer a formidable question about the meaning of a passage by Leo Tolstoy.

"No matter what one thinks of America's history of state primacy in K-12 education, this study underscores the folly of a big, modern nation, worried about its global competitiveness, nodding with approval as Wisconsin sets its eighth-grade reading passing level at the 14th percentile while South Carolina sets it at the 71st percentile," Chester E. Finn Jr. and Michael J. Petrilli, both of the Fordham Institute, said in a foreword to the study.

The study's findings regarding Wisconsin are separate from, but dovetail with, results from a national testing program released last week by the U.S. Department of Education. Federal researchers concluded that 35% of Wisconsin fourth-graders were proficient in reading, while the figure given by the Wisconsin Department of Public Instruction for testing done a year ago was 81%. In four areas - fourth-grade reading and math and eighth-grade reading and math - federal researchers had a much less cheery view of how Wisconsin kids are doing in school.

Petrilli, vice president for policy at Fordham, said a low bar for proficiency signals to educators and parents that things are well when they are not. In addition, he said, there are indications nationally that some schools focus their teaching on students who need a push to get over the proficiency line, which can mean they aim for less ambitious results if the bar is low.

Petrilli said several recent national reports raise questions about what is happening in Wisconsin education.

"Wisconsin is one of the few states that is showing barely any progress over time," he said. "It is just breathtaking to me that the DPI - I would think someone there would say, 'Oh my God, we're hardly making any progress. This is a crisis.' Instead, they want to spin the results to say that everything is just fine."

"That's the Fordham party line, and we reject that," said Tony Evers, deputy state superintendent of public instruction. "We're not spinning anything. . . . We take testing very seriously in this state. We take setting cut scores very seriously. . . . To say we're not doing anything is baloney."

Evers said the DPI agreed with a statement in the study that cut scores are subjective and agreed that more needs to be done to make proficiency standards consistent across the country. He said Wisconsin is among more than 30 states taking part in an effort called the American Diploma Project Network, which has similar goals.

But he said DPI thought "absolutely" that a student rated as proficient in Wisconsin is, indeed, proficient and that cut scores had been set in an appropriate process.

Asked if Wisconsin's cut scores were comparatively easy, he said, "I don't think you can make that statement."

He also said DPI specialists had questions about the methodology of the study.

The report was based on research by the Northwest Evaluation Association, an organization that works with about 2,700 school districts across the United States, including some in Wisconsin, on improving educational programs. The organization has its own testing program, the Measures of Academic Progress.

Researchers used results from those tests to compare the difficulty of tests such as the Wisconsin Knowledge and Concepts Examination, known as WKCE, taken each year by hundreds of thousands of students in Wisconsin, and what it takes to be labeled as proficient on those tests.

The resulting report was titled "The Proficiency Illusion" because of the range of standards for proficiency.

"Proficiency varies widely from state to state, with 'passing scores' ranging from the 6th to the 77th percentile," Finn and Petrilli said in summarizing the results. " . . . It's not just that results vary, but that they vary almost randomly, erratically, from place to place and grade to grade and year to year, in ways that have little or nothing to do with true differences in pupil achievement."

The Fordham Institute sponsored the study, and the Joyce Foundation, based in Chicago, provided much of the funding. The Fordham Institute - which is not connected to Fordham University - is generally regarded as conservative in its views, and it supports national standards and assessments.

Proficiency standards

Overall, Wisconsin had the second-easiest standards for reading proficiency among the 26 states in the study, and the fourth-easiest in math.

To get a proficient rating in reading, a fourth-grade student in Wisconsin could score as low as the 16th percentile nationally, compared with an average among the 26 states of the 29th percentile. The 16th percentile would mean that in a representative group of 100 students nationally, that student would score better than 15 of them.

For eighth-graders, the bar in Wisconsin was set at the 14th percentile, while the average among states in the study was the 36th percentile.

The authors of the study concluded that "Colorado, Wisconsin and Michigan generally have the lowest proficiency standards in reading, while South Carolina, California, Maine and Massachusetts have the highest. In math, Colorado, Illinois, Michigan and Wisconsin have the lowest standards, while South Carolina, Massachusetts, California and New Mexico have the highest."

Evers, of the DPI, expressed doubts about whether the tests used by the Northwest Evaluation Association for its comparisons were valid for that purpose. He said the tests were intended for diagnostic purposes and not for assessing proficiency.

He also said the example of questions cited in the study - dogs and cats vs. Tolstoy - reflected questions used by Northwest Evaluation and not those used on Wisconsin's actual tests. (The report used the questions as example of levels of difficulty and did not say they had actually been given to children in Wisconsin or Massachusetts.)

The study found that it was somewhat tougher to get a proficient ranking in math than reading in Wisconsin. The lowest proficient score in math in Wisconsin was in at least the 21st percentile nationally for each grade from third through eighth. But there were still gaps at every grade when it came to comparing the Wisconsin standard with the average for all states studied. For example, in eighth-grade math, a score in the 23rd percentile in Wisconsin was proficient, while the average in the 26 states was the 44th percentile.

In many states, including Wisconsin, state math tests are harder than reading tests, the study concludes.

"Americans may wrongly think their children are doing better in reading than in math - when in fact less is expected in (reading)," the study says.