The Trouble With the Test
by David Bacon
SAN FRANCISCO, CA (11/12/01) - Remember spring fever? Spring used to be the slow time at school, when sleepy students looked out the window after lunch, waiting for the bell.
Well, these days spring has become a deadly serious season, and daydreaming is definitely not an option. For the last two years, when warm weather arrives in California classrooms, schools have gone test-crazy as students prepare for the new state-mandated STAR (Standardized Testing and Reporting) exam. Teachers begin "teaching the test," as they call it, although many do so with great trepidation. Classroom walls, normally filled to overflowing with education aids and student work, are suddenly transformed as anything that might be construed as a hint to a student test-taker is pulled off.
When the results finally come in, every school in California gets rated and placed on a scale from the lowest to the highest. And what might otherwise be viewed as a form of academic Olympics acquires a sharp financial edge as the state begins handing out big-time bucks to teachers and school personnel based on test scores.
This October the state of California gave financial awards to teachers in 304 schools. Some received $5000 apiece, a smaller number $10,000, and an even more select group $25,000 each. Governor Grey Davis, when he announced the program, obviously thought teachers would be overjoyed by the largesse. Instead, when the first round of smaller awards were handed out over the summer and fall, many educators said they were being bribed. Some even donated their money to different redistribution efforts as a protest.
And at a November hearing called by State Senator Dion Aroner to investigate the impact of testing, an English-immersion teacher at Berkeley's Cragmont school announced she had decided to donate her $10,000 award to purposes obviously other than those which the governor intended. "It's dirty money," Reva Kidd charged. "We've had to fight hard for adequate salaries, but this money is a bribe, to make us complacent in the face of changes that are hurting students and teachers alike." Kidd donated some of her money to a fund which is redistributing awards among all Berkeley teachers, and some for a field trip for one of her colleague's classes.
But her third choice must have really given the governor heartburn. The last part of her money she directed to Cal CARE, an advocacy group which organizes parents and faculty against high-stakes testing itself.
Kidd wasn't an isolated loner, and the doubts she expresses are common to parents and educators nationally. By last year, every state in the U.S. but one had adopted curriculum standards for public school students, and forty-one states had gone on to adopt standardized tests to measure student performance. At the same time, however, rising protests have challenged high-stakes testing around the country. Students at schools in New York and Massachusetts have refused to take them, risking their academic future. A rebellion by Wisconsin parents forced the state's legislature to kill a proposed high-school graduation exam. In Cleveland, Ohio, the NAACP charged the Ohio Proficiency Test with being racially biased, one of many such legal challenges. A Youngstown State University study, in fact, found that the poorer the family of the student, the lower the test score was likely to be.
But the actions by Bay Area teachers mark the first time teachers themselves have taken such an active role in challenging the exams.
Both the larger Certificated Staff Performance awards (like Kidd's) and the smaller Academic Performance Index awards are made by the State Department of Education, and go to schools which made the largest increases in student standardized STAR test scores. And that's the problem - the tie to the test. A large group of teachers feel that most classroom instructors and school personnel work hard, as do students themselves. To single out one particular school based on test scores ignores what teachers and students accomplish in hundreds of other schools throughout the state.
A smaller group goes further. These educators question the value of high-stakes, standardized testing itself. They argue that the tests are biased against the poor and people of color, and transform education into a kind of testing Olympics.
San Francisco's Lowell High School was one of the first California schools to voice opposition to the awards. That was an irony itself. Governor Grey Davis, in announcing the program, implied that the most deserving schools would be those in the poorest communities. Teachers there, presumably motivated by cash prizes, would inspire pupils to make big jumps in test scores. Instead, Lowell is San Francisco's premier elite campus, whose students are selected based on their previous high academic achievement.
Under the Academic Performance Index award program, 4800 schools like Lowell were selected to receive a total of $350 million as rewards for increased scores. All personnel at the chosen schools shared in the money - each receiving $591. "Don't get me wrong - we've got great faculty here at Lowell, and as teachers we certainly deserve more money," said Ken Tray, site representative for United Educators of San Francisco. "But our friends and colleagues at Balboa High, for instance, also work their tails off. The awards are a slap in the face for them, not recognizing the hard work they do." Lowell teachers decided to encourage voluntary donations to a scholarship fund for students at schools which didn't receive the award,
Tray says teachers supported the idea because the awards "seem like a back-door merit pay system." Even Lowell's principal, Paul Cheng, contributed his award to the fund.
In Berkeley, teachers went further. There the Berkeley Federation of Teachers set up an award redistribution committee. The union won agreement from the district that teachers would be allowed to indicate on a payroll checkoff form their preference for using their money as Kidd did - redistribution, a field trip fund, or donating it to Cal CARE .
The educational issues committee of the union then drew up a petition opposing the use of standardized tests entirely. Some 300 of Berkeley's 600 K-12 teachers signed on. Among other objections, the petition declared that "the SAT 9/STAR test is racially, culturally and socio-economically biased, unfair, and inappropriate for our students. It does NOT measure student achievement or teacher quality. The test itself and the API reward system perpetuate the huge economic, soical, cultural and language disparities among the stete's students."
"High-stakes tests force us to teach in a way in which high scores become the most important goal," explained Terry Fletcher, a third-grade teacher from Thousand Oaks Elementary School. "Teachers are forced to cram information into students, but not to encourage critical thinking or broader knowledge. There's no emphasis on art or music, or even social studies. Testing really turns us into worse teachers."
Berkeley teachers have turned in about $20,000 so far, according to Barry Fyke, president of the Berkeley Federation of Teachers. "I think some appreciate the money in a profession in which we aren't paid as professionals," he says. "But a majority of teachers don't think testing and prizes are a very effective way to ensure accountability. Teaching is a combination of a science and an art, and should be evaluated in the classroom, rather than using money as a bribe to get kids to perform well on tests."
Teacher salaries in the district average $34,000, which is not enough to buy a home in a city with some of the highest housing osts in the state. "It would be better if the state used the money to increase pay." Fyke adds.
Ironically, California's test publisher this year again flunked its own process for test scoring, according to a September 12 memo by state Superintendent of Public Instruction Delaine Eastin. In it she explained that Harcourt Educational Measurement, publisher of California's STAR test, had botched scoring in eight districts. By using norms from the wrong period, student scores were artificially elevated. "These types of mistakes undercut our confidence in the integrity of our assessment and accountability system," Eastin said. It wasn't the first time. Harcourt was penalized $1.1 million in August, 1999, for late reporting of test results and 100,000 mistaken reports sent to parents, which had to be recalled.
In November 1997, then-Governor Pete Wilson's appointees to the California Board of Education overruled Eastin, and granted Harcourt a 5-year, $12 million/year contract to administer and score tests for California's 4 million students. In doing so, he cut short a process in which state educators spent years developing a set of standards for core curriculum. The test designed to assess knowledge of that curriculum, the CLAS test, was attacked for efforts to incorporate cultural diversity, and dumped. To force the legislature to immediately adopt an off-the-shelf test, Wilson withheld $200 million in school spending until lawmakers agreed. The year following adoption, Harcourt's revenues from its Education Group division shot up $85 million (18%), and its profits jumped $34 million (58%).
Such a widespread and rapid change in education has rarely swept through California's schools, and for that matter, those of almost every state. Politicians today vie with each other to appear more concerned about education, and position themselves as would-be "education candidates." Driving this almost obsessive interest in testing are factors ranging from political ambition to genuine frustration by parents and teachers with the ability of the public school system to teach its students. But testing is getting a big push from another important source, which gets much less media coverage - the testing companies themselves.
Districts and states are spending huge sums on testing and standards, which go to a few large companies who also publish the books schools use for instruction. Dominating the field are three big publishers. McGraw-Hill and its subsidiary, CTBS, publish the Terranova test series. Harcourt Inc.'s Education Group publishes the STAR test, and Houghton-Mifflin's Riverside division publishes the Iowa Test of Basic Skills and the Metropolitan Achievement Test.
Test-scoring is also a source of revenue. National Computer Systems, a company with a close relationship to a Harcourt, scores Ohio's tests for about $10 million a year. Ohio parents got a surprise in 1998, when they discovered that ninth and twelfth grade students had their essays graded by a subcontractor, Measurement, Inc. For $1.4 million a year, the company employed temporary workers at close to minimum wage in a North Carolina strip mall. These workers, who had no teaching experience or education credentials, spent about two minutes looking over each paper.
Together, test publishers divide a testing market that was estimated at $218.7 million in 1999 by the Association of American Publishers. The publication of standardized tests is considered part of the market for instructional materials, which, at $3.4 billion, is over 15 times as large. But the market for tests has been growing at an average of 7% a year for over a decade -- much faster than the market for textbooks, whose annual growth rate was 3% over the last four years, according to the executive director of the association's school division, Steven Dreifler. That growth rate alone makes it an attractive market for publishers, and one which promises to become much larger and more important in relation to their traditional business of publishing books.
Houghton-Mifflin's Riverside testing operation, which sells the Iowa test, grew at a phenomenal 17% in 1999, while its overall textbook division grew at 9.2%. In 1997, McGraw-Hill's testing division had gross income of $95 million, and its overall educational publishing group grew 5.7% to $832 million.
Test contracts are lucrative. McGraw-Hill won a $30 million contract for tests in Kentucky last year. In Mississippi, with one of the lowest per-pupil spending levels in the country, the state signed a 10-year testing contract with McGraw-Hill for a total of $29.4 million.
During his presidential campaign, George Bush Jr. touted Texas' "education miracle, where beginning in 1985, Harcourt developed the now-famous Texas Academic Assessment Skills test. Texas currently contracts for test development with National Computer Systems for $20 million a year. NCS in turn subcontracts to Harcourt, which also gets another $2.8 million a year for developing TAAS study guides.
Being the test developer can be very advantageous. Harcourt's textbooks were marketed to local districts around the state with a flier stating "Why choose Harcourt Brace for your math program? . . . (It is the) only program to have texts written by the same company that helps to write the TAAS tests..." Stating the connection so baldly is considered bad form, and Harcourt later said it would discontinue the promotion. But according to the Texas Education Agency, the company sold $25 million of elementary school math textbooks in 1999.
Despite the money spent, uncertainty is rampant over what the test scores actually mean. Sandra Stotsky, a researcher at Harvard, says that in Texas "there may have been no real improvement in reading skills. There may even have been a decline." She alleges the test is made easier so more students pass. Underlying the debate are even more basic questions about whether standardized tests reflect a bias which favors white children over racial minorities, and English-speakers over immigrants.
These tests are not recent inventions. The Stanford-9 and Iowa tests go back over 60 years. Stanford psychologist Louis Terman, who wrote the first test in the Stanford series before World War One, was notorious for regarding racial minorities and Jews as "feeble-minded." Other early test developers held similar racist views, and saw the tests as instruments to weed out the less intelligent.
Allegations of racial bias continue to dog the tests today, despite publishers' claims of an unprecedented level of objectivity. In Cleveland, the NAACP charged the Ohio Proficiency Test with being racially biased after no student in five schools in poor urban areas passed all sections.
Two Harcourt tests were recently charged with being Eurocentric and discriminatory to New Orleans's African-American students, when they were used as a basis for admission to a local high school. For admission during the 1997-98 school year, 763 students took the tests, of whom 44 percent were black and 42 percent were white. Of the 347 who passed, 27 percent were black and 59 percent were white.
New Orleans was sued, a common experience - most states are sued over bias or problems with mistaken test results. The Mexican American Legal Defense and Education Fund, for instance, mounted a major legal challenge to the TAAS test, saying it has a discriminatory impact on Black and Hispanic children. According to Maureen DiMarco, formerly California's Secretary of Education under Wilson, and now vice-president for education and government at Houghton-Mifflin, "it's hard to have a test that doesn't get sued." But, she notes, it's the state or school district that has to mount a defense and bear the legal costs, not the publisher.
"Tests do measure social and economic conditions," she admits. "Children from poor communities go to schools which don't have resources, and use less effective methods of instruction. Lots of test scores can be explained by the lack of books. Poor children also move more often. The implications of what's being measured are very deep. Poor kids can learn just as well as higher income kids. They're just not getting the resources they need to learn."
Despite these questions, and growing opposition by parents and teachers, the test market looks good to publishers. Twenty states now work with publishers to come up with standards for what students are expected to know. "It's a wise state that seeks the advice of a publisher when formulating standards, to ensure they're rigorous, and not too vague," DiMarco explains. The company that helps develop the standards, presumably, has a better chance at getting the bid for the test, and an advantage in selling textbooks which meet the curriculum requirements as well.
Backing the growth in testing is also what DiMarco calls the standards and accountability movement. This is not a grassroots wave from below, but includes organizations like the Pew Charitable Trust, the Heritage Foundation, and the Hudson Institute, and corporate CEOs like IBM's Lou Gerstner. The National Governors' Conference and the Business Roundtable set up an education thinktank called Achieve, Inc., which produced a study of education used as the basis for implementing standardized testing in Ohio.
According to a leader of the Ohio parent protests which followed, Mary O"Brien, "big corporations like IBM, Proctor and Gamble and Eastman Kodak are very up-front about their agenda. They want schools to educate students to their specifications. They want education centered on testing, and curriculum aligned to the tests. They've even moved to pressure university education schools to adopt this way of thinking, and to join their school reform movement."
O'Brien's estimate was shared by many teachers who appeared at Aroner's November 9 hearing at the new Oakland State Building. They came, not just from Berkeley, but from almost every school district in the area she represents. The problem, Aroner said, is that the people devising the methods for assessing student progress aren't listening to the people who know the most about it - teachers. "These awards are a way for the legislature to pay teachers off," she charged. "It's time to hear from teachers themselves - why do't we want to listen to the experts? They're our eyes and ears in the classroom."
One big problem with the STAR test, according to teachers, is that it is norm-referenced. In other words, the scores are plotted on a bell curve, and students are told their results in terms of how they rank in comparison to other students, not to a defined set of criteria. Rick Rubina, a 14-year school principal who today is in charge of certified personnel for the Mt. Diablo School District, told Aroner's public forum that "a criteria-referenced test compares students to a set criteria. It shows individual growth in knowledge for each child. But in a norm-referenced test, half of the students are always, by definition, in the bottom half of the test results."
"We need more than just a gimmick," Fyke responded. "What people want is accountability from both teachers and students, and that's good. But what we have is a method designed by people far from the classroom."
Fyke, cautious in the way he frames the reasons for opposition, also proposes an alternative. "We have to identify what the expectations really are," he says. "These should be developed by teachers, parents and researchers. We need to assess more difficult things - the ability of students to solve math problems and write creatively, their knowledge of social studies, their highest thinking skills, their ability to take initiative and accept responsibility, and their emotional intelligence. We expect teachers to be able to impart all these things, and so they should. But I would welcome a process of assessment which actually measured this."
Ted Sugarman, a fifth-grade teacher at Oakland's Cleveland Elementary School, added that"to teach math to kids, you need to teach them how to think, not just the right answers to questions."
But critical thinking is not only impossible to assess on a multiple-choice test, but seems to be falling on the priority list of the state's educational system entirely. Berkeley parent Juana Alicia Rodriguez, whose eight-year old daughter, Mayahuel, goes to third grade at Rosa Parks Elementary School, calls it "a war against the imagination."
Rodriguez is a muralist whose gigantic works are painted on walls from Mexico to the new international arrival building at San Francisco International Airport. Despite her global recognition, however, Berkeley can only afford to give her two days of work a week as an arts instructor at Arts Magnet School. The other days she travels to Fremont, where she teaches and supervises other arts teachers.
Arts education is on the block in the current educational environment, which Rodriguez says is a conscious choice against emphasizing critical thinking skills. "When children are offered arts education at an early age, their critical thinking develops," she says, adding that the same skills are fostered by bilingual education. "Children have to code-switch - take one body of knowledge and relate it to another."
Standardized tests have no way to measure the development of that skill, and a system based on testing devalues it. A big mistake, Rodriguez says - "this is the heart of learning, not frosting on top."
Many students are even more direct. For Fadeelah Muhyee, a twelfth-grader at Oakland High School "we know what needs to be done to make our school better, and the test doesn't help at all. We don't have enough books," she charges. "There are no counselors. There's a lot of unevenness among teachers, and there's no ethnic studies. Now they just want teachers to teach to the test. We don't need a test to show us that we're at the bottom - we already know that. This test is racist and biased. It's set up for the rich schools to get richer, and the poor to get poorer."
Those sentiments could lead to a boycott of the test this spring, as has happened in other parts of the country. If that happens, Berkeley and Oakland will be a likely point of origin.
Maureen Katz, mother of another Rosa Parks third-grader, says the only reservation she has about that prospect is the fact that Rosa Parks teachers donated their prize money this year to the school itself, which, she says, it desperately needed. The school, rebuild as a result of parent pressure, is the only elementary school in Berkeley west of San Pablo Avenue, in one of the city's poorest communities.
"As parents, it's hard to say we don't want that money," Katz says. "But if that were out of the equation, I'd like to see parents organize a boycott. Then we could call in consultants who aren't tied to the testing companies, and find a reasonable way to evaluate educational programs, student progress, and teacher performance."
Katz' daughter Alexandra takes one week of tests in Spanish (Rosa Parks is a bilingual school), and two week in English - almost a month out of the instructional year. "She hates it," Katz comments bitterly. "The tests are emblematic of the new educational goal - regurgitating facts, not creative thinking. There have to be other ways to teach."
That's also the conclusion of Naomi Katz (no relation), whose child goes to Oakland's Crocker Highlands Elementary School, in the slope between the flatlands and the hills. Naomi Katz works part-time for Cal CARE,, and hopes to organize a mass opt-out in Oakland in the spring. Katz sees testing also as emblematic of deeper problems. "It just highlights in inequity of resources," she charges. "It absorbs huge amounts of money we need to fix deteriorating schools, to hire and train fully-credentialed teachers, and for developing a meaningful curriculum. Instead, what we got this year was yet another corporate product - the Open Court reading program, which McGraw Hill sells and calls 'teacher proof.' It's just another step in rote-learning."
Turning in the award money is just one indication that many teachers are likely to support such a move. They are already notifying parents in a number of East Bay classrooms that they can fill in forms which allow their children to opt out of the test.
Gail Mendez, a teacher at Bayview Elementary School in Richmond, says that despite some of the lowest teacher salaries in the state, she couldn't in good conscience accept her $591. "I tell my fourth graders that you have to stand up for what you believe in. How could I face them if I took this money?" she asks.
Top of Page
PEACE & JUSTICE
WORKPLACE | STRIKES | PORTRAITS | FARMWORKERS | UNIONS | STUDENTS
Special Project: TRANSNATIONAL WORKING COMMUNITIES
HOME | NEWS | STORIES | PHOTOGRAPHS | LINKS