No Students Left Behind: Literacy Measurements and the Spellings Commission on the Future of Higher Education in America
by Dennis Baron
Last September, Secretary of Education Margaret Spellings appointed a Commission on the Future of Higher Education in America to consider just how well the nation’s colleges are preparing students for the 21st-century work force and “whether the current goals of higher education are appropriate and achievable.” As Commission Chair Charles Miller put it, “There is no solid, comparative evidence of how much students learn in college, or whether they learn more at one school than another.”
Miller, a Texas businessman who served as head of the University of Texas Board of Regents and put in place an elaborate assessment system for that state’s colleges during the Bush governorship, works from the paradoxical premise that higher education in America is both the best in the world, yet in need of significant remediation. His Commission’s charge is to focus attention on issues of access, affordability, achievement, and accountability. Secretary Spellings hopes that the Commission will find a measuring stick to allow objective comparisons of the nation’s two- and four-year public, private, non-profit, and for-profit colleges and universities. And though the Commission will not report until August, Chairman Miller has already indicated that such a measuring stick should include tests of student outcomes similar to those mandated by No Child Left Behind legislation. He further anticipates tying such value-added measures to a radical restructuring of the college accreditation process.
In the Commission’s view, accreditation and outcomes testing must go hand in hand. The 2003 National Assessment of Adult Literacy reports that as many as one third of recent college graduates do not qualify as “proficient” in literacy. A Commission white paper calls this a “shameful outcome” and lays the blame partly on inadequate instruction and testing, and partly on the current accreditation process, conducted by colleges themselves under the direction of a number of private regional accrediting agencies. Miller wants to replace this system of educational self-monitoring with a National Accreditation Foundation and link each institution’s accreditation to defined learning outcomes as measured by standardized tests.
The Commission also wants to marshal data furnished by the colleges so that the Department of Education can produce National College Report Cards. But unlike those now issued for K-12 schools, education consumers would be able to sort this college data to create individualized profiles that will help them decide where -- and whether -- to spend their higher education dollars, and the nation’s employers would be able to estimate with some precision the impact of a given college on the brains of their job applicants.
The unspoken promise of the Spellings Commission is that all colleges will be ranked – though it may be necessary to put some on watch lists – and as with our K-12 schools, no students will be left behind, particularly on testing day. For there seems to be no way to collect the necessary data short of not one, but multiple batteries of standardized tests, tests that will give our institutions of higher education the vision and direction that the Commission feels they currently lack.
Transcripts of the Commission’s meetings show that its report is also likely to characterize American higher education as overpriced, unfocused, poorly managed, and seriously under-regulated. Many Commission members view the variety of colleges and the diversity of their missions with alarm. The Commissioners, many of whom are corporate executives who would surely resist government interference in their own highly-diverse market endeavors, see the free marketplace of ideas that constitutes American higher education as sorely in need of monitoring and regulation. One Commissioner complained, there is no bottom line in education, no way to tell if an institution had a good year. His unstated message was, “If I ran my business like you run your university, I’d have to sell out to someone who actually made money, like the University of Phoenix, and close my doors.”
Of course he’s wrong, though none of the college presidents on the Commission reminded him that even nonprofit organizations have budgets and measure their years in dollars – appropriations, grants, contracts, and gifts received are certainly factors in any institution’s sense of achievement or failure. But colleges also measure bottom-line “profits” by published research; by the ability to attract talented high school graduates; by retention and graduation rates; and by where students go next. And while colleges have become increasingly digital, the size of the library still matters as well.
Scripted testimony before the Commission from specially-invited speakers seems to confirm what the commissioners already believe, that the curriculum needs more math and science, while reading and writing, the cornerstones of literacy, even in technical fields, doesn’t get much play. Witnesses and commissioners rehearse some other common themes: that more people need to go to college; that it costs too much; and that students don’t learn enough once they get there, because traditional education has failed. The model of teachers chaining students to desks just seems so twentieth-century to the Commissioners, who warm to the idea of replacing teachers with coaches on the assumption that the coach is a more user-friendly figure who won’t drive away the paying customers.
I happen to live next door to a coach, and it seems to me that coaching is not a kinder, gentler way to boost either literacy or performance on the field. Coaches start practice at 6 a.m. when players would rather be sleeping. They cut players who don’t measure up and punish them if they break the rules. Coaches make players do their homework and go to bed at a reasonable time. Coaches do all this using carefully-researched motivational strategies that involve a lot of yelling and screaming. And then there are coaches like Bobby Knight.
But what the Commission finds attractive in this model is the knowledge that when the team fumbles, it’s the coach who’s fired. If literacy levels plummet, they seem to suggest, we should just get new coaches. The Commission also seems to feel that today’s students need to learn not in stuffy classrooms but free and unfettered on the web, where they can take courses at a variety of nontraditional institutions, work at their own pace, IM their friends and download movies while sitting home in their underwear or chugging lattes at Starbucks. If we believe the testimony, a combination of for-profit schools, on-line classes, and standardized tests will fix all the ills of American higher ed. That and a subscription to iTunes.
Chairman Miller is optimistic that new testing instruments developed by such groups as the Educational Testing Service will reveal whether college improves students’ “critical thinking, analytic reasoning, problem solving, and written communications.” Commission member James Hunt, who as governor of North Carolina instituted that state’s rigorous school testing strategy, commented at one Commission meeting, “We need to teach creativity and inventiveness and figure out how to measure it.” Think outside the box, just draw inside the lines when you fill in the answer sheet.
In the face of a literacy crisis in which our best and brightest students learn to take tests instead of learning to think, Chairman Miller lauds “new educational delivery models” such as Western Governors University, an on-line “competency-based” school whose president is a Commission member. Competency-based is a code word for “fill-in-the-bubbles.” Also on the Commission is the CEO of Kaplan, Inc., a company which not only makes its money by prepping students to take standardized tests, but which also runs Kaplan University, the nation’s first on-line, for-profit law school, where all that students need to bring with them to their virtual classes is the digital equivalent of a no. 2 pencil.
And speaking of no. 2 pencils, the SAT was recently in the news for inaccurate scoring of tests, an error the company blamed on wet answer sheets caused by rainstorms in the Northeast. That seems to be the least of America’s testing woes, however. A recent report by EducationSector charges that testing companies are overextended and under-regulated. Our testers can’t keep up with the 45 million tests that must be administered to satisfy No Child Left Behind, not to mention the millions of certification and admissions tests they also field.
Among its panoply of tests, the SAT currently administers a national writing sample to college-bound students. Whatever you think about the wisdom of pegging college admissions to a twenty-five minute essay on a general topic where poor handwriting may result in low grades but students are not marked down for getting their facts wrong, at least the scoring of these essays is not affected by the humidity level of the paper that they write on. But it is simplistic to think that a second writing sample administered just before graduation will allow consumers to see in concrete terms the value added by higher education.
Or the value subtracted, for according to the National Assessment of Adult Literacy, adults with postsecondary degrees saw their prose literacy actually drop 9 points, while adults with graduate degrees dropped 10 points for prose literacy and a whopping 14 points for document literacy. Not surprisingly, such figures are producing warnings that college might actually be harmful to the nation’s intellectual health.
But standardized tests like the NAAL don’t tell the whole story. The 1992 National Adult Literacy Survey that served as a baseline for the NAAL was created and administered for the Department of Education by the Educational Testing Service. It led to headlines screaming that almost half of America’s grown-ups – 41 million people – were functionally illiterate. But the results of that survey were wrong. When the government’s experts actually took a minute to check their work, they discovered flaws in their own testing literacy.
They had set the pass rate too high. To be proficient you had to get 80% of your answers right, which meant that people who passed probably knew how to use the skills being tested. But the 80% passing grade also produced an unacceptable number of false negatives: many people who were marked wrong on the test could probably perform the tasks in question in non-test situations. They really could figure out where to sign a check, what a newspaper article meant, or how to take the bus so that they’d get to work on time. So the government revised its calculations and quietly reported that not half but only 13% of adult Americans demonstrated significant reading and writing problems. And if all the false negatives were eliminated, only 5% of the population had really serious literacy issues.
Like its predecessor, the NAAL doesn’t give the whole picture. Using the same test as the NAAL, a new study, “The Literacy of American College Students,” finds that current college students have significantly higher literacy levels than adults who had previously received a college degree. Two studies, using the same test, come up with conflicting data. The Department of Education has yet to weigh in on that paradox, but I wouldn’t be surprised if they concluded that further testing was warranted.
But more tests are not the answer. If we’ve been testing reading for well over a century and we still haven’t figured out how to measure national literacy levels accurately, how are we going to come up with dependable value-added tests for college? Even if we could, I think that neither the benefits nor the downsides of college can be quantified and translated into the kinds of charts and graphs that comparison shoppers will find useful. Sure, I want to know the likelihood that an applicant will be admitted; how many students drop out before graduation; and the campus crime stats. But I don’t see the use of dividing sentence complexity by cost per credit hour to produce a National Literacy Intelligence Quotient. Yes, I’m exaggerating, but my point is this: We don’t know how to produce – we shouldn’t even want to produce – a National Report Card for postsecondary institutions that’s comparable to the ways we rate our K-12 schools.
Those user-friendly DOE report cards won’t really show you what college will do for you. Prospective students won’t see much that they can’t already get from commercially-available college guides. But the rankings will give education consumers the illusion that national education standards exist, when in fact they don’t, and they’ll give consumers and legislators the false impression that choosing college can be as straightforward as choosing what to wear in the morning.
I believe that higher education must always be accountable to its various constituencies, and that despite Secretary Spellings’ claims to the contrary, we already produce reams of statistics to demonstrate our strengths and shortcomings. Adding standardized testing to the existing mix presumes a standard body of skills and knowledge that is being tested. In writing this is certainly not the case. I don’t think it’s true in reading either. Valid assessments of literacy practices are those which are situated in actual in-school and out-of-school contexts, not those gleaned from asking students to pick out grammatical errors in a sentence or to reorder the sentences in paragraphs that only exist on standardized tests or in study guides for taking those tests.
I used to think that my colleagues in science and engineering placed great faith in the ability of standardized tests to measure their students’ quantitative literacy, while those of us in the humanities were the skeptics who saw talents and flaws in our students that no test could ever reveal. But it seems that the numbers really do lie. My sci-tech friends complain about the false positives of high-scoring students who can calculate the point of impact of two trains leaving their respective stations at different times and traveling at different velocities along the same track, but who can’t figure out what made the trains embark on this suicidal mission in the first place. Recently I complained to my neighbor on the other side, not the coach but the electrical engineer, about our many high-scoring students who can’t read with insight or write convincingly. He looked at me in surprise: “These test scores are meaningless to me. I thought you people in English used them for something.”
In its report, EducationSector warns that standardized tests are being increasingly dumbed-down to meet federal testing demands, and that as more and more students are tested, schools turn to fill-in-the-bubble tests rather than essays or problem-solving exercises to control grading costs. This makes it even more likely that scores on standardized tests won’t give us as education providers the information we need about our students, either when they come to us or when they leave. And it means that no National College Report Card based on exit testing can hope to give education consumers the information they need to judge the effectiveness of their tuition and tax dollars.
Dennis Baron is professor of English and linguistics at the University of Illinois and chair of the CCCC Public Policy Committee. These remarks are adapted from a session that he led at the CCCC Convention last month in Chicago which focused on the impact of the Spellings Commission.