Wednesday, January 12, 2005

Not Searching for Skeletons


Last fall I was asked to serve on a team of external reviewers evaluating an English department. I've done such reviews before, and I've been on the other side too, when the basic-writing division in my own department was the object of some close scrutiny by visitors.

External reviews are a fact of academic life. Provosts routinely assemble the review teams to gauge how a department is performing or how it compares with its peers nationally. But while some institutions publish detailed procedures for evaluating programs, the nuts and bolts of the actual review visits are never discussed in polite company, certainly not in front of the children.

The first time I served on a review team, I had no way of knowing what to expect, and sometimes the programs being scrutinized are equally in the dark. In one review I did, the chairwoman had never seen the provost's charge to our committee and could only fear the worst. We gave her a copy of the questions that we were supposed to answer, and the level of tension dropped significantly, at least until the last day of our visit. That last day is a killer, but more on that later.

There are lots of reasons to review a program, but just as most people don't volunteer for strip searches, departments seldom elect the full-frontal exposure that a review entails. A few universities mandate reviews of all departments in 5- or 10-year cycles. At others, the provost may seek the reassurance of a look-see by outsiders before committing big bucks to a program, or as a step toward closing it down.

The review of the basic-writing program in my own department began with termination as its goal, but as it turned out, the evaluators actually convinced the dean that the program was both essential and successful, and basic writing got an infusion of cash instead of the ax.

Sometimes external-review teams are treated to department receptions, tours of the campus, fine dining, rooms in posh hotels. At other times we're put in dormlike cells in the student union and asked to arrange our own transportation and meals.

Underneath such trappings -- which may or may not suggest how much the institution values the review team's effort -- external reviews share a basic structure. All of them feature a tightly packed schedule of meetings with everyone in the department, and with virtually everyone on the campus who does business with the department, followed by a series of final interviews with the higher-ups and a written report.

All of the reviews I've participated in have been remarkably similar in purpose as well. Tell me, the provost asks us, how does the department stack up against its peers? Are the faculty members productive and visible in the profession? Is undergraduate education effective? What about advising? Where does the unit draw its graduate students from? Do they finish their degrees? Where do they go once they graduate? Which programs in the department are strong? Which are weak?

Seldom in the written charge, but often on the provost's mind, is the subject of the department's ethos. Does it have energy and a future? Or has it gone into stasis? Always there's the reality check: The provost wants to compare the face that the department shows to the institution with the face that it shows the profession.

The charge letter to members of the review committee often runs three to four pages. If we're lucky, we get it before we get to the campus. We're supposed to gather the information we need and report our preliminary findings all in the space of two days, sometimes less.

Surprisingly enough, that turns out to be do-able. For one thing, the review team gets a copy of the detailed self-study that the department has already completed for accreditation purposes. We usually get it before our visit, though in more than one case, I received it so late I had to read it on the plane. Cast in turgid administrative prose, self-study reports are about as appetizing at 32,000 feet as the packet of fat-free pretzels that passes for lunch on most flights.

While the self-study offers a snapshot of a department, it's the hurried face-to-face meetings with administrators, professors, students, and staff members that really show us what's going on -- and what isn't.

A self-study by the English department at a university in the South announced a sexy new Ph.D. program in postmodern literacies, but when we met with the faculty members, it was clear that they hadn't thought through what courses to offer or what sorts of jobs students might apply for once they graduated.

In another instance, the administration at Soybean State College (all of the names of institutions in this column have been changed) described a happy department filled with faculty members working in harmony toward common goals. But in our meetings, those happy faculty members whined about their declining institutional influence and generally longed for what they perceived to have been the good old days.

Then there was Hudson University, a commuter campus that celebrated its ability to reach out to the metropolitan area that it served. But some of the English department's star faculty members stretched the definition of commuter, listing addresses that were not miles but whole states away from the campus.

When we met them, these expats assured us that through the magic of technology they were actually more accessible to their students during the eight months they spent at a lakeside veranda or mountain aerie than during their semester in residence on the campus. But the students told a different story, one of e-mail messages ignored, papers and dissertations unread.

The purpose of a program review is not to look under rocks but to suggest improvements that will help departments move up in the rankings.

Sometimes an outside team can see strengths and shifts that the department has not been able to articulate. The dean at Soho City College wanted the English department to identify an area for additional hiring. But faculty members were reluctant to advance their own specialties at the expense of someone else's.

After two meetings with these remarkably collegial professors, our evaluation team detected a common thread in their research: Many were working independently on projects that could fit under an umbrella of colonial and postcolonial studies. We urged the department head to pitch a cluster of new hires in that area to the dean.

Financial carrots from deans come with a limited shelflife. A department that isn't ready to pounce may find itself waiting a long time before the next such opportunity comes around.

Sometimes it's the university administration, not the department, that raises concerns among the reviewers. At the university in the South where I was an external reviewer, the dean told us that a department program we were about to single out for its great potential was in her eyes worthless. "I took a course on this in college," she explained. "It made no sense then, and it makes no sense now."

Had she taken the time to look at our vitae -- the qualifications of the external reviewers are sent to everyone who will meet with us before we arrive on the campus -- she would have seen that the field in question was the bread and butter of a team member who thought he had been chosen for his role because of that very expertise. He told the dean so, while the rest of us looked on speechless. In our written report to the provost we ever-so-politely suggested that the program should be refocused rather than eliminated.

The members of review teams really are chosen for their expertise. Provosts, who organize most unit reviews, solicit names from the department under review. They also contact institutions up the food chain for suggestions. Some reviewers are seasoned administrators; most are full professors; most are well-known for their scholarship. Occasionally one has lectured at the institution, worked with one of its programs, or served on an earlier review of the unit.

In one case a reviewer on one of the external teams on which I served had a degree from the department being reviewed. I was surprised that the provost didn't consider that a conflict of interest, but the reviewer showed a keen sense of independence, and it didn't prove to be a problem.

I don't know if I've just been lucky, but each group of reviewers I've been part of has quickly coalesced into an efficient team, even though we were usually strangers meeting for the first time. Frictions never emerged, and each of us managed to come to remarkably similar conclusions about the department and its institutional context. So far, I've never been on a review team that produced a minority report.

Our campus visits end with a series of final interviews, one with the department head, another with the dean, and a final sit-down with the provost and other higher-ups. Our report will come to the department not directly from us, but by way of the provost, and usually some time after our visit is concluded.

That gives most members of the department a chance to gain some distance, see the broader context, and begin to move forward. But chairmen often approach our final meeting with apprehension. Inevitably some of what we will say is news that they were hoping not to hear, and while we try to be upbeat and optimistic, and they put on a brave smile to face the moment of truth, chairmen greet us on that last day like we're bringing blindfolds and cigarettes.

Deans and provosts approach the final interview with more equanimity. They thank the review team profusely and assure us that our review won't wind up buried in a file drawer.

That may be a little optimistic. I know that at least some of our recommendations will be ignored. But that's fine with me: Review teams aren't out to break eggs and make omelets. Our job is not to engineer change, but to provide a point of view that will stimulate local reflection and discussion. If that discussion winds up improving the position of a department on the campus and among its peers, then we've done a service to the profession, and the grueling, whirlwind review process has been worthwhile.

Dennis Baron is a professor of English and a former department chairman at the University of Illinois at Urbana-Champaign.

Copyright © 2005 by The Chronicle of Higher Education