Faces of Success Reflect the Need for New Tests

He said, “I picked up some test-prep books and I’ve been working my way through some of the quizzes.”

I thought to myself, “Dick, you’ve got to get a life,” but I didn’t say anything, and he said, “Why do they have these verbal analogy questions?”

Now Pat Hayashi is to verbal analogies as Tiger Woods is to golf. So I said, “Dick, analogies help gauge a person’s verbal adeptness. They test a person’s deep understanding of language.”

He looked at me and said, “What research, what theory of cognitive development justifies that nonsense?”

Before I had joined Dick’s staff, I had done some research of my own. I knew that in his day Dick Atkinson was to cognitive psychology as Michael Jordan was to basketball. He was the most frequently cited cognitive psychologist in the world. So I decided maybe I shouldn’t argue theories of cognitive development with him and I tried another tactic. I said, “These questions test a student’s ability to reason analogically. Analogical reasoning is extremely important in modern society.” And for good measure I threw in, “Lawyers think that way.”

He looked at me as if he was trying to figure out a way to send me back to Berkeley. But he patiently explained. He said, “These questions are nothing more than a vocabulary test. If a student knows the definition of the words, the reasoning involved is trivial. If a student doesn’t know the definition, no amount of intelligence or reasoning ability will help.

“We all know that language acquisition depends heavily on family environment. And we all know students, particularly poor students and recent immigrants who haven’t been exposed to rich, college-level English, but who are hard working and smart and who have great potential. What about those students?”

At that point I decided to just listen and learn. Over the next 18 months I learned about the power of standardized tests. I learned about how they influence what teachers teach, how they define America’s notion of merit, and how they can open up or close off opportunities. I learned how they affect a student’s sense of self-worth, how they shape a student’s dreams. After a year and a half of listening, I could distill President Atkinson’s thoughts into one sentence: In America, students should be tested on what they are taught.

On Feb. 18, 2001, at the national meeting of the American Council on Education, President Atkinson gave a speech based on that simple principle. What followed was amazing.

He was flooded by invitations to appear on “Good Morning, America” and other talk shows. Newspapers put his speech on the front page above the fold. Time magazine and Newsweek ran cover stories. People wrote op eds, pro and con, and hundreds of letters poured in.

During all of this excitement I became a teacher again. Teachers all do the same thing when we think about educational policy questions. We put a human face on the issue by asking, “How will this policy affect my students?” And as I listened to Dick talk about his idea—that students should be tested on what they have been taught—that is the question that I asked. Would such a change help or hurt the students that I know and care about?

Let me introduce you to three of them. I’ll begin with Yvette.

Yvette Takes Action

Yvette had suffered her first stroke when she was three. She had another stroke when she was twelve. As a result, her entire right side was paralyzed, her leg, her arm, the side of her face. Because of brain damage she had a very hard time with language skills. She was in and out of hospitals all of her life. She struggled to keep up with her classmates, and she did well because she worked two, three, four times harder than anyone else.

She always got good grades, but her test scores were abysmal. And worst of all, there was no way she was able to connect the information she received from her test to any kind of personal plan of action. So whenever she took a test, she would become depressed, because they seemed to say that she was nobody and would never amount to anything. She got into Berkeley not because of, but in spite of, her test scores. We looked at her entire record and saw that against great odds she had succeeded in everything she had tried.

I met her at a conference. We sat together at dinner. Hoping that we would have a nice, light conversation, I said, “Hi, how are you?” She said, “I’m really angry.”

I took a deep breath and I asked her why. She said she was upset by the Daily Cal, our student newspaper. She said a student had just killed herself by jumping off of our student union and the Daily Cal explained the suicide by saying that the student had multiple sclerosis.

She went on. “The Daily Cal was saying that the disabled don’t have as much to live for as able-bodied people, that the disabled somehow aren’t fully alive, aren’t fully human.”

I asked her what she was going to do with her anger. She said, “I’m going on a one-woman crusade to get people to stop locking their bikes on disabled access ramps.” She said that the bikes make it really hard to get into buildings. And to her, it was just a simple matter, teaching people to be civil, to think about others when they act and speak.

I learned a lot from her that night.

Later, I saw Yvette again. There was a big protest brewing on admissions. I heard the students were planning to take over my office.

I worked in Sproul Hall, a favorite target for protesters. This didn’t make me happy. Frankly, I was very worried. So I called my boss at the time, Vice Chancellor Russ Ellis. He worked in California Hall. I said, “Russ, what should I do? Students are planning to stage a sit-in right in my office.” He said, “Look, Pat, this is Berkeley. You’ve got to go with the flow.”

The next morning two hundred students took over California Hall. Russ called me in a panic. “Pat, they’ve taken over the building.”

I said, “Russ, this is Berkeley. Go with the flow.”

He said, “I’m glad you feel that way because they’re chanting your name, and the police want you here now to negotiate with them.”

The chief of police came by to pick me up and as we were walking across campus I asked, “Where were you guys? How did they get in the building?” He said, “Oh, they were really clever. They used this disabled black student. She was friends with a student aide named Tyrone, who was guarding the door, and she asked him to let her in the side door for the disabled. But she was just a decoy, and when he opened the door for her, all the protesters rushed in.”

Yes, it was my friend, Yvette.

Two weeks later, I took her out for coffee.

I was really mad at Yvette for using her friend, Tyrone, and I was mad at the rest of the protesters for using her. So I looked at her and I asked her, “Yvette, is it true that you acted as a decoy?” When I asked, she sat up straighter, she started smiling, and her eyes glistened. She said, “Pat, it was so exciting. I felt just like a spy. I’ve never been able to do anything like that. I’ve never been able to do anything daring, anything courageous. It was the best day of my life.”

I had planned to shake my finger at her, but I just mumbled, “Well, you should consider apologizing to Tyrone.” “Oh, I did that right away,” she said, “He understood.”

Robert Does the Write Thing

Now let me tell you about another student, Robert. I hired Robert to help me clean up our office files. We worked together over the winter break so I had a chance to get to know him. We hit it off because he was an English major, just as I had been. I found out that he had a complicated life. He had faced the problems of a student who had jumped from school to school. And at each new school he started at the bottom of the class and pretty much stayed there. His grades were just fair. He did OK on tests, but they didn’t mean anything to him because they never allowed him to take stock of where he was. They never helped him to plan where he wanted to go. But he got lucky and joined an Upward Bound program.

An Upward Bound staff member noticed this quiet kid who always had his nose stuck in a book. He originally was rejected by Berkeley, but he got in on appeal because the Upward Bound director said, “Robert loves to read and loves to write. If we give him a chance, he will make it.”

Robert’s first year at Berkeley was rough. His father was in prison. His stepbrother had just been arrested for manslaughter. His mother was on welfare and, as the oldest son, it was Robert’s responsibility to keep everything together. He was on the verge of flunking out. He was very bright, but he had a lot on his mind. The thing that was bothering him at the moment was that his aunt was a crack addict, living with another crack addict, and they would punish their little three-year-old son, Santos, by locking him in a closet. Robert’s mother called a social worker, who took Santos into protective custody. This infuriated Robert’s aunt, and her boyfriend went over to Robert’s mother’s apartment, tore the place apart, and told her that she had better watch herself. Robert felt that it was his responsibility to go home, to confront this guy, to call him out, and to protect his mom.

I asked him what he planned to do. He was scared, but he made it clear that he could not, he would not, back down. I knew that there was nothing I could say that would change what he felt he had to do. So, in the end, I lamely suggested that he write about what he was thinking and feeling and describe what happened.

Well, he did. And he ended up writing a short story. He stayed in school, enrolled in Maxine Hong Kingston’s creative writing class and got an A+. Later he entered a short story in a campuswide competition open to students, staff, and faculty and won second place. He managed to work himself off academic probation and, once he felt a part of Berkeley, like all good Berkeley students, he became a social activist. He started Berkeley’s first Chicano fraternity.

Jenny Covers Enormous Ground

Finally, let me tell you about Jenny. Jenny was born in Cambodia. When she was seven, her mother took two family photos, punched holes in the top of the photos, strung some string through the holes, and then hung one photo around her neck and one around her little sister’s neck. And then said good-bye. Their two older brothers, who were guerrillas fighting against the Vietnamese who had invaded Cambodia, took these two little girls into the jungle so that they could work their way out of the killing fields of Cambodia and into Thailand and the safety of a refugee camp. As they left the village, she told me, they had to walk around the body of their neighbor, who was lying dead in a creek bed being eaten by small fish.

They stayed in the jungle for two weeks. They were scared the entire time. They had to hide from the Vietnamese, but also they had to hide from their own countrymen, the Khmer Rouge. They had some rice, but mostly they had to scavenge food in order to eat. All this was horrible. But Jenny told me that the most horrible thing of all was that she and her little sister saw their brothers torture and then kill a Vietnamese soldier.

By the time she was 11, Jenny had somehow made her way to Oakland, where she lived on welfare and attended her first American school. She and her sister started school in the middle of the year just before Christmas. That was hard, because they didn’t know anyone, but they got lucky. They found some wonderful teachers. Jenny’s first teacher made a change in the school’s Christmas program. She had Jenny and her little sister sing “Twinkle, Twinkle, Little Star” because those were the only English words they knew. This teacher created a place for them in the Christmas pageant because this fine teacher, this fine person, was determined that these two little Asian girls would not be left out in their new country.

Jenny went to high school, fulfilled her UC requirements, and went on to Berkeley. There was absolutely no doubt she would succeed. She had come way too far to stumble now. She had very good grades—nearly straight A’s—but her test scores told a quite different story. She had terrible test scores.

She was making great strides in school, just inhaling everything that was presented to her—we’ve all known students like her—but her test scores hardly budged. That was bad, but what was worse was that the test scores didn’t provide any information to her or her teachers about what she might focus on. They just told her that she was at the bottom of the heap.

When she applied to Berkeley she had the lowest verbal scores that I have ever seen. But she had as much to say as any person I’ve ever known. Once again we admitted her on appeal. Our Upward Bound director, again, told us her whole story.

The Moral of the Stories

Now, why do I tell you these three stories? What’s the moral of these three stories? For me, their stories tell me a great deal about what tests should and should not do.

Tests did not help these three students. They all started in different places. Jenny started in Cambodia. Robert moved around so much he doesn’t know where he started. Yvette began her schooling in a hospital. The tests they took, which purported to test their aptitude, only told them that they didn’t have any. But when we look at their lives, we realize that any university in the United States should be eager and proud to admit Jenny, Robert, and Yvette. They are hard working and smart, they are brave and good. By listening to their stories we learn that rather than create tests to judge them, we should think about students like Jenny, Robert, and Yvette, and use them to take our measure. And if our tests do not do them justice, then we must find our own policies wanting and choose new tests.

And that is exactly what the University of California is doing. The faculty committee that oversees admissions has recommended that UC require only curriculum-based achievement tests. In other words, UC should test students only on what we’ve asked them to study and master. The faculty admissions committee has said that such a curriculum-based achievement test creates great possibilities for individual students, for schools and colleges, and for society.

We all know that students start at very different places and have to take very different paths. Rather than a barrier, a curriculum-based test can be a road map. A curriculum-based test provides students with specific guidance on how to prepare for college. What the test covers is not a mystery or a secret. On the contrary, a curriculum-based test covers just that—the college-prep curriculum. The test itself tells students what to focus on. It tells them to concentrate not on the tricks taught in test-prep courses, but to concentrate instead on what their teachers are teaching them in the classroom. Test scores tell students much more than just whether they have done better or worse than other students. Test scores from a curriculum-based achievement test provide students with direct feedback about their particular academic strengths and weaknesses. The scores provide them with information on how they might improve. And that is exactly the type of information that would have benefited Jenny, Robert, and Yvette, and students like them.

Because curriculum-based tests have this diagnostic power, students can and should take them early so that they can plan their course of study and focus their attention. Curriculum-based tests also help teachers in schools identify the particular areas that they need to focus on, help them gauge their own effectiveness. In the short run, a curriculum-based test will not eliminate the disparities among different racial and ethnic groups, or among students from different socioeconomic backgrounds. But it will make a big difference in the long run.

First, immediately, it will change the discussion, the public discourse, about these disparities, about why the gaps exist. Right now, tests that are not curriculum-based, those that purport to measure aptitude, lend themselves to pernicious, superficial interpretations. Some people take differences in test scores and attribute them to differences in native intelligence, or to differences in how much different groups value education, or to so-called pathologies in family structure.

Curriculum-based achievement tests force leaders to ask different, more powerful questions: What courses did these students take? What books did they read? How hard were the students asked to work? How qualified were their teachers? In other words, a curriculum-based achievement test focuses attention on the distribution and use of opportunities, and that’s exactly where America should focus its attention.

Most important of all, a curriculum-based test focuses on individual student progress. We all know that the defining feature of American education is unevenness. Some students have the good fortune to go to well-endowed schools staffed by excellent teachers. Others have the misfortune of having to attend poor schools where very little teaching or learning takes place. Because the playing field is not level, American universities must pay close attention to a student’s trajectory. We have to know, we have to ask, about a student’s progress. Of course we need to know where they are right now, what they have actually mastered. But we should also ask, “How much have they improved?” so that we can get a better sense of their ambition and their potential.

Think again about Jenny, Robert, and Yvette. Their test scores only gave a snapshot of who they were. They provided no information about where they started, how much ground they had covered, or where they might end up. And to my mind, that’s the most important thing about a curriculum-based achievement test. It would help everyone—students and their teachers, schools and universities—to understand who these students might become.

I realize that nothing that I have said is new to the ACT community. I am familiar now with the history of the ACT. I know that Dr. Lindquist envisioned the ACT as more than a common yardstick. He saw the ACT as a common staircase. And because of the debate sparked by President Atkinson, one of the wonderful things that has happened is that we, at the University of California, have started to work very closely with the ACT. I’d like to express my personal admiration and appreciation for the work done by Ernest Valdez, Cyndie Schmeiser, and Dick Ferguson. They have been extremely articulate in describing to the UC administration, faculty, and our regents what the ACT offers to our students in California. I chose my words carefully, just as they do. Notice that I did not say “what the ACT offers UC.” They have stressed that the ACT’s first concern is for students. They emphasize that the ACT offers more than a test. The ACT offers a philosophy that is reflected in an integrated program of services that, in turn, reflects a democratic vision of society and of education. It reflects and offers a vision that shows us who students might become.

Who They Became

Let me finish my story by telling you who Yvette, Robert, and Jenny have become.

Robert got married and dropped out of school. But he came back and graduated. He ended up going to my school, the school of public policy at Berkeley. He did very well and is now working for the city of San Francisco. He’s still having a hard time, but I realize that Robert will always have a hard time because he likes having hard times. They give him more to write about. And write he does. He’s published several poems and short stories; most are good, some are wonderful, all are honest.

Jenny sailed right through Berkeley as we knew she would, and while at Berkeley she helped some of our faculty gain entrée into the southeast Asian community. After she graduated she worked with a community agency that worked with at-risk Asian youth, which is a euphemism for Asian gangs, and now she’s at grad school in Massachusetts, studying education.

Yvette had a tough time. She did very well at Berkeley and had been admitted to a graduate program at Harvard. But there was one really cold winter at Berkeley, the coldest we had had in 50 years, and she told me then that she found her body just shut down in the cold. She had to take 1,000 milligrams of painkillers just to be able to walk from class to class, and the painkillers were beginning to have awful side effects. But, worst of all, she knew she could never last a winter at Harvard. So she had to make other plans. But before she could make those plans, she got cancer. This really upset her because she got it in what she called her “good leg.” Despite what she might think, Yvette has more courage than anyone I know. She went into the hospital and went through radiation and chemotherapy, and while in the hospital, at the ripe old age of 22, for the first time she became a grandmother. The hospital had a program where people adopt terminally ill children who have nobody else and love them as only a grandmother can. In addition to adopting two small children she also became the hospital ombudsman and promptly began suing the place. I like to think she got her training at UC, and I like to think our new test policies will allow us to meet many more students like her. Thank you.