CAAP: A Tool for Colleges to Measure Academic Achievement

A number of postsecondary systems and institutions find ACT’s CAAP to be a helpful tool for providing accountability, and for refining the courses that form the foundation of their academic programs.

CAAP offers tests in Reading, Writing Skills, Writing Essay, Mathematics, Science, and Critical Thinking. Each may be used as a stand-alone exam or in combination with other CAAP exams. CAAP scores also can be compared to ACT Assessment^®, COMPASS^®, or ASSET^® scores to show growth over time.

South Dakota

In 1998, the South Dakota Board of Regents and the six public universities it oversees decided to assess the quality of the postsecondary general education program. Regents also wanted to provide students with information about their progress and compare the performance of students in South Dakota to students nationwide.

The Regents chose to administer four of the CAAP tests—Writing Skills, Math, Reading, and Science Reasoning—to all students who had completed 48 credit hours.

“The state and the Regents have used the CAAP gains reports to provide incentive funding for the institutions. It is a pat on the back, a reward for providing your students with an education that resulted in expected and higher- than-expected gains,” said Lesta Turchen, senior administrator and chief academic officer for the South Dakota Board of Regents.

Term reports and an annual report, The CAAP Gains Report, show how each university fares on each subject tested.

“Policymakers want to know that we are making efficient use of the state resources,” said Turchen. “Before CAAP, we really did not have the mechanism other than anecdotes to show that our universities have increased the knowledge of students. But if you want to use data-driven decision making, anecdotes just don’t do it.”

Data from CAAP administrations also have helped the state focus its programs.

After falling below the national mean in English for five out of six years, one university implemented an additional writing requirement for all of its students. Another placed a new focus on math after receiving lower-than-expected scores in that subject.

South Dakota now also uses scores from the ACT Assessment or COMPASS to place incoming students appropriately. Faculty members told the Regents that they “can’t wait until students sit for the CAAP test to deal with those who need remediation,” Turchen said.

“So they recommended and we have adopted a systemwide placement process in English and math.”

Identifying students who need assistance and providing them with a way to monitor their personal progress is what Turchen calls CAAP’s biggest success.

“The CAAP exam is a primary tool we use in working with individual students to improve their skill levels so that they can be successful in upper-division courses in their majors,” she said. “If you can better prepare students to be successful in their coursework, they’re more likely to reach their personal goals.”

The Board of Regents intends to expand the use of CAAP to test students enrolled in its Electronic University Consortium.

“Our regional accrediting agency certainly scrutinizes how you establish student learning outcomes for general education for distance students,” Turchen said. “The CAAP tests allow us to substantiate that the education those distance students receive is equivalent to the education they would receive in a face-to-face situation.”

San José State University

At San José State University in San José, California, students must pass the CAAP Writing Skills Test before they can enroll in an upper-level writing course required for graduation.

The California State University system, which includes San José State, requires a writing assessment for graduation, although it lets each campus select its own assessment tool. San José State wanted a test that would be objective, fair, easy to implement, and inexpensive for the students. CAAP works well for the school, said Bob Cooper, associate vice president of Undergraduate Studies.

“Now, with CAAP, students come into the upper-level courses with a set of skills that every instructor can count on for every student. We get to teach those more demanding courses at a higher level, and spend less time trying to fix up basic writing problems,” Cooper said.

“It’s part of the student culture to say in those first two years, ‘You’ve got to get yourself ready to take this test so that you can go on and take those upper-division courses.’ Having CAAP in place means that students take learning their writing skills more seriously from the very beginning. It’s a great motivational tool.”

The results from the CAAP test also have helped San José State to improve its lower-level writing courses.

“One of the pleasant things for us to discover is that the pass rate for students who take their lower-division English courses here is substantially higher than the pass rate for transfer students,” Cooper said. Now the university works with the community colleges in the area to improve the writing skills of students who plan to transfer to the university.

Results also have shown San José State that its remedial classes are not as effective as the university wants them to be in developing the writing skills students need to pass the test and be successful in upper-level courses. San José State is working to improve the remedial courses, especially for students who speak English as a second language. Reliable skills assessment from the CAAP test aids the effort.

Front Range Community College

When results of an accreditation visit in the late 1990s indicated Front Range Community College in Westminster, Colorado, needed to strengthen its assessment plan, staff started looking for a tool to measure FRCC’s general education program. The college, which serves more than 28,000 students at three campuses in north central Colorado, wanted to be able to demonstrate to students, state legislators, and taxpayers the value of an education from FRCC.

In 1999, FRCC chose CAAP as a mandatory test for all of its graduates. Though CAAP scores are not recorded on students’ transcripts and do not count toward students’ grades, the college uses aggregate scores as a measure of its general education program and as a means to ensure that students at each campus receive the same quality of education.

FRCC’s CAAP scores have never been significantly lower than the national average, and in most cases are significantly higher—an outcome that bolstered the college’s reputation with faculty, students, and the community.

“We are definitely validating our general education program with it,” said Laura Jensen, a research analyst for FRCC. “We haven’t had to implement a lot of change because of the CAAP, because it’s showing that we are strong in our general education program.”

Because Colorado requires all high school juniors to take the ACT Assessment, FRCC recently began using the ACT Assessment/CAAP linkage reporting service to show performance gains. In the first year, “we look great,” Jensen said.

Results showed higher-than-expected gains of 18 percent in reading, 12 percent in writing, 14 percent in math and 11 percent in science—3 percent to 9 percent higher than the national average scores. FRCC believes these data could be an important piece of information for performance reporting in the state.

Jensen says that it’s a huge boost for the faculty to see evidence of student improvement. “With the CAAP results, we are able to show our stakeholder groups, our students, our taxpayers, our businesses, our parents, and the communities we serve what they’re getting for the investment that they’re putting into community colleges, particularly Front Range Community College.”