Annotated Bibliography

This annotated bibliography identifies publications and presentations regarding the development, norming, validation, and implementation of the ACT Engage® (formely known as SRI). It also includes references to important policy reports that are relevant to ACT Engage. This bibliography will continue to expand and grow as ongoing studies are published and as new studies and implementation efforts are created.

Section 1: Development

Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508.

Details the rational-empirical approach involved in the construction of a battery of psychosocial factors for which the final product was ACT Engage. Development information includes scale construction, methodology, initial pilot study, exploratory and confirmatory factor analyses, and the resulting model of scales and constructs. Scales displayed good to excellent internal consistencies (evidence of reliability).

Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288.

This meta-analysis was conducted as an attempt to integrate the psychological and educational literatures regarding the postsecondary outcomes of achievement and retention. The authors specifically examined the relationship between psychosocial and study skill factors (PSFs) across 109 studies. Nine broad constructs of PSFs were categorized from the literature: achievement motivation, academic goals, institutional commitment, perceived social support, social involvement, academic self-efficacy, general self-concept, academic-related skills, and contextual influences. Results indicated moderate relationships between retention and academic goals, academic self-efficacy, and academic-related skills. Regarding achievement (GPA), academic self-efficacy and achievement motivation were the best predictors. This study indicates that there are incremental predictive contributions by PSFs above that of socioeconomic status, standardized achievement, and high school GPA for predicting these important college outcomes.

Robbins, S., Le, H., & Lauver, K. (2005). Promoting successful college outcomes for all students: Reply to Weissberg and Owen (2005). Psychological Bulletin, 131, 410-411.

This article is a reply to Weissberg & Owen (2005), who claim meta-analytic findings (Robbins et al., 2004) did not generalize to certain institutions (primarily commuter students) or to certain demographic subgroups. The article discusses how meta-analytic methodology is generalizable to both four and two year institutions. Additionally, PSFs are robust predictors of important college outcomes.

Section 2: Norming and Validation

Allen, J. & Robbins, S. (2010). Effects of interest–major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23–35.

This study used longitudinal student data from a range of postsecondary institutions (4-year and 2-year), to test the effects of interest–major congruence, motivation, and 1st-year academic performance on timely degree completion. Findings suggest that interest–major congruence has a direct effect on timely degree completion across institutional settings and that motivation has indirect effects (via 1st-year academic performance). The total effects of both interest–major congruence and motivation on timely degree completion underscore the importance of including both constructs (i.e., congruence and motivation) in understanding student adjustment and postsecondary success. Implications for theory and counseling practice are discussed.

Gore, P. A. (2006). Academic self-efficacy as a predictor of college outcomes: Two incremental validity studies. Journal of Career Assessment, 14(1), 92-115.

Details of two incremental validity studies, conducted in an attempt to examine the extent to which academic self-efficacy beliefs predict college outcomes beyond that accounted for by standardized test scores. Although findings do provide evidence supporting the importance of academic self-efficacy in predicting college outcomes, this relationship is not unaffected by additional factors. The types of efficacy beliefs measured, when efficacy beliefs are measured, and the nature of the criteria used all affect the nature of the predictive relationship between academic self-efficacy and college outcomes. For example, self-efficacy beliefs are more strongly related to college performance and persistence after students have gained college experience.

Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social desirability and college academic performance. Personality and Individual Differences, 41, 663-673.

This study examines the relative ability of the Big Five Inventory (BFI) and ACT Engage, formely known as the Student Readiness Inventory (SRI), to predict college GPA. ACT Engage scales accounted for a larger proportion of the variance (range = 24% to 29%) than the BFI (range = 3% to 8%). Social desirability effects were also examined and found to be minimal. Hypothesized relationships between ACT Engage and the BFI were found, providing further evidence of the construct validity of ACT Engage.

Porchea, S., Allen, J., Robbins, S., Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education.

This study examined the effects of student (academic preparation, socio-demographic, situational, and academic preparation) and institutional characteristics on long-term enrollment outcomes for students who originally enrolled at a range of two-year institutions. Using over 4,000 community college matriculates, we tracked five years of enrollments and degrees awarded. The findings point to the central importance of academic preparation and degree expectations across outcomes. Also the results reveal differential relations of parent's education, parent's income, and motivation. These findings have implications for identifying entering community college students who are at high risk of not fulfilling their educational aspirations and for tailoring efficient intervention strategies. Implications for community college accountability systems also are discussed.

Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the Differential Effects of Motivational and Skills, Social, and Self-Management Measures from Traditional Predictors of College Outcomes. Journal of Educational Psychology, 98, 598-616.

This study attempts to answer the multiple questions, all of which deal with the effects of psychosocial and study skills factors (PSFs) in predicting college outcomes of retention and achievement. First, there are differential effects of PSFs when predicting college outcomes. Specific scales were found to be differentially predictive of academic success and retention. PSFs provide additional predictive ability beyond that of traditional predictors such as cumulative GPA and achievement test scores. The study also examines curvilinear effects of some of the scales and differences in PSFs across two- and four-year institutions.

Allen, J., Robbins, S., Casillas, A., & Oh, I.-S. (2008). Third-year college retention and transfer: effects of academic performance, motivation, and social connectedness. Research in Higher Education, 49, 647-664.

This study examined the effects of academic performance, motivation, and social connectedness on third-year retention, transfer, and dropout behavior. To accommodate the three outcome categories and nesting of data within institutions, the authors fit a hierarchical multinomial logistic regression path model with first-year academic performance as a mediating effect. The sample included almost 7,000 students representing 23 4-year universities and colleges. This work expands the current state of persistence research by (1) considering the effects of motivation and social connectedness on college persistence beyond the first year of college, (2) testing whether the effects of motivation and social connectedness on third-year retention and transfer are direct, indirect, or both, and (3) testing whether the effects of academic performance, motivation, and social connectedness are different for retention and transfer. The results show that (1) academic performance has large effects on likelihood of retention and transfer; (2) academic self-discipline, pre-college academic performance, and pre-college educational development have indirect effects on retention and transfer; and (3) college commitment and social connectedness have direct effects on retention. Practical and theoretical implications of these findings and directions for future research are discussed.

Section 3: Implementation

Allen, J., Robbins, S., & Sawyer, R. (2010). Can measuring psychosocial factors promote college success? Applied Measurement in Education, 23, 1–22.

Research on the validity of psychosocial factors (PSFs) and other noncognitive predictors of college outcomes has largely ignored the practical benefits implied by the validity. The authors summarize evidence of the validity of PSF measures as predictors of college outcomes and then explain how this validity directly translates into improved identification of high-risk students and improvements in academic success and persistence rates. The authors discuss how the value of measuring PSFs hinges on the effectiveness of institutional support programs designed to help high-risk students. The authors recommend that postsecondary institutional research focus on the net effect of systems for identifying and intervening with high-risk students.

Cole, R.P., Saltonstall, M., & Gore, P.A. (2008). Assessing student readiness to promote student success: A campus collaboration. In W. G. Troxel & M. Cutright (Eds.). Exploring the evidence: Initiatives in the first college year (Monograph No. 49, pp. 27–34). Columbia, SC: University of South Carolina, National Resource Center for the First-Year Experience and Students in Transition.

Detailed case study of the implementation process and applications of ACT Engage at a large southwest state university. Components include a program description that was conceptualized as systemic, coordinated, and integrative. ACT Engage meshed with the institution's goals of having a common focus and coordination of efforts, as well as implementing proactive intervention strategies for incoming students. Also included are critical elements of the implementation, lessons learned, and recommendations.

Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney, E. & Gore, P.A. (2009). Associations of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50, 101-118.

This study tracked resource and service utilizations and first-year college outcomes for the entering first-year class at a traditional 4-year postsecondary institution. Resources and services offered by the institution were grouped into four general categories: academic services, social resources, recreational resources, and advising sessions. The authors investigated the interrelation of risk, resource and service utilization, and first-year GPA and retention. The results show that utilization of each resource/service category was positively associated with GPA and/or retention. The authors further investigated whether the associations of resource and service utilization and outcomes were moderated by risk. The findings suggest that the associations of academic services and advising sessions with GPA were more pronounced for higher-risk students. The paper ends with a discussion of the findings, including how the differential associations of resource and service utilizations and outcomes can affect intervention decisions with high-risk students.

Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184.

Using both organizational and educational perspectives, the authors proposed and tested theoretical models on the mediating roles that psychosocial factors (PSFs; motivational, emotional, and social) play between college interventions (academic skill, self-management, socialization, and first-year-experience) and college outcomes (academic performance and retention). The authors first determined through meta-analysis of 404 data points the effects of college interventions on college outcomes and on PSFs. These meta-analytic findings were then combined with results from Robbins et al.'s (2004) meta-analysis to test the proposed models. Integrated meta-analytic path analyses showed the direct and indirect effects (via PSFs) of intervention strategies on both performance and retention outcomes. The authors highlight the importance of both academic skill and self-management-based interventions; they also note the salience of motivational and emotional control mediators across both performance and retention outcomes. Implications from organizational and educational perspectives, limitations, and future directions are addressed.

Section 4: Policy Reports

ACT, Inc. (2008). What we know about college success: Using ACT data to inform educational issues. Iowa City, IA: Author.

This policy issue brief presents the argument that, if students are ready for college, dropout rates and the costs of remediation are reduced and more students persist in and graduate from college. Existing ACT research related to college readiness and college success offers insights about the impact of readiness on success during (and after) college. The report ends with recommendations designed to maximize students' chances of succeeding in college.

Habley, W. R., & McClanahan, R. (2004). What works in student retention – All survey colleges. Iowa City, IA: ACT.

This report investigates the impact of campus practices on college student retention and degree completion. It reviews the results of the 2003 survey sent to over 2,000 two- and four-year institutions. It highlights current retention programs that were identified as having the greatest impact on students. It concludes by suggesting recommendations to help college administrators with enhancing existing student retention practices.

Lotkowski, V. A., Robbins, S. B., & Noeth, R. J. (2004). The role of academic and non-academic factors in improving college retention. Iowa City, IA: ACT.

This policy report provides information from a major technical study about the influence of non-academic factors, alone and combined with academic factors, on student retention and performance at four-year colleges and universities. It highlights examples of successful retention strategies and stresses the need to evaluate the bases on which retention policies and programs are created. It concludes by offering recommendations to help administrators and policymakers consider both academic and non-academic factors in the design and implementation of retention efforts.