Research for Mission-Driven Innovation
Research for Mission-Driven Innovation (RMDI)
Research for Mission-Driven Innovation (RMDI) is a new program for making more explicit connections between important customer needs and scientific inquiries at ACT Research. RMDI is funded by Research discretionary funds.
The goal of RMDI is for every funded project to:
(1) address the intersection of an important customer need and an important research question and to
(2) be executed by an interdisciplinary team in order to generate more ideas, increase relevance, and accelerate innovation and transformation.
"RMDI is a natural extension of our mission: by addressing important customer needs through rigorous scientific inquiry, we are developing the next generation of solutions for helping people achieve education and workplace success."
Alex Casillas, Principal Research Psychologist, Design Based Research
Projects Awarded Funding for Iteration 1
A Diagnostic Prototype for Measuring Mathematics Learning
Pamela Paek, Sweet San Pedro, Yile Zhou, Randy Weiner, Keith Devlin, Carolyn Brown, Jerry Zimmerman
We will develop a diagnostic assessment prototype for math learning that will focus on the how and why students struggle in order to build a profile of student learning characteristics and expand a current game-based assessment. Foundations in Learning will be a partner to help to flesh out the learner and learning characteristics. We will partner with BrainQuake to expand their current game-based assessment to develop a more robust profile of the learner, the way they problem solve, and formative feedback in the fine-tuned places they struggle.
The Arts and Social and Emotional Learning (SEL) in the Context of Underserved Populations
Pamela Paek, Dana Murano, Sweet San Pedro, Cassi Barker-Carr, Scott Payne, Jeremy Burrus, Becky Bobek, Maria Vasquez, Lew Montgomery
Partnering with middle schools in Watts and Eastside Theater Works, we will co-create resources to better prepare adults and students to effectively engage in social emotional learning (SEL) while also embedding SEL within academic content. This project will use continuous improvement to create pre-SEL activities to prepare adults/students for SEL.
Using Social and Emotional Learning (SEL) to Increase Diversity
Cristina Anguiano-Carrasco, Kate E. Walton, Jason Way, Krista Mattern, Amy Nicknish, John Whitmer, Ruitao Liu
Our goal is to gain insights to make evidence-based decisions for future research and product development and investigate the impact of including SEL for admissions purposes on diversity. Research questions include: 1) How can we combine ACT and Tessera scores to optimize cumulative college GPA prediction? and 2) What will be the impact on the diversity of admitted students? We created a cross department team with diverse backgrounds that includes experts from the Validity and Efficacy, the SEAL, and the CEL teams. We aim to build a solid research base to inform future product development.
SEL in the Workplace - Assessments, Training, Credentials. What do Employers Really Need? How Can ACT Position Itself to Satisfy those Needs?
Sandy Greenberg, Carla Caro, Jacqueline Carpenito, Alex Casillas, Ken Doucet, Helen Palmer, Cindy Hill, Kyle Jacobsen, Pat Muenzen, Jason Way
This is a multi-pronged approach to validate employers’ needs regarding essential SEL skills and develop potential solutions by conducting desk research and interviews with key stakeholders, framing potential solutions responsive to employers’ SEL challenges, facilitating focus groups to obtain specific feedback about the potential solutions, and developing conclusions about what employers value and what they
need to solve their SEL problems.
Implicit Social and Emotional Learning (SEL) Assessment Using Learning Analytics
John Whitmer, Kate E. Walton, Sweet San Pedro, Alejandro Andrade Lotero, Jeremy Burrus, Dana Murano, Saad Khan, Kyle Jacobsen, Joann Moore, Brian LaMure, Pamela Miler
What if we could observe social and emotional skills through student behaviors in online environments? For example, is grit associated with accessing course materials with greater regularity? Does that behavior, in turn, predict academic performance? Through a partnership between ACT, the University of Maryland Baltimore County, and VitalSource (an eTextbook provider), we will examine these questions. This project will provide immediate validity evidence for Tessera College and could be used to create functionality for feedback for students, teachers, and counselors through educational technology platforms.
ACT Online Prep (AOP) A Window into Test-Prep Practices
Edgar Sanchez, Raeal Moore, Sweet San Pedro, Scott Payne
We will examine how AOP is practiced and supported in high schools (B2B) to uncover ways in which schools use AOP in their test preparation initiatives. Using a three-phrase approach, AOP clients will first be classified based on demographic and product usage data, then they will be asked to complete a survey about the implementation of test preparations activities school-wide, and finally, select educators will be invited to participate in a phone interview to explore test preparation program design decisions at schools of interest. This work will support the development of an AOP implementation guide for schools and identify recommendations for AOP product improvement by developing a deeper understanding of test preparation practices in school and exploring trends between ACT scores and test prep activities and practices.
Supporting User Interpretations of Score Reports in a Technological Ecosystem
Daniel Lewis, Mark Kopulos, Becky Bobek, Pamela Paek, Sweet San Pedro, Yile Zhou, Rob Cook, Pamela Miler
ACT score reports are viewed by millions annually and communicate the results of, and inferences made from, examinees’ performance. But do our score reports do what they are supposed to do? This project will study whether we can use technology to free ourselves from the constraints of 8.5”x11” thinking and increase users’ comprehension of reported information and increase the perceived value of the reported information. The results of the study may enhance the value of ACT reports and support better decision making for our customers. This study will also allow us to address the following questions: Will operationalization of reporting principles increase appropriate interpretation of reported information? And will operationalization result in an increase in user satisfaction compared to the status quo?
Informing Tessera Playbook Development
Dana Murano, Jeremy Burrus, Kate E. Walton, Amy Nicknish, Kyle Jacobsen
College students report feeling insecure and unprepared for the workforce, while employers struggle to find new hires with adequate skills. Social and emotional skills (SEL) are key for workforce readiness and success, yet no tool on the market exists that takes a holistic, student-centered approach to implementing social and emotional learning at the college level. This project will allow us to create a product-plan for Tessera College SEL curriculum that aligns best practices according to research with students and instructor input. It will also support the need for a student- and instructor-centered system that combines SEL assessment and intervention for college students.
Projects Awarded Funding for Iteration 2
To Contextualize or Not to Contextualize? A Two-Part SEL Study
Jeremy Burrus, Alex Casillas, Lori Beckerman, Joann Moore, Raeal Moore, Louis Tay (Purdue University)
Student social and emotional (SEL) skills may sometimes manifest themselves differently across various contexts. For example, a student who displays strong teamwork skills at school may struggle with teamwork in out-of-school situations. If this is true, it implies that we may be able to improve the validity of our social and emotional skills assessments by varying the contextualization of items. For example, some items may refer to student behaviors at school, whereas others may refer to student behaviors at home. In a two-part study, we will vary the contextualization of our social and emotional skills items by: (a) having students answer typical survey items framed around multiple contexts during a single assessment period, and (b) having students answer a smaller set of items several times a day over the course of a few days on their smartphones. In addition to improving our assessment validity, this project may also help us to better target and personalize social and emotional skill interventions.
Automatic Smoothing in Equating
Automation plays an important role in our society by freeing up resources and providing fast results. The current equating process – a mathematical procedure to ensure scores on different test forms are comparable – includes manual procedures that take time. The goal of this project is to automate the manual procedures to save resources and enable faster score reports.
Transadaption of Tessera for Total Market Domination
Alex Casillas, Cristina Anguiano-Carrasco, Jeffrey Steedle, Elkin Arrendondo, Changhua Rich, Cory Galano, Capstan, Yi-Lung Kuo, Juan Antonio Valdivia Vasquez et al.
The need for a suitable social and emotional learning (SEL) measurement tool is cited among one of the biggest hurdles to implementing SEL programs throughout the world. Unfortunately, the process for the translation and adaptation (i.e., transadaptation) of assessments can be time and resource consuming and often focuses on country-specific versions. This project will create transadapted versions of ACT Tessera by focusing on an approach that is expected to make the process more affordable, efficient, and yield regional—rather than country-specific—language versions for Latin America (Spanish) and China (simplified Chinese).
RMDI Partnership Opportunity
ACT recognizes the value of partnerships in research and welcomes the opportunity to collaborate with academic and industry organizations nationally and internationally to promote education and workforce success.
We invite you to contact our RMDI team if you believe you can help ACT Research address pressing scientific questions in a way that offers solutions to important customer needs or problems.
Please contact us at RMDI@act.org to discuss potential partnership opportunities.