STANDARDS FOR PREDICTIVE INDICATORS

EWS 2.0 Workgroup Guidance

WORKGROUP GUIDANCE

Students typically send strong signals to educators about whether they are on track to graduate high school ready to continue their education. Valid indicators help us identify the most important signals. PAS workgroups established the following standards to provide guidance on identifying the most useful predictive indicators to promote student and adult success.

~ Indicators are quantifiable measures of behaviors, skills, and characteristics that are highly predictive of students’ being on track for high school graduation, postsecondary readiness, and adult success.

~ Indicators are most useful when they directly measure behaviors or outcomes that are important to students’ well-being and progress in school and training. This makes them actionable.

~ Indicator data can be obtained easily and regularly.

~ Collecting, recording, and reporting the indicators does not require substantially more effort from school-level personnel than is already required by federal and state laws and regulations,and district policies.

~ Indicators are timely.

~ Patterns and trends can be quickly observed and acted on; outcomes of actions can be promptly monitored, enabling rapid revision.

~ Indicators are reliable.

~ It has been demonstrated that they send dependable signals, over time and through repeated statistical analysis of large-scale, longitudinal databases in different settings.

~ Indicators point to underlying conditions that are “malleable” or modifiable.

~ In general, a few good indicators are more actionable than many.

~ Indicators can serve both a formative function (leading to immediate actions) and a summative function (leading to monthly, quarterly, mid-year, or end-of-year redesign of action systems). Essentially, however, indicators lead to actions.

~ Thresholds that indicate the type of action needed — indicating whether to take action now, not yet, or not at all — may vary by context, but will fall within a numerical range established by evidence.

NEW INDICATORS

New indicators, beyond those already in use and meeting the above criteria, must also:

~ Identify and prioritize substantially more youth in need of support than are identified by the existing indicators, and/or

~ Identify behaviors, skills, and characteristics that support adult success and are not captured be existing indicators; and

~ Lead to new solutions.

EXTENDING THE “ABCs”

Current research shows that students who attend high school nearly every day have the strongest track record of college persistence. Students who attend high school 97 percent of the time or more have the highest odds of college degree attainment, particularly for four-year degrees.

There is also evidence that students who attend less than 95 percent of the time and miss 10 or more days of schools begin to see diminishing odds of college degree attainment. Some schools and districts may initially want to see attendance between 95 and 97 percent as an “alert zone” in which students are more closely monitored and other indicators examined, but not as a signal for the school to take immediate action to improve college readiness.

We continue to deepen our understanding of social-emotional learning measures and students’ likelihood of college success. The behavior category in the ABC indicators is where states, districts, and schools can employ social-emotional indicators as they become sufficiently reliable and valid. 

In the meantime, all available evidence suggests that for students to succeed in college or career training, they need to develop the ability to self-manage their learning in such settings.

Thus, in the absence of established indicators, observation and assessment of students’ success in self-managing their learning could prove an effective means to monitor and support the development of behaviors important for their eventual college and career success.

While this EWS indicator for high school graduation is whether students are passing core academic courses (English and mathematics in middle school, and required credit bearing classes in high school), GPA or grade-point average is the most predictive indicator of college readiness and persistence. 

Research suggests that a B or higher average or 3.0 GPA for core academic courses is a dependable threshold indicating how likely students are to succeed in college studies. Studies at the local and state levels, however, have identified GPA thresholds ranging from 2.7 to 3.2 as being the most predictive of college success. This shows that districts and states should conduct their own analysis and set levels appropriate for their students. Such analysis, however, is not always possible, so PAS recommends using a GPA of 3.0 or a B average as the starting point.

Just as with attendance, districts and schools may want to first more closely monitor students near the cut point — i.e., those with 2.7 GPAs — and consider how they fare on other ABC indicators before devoting resources to improve their odds of college success.

COLLEGE READINESS AND PERSISTENCE INDICATORS

Workgroup Recommendations

Continue to use a limited number of valid, powerful indicators. Resist the temptation to add indicators simply because the outcome seems of some value to postsecondary success. Doing so is likely to add complexity without improving indicators’ usefulness.

Extend the most proven EWS for high school graduation indicators (the ABCs of Attendance, Behavior, and Course Performance) to predict students’ on-track status for college readiness and persistence.

Add two additional types of indicators to effectively monitor whether youth are on track for college success.

First, a checklist of students’ key postsecondary preparation milestones. These are the navigation steps students need to take in secondary school to keep all their postsecondary options open. They include

~ visits to postsecondary institutions

~ meeting with a college advisor or counselor

~ test-taking as required (SAT and/or ACT)

~ applying to appropriate postsecondary institutions

~ completing financial aid forms

~ accepting an offer to a postsecondary institution

Many students also need additional support once admitted to college to help them register for classes and campus housing, and navigate other challenges in the postsecondary transition.

Second, a composite measure — or checklist — that shows the postsecondary preparation “intensity” of a student’s high school experience. This may include

~ taking required courses for admission into the state university system

~ participation in Advanced Placement (AP), International Baccalaureate (IB), ~ dual-enrollment college courses, and/or three or more linked career electives

~ performance on state achievement assessments

~ developing stronger writing skills

A separate checklist may be kept for middle grades students, with an emphasis on the rigor of courses taken, since course selection, sequence, and performance in grades 6 – 8 can be critical in determining high school and postsecondary paths.

MEASURING STUDENTS’ SOCIAL-EMOTIONAL LEARNING ON THE PATHWAY TO ADULT SUCCESS

The many educators, researchers, and other advisors involved in PAS broadly agree that students’ integrated social-emotional-academic development is important for postsecondary and adult success. They also recognize that social-emotional learning in education is a relatively young but rapidly advancing area of research.

Therefore, PAS does not currently recommend that specific social-emotional skill levels should be used as indicators in EWS 2.0. When a state or district decides that research has determined how to measure social-emotional skills with sufficient reliability and validity, such measures could be included in the “Behavior” category in an ABC indicator system — as the behavior category includes not only measures of involvement with school disciplinary systems, but also beliefs and actions associated with school success.

PAS further believes that student surveys and data from teachers’ observations of students’ social-emotional behaviors and beliefs can be helpful in the analysis phase of EWS 2.0, when teams of educators work to understand why students may be off track for postsecondary success and decide on the action(s) to take to provide additional support. 

PAS participants recommend that schools regularly administer student and teacher school climate surveys and include key outcomes in the data reviewed by EWS 2.0 teams. This will help identify situations where actions at the class, grade, school, or district level may help multiple students with indicators, and show where preventative and generative actions can and must occur at the systems level.