The federal government recently released a new set of student outcome measures that are an improvement over traditional graduation rate metrics but still fall short of the data needed to better understand community college student progress or shape institutional improvement.
Below, I present an overview of the new Outcome Measures and some of the important nuances of how to measure community college effectiveness to support improvement.
The U.S. Department of Education (ED) released this fall a new set of metrics, known as Outcome Measures (OM), for all postsecondary institutions. The new Outcome Measures are an improvement on the statutorily defined completion measures that were collected only for students who began their postsecondary education at an institution and enrolled full-time. They broaden the number of students included and changed the tracking period for measuring outcomes for those students.
While the new OM metrics are an improvement, they still do not provide community colleges with the detailed information of importance to effectively measure student outcomes tied to their missions as are defined in the Voluntary Framework of Accountability (VFA).
What the new measures do
The new OM metrics track four sub-populations of “degree/certificate seeking” students who begin their education at an institution in a given fall for eight years. The four degree seeking groups are:
- full-time, first time in college (as defined in the student right-to-know legislation)
- part-time, first-time in college
- full-time, not first-time in college
- part-time, not first-time in college
The metrics reported for these four groups are the number who earned any credential by the end of six years, and after eight years the number who either:
- earned a credential (completed)
- did not earn a credential but were still enrolled in the eighth year (still enrolled)
- did not earn a credential, were not still enrolled, but had evidence of enrolling at another postsecondary institution after leaving the college (transferred)
- met none of the above criteria — their current status was unknown (status unknown)
The cohort of students is supposed to be identified in the first year of the tracking period and tracked for the full eight years. The same criteria are used for all postsecondary institutions who award degrees.
More outcomes via the VFA
The VFA provides more outcomes for community colleges that better reflect the community colleges. First, the VFA provides early indicators of student progress that are not collected in federal data collection (with the exception of fall-to-fall retention for first-time, full-time students). These early indicators are important for colleges to better understand how well student are progress at an earlier point in time than eight years after starting at the institution.
The VFA collects outcomes for student who began six-year earlier (six-year outcomes) in a more nuanced manner than OM. Colleges report outcomes in the following hierarchical, mutually exclusive categories:
- Earned a baccalaureate (from that institution)
- Earned an associate degree with subsequent transfer
- Earned an associate degree without transfer
- Earned a certificate with subsequent transfer
- Earned a certificate without transfer
- Transferred without earning a credential
- Had not earned a credential but was still enrolled in the sixth academic year
- Left without earning a credential but earned 30 or more college-level credits
- Left without earning a credential but and earned fewer than 30 college-level credits
Critical differences
There are some key differences between the VFA metrics compared and those in the OM that are important for community colleges. First, the VFA outcomes are reported separately by level of award.
Second, transfers are reported for student who earned a credential, as well as for those who did not. This is a critical difference, as there is no way to calculate a true transfer rate using ED data—only transfer for those who did not earn a credential. For colleges with transfer as a critical part of their mission, it is important to know not only those who transferred without a credential, but also those who have earned a credential, and subsequently enrolled in another institution.
A third difference, is a breakdown of student who left without a credential into two groups — those with at least one year’s worth of academic experience, and those with less than one year.
Another key difference between the VFA and OM is who is included in the cohorts. A critical delineation in the OM data collection is that only “degree/certificate seeking” students are included in the cohort, and this “degree seeking” is determined in the initial fall term of enrollment. The VFA collects a variety of student cohorts in order to better understand all student who attend community colleges, and more appropriately identify degree/certificate seeking students.
Activate your AACC benefit and register to participate in the 2018 VFA.
All VFA cohorts are tracked for six years. The VFA main cohort tracks all students who first enter an institution in a given fall and take at least one course for credit (including developmental education credits).
To put this in perspective, based on ED data, approximately 18 percent of students enrolled in fall 2015 were classified non-degree/certificate seeking students. As a result, one cohort in the VFA tracks all students new to the institution for six years.
Not all students attending a community college seek to earn a degree or credential. However, defining that group of students based on first-term designation is likely to lead to many false positives and false negatives in defining a credential-seeking cohort.
The VFA has chosen to use student course-taking behavior (completion of 12 credits within two years) to identify a sub-population of credential-seeking students. Student behavior is a more reliable measure of student intent, and this cohort provides colleges with a separate view of student attending their institution.
Finally, unlike the OM data, the VFA disaggregates the cohorts into several important populations. College receive the VFA progress and outcome measures disaggregated by race/ethnicity, age, gender, college-readiness status, first-term enrollment intensity, and whether or not the student ever received a Pell Grant. Community colleges are committed to equitable outcomes for all students, and the ability to analyze disaggregated data such as this is critical for helping colleges achieve improved student success across all student populations attending their campuses.
