Search EdWorkingPapers

Search EdWorkingPapers by author, title, or keywords.

Bringing Transparency to Predictive Analytics: A Systematic Comparison of Predictive Modeling Methods in Higher Education

Colleges have increasingly turned to predictive analytics to target at-risk students for additional support. Most of the predictive analytic applications in higher education are proprietary, with private companies offering little transparency about their underlying models. We address this lack of transparency by systematically comparing two important dimensions: (1) different approaches to sample and variable construction and how these affect model accuracy; and (2) how the selection of predictive modeling approaches, ranging from methods many institutional researchers would be familiar with to more complex machine learning methods, impacts model performance and the stability of predicted scores. The relative ranking of students’ predicted probability of completing college varies substantially across modeling approaches. While we observe substantial gains in performance from models trained on a sample structured to represent the typical enrollment spells of students and with a robust set of predictors, we observe similar performance between the simplest and most complex models.

Keywords
Predictive analytics, college success, algorithmic bias, inequality, machine learning, data science
Education level
Document Object Identifier (DOI)
10.26300/hd2e-7e02

EdWorkingPaper suggested citation:

Bird, Kelli A., Benjamin L. Castleman, Zachary Mabel, and Yifeng Song. (). Bringing Transparency to Predictive Analytics: A Systematic Comparison of Predictive Modeling Methods in Higher Education. (EdWorkingPaper: 21-438). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/hd2e-7e02

Machine-readable bibliographic record: RIS, BibTeX