College success requires students to engage with their institution academically and administratively. Missteps with required administrative processes can threaten student persistence and success. Through two experimental studies, we assessed the effectiveness of an artificially intelligent text-based chatbot that provided proactive outreach and support to college students to navigate administrative processes and use campus resources. In both the two-year and four-year college context, outreach was most effective when focused on discrete administrative processes such as filing financial aid forms or registering for courses which were acute, time-sensitive, and for which outreach could be targeted to those for whom it was relevant. In the context of replicating studies to better inform policy and programmatic decision making, we draw three core lessons regarding the effective use of nudge-type efforts to promote college success.
We examine through a field experiment whether outreach and support provided through an AI-enabled chatbot can reduce summer melt and improve first-year college enrollment at a four-year university and at a community college. At the four-year college, the chatbot increased overall success with navigating financial aid processes, such that student take up of educational loans increased by four percentage points. This financial aid effect was concentrated among would-be first-generation college goers, for whom loan acceptances increased by eight percentage points. In addition, the outreach increased first-generation students’ success with course registration and fall semester enrollment each by three percentage points. For the community college, where the randomized experiment could not be robustly implemented due to limited cell phone number information, we present a qualitative analysis of organizational readiness for chatbot implementation. Together, our findings suggest that proactive outreach to students is likely to be most successful when targeted to those who may be struggling (for example, in keeping up with required administrative tasks). Yet, such targeting requires university systems to have ready access to and ability to make use of their administrative data.