Model Drift in the Admissions Process

AE Rodriguez
March 17, 2022


  • Professor & Chair
  • Department of Economics & Business Analytics
  • Pompea College of Business
  • The University of New Haven

My Task


More Specifically: the Admissions Intake Process

AdmissionsIntake

  • To discuss dysfunction in the intake processes. Intake processes (an element of supply chains) have come under scrutiny lately and will probably remain an issue in the future near and not so near.

  • To examine machine learning processes: their uses, applications and limitations in business operations. Here I focus on OneClass algorithms and Propensity Score Matching.

  • Possible alternatives: Semi-Supervised Learning, Misclassification

The Big(-ish) Picture


Counterfactual


  • partial view
  • need to understand what happened
  • incomplete data

A Perfect Storm


  • The Demographic Cliff
  • Covid
  • The George Floyd Social Upheaval

storm

The Demographic Cliff

Cliff

Between 2025 and 2029, the number of college-going students will decline by 15%—that’s over 400,000 fewer students in a span of four years, an average loss of 100,000 students per year.

Afia Tasneem, EAB.com (Feb 2019)

Covid Impact

covid

After a 4-year slide due to Trump administration immigration policies, enrollment of new international students fell dramatically in fall 2020 because of COVID-19, according to the Institute of International Education.

Andrea Widener, C&EN (Nov 2020)

Dysfunction in GPA/SAT/GRE?

alt text

Source: Forbes

- Since their inception a century ago, standardized tests have been instruments of racism and a biased system.
- Students of color, particularly those from low-income families, have suffered the most from high-stakes testing in U.S. public schools.
- Today, a movement is growing across the country to resist testing abuse and overuse, and to promote authentic assessment.

National Education Association, By: John Rosales and Tim Walker
Published: 03/20/2021

RIP SAT?

CNN

Source: CNN

Link: RIP SAT?

RIP SAT? Harvard makes standardized tests optional through 2026

Harvard announced it won’t require SAT or ACT scores through 2026. If other schools follow suit, admissions offices will be ‘looking at applications they would not have seen previously,’ says Jeff Selingo, author of ‘Who Gets in and Why’

Source: CNN, SMerconish, Dec 2021

Simulations

Counterfactual


  • examine what has happened?
  • what to do in the future of it were to happen again?
  • what to do if there are changes in the components?
  • examine possible and plausible alternatives
  • help discriminate across competing scenarios

The Logistic Model


Here’s the general logistic regression model:

Prob{Y=1|X}=11+exp(−Xβ)

The X represents our predictors. The β represents weights or coefficients for our predictors.

Our three predictors, age, experience, and gpa

then Xβ=β1*age+β2*experience + β3*gpa

Structure

alt text

One Class Solvers: SVM & Random Forests

alt text

alt text

One Class of Students

alt text

Scatter Age v GPA Assigned Classes

alt text

GPA by Assignment Status

alt text

Propensity Matching


alt text



In order to reduce the selection bias, Propensity Score Adjustment can be used in this case for reweighting.

Fit a New Admissions Algorithm


alt text

The Resolution



alt text


  • Best-Practices
  • Analytic Reasoning

For Further Work


  • Semi-Supervised Learning Approach
  • Misclassification Approach



alt text

arodriguez@newhaven.edu