Loan Default ML Pipeline

Approving a loan is not a binary classification problem. A lender has to answer a chain of questions, and only the first is a model output: how likely is this borrower to default, how far can that number be trusted as a probability, what is driving it in terms that can go in a letter to the applicant, how much is lost if it happens, is the loan still profitable, and what happens to the whole book if house prices fall thirty percent.

Most loan-default projects answer the first question and stop. I wanted to build the whole decision.

The first working version scored a ROC-AUC of 1.0000. Logistic regression, random forest and XGBoost all did, with zeros off the diagonal of the confusion matrix. A linear model cannot separate a real credit population perfectly, so the cause was never the model. It was the data.

A column called interest rate spread was blank on about a quarter of rows, and its blankness matched the default label on every one of the 148,670 rows. The column is only populated for loans that were actually originated and priced, so being empty is a consequence of the outcome, not a predictor of it. A missing-value indicator built from it had handed the model the answer key: two such indicators carried 97.5% of total feature importance, and 82 of 87 features, including credit score, loan-to-value and income, carried none.

Finding that out is the most useful thing in the repository. The leaking notebook is preserved unmodified as evidence, and a regression test now fails the build if any excluded column returns to the feature set or if held-out AUC ever climbs above a plausible ceiling again.

Build the lending decision, not the classifier. A leakage-controlled, calibrated probability of default feeding a risk grade, an expected-loss calculation, SHAP-derived reason codes and a decision rule, exposed through an API with the tests and documentation to keep it honest.

One implementation of feature engineering serves both training and inference, so the model in production sees exactly what it was trained on. Input validation uses enums generated from the data. Gender and age are excluded as a fair-lending safeguard, and the API rejects requests that contain them outright rather than merely ignoring the fields.

Three candidates were compared under stratified five-fold cross-validation on the training split only, and selected on PR-AUC rather than accuracy. At a sixteen percent base rate, a model that predicts no default for everyone is eighty-four percent accurate and worthless. XGBoost won, then was calibrated with isotonic regression so that an eight percent prediction means eight in a hundred such loans actually default.

Downstream, the risk engine computes expected loss as PD × LGD × EAD, maps probabilities to grades A through G, and applies a hurdle rate where risk-adjusted return meets the cost of equity to produce an approve, review or decline. Portfolio and stress-test endpoints run the same logic across a book of loans.

Several choices were made deliberately and are defended in the repository. No SMOTE and no positive-class weighting, because both improve separation metrics while destroying calibration, and calibration is the whole point. Complete-case filtering instead of imputation for the leaking columns, because an imputation mask would have smuggled the same signal back in.

Isotonic calibration was chosen over Platt scaling even though it is marginally worse on Brier score and AUC, because it halves the worst bucket-level error, from 3.09 to 1.36 percentage points, in the high-PD region where the decision engine actually operates. Removing the leak also meant dropping 24,123 rows and moving the observed default rate from 24.6% to 16.3%. The two ROC curves are plotted on different populations on purpose, because hiding that would have been its own small dishonesty.

An honest model with an honest number. On 24,910 loans never seen during training or calibration, the system reaches a ROC-AUC of 0.8244 with a mean predicted default rate of 16.54% against 16.32% observed. Risk grades are monotonic from A to G. The API is live, documented, and pre-filled with a real application so anyone can try it without cloning anything.

A perfect score is a bug report. Calibration matters more than separation when the number feeds a decision. And the most valuable artefact in a machine-learning repository is often the document explaining what you removed and why.

The drift-monitoring endpoint exists; the retraining policy behind it does not yet. Segment-level models are the other open thread, so that the book is not scored by one global model.