Pitcher Injury-Risk Model

The Problem

The headline finding of this project is a number that looks like a failure: the best honest model I built predicts pitcher injury at a walk-forward cross-validation AUC of 0.574, barely above the 0.5 you’d get from a coin flip. That number is the most important thing here, and I lead with it on purpose. Injury prediction from public pitch-level data has a low ceiling, and most of the work was figuring out where that ceiling sits and what stays useful once you accept it.

Three properties of the data make a strong discrimination score suspicious rather than impressive:

• The outcome is rare and binary. The 30-day injury positive rate across pitcher-games is roughly 6.6%. A model that “just predicts no injury” is right ~93% of the time and useless, so accuracy is meaningless and AUC/PR-AUC are the only honest lenses.
• The structure is temporal, not IID. Injury risk accumulates over a season; feature lookback windows (trailing 7/28/90-day workload) reach backward in time. Any evaluation that shuffles rows leaks the future into the past. Walk-forward CV (train on prior seasons, test forward) is mandatory, not a nicety, and it produces a lower number than stratified splits precisely because it stops the leak.
• The real causes are invisible. The mechanisms that actually blow out an elbow (biomechanics, arm-path deterioration, proprioceptive deficits, off-field load) don’t appear in Statcast. Statcast measures pitch outcomes, not muscle recruitment. A model that scored, say, AUC 0.85 on this data would be evidence of a leak, not a breakthrough.

So the framing throughout is: a high AUC would be a red flag. The interesting question is what you can extract when raw ranking is weak.

Why It Matters

Once you accept that per-game discrimination has a ceiling, the value moves from classification accuracy to decision support. Three outputs survive the weak-ranking regime and are directly usable by a baseball-ops group:

• Calibrated probabilities for load management. Isotonic calibration is applied to the raw predicted probabilities (raw ECE 0.046 on the 2024 holdout). A probability anchored to an observed rate can be thresholded (“hold anyone above X% projected 30-day risk”) even when the model can’t cleanly rank two pitchers against each other.
• A season-stable risk score for planning. The Injury Risk+ (IR+) composite correlates r = 0.645 year-over-year across 9,752 pitcher-seasons. Per-game predictions are noisy; a score that holds from one season to the next is something a front office can plan a roster around months ahead.
• A validated dose-response. Pitchers whose velocity declines more than 2 mph year-over-year show roughly double the observed injury rate. That’s not a ranking model. It’s an actionable, directly interpretable relationship.

This is a canonical hard sports-analytics problem: real data, severe class imbalance, strong temporal dependence, invisible true causes. Building on it honestly meant confronting the limit of the signal instead of chasing a leaderboard number.

My Approach

Data

• Statcast (2015–2024): every MLB pitch: velocity, movement, release extension, pitch mix.
• Transaction and injury logs: 10-day and 60-day IL stints, cross-referenced to build a binary injury outcome per pitcher-game plus a severity label for multi-task modeling.
• Workload features: pitches in the trailing 7/28/90 days, rest-day gaps, appearances per rolling window, and acute-to-chronic workload ratio (ACWR).
• Velocity-trend features: per-pitcher rolling mean, rolling standard deviation, and year-over-year delta, computed separately so chronic decline is distinguishable from game-to-game noise.

Final dataset: 3,249 pitchers, 205,911 pitcher-game rows, 82 engineered features.

Why four model tracks

I didn’t build four models to pad the project. Each one answers a different sub-question that the others can’t:

1. Baseline classifiers: is there any signal? Logistic regression and random forest / gradient boosting on pitcher-game rows. The random forest hit held-out AUC-ROC 0.590 and PR-AUC 0.134 (against a ~6.6% base rate), which is the “yes, but barely” answer that sets up everything downstream.

2. Survival ensemble (respecting the censoring). Binary classification treats a pitcher who finishes the season healthy as a negative example, which is wrong: he’s a censored observation, not a confirmed non-injury. I modeled days-until-next-IL-stint with an ensemble of Gradient Boosted Survival Analysis, Cox proportional hazards, and a Random Survival Forest. Ensemble C-index 0.556: same modest-but-real signal as the classifiers, but on the correct data structure.

3. Multi-task severity: 10-day vs. 60-day. Predicts injury occurrence and, conditional on injury, whether it’s a short (10-day) or extended (60-day) stint. “Probably misses two weeks” and “heading toward a season-ender” are different operational decisions, so I separated them.

4. Injury Risk+ (IR+): convert weak per-game predictions into a stable season number. An ERA+-style composite scaled to a mean of 100 per season and archetype (starter vs. reliever), computed across 9,752 pitcher-seasons. This is the track that turns noisy game-level output into the year-over-year-stable operational signal (r = 0.645).

Two decisions I had to defend

• Walk-forward CV over stratified splits. Stratified time-series splits let the feature-engineering lookback windows see future seasons, inflating the score. I chose temporal walk-forward CV (train only on seasons prior to the test season) even though it produces a lower headline number (mean AUC 0.574, std 0.023). The honest number is lower, and that’s the point.
• Interventional SHAP over path-dependent SHAP. Path-dependent (tree-path) SHAP splits credit by the tree’s structure and can hand attribution to correlated features arbitrarily. Under interventional SHAP, conditioning on counterfactual workload, pitches_90d genuinely outranks ACWR. That flips a common sports-science heuristic: raw 90-day cumulative load carries more signal than the acute-to-chronic ratio everyone reaches for first.

Results

I lead with what’s validated, then the ceiling, because the validated findings are what a team can act on, and the ceiling is what keeps the findings honest.

What’s validated

• Velocity-decline dose-response. In the velocity analysis (~89K no-decline vs. ~13K declining pitcher-games, from NB11), pitchers whose year-over-year velocity fell more than 2 mph show an observed 30-day injury rate of about 12%, versus roughly 6% for pitchers without that decline: a near doubling. In IR+ terms the declining cohort scores 129.6 against 102.0 for non-decliners. This is the cleanest, most directly actionable finding: not a ranking, a relationship.

• IR+ is season-stable. Year-over-year correlation of r = 0.645 across 9,752 pitcher-seasons. A per-game probability is too noisy to plan around; a score that holds from one season to the next is something a front office can build a roster on.

• pitches_90d outranks ACWR under interventional SHAP. Absolute 90-day cumulative load carries more signal than the acute-to-chronic ratio the sports-science literature leans on. A concrete, checkable challenge to a common heuristic.

• Injury history dominates the binary model. The top random-forest importances are prior_il_total (0.258), days_since_last_injury (0.147), and prior_il_days_lost (0.136). Prior injury, not current workload, is the strongest single binary predictor.

The ceiling — and how I know it’s real

• Discrimination tops out near chance. Walk-forward CV mean AUC 0.574 (std 0.023); the survival ensemble lands at C-index 0.556. Both consistently beat 0.5 across forward folds, so the signal is real, but it’s not strong enough to replace expert judgment, and I don’t claim it is.

• I tested the ceiling instead of assuming it. The improvement log is a record of experiments that were tried and reverted because they didn’t help:

  • Weibull AFT: attempted as a parametric survival model; reverted after it failed to improve the ensemble.
  • ExtraSurvivalTrees: scored C = 0.5381 on its own and dragged the ensemble down; dropped.
  • IPCW least-squares loss: incompatible with 95% censoring in the survival data; abandoned.

  These reversions are the evidence that 0.574 is a discovered ceiling, not a lack of effort.

What the model can’t see

• Biomechanical breakdown: arm-path mechanics and muscle recruitment don’t appear in Statcast; in-game form deterioration is invisible.
• Off-field factors: training load, sleep, prior surgery detail, proprioceptive testing, all unavailable.
• Individual resilience: two identical Statcast profiles can carry very different injury susceptibility. The model estimates a population-level conditional probability, not an individual’s fate.

Shipping the useful part

The findings live in a Streamlit dashboard with four panels: a season IR+ leaderboard, an individual pitcher profile, multi-pitcher comparison, and archetype analysis. It surfaces the outputs that survive the weak-ranking regime (the season score and the dose-response) rather than a per-game prediction the data can’t support.

Key Takeaways

What I’d want a reader to leave holding is the reasoning, not a metric:

1. The weak AUC is the finding. 0.574 walk-forward isn’t a failure to hide. It’s the honest ceiling of injury prediction from public pitch-level data, and stating it up front is what makes the rest of the project trustworthy. A strong AUC here would have meant a leak.

2. When ranking is weak, calibration and stability carry the value. Isotonic calibration applied to raw probabilities (raw ECE 0.046 on 2024 holdout) gives thresholdable probabilities; IR+‘s r = 0.645 gives a season-level planning number. Both are usable even though per-game discrimination isn’t.

3. The method choices were forced by the data. Walk-forward CV because lookback windows leak the future; survival modeling because 95%-censored healthy seasons aren’t negatives; a composite because per-game output is too noisy to act on. Each decision has a specific reason and a rejected alternative.

4. The ceiling was tested, not assumed. A Weibull AFT, ExtraSurvivalTrees, and an IPCW loss were all tried and reverted. That’s how I know 0.574 is where the signal actually stops.

5. What a baseball-ops team can use today: the calibrated risk threshold for load management, the IR+ season signal for roster planning, and the >2 mph velocity dose-response as a validated red flag.

---

Next steps: Prospective validation against 2025 IL transactions; export the velocity dose-response curves for the dashboard; integrate IR+ into a team-level roster-health view.