Cliffton
Back to projects

Crop Yield Prediction

SoloMay 2025 – Aug 2025Try the demoView code

0.967
adjusted R² over 42,306 records, measured on a fresh retrain against a held-out 20% split
0.942 / 0.923
adjusted R² for the specialised Wheat and Rice models
0.170
adjusted R² for Sugar Cane, evidence a linear model doesn't fit every crop equally

The problem

Predict Indian crop yield from climate and agricultural-input data across 81 crops. A single model that ignores crop type will mostly be measuring the difference between sugar cane and lentils, not the effect of rainfall or fertiliser.

Approach

  • Built a from-scratch multivariate linear regression with gradient descent (no scikit-learn for training) across 87 features: 7 continuous climate/input variables plus 80 one-hot crop indicators, over 42,306 records merged from five separate FAO, World Bank and India Open Data sources.
  • Diagnosed the baseline honestly rather than declaring victory on its 0.966 adjusted R²: coefficient analysis showed the model was dominated by which crop a row was, not by climate or inputs, so a high aggregate score was hiding the actual question being asked.
  • Trained per-crop specialised models to isolate that effect. Wheat and rice fit well (adjusted R² 0.942 and 0.923); sugar cane did not (0.170), which is itself a finding: a linear model can't capture how sugar cane's yield actually responds to these features.
  • Added degree-2 polynomial features and grid-searched the learning rate and iteration count on a 75/25 split, which is the model actually retrained and shipped as the browser demo on this site.

The saved model wasn't the model the README described

The repo's committed model_artifacts.pkl turned out to be trained on a single crop, Unmanufactured tobacco, not the 81-crop baseline the README reports. A per-crop loop in the training notebook left a variable bound to its last iteration, and the next cell reused it without rebinding, so the artefact that got saved and wired into the Streamlit app was quietly wrong for every other crop. The demo on this site is a fresh, from-scratch reproduction of the honest 87-feature baseline using the same published dataset and the same custom regression code, evaluated on a real held-out split rather than the broken artefact.

Stack

  • scikit-learn
  • NumPy
  • Streamlit