Pearls AQI Predictor - Multi-City Forecast Lab

Pearls AQI Predictor is my end-to-end Data Science internship project for forecasting the next 3 days of AQI across four Pakistani cities Islamabad, Rawalpindi, Lahore, and Faisalabad.

I built this as a working ML product, not just a notebook. The system collects live weather and pollutant data every hour, engineers features, stores them in MongoDB Atlas, trains multiple models automatically every day, registers champion models per forecast horizon, and serves live predictions through a multi-page Streamlit dashboard.

Live Links

Item	Link
Live dashboard	https://pearls-aqi-multicity.streamlit.app
Forecast page	https://pearls-aqi-multicity.streamlit.app (Dashboard tab)
EDA / Trends page	https://pearls-aqi-multicity.streamlit.app (Trends tab)
Source repository	https://github.com/shehroz53531/Pearls-AQI-Predictor
Final report	Pearls_AQI_Report.pdf

Project Screenshots

What The System Does

• Predicts AQI for Day +1, Day +2, and Day +3 across 4 cities.
• Uses live API-based weather and pollutant data from Open-Meteo.
• Stores processed features in MongoDB Atlas as a cloud feature store.
• Trains multiple models: Ridge Regression, Random Forest, Gradient Boosting, and a TensorFlow/Keras neural network.
• Evaluates every model using RMSE, MAE, and R\u00B2.
• Selects a champion model per forecast horizon dynamically instead of hardcoding a winner.
• Stores model registry metadata and model binaries in MongoDB Atlas via GridFS.
• Runs automated feature and training pipelines using GitHub Actions.
• Serves live predictions, current pollutant/weather conditions, and hazard alerts through a Streamlit dashboard deployed on Streamlit Community Cloud.
• Includes EDA (30-day trend, pollutant correlation, diurnal pattern), SHAP-based feature attribution, and an honest write-up of pipeline incidents.

Architecture

Open-Meteo APIs
   |
   | hourly GitHub Actions feature pipeline
   v
MongoDB Atlas Feature Store
   |
   | daily GitHub Actions training pipeline
   v
Model Metrics + Model Registry (MongoDB Atlas + GridFS)
   |
   | loaded on demand
   v
Streamlit Dashboard on Streamlit Community Cloud
   |
   | public access
   v
End User (any device, anywhere)

Latest Forecast Snapshot

Fill this in from the live dashboard's Champion Models panel whenever you refresh the README \u2014 values change as the pipeline retrains daily.

Horizon	City	Predicted AQI	Risk	Champion model
Day +1	Islamabad	TBD \u2014 see live dashboard	\u2014	Gradient Boosting
Day +2	Islamabad	TBD \u2014 see live dashboard	\u2014	Gradient Boosting
Day +3	Islamabad	TBD \u2014 see live dashboard	\u2014	Gradient Boosting

The dashboard supports switching between all 4 cities, and each city's forecast is generated from that city's own latest feature row against the shared champion models.

Model Training Summary

The training pipeline reads historical features from MongoDB Atlas, builds one training set per forecast-horizon bucket, trains all four candidate models, evaluates them on a time-based holdout split, and stores the full result in the cloud model registry.

Horizon (bucket)	Selected champion	RMSE	R\u00B2
Day +1 (1\u201324h)	Gradient Boosting	21.64	0.647
Day +2 (25\u201348h)	Gradient Boosting	33.16	0.204
Day +3 (49\u201372h)	Gradient Boosting	36.73	0.028

Overall leaderboard winner across all three buckets: Gradient Boosting. Short-term accuracy is strong and actionable; accuracy tapers off for longer horizons, which is expected \u2014 3-day-ahead, multi-city air quality forecasting is a genuinely hard problem, and the system still clearly beats a naive "predict the average" baseline even at Day +3.

Cloud Evidence

MongoDB Atlas

GitHub Actions Automation

Deployment

EDA and Explainability

Automation Details

Feature Pipeline

Workflow file: .github/workflows/feature_pipeline.yml

• Runs on a GitHub Actions hourly schedule.
• Fetches current weather and pollutant data for all 4 cities from Open-Meteo.
• Engineers time-based, cyclical, and lag features, then writes them to MongoDB Atlas.
• Uses (city, timestamp) as a natural key so repeated or overlapping runs overwrite cleanly instead of creating duplicates.

Training Pipeline

Workflow file: .github/workflows/training_pipeline.yml

• Runs once daily.
• Fetches historical feature data from MongoDB Atlas for all 4 cities.
• Builds three forecast-horizon buckets (Day +1, Day +2, Day +3) and trains Ridge Regression, Random Forest, Gradient Boosting, and a neural network for each.
• Evaluates every candidate with RMSE, MAE, and R\u00B2.
• Saves each bucket's best-performing (lowest RMSE) model, its scaler, and its metrics to the MongoDB Atlas model registry via GridFS, replacing the previous champion.

Repository Structure

feature_pipeline/          Hourly feature collection + engineering script
training_pipeline/         Daily model training, evaluation, and champion selection
backfill/                  One-time historical backfill script (1 year x 4 cities)
app/                       Streamlit dashboard (streamlit_app.py) and shared utils (utils.py)
.github/workflows/         Feature pipeline and training pipeline automation
assets/readme/             Public screenshots used inside this README
documentation/             Final internship report
requirements.txt
README.md

Local Setup

git clone https://github.com/shehroz53531/Pearls-AQI-Predictor.git
cd Pearls-AQI-Predictor
python -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate
pip install -r requirements.txt

Required environment variables (create a .env file in the project root):

MONGODB_URI=your_mongodb_atlas_uri

Run the pipelines locally:

python backfill/historical_backfill.py           # one-time: backfill 1 year of history
python feature_pipeline/fetch_and_store_features.py   # fetch the latest hour of data
python training_pipeline/train_model.py          # train all models, register champions

Launch the dashboard:

streamlit run app/streamlit_app.py

The app opens at http://localhost:8501.

Final Submission

This repository was submitted as the final deliverable for the 10Pearls Shine Program Data Science internship track. It contains the working project code, the deployed live dashboard link, automation workflows, screenshots, cloud evidence, and the final internship report.

Final report:

Pearls_AQI_Report.pdf

Built By

Shahroz Khalid Data Science Intern at 10Pearls Shine Program CS Student, Riphah International University

• GitHub: https://github.com/shehroz53531
• Live project: https://pearls-aqi-multicity.streamlit.app

Mentors: Umema Ashar, Hafsa Imtiaz