# Scripts — ML Pipeline & ETL

## Purpose

All data processing, feature engineering, model training, and inference scripts.

## Stack

- pandas, numpy, scipy (data processing)
- torch, torch_geometric (GCN model)
- MLflow (experiment tracking)
- geopandas, rasterio (spatial data)

## Key Scripts

| Script | Purpose |
|--------|---------|
| `etl_weather.py` | Weather data ETL (wide→long, interpolation) |
| `etl_medical.py` | Medical case ETL (address standardization, geocoding) |
| `generate_grid.py` | 100m grid generation |
| `generate_grid_features.py` | Grid-level feature engineering |
| `resample_spatial_features.py` | DEM/raster resampling to grid |
| `aggregate_cases_to_grid.py` | Aggregate cases to grid cells |
| `train_model.py` | Full training pipeline (PyTorch + MLflow) |
| `inference_grid.py` | Batch grid-level inference |
| `inference_daily.py` | Daily inference runner |
| `alert_engine.py` | Risk alert generation |
| `evaluate.py` | Model evaluation & metrics |
| `deploy_schema.sql` | PostGIS database schema |

## Patterns

- Scripts are standalone: `if __name__ == '__main__': main()`
- Paths use `Path('processed/...')` relative to project root
- Run from project root: `python scripts/train_model.py`
- MLflow tracks experiments in `mlruns/` and `mlflow.db`

## Data Flow

```
Datas/ → etl_* → processed/ → train_model.py → models/
                          ↘ inference_*.py → PostGIS → API
```

## Anti-Patterns

- Don't hardcode absolute paths — use `Path` relative to project root
- Don't skip MLflow logging for new experiments
- Don't modify `processed/` files manually — re-run ETL scripts
- Don't import from `backend/` — scripts are independent