Sentinel is a fraud-scoring research project for anonymized transaction data. I built it to test the parts that are easy to hand-wave in ML demos: imbalanced evaluation, threshold choices, per-score explanations, and an analyst review loop that actually saves its state.
No real PII or real cardholder data belongs in this repository. Sentinel is designed for the public anonymized Kaggle ULB credit card fraud dataset or synthetic data generated by the project itself.
Research / demo project. Not certified for production fraud prevention.
The primary dataset is the Kaggle "Credit Card Fraud Detection" dataset from the Machine Learning Group at Universite Libre de Bruxelles. It contains anonymized PCA components (V1 through V28), Time, Amount, and binary Class. Download it from Kaggle and keep it outside Git-tracked paths.
Docker:
docker compose -f infra/docker-compose.yml up --buildBare metal:
make install
make data
make train
make serveChecks:
cd ml && python -m pytest
cd backend && python -m pytest
cd frontend && npm run lint && npm run typecheck && npm test -- --run && npm run buildFrontend only:
cd frontend
npm install
npm run devBackend only:
cd backend
python -m venv .venv
. .venv/Scripts/activate
pip install -e .[dev]
uvicorn app.main:app --reload- Synthetic data is the default path so CI and local demos do not require a Kaggle account.
- Accuracy is intentionally not a primary metric. The dashboard and reports lead with PR-AUC, recall at fixed precision, cost-weighted loss, and threshold confusion matrices.
- Logistic regression, random forest, and isolation forest are implemented as sklearn model adapters. XGBoost, LightGBM, PyTorch autoencoder, stacked ensemble, and true SHAP remain tracked production gaps in
docs/REQUIREMENTS_AUDIT.md. - Scored rows and analyst reviews use SQLite locally and Postgres in production.
ml/ data generation, feature transforms, model adapters, evaluation, reports
backend/ FastAPI app, scoring, transactions, drift, jobs, auth, database models
frontend/ React dashboard with overview, scoring, triage, models, drift, settings
infra/ Docker Compose, service Dockerfiles, deployment files
notebooks/ exploratory notebooks for baselines, boosters, and thresholds
reports/ generated reports, ignored by Git
Primary model metrics are PR-AUC, ROC-AUC, recall at precision 0.5/0.7/0.9, precision at recall 0.5/0.7/0.9, F2, Brier score, and cost-weighted loss. Confusion matrices are always tied to a named threshold.
- Add a class under
ml/sentinel_ml/models/that implementsfit,predict_proba, andmetadata. - Register it in
ml/sentinel_ml/models/registry.py. - Add a smoke threshold in
ml/tests/test_model_smoke.py. - Run
make trainand inspectreports/{run_id}/report.md.
Implement ExperimentTracker in ml/sentinel_ml/tracking/base.py, add the provider to ml/sentinel_ml/tracking/registry.py, and set SENTINEL_TRACKER. The default is MLflow when configured, otherwise a local no-op tracker.
- Fly.io: use
infra/deploy/fly.toml, set Postgres, Redis, MinIO, and admin-key secrets. - Railway: follow
docs/RAILWAY_DEPLOYMENT.md. Production uses the web app, API, worker, Postgres, Redis, and a private artifact bucket. - VPS: run Docker Compose behind Caddy or nginx with TLS, nightly backups, and rotated API keys.
- GitHub Pages:
.github/workflows/pages.ymlbuilds and deploys the frontend. It is a static demo unlessVITE_API_BASE_URLpoints to a deployed API.
- Online learning is planned but not enabled by default.
- Real-time streaming via Kafka is not included.
- Graph models over merchant networks are future work.
- Adversarial robustness tests are planned.
- Multi-tenant SaaS mode is behind a future feature flag.
- The API falls back to a small local scorer only when no bundle exists in development. Production refuses to start without the trained sklearn bundle included in its image.
- Production retraining runs through Redis and a worker, writes the new versioned bundle to the artifact bucket, and is picked up by the API within one minute.
- XGBoost, LightGBM, a PyTorch autoencoder, stacked ensembles, true SHAP, and MLflow/MinIO artifact storage are still research work, not shipped features.