Workflow-CI - BankChurners

Repository ini digunakan untuk Kriteria 3 submission Dicoding MSML: menjalankan re-training model secara otomatis dengan GitHub Actions dan MLflow Project.

Struktur Project

Workflow-CI
|-- .github
|   `-- workflows
|       `-- train.yml
|-- MLProject
|   |-- BankChurners_preprocessing.csv
|   |-- DockerHub.txt
|   |-- MLproject
|   |-- conda.yaml
|   |-- modelling.py
|   `-- requirements.txt
|-- api
|   |-- inference.py
|   `-- serving_input_example.json
|-- monitoring
|   |-- alert_rules.yml
|   |-- prometheus.yml
|   |-- prometheus_exporter.py
|   `-- grafana
|       |-- dashboards
|       `-- provisioning
|-- Dockerfile
|-- docker-compose.yml
|-- requirements.txt
`-- README.md

Dataset

Dataset yang digunakan adalah hasil preprocessing dari repository eksperimen:

MLProject/BankChurners_preprocessing.csv

Target klasifikasi:

Attrition_Flag

Menjalankan Training Lokal

Install dependency:

python -m pip install --upgrade pip
python -m pip install -r MLProject/requirements.txt

Jalankan MLflow Project:

mlflow run MLProject --env-manager=local

Setelah training selesai, file helper untuk CI akan dibuat di:

MLProject/outputs/run_id.txt
MLProject/outputs/model_uri.txt

GitHub Actions CI

Workflow berada di:

.github/workflows/train.yml

Trigger workflow:

push ke branch main
pull_request ke branch main
workflow_dispatch manual

Workflow akan:

install dependency
menjalankan mlflow run MLProject --env-manager=local
mengupload artifact MLflow
membangun Docker image dengan mlflow models build-docker --env-manager=local
push image ke Docker Hub

Docker Hub

Image Docker Hub:

iniamin12/workflow-ci

Link Docker Hub:

https://hub.docker.com/r/iniamin12/workflow-ci

GitHub Secrets

Tambahkan secrets berikut di repository GitHub:

DOCKERHUB_USERNAME
DOCKERHUB_TOKEN

DOCKERHUB_TOKEN disarankan berupa Docker Hub access token, bukan password utama.

Kriteria 4: Serving, Monitoring, dan Logging

Komponen Kriteria 4 yang disiapkan:

FastAPI inference service
Prometheus metrics endpoint
Standalone Prometheus exporter
Prometheus scrape config
Prometheus alert rules
Grafana datasource provisioning
Grafana dashboard provisioning
Docker Compose untuk menjalankan semua service

Endpoint FastAPI:

GET  /health
POST /predict
GET  /metrics

Jalankan training terlebih dahulu agar MLProject/outputs/model_uri.txt dan mlruns/ tersedia:

mlflow run MLProject --env-manager=local

Jalankan API lokal tanpa Docker:

python -m pip install -r requirements.txt
uvicorn api.inference:app --host 127.0.0.1 --port 8000

Contoh request prediksi PowerShell:

$body = Get-Content api/serving_input_example.json -Raw
Invoke-RestMethod -Uri http://127.0.0.1:8000/predict -Method Post -Body $body -ContentType application/json

Jalankan serving dan monitoring dengan Docker Compose:

docker compose up -d --build

URL service:

FastAPI:    http://127.0.0.1:8000
Metrics:    http://127.0.0.1:8000/metrics
Prometheus: http://127.0.0.1:9090
Grafana:    http://127.0.0.1:3001

Login Grafana lokal:

username: admin
password: admin

SMTP email alert memakai variabel .env berikut. Jangan commit .env karena berisi app password:

GF_SMTP_HOST
GF_SMTP_USER
GF_SMTP_PASSWORD
GF_SMTP_FROM_ADDRESS
GF_SMTP_FROM_NAME
GF_SMTP_SKIP_VERIFY

Contoh format aman tersedia di .env.example.

Contact point Grafana otomatis tersedia dengan nama:

Email Alert Receiver

Dashboard Grafana otomatis tersedia dengan nama:

dashboard-Aminadab Yunus

Dashboard memakai UID aman berikut agar valid di Grafana:

bankchurners_monitoring_ini_amin

Metric utama untuk bukti Prometheus:

http_requests_total
system_cpu_usage
system_ram_usage

Metric tambahan yang tetap tersedia:

api_request_total
api_latency_seconds
model_prediction_total
model_loaded
model_drift_score

Alert rules Prometheus berada di:

monitoring/alert_rules.yml

Alert yang disiapkan:

HighLatency
HighErrorRate
HighCPUUsage

Bukti submission Kriteria 4 yang perlu diambil setelah service berjalan:

1. Screenshot FastAPI /health atau /docs
2. Screenshot Prometheus query untuk http_requests_total, system_cpu_usage, dan system_ram_usage
3. Screenshot Grafana dashboard dashboard-Aminadab Yunus
4. Screenshot Prometheus alert rules