EdgeML

Enterprise Edge Machine Learning Prediction Platform.

EdgeML is a self-hosted, plugin-oriented platform for batch predictions from CSV files. Its first milestone supports selecting a deployed model, uploading a CSV, previewing results, and downloading a CSV with predictions.

Implementation progress

Runtime optimization now includes a bounded per-process model cache and a dedicated, bounded dataset-profiling pool. Registry checks remain authoritative; changed/deleted artifacts are not served from old cache entries. Overload returns 503 with Retry-After: 1. See runtime limits and measured synthetic performance; artifact disk-byte budgets are not RAM limits.

Milestone	Status	Current scope
v0.1 Prediction Server	Completed	Model selection, CSV prediction, missing-row cleanup, optional Ground Truth evaluation, metrics, preview, and download
v0.2 Prediction History	Completed	Stateless prediction metadata history with injectable repository storage
v0.3 Classification and training controls	In progress	Classification training and Logistic Regression are implemented; Ridge and richer model controls remain planned
v0.4 Explainability	Deferred	SHAP-backed explainability and prediction insights
v0.5 Model Registry	Completed	Trusted model registry with publish, enable, disable, and unregister lifecycle controls
v0.6 API Authentication	In progress	SQLite-backed API token generation, listing, one-time display, scoped access, and revocation
v0.7.1 Observability	Completed	Health endpoints, request logging, and Prometheus metrics
v0.7.2 Queue Workers	Completed	Redis-backed asynchronous training worker with Docker end-to-end verification
v0.7.3 Queue Operations	In progress	Backend retry, dead-letter routing, graceful shutdown, queue-control APIs, Queue Operations UI, and worker-capacity controls; idempotent output, SIGTERM and Redis restart recovery tested; richer capacity controls remain planned

The latest Prediction update rounds regression prediction and prediction_error values to four decimal places in the returned CSV. Full-precision values remain in the evaluation calculations. Long prediction-preview headers and cell values expose tooltips so Chinese and long feature names remain readable.

Prediction clients can use either POST /api/predict for CSV upload/download or POST /api/predict/json for direct JSON data from a database or service integration. The JSON endpoint returns prediction records and evaluation metadata without creating a temporary CSV file.

JSON Prediction enforces the configured body limit before decoding (10 MiB by default), including chunked requests. Data values must be scalar strings, finite numbers, booleans, or null; nested values and numeric NaN/Infinity return 422. Existing record/column limits remain, and oversized strings or column names return 413. See API limits.

Ground Truth evaluation uses the model's canonical problem type: regression metrics for regression, and accuracy/weighted precision/recall/F1 for classification. Legacy binary/multiclass classification manifests load as classification automatically, including the bundled CreditRisk and CustomerChurn examples; CSV and JSON support numeric and text class labels.

predict_api_example.py includes get_model_id_by_name("HousePrice"), a requests-based example of calling GET /api/models/by-name/{model_name} before submitting a JSON prediction.

For integrations that need to discover models, use GET /api/models/ids to retrieve all active model IDs, or GET /api/models/by-name/{model_name} to resolve a display name such as HousePrice to its stable model ID. The ID-list endpoint returns a JSON array such as ["house-price-v1", "credit-risk-v1"], which can be used to populate a client-side model selector. The by-name segment intentionally uses kebab-case for a readable and consistent URL; keep this path unchanged in clients.

Completed v0.1 Prediction Server

• Model discovery from backend/ml_models/
• CSV batch prediction for trusted sklearn artifacts
• Optional Ground Truth scoring during prediction (regression metrics and row-level errors)
• Prediction preview keeps compact columns and provides tooltips for full headers and values
• Prediction page shows CSV upload progress and processing status
• Stateless API: prediction results are returned directly as CSV
• Vue 3 interface with upload, preview, and download
• Dataset upload and profile pages with Chinese CSV encoding support
• Regression training for Random Forest, Gradient Boosting, XGBoost, and AdaBoost
• Draft-to-published model workflow; published models appear in Prediction automatically
• Dataset and trained-model display-name editing without changing original CSV filenames
• Traditional Chinese / English UI switching with a persisted language preference
• Background training progress and optional Random Forest, Gradient Boosting, and XGBoost hyperparameter overrides
• Docker Compose deployment

Completed v0.2 Prediction History

• Successful prediction metadata is recorded through an injectable PredictionHistoryRepository.
• GET /api/prediction-history returns prediction history in reverse chronological order.
• The initial file adapter persists metadata only; uploaded CSV files and prediction outputs are not retained.
• The frontend provides Traditional Chinese / English switching with a persisted language preference.
• Prediction, history, dataset, training, and trained-model pages share a responsive visual design system.
• Tables support compact layouts, readable typography, and full-value hover tooltips for truncated content.

v0.3 in progress

• Classification training with stratified cross-validation and classification metrics is implemented for Random Forest, Gradient Boosting, XGBoost, AdaBoost, and Logistic Regression.
• Training provenance, explicit validation/test confusion matrices and problem-aware model lists are implemented; dataset content preview is paginated and preserves raw text.
• Ridge regression and richer regularized linear-model controls remain planned for the standalone AutoML platform.
• Keep regression and classification model manifests compatible with the existing trained-model and prediction workflows.

Completed v0.5 Model Registry

• Replace runtime folder scanning with a file-backed model registry while keeping ModelCatalog as the Prediction application boundary.
• Add a Model Registry page for viewing, enabling, disabling, and unregistering trusted model packages.
• Refine the registry table with compact responsive columns, clear status badges, action buttons, and hover details for truncated values.
• The registry's compact Model ID column displays the API identifier, not the package directory. Long IDs are visually shortened to leave space for model names and actions; click the copy icon (tooltip: Copy model ID) to copy the complete ID into the CSV or JSON Prediction API's model_id field. Hover shows the full ID, and copying never includes the displayed ellipsis. IDs are selectable for manual copying if browser clipboard policy blocks automatic copying. Only active models can be used for prediction.
• Published model artifacts remain operator-controlled files; registry removal never accepts or deletes serialized artifacts through HTTP.

v0.6 in progress

• Add a SQLite-backed API Token management page for creating, listing, and revoking tokens.
• Store token hashes only; the complete token is returned once when it is created.
• Support api and tokens:manage scopes, with EDGEML_API_TOKEN retained as the bootstrap management token.
• Keep health probes anonymous and protect business /api/* routes by default, including when all managed tokens expire or are revoked. Anonymous development requires explicit EDGEML_ANONYMOUS_API=true.
• A reverse proxy remains recommended for production authentication because browser-held tokens are not secrets.

Completed v0.7.1 Observability

• Structured JSON request and training-job logs with request IDs.
• Liveness/readiness endpoints at /health/live and /health/ready.
• Prometheus metrics at /metrics for HTTP requests, predictions, and training jobs.

Completed v0.7.2 Queue Workers

• Replace local background jobs with a Redis-backed training queue and independent worker.
• Keep the existing training-job polling API stable while jobs run asynchronously.
• Verify the complete Docker Compose training flow end to end, including persisted job records and trained artifacts.

v0.7.3 in progress

• Backend reliability controls for bounded retry, dead-letter routing, and graceful worker shutdown are implemented.
• Queue Operations APIs now expose queue depths, dead-letter metadata, manual requeue, and cancellation of queued jobs.
• The frontend now includes a Queue Operations page for queue depth, queued/processing IDs, retry attempts, dead-letter inspection, requeue, and cancellation.
• Queue Operations refreshes automatically and supports running multiple Docker worker replicas against the shared Redis queue.
• Windows launchers cover the hybrid Redis/local workflow, full Docker startup, and rebuild/deploy flows.
• Remaining v0.7.3 work is richer capacity controls and operational monitoring.

Optimization backlog

The ordered project-wide review and acceptance checklist is maintained in ROADMAP.md. The first reliability batch implements ID-based safe publication, persistent Docker model storage with legacy migration, live-worker-safe recovery on a shared volume, and cross-process Registry transactions. See deployment upgrade instructions before rebuilding an existing installation.

The second batch (R05–R08) adds explicit anonymous-development configuration, exact integer/range validation for CSV and JSON prediction, recoverable dead-letter replay with state persisted before dispatch, and consistent missing-row handling for training and both external-test entry points. Job JSON files now use atomic replacement; dataset/history/job state and outbox transactions are now implemented in R11 (see CHANGELOG).

The current implementation is suitable for local and controlled self-hosted use. The following items are recorded before expanding toward multi-user production deployment:

• P0 reliability: shared-volume ownership locks protect live jobs during recovery; dead-letter replay now has real-Redis concurrency, crash, and error-injection coverage. Idempotent output, Worker SIGTERM and Redis restart recovery have isolated integration coverage; capacity/monitoring improvements remain in ROADMAP.
• P0 data durability: Redis AOF, published packages, dataset transactions, job-state/outbox transactions, and history recovery are implemented. Full backup/restore verification remains in ROADMAP.
• P0 access control: refine token scopes and add an external identity/reverse-proxy integration before exposing model registry and queue operations outside a trusted network.
• P1 operations: add service healthchecks, resource limits, log rotation, Redis authentication, backup/restore procedures, and immutable Docker image tags.
• P1 frontend quality: split the monolithic App.vue, move all labels to structured i18n keys, add frontend unit/E2E tests, and code-split the large production bundle.
• P2 product roadmap: complete Ridge and richer AutoML controls where they belong, then continue the planned standalone AutoML extraction and v0.8 observability dashboard.

Follow the prioritized outstanding work in ROADMAP.md and completed changes in CHANGELOG.md. Dataset IDs use canonical UUIDs; read, rename, delete, training, and evaluation reject paths and linked files outside their storage boundary. Dataset CSV uploads reject empty/header-only files, duplicate headers, malformed rows, and infinite numeric values with 422, while retaining Chinese encoding support and missing-value profiling. See the API contract. Redis AOF is already enabled; broader concurrency protection and stronger external authentication remain planned. These changes preserve the ModelCatalog and PredictionService application boundaries.

Runtime selection

Dataset read/rename/delete and transaction recovery share a cross-process lock. Prediction History is paginated (100 rows per UI page) and archives damaged originals before repair. Retention is opt-in with EDGEML_PREDICTION_HISTORY_MAX_RECORDS; 0 keeps all active records. Prediction audit-write failure returns 503. See storage details and completed work.

Choose one of these runtime modes for normal development:

Runtime	Launcher	Services	Host endpoints
Prediction-only	start-dev.bat	Local Backend + Frontend	Frontend 5173, API 8000, Redis not required
Hybrid local training	start-dev-redis.bat	Local Backend + Frontend + Worker, isolated Docker Redis	Frontend 5173, API 8000, Redis 6381
Full Docker	start-dev-docker.bat / deploy-docker.bat	Docker Backend + Frontend + Redis + Worker	Frontend 5180, API 8010, Redis 6380

The Docker host mappings are different from the container ports. Containers use Backend 8000, Frontend Nginx 80, and Redis 6379 internally. Redis uses a named volume with AOF enabled so queue and dead-letter state survives container recreation. The hybrid launcher uses a separate Redis project and queue, so it can run beside the full Docker runtime.

For authentication, copy .env.example to .env. Set EDGEML_WEB_USERNAME (default admin) and a separate strong EDGEML_WEB_PASSWORD for browser login; retain EDGEML_API_TOKEN for Python/curl integrations and bootstrap token management. Docker Compose and the local launchers load these settings. The browser uses an HttpOnly session cookie with CSRF protection, including CSV uploads; API tokens are no longer embedded in frontend builds. Existing token-enabled installs must configure the web password before using the new login screen. Restart local services or redeploy Docker after upgrading. See Web login and API authentication for setup, HTTPS, session expiry, and migration.

R05 upgrade: API protection is now the default even when no credentials exist. For an intentionally anonymous local development environment, explicitly set the following in the root .env, then restart start-dev.bat or start-dev-redis.bat:

EDGEML_ANONYMOUS_API=true
EDGEML_API_TOKEN=
EDGEML_WEB_PASSWORD=

Use EDGEML_ANONYMOUS_API=false (default) with web/API credentials for normal deployments. A configured bootstrap token or web password takes priority over the anonymous flag. Managed-token creation, expiry, and revocation never switch the configured anonymous policy; explicit invalid credentials are still rejected. Docker Compose also passes this setting to Backend and requires recreation after an environment change.

First-time setup

New Windows computer: local or hybrid development

Install Git, Python 3.12, Node.js LTS, and Docker Desktop first. Then run the following commands from PowerShell:

git clone https://github.com/Zimkjdv/EdgeML.git
cd EdgeML

cd backend
py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
# Optional: rebuild deterministic example model packages.
.\.venv\Scripts\python.exe scripts/build_example_models.py

cd ..\frontend
npm.cmd install

cd ..

Start Docker Desktop before using start-dev-redis.bat. The launcher starts isolated Redis on 6381, sets EDGEML_REDIS_URL=redis://localhost:6381/0, and uses the local .venv for Backend and Worker. This setup is required once per computer; later starts only need the launcher.

New Windows computer: full Docker

The full Docker runtime does not require a local Python virtual environment or frontend node_modules; Docker builds those dependencies inside the images. The Backend and Worker share the edgeml-ml-base image, which contains the common Python ML runtime. edgeml-ml-base is a build image, not a running container. After Docker Desktop is running, perform the first build and start with:

.\deploy-docker.bat

This builds the shared edgeml-ml-base first, then builds and starts Backend, Frontend, Redis, and Worker. deploy-docker.bat is reusable, not a one-time initializer: whenever local development reaches a stable milestone, run it again to rebuild the base and application images and deploy the current workspace. start-dev-docker.bat starts existing images only; run deploy-docker.bat after cloning or whenever the base image is missing.

Shared ML base image

The Docker runtime separates the common Python/ML environment from the application services:

edgeml-ml-base
        ├── edge-backend
        └── edge-worker

edgeml-ml-base contains the pinned Python dependencies used by Backend and Worker, including Pandas, Scikit-learn, SciPy, and XGBoost. It is a build image, not a running container. The running containers remain independent:

edge-backend    FastAPI prediction and training APIs
edge-worker     Redis-backed asynchronous training worker
edge-frontend   Vue/Nginx frontend
edge-redis      Redis queue and state store

This layout keeps Backend and Worker on the same dependency versions, speeds up later builds when ML dependencies have not changed, and allows Docker to share the common read-only layers. Docker Desktop may show the base, Backend, and Worker images with similar virtual sizes; use docker system df to inspect actual shared disk usage.

When requirements.txt changes, rebuild the base and application images with:

.\deploy-docker.bat

The shared base improves dependency consistency and build caching; it does not reduce the RAM used by the Backend and Worker processes, and it does not create an additional running service.

Endpoints and port mapping

Runtime	Frontend	API docs	Redis
Prediction-only (start-dev.bat)	http://localhost:5173	http://localhost:8000/docs	Not required
Hybrid training (start-dev-redis.bat)	http://localhost:5173	http://localhost:8000/docs	localhost:6381
Full Docker (start-dev-docker.bat)	http://localhost:5180	http://localhost:8010/docs	localhost:6380

Quick start

Prediction-only local

.\start-dev.bat

Hybrid local training

.\start-dev-redis.bat

Full Docker: first build or rebuild

.\deploy-docker.bat

Use the same command after any later local milestone to redeploy the latest code. It does not create a Git commit or push to GitHub; commit and push separately when the change is ready.

Full Docker: subsequent starts

.\start-dev-docker.bat

To increase Docker training capacity, pass a worker replica count to the deploy or start launcher, for example:

.\deploy-docker.bat 3

The equivalent Compose command is:

docker compose up -d --build --scale worker=3

Each worker consumes the same Docker Redis-backed queue. The Queue Operations page shows combined queue state; use docker compose ps worker to inspect replicas. All Docker services use restart: unless-stopped, so they restart after a Docker daemon or host reboot and remain stopped only after an explicit docker compose stop/down. To stop the full Docker runtime without deleting volumes, run docker compose down.

Three deterministic example model packages are included:

Model	Task	Sample CSV
HousePrice	Regression	backend/sample_data/house_price_input.csv
CreditRisk	Binary classification	backend/sample_data/credit_risk_input.csv
CustomerChurn	Binary classification	backend/sample_data/customer_churn_input.csv

JSON Prediction API example

POST /api/predict/json accepts records from a database or another service and returns JSON. The request uses data as the input array:

{
  "model_id": "house-price-v1",
  "data": [
    {"Area": 80, "Room": 2, "Age": 15},
    {"Area": 120, "Room": 3, "Age": 8}
  ]
}

source_name is optional and is only used to label JSON prediction history; when omitted, the API records the source as json-api.

For a runnable Python client, install requests in the environment where the script runs and execute:

python -m pip install requests
python predict_api_example.py

The response includes model_name, prediction_column, a records array containing predictions, optional evaluation metrics, and dropped_rows. The legacy records request field is still accepted for compatibility, but new integrations should send data.

CSV prediction

Prediction CSV parsing supports UTF-8/BOM, quoted commas/newlines and escaped quotes. Upload counts skip blank physical lines but count empty CSV records; dropped-row estimates check only required model features and selected Ground Truth, using the same explicit NA markers as the backend. Changing model/Ground Truth recalculates counts and clears stale results. Duplicate/empty headers or inconsistent row widths are rejected. Extra-column missing values do not remove rows.

Docker Nginx now allows an 11 MiB request body (default backend limits: 5 MiB CSV file, 10 MiB JSON body) and 300-second send/read inactivity timeouts. Configure EDGEML_PROXY_MAX_BODY_SIZE, EDGEML_PROXY_SEND_TIMEOUT, and EDGEML_PROXY_READ_TIMEOUT together with backend limits if needed. Rebuild the frontend image to apply the template change; see Deployment.

CSV prediction parsing and inference run in Starlette's bounded worker thread pool, keeping the API event loop available for health checks and other requests during prediction. This does not imply unlimited parallel model inference.

MySQL prediction client

For database-backed examples, see MySQL → EdgeML → MySQL: whole-table client and time-range client. Both process all matching source rows by default, in batches of 200; there is no 1,000-row cap. --limit is optional for small trial runs. Read piapi_fd.pidata1_merged, map Chinese features using pidata1_mapping, exclude missing/invalid/out-of-training-range rows, and write full source fields plus nullable actual values and predictions to a new result table, piapifd_edge.pidata1_predict. The source stays read-only. Preview is read-only by default; --mode write runs predictions and saves results, skipping completed (timestamp, model_id) keys without overwriting measured values. Tokens/passwords come from environment variables, the private client .env.local, or masked prompts, not code.

CSV and JSON Prediction now also accept optional training_range_policy="drop": the server checks numeric inputs against the trusted model package's saved training min/max. The default none preserves extrapolation for existing callers. JSON out_of_range_rows (CSV X-Prediction-Out-Of-Range-Rows) is a subset of total dropped_rows. See Prediction API for snapshot requirements and error behavior. This adds no database dependency to the server.

The client defaults to the existing Docker URL http://127.0.0.1:8010/api; setting this URL does not build or redeploy any service. Run .\deploy-docker.bat to apply changed backend code to Docker, or run the updated local backend and pass --api-url http://127.0.0.1:8000/api. The example refuses to write results when an older API lacks the new training-range response metadata.

Both MySQL entrypoints share explicit Chinese-name aliases in examples/piapifd_feature_aliases.json (e.g. model 前段水份 → DB mapping 前段水分, plus observed spacing differences). Exact names take priority; aliases never rename model inputs or modify the source DB. Override with --feature-aliases if needed.

Non-secret settings are in examples/piapifd_prediction.ini: API URL/model ID, MySQL connection/timeouts, source/mapping/result tables, batch size and optional row limit. Both entrypoints load the INI next to the script, independent of the working directory. Priority is CLI → existing environment variables → INI → built-in defaults. Use --config path/to/client.ini for another profile; examples/*.local.ini is Git-ignored. INI alias paths are relative to the INI's directory; CLI alias paths are relative to the working directory. Tokens/passwords are not allowed in the INI; the client does not read the Server .env. Preview remains the Python default; write/table creation and time bounds require CLI options. See client configuration.

For credentials, copy examples/.env.local.example to examples/.env.local only if that private file does not already exist, and fill in EDGEML_CLIENT_API_TOKEN and DB_PASSWORD once. Both Python entrypoints and the BAT read this explicit script-relative file automatically; they never search for or load the Server .env. Credential priority is nonempty process environment → client .env.local → masked prompt. --env-file path/to/client.env selects another file (which must exist; exclude it from Git yourself). Blank or absent credentials prompt only for the missing values. Quoted passwords preserve spaces/#; ${...} is literal, not expanded. The file is Git-ignored but not encrypted: restrict access, never share it or commit a filled template. See private client credentials.

For the local API on port 8000, run .\run-piapifd-prediction.bat from PowerShell. It uses backend/.venv, processes the whole source table in batches, and creates the missing destination table before writing eligible predictions. Start the API and MySQL first; the launcher does not start services, install dependencies, or drop tables. API tokens and DB passwords use the credential sources above. Additional client arguments are forwarded, e.g. .\run-piapifd-prediction.bat --help (help only, no API/DB calls or credential-file reads). Existing (timestamp, model_id) results and measured actual values are not overwritten.

The BAT explicitly passes --api-url http://127.0.0.1:8000/api --mode write --init-target; these override INI/environment defaults. Model, DB and batch settings come from the INI. Override the URL by passing --api-url to the BAT, or use Python directly to use the INI URL unchanged. A nonempty INI limit also limits the BAT run; keep it empty to process the whole table.

Training workflow

Cross-validation trains once per fold and collects scores, predictions and probabilities together. Five folds plus the final model need six fits. Progress shows completed folds; preprocessing is still fitted separately inside each fold.

Model evaluation, rename, publication, and deletion are serialized per model across API/worker processes. Model records and manifests use atomic replacement to prevent partial reads; local lock files must not be removed while services run. See Storage.

Validation supports random folds, time-ordered expanding windows, and batch-separated folds. Choose a time/batch column when appropriate; timestamps/batches must not be missing. Time validation keeps equal timestamps together and optionally leaves a gap measured in distinct timestamps. Warm-up rows are excluded from validation scores; coverage is shown with the model. Existing models display a visible legacy-evaluation warning and require retraining for current metrics. See Regression evaluation.

1. Upload a CSV in 數據集管理, inspect columns and the paginated content preview (50 rows/page), and optionally rename its display name.
2. In 模型訓練, choose a numeric regression target or classification labels and check the feature columns to use.
3. Select an algorithm. Random Forest exposes tree-count and split/leaf controls; Gradient Boosting exposes n_estimators and learning_rate; XGBoost exposes its optional hyperparameter fields. Logistic Regression exposes penalty, solver, and C for classification. Leaving fields empty uses the estimator defaults.
4. Review the background-job progress, then inspect the Draft model metrics.
5. Publish the model to make it available in Prediction.

Select a model in Trained Models to display its detail/evaluation card, including separate validation and test metrics. Optional feature-importance or classification cards do not hide these details; visibility follows model selection rather than card order.

New training saves Training provenance: exact source CSV SHA256, selected fields, resolved estimator/preprocessing parameters (including fitted XGBoost booster configuration), Python/library versions and split counts/index hashes. Expand its card to inspect it. Published metadata retains the audit; legacy models show unavailable provenance rather than guessed historical values. Identical metadata does not guarantee bit-identical results across hardware.

The model list filters Regression/Classification and uses R²/RMSE or Accuracy/F1 accordingly; mixed lists label each value. New classification runs use pooled OOF accuracy and weighted precision/recall/F1, with fold means saved separately. Validation/test confusion matrices show explicit label order, rows=actual and columns=predicted; binary TN/FP/FN/TP names the positive class, not a presumed semantic label. Later evaluation updates only test scores/matrix and records the test CSV checksum. The configurable classification label cap defaults to 100 to bound matrix size. See Training audit and classification contract.

Inside Model details and evaluation → Additional test evaluation, upload a test CSV or choose an existing dataset and click Evaluate test data. This works for Draft and Published models without retraining. The CSV must contain all selected training features and the model's target column as Ground Truth. Use data excluded from training to measure performance on new data. The upload is evaluated in memory and is not added to dataset management. A successful evaluation replaces the previous test metrics, updates list scores and saves source/row-count metadata; validation metrics and model artifacts remain unchanged. Failed evaluation preserves previous scores. See Regression evaluation.

Training, its optional external test, and later Trained Models → Evaluate share the same missing-row rules: remove missing targets; with numeric_imputer=drop, also remove rows missing any selected numeric or categorical feature. Other imputation modes retain feature gaps for the fitted pipeline. Unselected columns never cause row removal. Cross-validation row/class counts are checked after cleaning; an empty evaluation set returns 422. Existing saved metrics are unchanged until the model is retrained or evaluated again.

Prediction has a separate row policy: it drops missing required input features and selected Ground Truth. Numeric values are validated first; integer inputs such as 1.9, nonfinite values and values outside the manifest dtype range return 422 instead of being truncated or wrapped. Integral values such as 2.0 are accepted. CSV and JSON preserve integer precision during parsing, including columns with missing rows.

Manual local development (advanced)

The launchers above are recommended for normal Windows development. Use these manual commands when you need to run each process yourself or customize the environment.

For the hybrid local workflow, start its isolated Redis project and set the URL in the current PowerShell session:

docker compose -p edgeml-hybrid-redis -f docker-compose.hybrid-redis.yml up -d redis
$env:EDGEML_REDIS_URL = "redis://localhost:6381/0"

Then run the FastAPI server in one terminal:

cd backend
python -m venv .venv
# Windows: If multiple Python versions are installed, explicitly select Python 3.12:
# py -3.12 -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
python scripts/build_example_models.py
python -m uvicorn app.main:app --reload --env-file ..\.env

Run the training worker in a second backend terminal when using the manual workflow:

cd backend
.venv\Scripts\activate
python -m app.workers.training_worker

Run the frontend in a third terminal:

cd frontend
npm.cmd install
npm run dev

If only the FastAPI server is started without Redis and the worker, Prediction remains available. Training submissions are durably saved and return 202 queued with a recovery message; they wait until Redis and a Worker are started. Poll the returned job ID instead of submitting the same work again.

When updating an existing local environment after pulling the training module, run pip install -r requirements.txt again. It installs XGBoost in addition to the original prediction dependencies.

Future roadmap

Trained models now include original-feature permutation importance, shown in a full-width card aligned with the model details below, with a compact chart (top 10 by default, with Show all and full CSV download). New training saves the report automatically; older models can compute it from their saved artifact and source dataset. JSON/CSV retrieval and explicit recomputation are available at /api/trained-models/{model_id}/feature-importance.

API integrations can query GET /api/models/{model_id}/feature-importance (or ?format=csv) with the same active Model ID as Prediction. This returns the complete published ranking without inference or source-data loading. Missing/invalid reports return 409; missing/disabled models return 404. The UI provides confirmed Recompute importance and Update Prediction snapshot actions, showing source, timestamps and synchronization/disabled state. The separate snapshot endpoint POST /api/trained-models/{model_id}/feature-importance/publish verifies artifact identity and updates only the report: model files, enabled/disabled status and detailed scores remain unchanged. Normal token/session/CSRF rules apply. See Feature importance ranking.

The workspace now uses a grouped, collapsible left sidebar instead of horizontal tabs, with a compact page-title toolbar and a stationary language switch. Desktop collapse preference is remembered locally; screens up to 900 px use a navigation drawer. See Workspace navigation.

The optimization editor includes compact feature rows, search and All/Adjustable/Fixed filters, highlighted adjustable parameters, restore-default controls, and a sticky simulation action bar. Fixed inputs can be imported from a UTF-8 CSV with feature-name headers: select and preview one data row, then apply it to all fixed features. Adjustable search bounds remain unchanged. The result table shows only selected adjustable features alongside prediction metrics; the API retains full input combinations. Model selection and target settings are grouped separately; detailed guidance and random seed settings can be expanded as needed.

Optimization now prefills numeric ranges and fixed median/mode values from the model's training dataset. New models save a statistics snapshot in metadata; old models resolve record.json dataset provenance. Defaults remain editable and missing source data is explicitly flagged. Language switching remains separate from page navigation.

Parameter Optimization is available as a new bilingual page: select a trained regression model (including drafts) or an active registry model, set target Y, fix unselected X features, and search for 1–5 input combinations within numeric bounds/steps or allowed categories. Results show predicted Y, target error, and tolerance status. See Parameter optimization simulation for setup, API examples, limitations, and follow-up plans.

• v0.7.3 in progress: bounded retries, exponential backoff, dead-letter routing, graceful worker shutdown, Queue Operations APIs, Queue Operations UI, and worker replica guidance are implemented.
• v0.7.3 follow-up: richer worker-capacity controls and operational monitoring.
• v0.8: add a frontend observability dashboard for API health, registry availability, training activity, prediction outcomes, and operational errors.

Documentation

Trained-model details now use explicit bilingual translation keys rather than rewriting DOM text. Switching language preserves original model/feature names and canonical copied IDs. Metric formulas and API payloads are unchanged. See UI translation and data safety.

The frontend loads workspace pages on demand. Dataset/model selection ignores obsolete replies; Training pauses status polling when left and can recover its saved job ID after refresh. Reopen Training from the reminder to check an existing job instead of submitting again. Full regression details and supplemental CSV testing remain available. See frontend lifecycle and isolated UI verification.

Parameter Optimization now prioritizes in-tolerance recommendations, validates inputs immediately, supports optional operating-baseline comparisons, and handles decimal steps consistently. See optimization review and remaining work.

The optimization page displays search elapsed time, the target interval, evaluated combination counts, and green/orange recommendation status. Candidate count, step and seed have explanatory tooltips. After completion, a convergence chart and exact per-iteration errors help assess improvement. Defaults show actual feature coverage and warn when category choices were truncated to 100. Real progress percentages and cancellation remain future backend work.

Regression evaluation now uses positionally paired predictions and unified OOF validation metrics. Undefined metrics display as —; legacy validation records require retraining. See Regression metric definitions for formulas, zero-target MAPE handling, NRMSE normalization, and fold statistics.

See docs/01_Project.md for the roadmap, docs/02_API.md for API contracts, docs/04_Architecture.md for the design, docs/09_Optimization_Method.md for the reverse-search method, and docs/05_Deployment.md for runtime instructions. See DEVELOPMENT_MEMO.md for local development, Docker verification, and Git workflow.