Skip to content

MLflow (ailab-ml)

What It Is

MLflow is an open-source platform for managing the ML lifecycle — experiment tracking, model registry, and artifact storage. On the ML platform, it tracks experiments whose run parameters contain hardcoded connection strings, API keys, and tokens. The tracking server is unauthenticated and network-accessible, and the lab now seeds both experiments and registered model versions so aipostex can validate deeper registry and artifact workflows.


Installation

Installed as mluser:

pip install --break-system-packages mlflow==2.21.3

--break-system-packages is required on Ubuntu 24.04 (PEP 668).


Systemd Unit

Unit: mlflow.service

/etc/systemd/system/mlflow.service
[Service]
User=mluser
ExecStart=/usr/local/bin/mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri sqlite:////opt/mlflow/mlflow.db --default-artifact-root /opt/mlflow/artifacts/
Restart=always

Four slashes in the SQLite URI

sqlite:////opt/mlflow/mlflow.db — the first three slashes are the sqlite:/// scheme prefix, the fourth begins the absolute path /opt/mlflow/mlflow.db.


Port & Config

Parameter Value
Host 172.16.50.20
Port 5000
Bind address 0.0.0.0
Authentication None
Backend store sqlite:////opt/mlflow/mlflow.db
Artifact root /opt/mlflow/artifacts/

Misconfiguration

The tracking server has no authentication. Anyone on the network can browse experiments, inspect run parameters, and download artifacts.


Seed Data

Three experiments with run parameters, metrics, and artifact trees containing embedded credentials:

Experiment Highlights
churn-prediction-v2 PostgreSQL and Snowflake connection strings, Redis feature store reference, model/deployment artifacts
fraud-detection-bert PagerDuty key, Kafka host, deployment metadata, multiple candidate and promoted runs
customer-embedding-model HuggingFace token, Qdrant linkage, deployment metadata, model card material

The lab also seeds registered models with versions linked back to real runs:/<run-id>/model sources:

Registered model Stages
acme-churn-ensemble Staging, Production
acme-fraud-bert Staging, Production

Credentials are logged as MLflow run parameters and tags, and the registry points back to artifact trees under the tracked runs. This lets aipostex validate enum, registry, model-versions, and model-artifacts without adding destructive write paths.


What aipostex Finds

  • Experiment enumeration — The /api/2.0/mlflow/experiments/search endpoint lists all experiments without authentication.
  • Run parameters with credentials — Inspecting individual runs reveals connection strings, API keys, and tokens logged as parameters.
  • Registry exposure — Registered model search and version enumeration expose model names, stages, version metadata, and runs:/... sources.
  • Artifact metadata — Artifact URIs and paths expose internal storage structure, S3 bucket names, and model/deployment material through artifacts and model-artifacts.

Verification

Confirm MLflow is running:

curl http://localhost:5000/
Expected response
OK

List experiments:

curl http://localhost:5000/api/2.0/mlflow/experiments/search
Expected response

JSON listing churn-prediction-v2, fraud-detection-bert, and customer-embedding-model.

List registered models:

curl http://localhost:5000/api/2.0/mlflow/registered-models/search
Expected response

JSON listing acme-churn-ensemble and acme-fraud-bert.