TorchServe Mock (ailab-ml)¶
What It Is¶
A lightweight mock of PyTorch TorchServe. It reproduces the three-port architecture (inference, management, metrics) with two registered models.
Surface¶
Inference API (port 8080):
| Endpoint | Purpose |
|---|---|
GET /ping |
Health check |
POST /predictions/{model} |
Model inference (acme-sentiment, acme-toxicity) |
Management API (port 8081):
| Endpoint | Purpose |
|---|---|
GET /models |
List registered models |
GET /models/{name} |
Model details (workers, batch config) |
POST /models?url= |
Register a model |
PUT /models/{name} |
Scale workers |
DELETE /models/{name} |
Unregister model |
Metrics API (port 8082):
| Endpoint | Purpose |
|---|---|
GET /metrics |
Prometheus metrics (inference requests, queue latency, worker count) |
Port & Unit¶
| Parameter | Value |
|---|---|
| Host | 172.16.50.20 |
| Ports | 8080 (inference), 8081 (management), 8082 (metrics) |
| Unit | torchserve-mock.service |
| Runtime | python3 server.py |
What aipostex Finds¶
- TorchServe fingerprint via management API
/models - Model enumeration with worker and batch details
- Unauthenticated model management (register, scale, unregister)
- Prometheus metrics exposing operational telemetry