HF TGI Gateway (ailab-app)¶
What It Is¶
A protocol-real Hugging Face TGI-facing gateway for the guided credential chain. Discovery
endpoints remain reachable, while /generate requires the HuggingFace token found in MLflow
run params/tags.
This models a common production shape: TGI itself stays simple, and auth is enforced by a fronting inference gateway.
Surface¶
| Endpoint | Purpose |
|---|---|
GET /health |
Basic health check |
GET /info |
TGI model metadata |
GET /v1/models |
OpenAI-compatible model listing |
GET /metrics |
Prometheus-style telemetry |
POST /generate |
Auth-gated text generation |
Port & Unit¶
| Parameter | Value |
|---|---|
| Host | 172.16.50.40 |
| Port | 8180 |
| Unit | tgi-gateway.service |
| Runtime | python3 server.py |
| Required header | Authorization: Bearer <HF token from MLflow> |
Guided Chain Role¶
- Ray leaks the MLflow Basic credential.
- MLflow exposes the HF/TGI token in run params/tags.
- The TGI gateway rejects missing or wrong tokens.
- The TGI gateway accepts the MLflow-looted HF token.