AI Infra Interviews logo

Trace inference requests with OpenTelemetry, Jaeger and Grafana

Build a local vLLM observability lab, follow native engine spans, verify request metrics, and test sampling and collector outages against an independent journal.

PremiumIntermediate

AI Infra Interviews editorial · Updated

How do you verify an inference tracing and metrics pipeline?

Send a known request cohort, keep an independent client journal, and compare it with received traces and metric counts. Preserve the request-to-attempt-to-engine parent chain and define each timing boundary. Disabling trace sampling should leave population metrics intact. Stopping the collector can remove telemetry while inference still succeeds, so collection health and request success need separate checks. This project executes those checks on a local vLLM, OpenTelemetry, Jaeger, Prometheus and Grafana stack.

Read the primary sources
What you build
A checked trace and metrics pipeline, four request journals, and an acceptance record that distinguishes missing telemetry from failed inference.
Environment
Linux · Python 3.12 · NVIDIA GPU · Docker Compose
Plan your session
2–3 hours plus environment setup. A planning estimate, not measured runtime.
Before you begin
Run the free Python streaming client and understand its completion contract. Use a Linux GPU host with a working NVIDIA driver, Docker Compose and enough free disk and device memory for the pinned small-model setup.
Local GPU and monitoring stack executed. Sixteen real requests on an RTX 5070: twelve successful streams and four deliberate model rejections. Native vLLM traces, Prometheus counts, Grafana queries, unsampled requests, collector outage and recovery checked. This is a local telemetry acceptance exercise; DCGM, production capacity and durable outage replay are not measured.
  1. 01Follow one request

    Propagate context from the streaming client into the native engine span.

  2. 02Check backend evidence

    Compare received spans and metric sums with the local request journal.

  3. 03Interrupt collection

    Disable sampling, stop the collector and verify what recovers.

Follow the changing state through the project. Each boundary has a check in the walkthrough.

Continue with Premium

Premium includes this guide’s complete walkthrough and downloadable project files. The technical overview, prerequisites and verification notes above are public.

View Premium access

Start with Stream vLLM responses with Python and detect incomplete answers (Free to read).

Primary sources

Checked . Source review and execution checks are described separately above.