AI Infra Interviews logo

Observe a GPU service and distinguish missing signals from healthy hardware

Capture real device samples, expose local metrics and correlate hardware activity with request failures. Build a dashboard that preserves missing data.

PremiumService project

AI Infra Interviews editorial · Updated

What should a GPU service dashboard measure?

Pair device evidence with request-level completion, correctness and latency. GPU activity is not SM occupancy; free memory does not prove requests avoid OOMs. Preserve unavailable values as missing and record collector failure explicitly. This project includes real device capture and a local exporter, with Prometheus, Grafana and DCGM deployment left as environment-specific extensions.

Read the primary sources
What you build
Device evidence, a local metrics exporter, dashboard configuration and an incident worksheet.
Environment
Linux · NVIDIA GPU for device capture · CPU fixture checks
Plan your session
90–120 minutes plus service measurements. A planning estimate, not measured runtime.
Before you begin
Run Python 3.12 and nvidia-smi on your Linux GPU host. Keep request records from the streaming client; device samples alone do not measure service quality.
Device capture executed · dashboard deployment pending. Real RTX 5070 samples, local exporter and CPU failure checks executed. The capture is an idle desktop observation. Prometheus/Grafana deployment, DCGM counters and a correlated model incident remain unexecuted.
  1. 01Capture

    Timestamp device samples and retain collection failures.

  2. 02Correlate

    Compare the same time window with request completion and latency evidence.

  3. 03Decide

    Identify the missing measurement before assigning a cause.

Follow the changing state through the project. Each boundary has a check in the walkthrough.

Continue with Premium

Premium includes this guide’s complete walkthrough and downloadable project files. The technical overview, prerequisites and verification notes above are public.

View Premium access

Start with Stream vLLM responses with Python and detect incomplete answers (Free to read).

Primary sources

Checked . Source review and execution checks are described separately above.