How should I study LLM Inference & Serving for a Red Hat interview?

Start with a question at your level, use the linked concepts to understand the mechanism, then practise explaining your decision and its limits. Review the company guide's sources and confidence notes, and confirm the current role and interview format with your recruiter.

Does Red Hat hire AI infrastructure engineers?

Look for software engineering roles associated with OpenShift AI, Red Hat AI Inference and upstream serving projects. The current careers page determines which openings and locations are available; product documentation explains the systems to study, not whether a particular team is hiring.

What does the Red Hat AI infrastructure interview test?

We do not have a verified AI-infrastructure-specific sequence of rounds. Based on the Red Hat AI Inference 3.5 engineering documentation, useful preparation includes request routing, inference pools, prefix-cache reuse, prefill/decode transfer and Kubernetes workload lifecycle. Treat these as preparation recommendations and ask your recruiter about the actual assessment.

Red Hat LLM Inference & Serving interview questions

Red Hat AI Inference identifies vLLM and llm-d as parts of its serving stack. Use this context to study when splitting inference phases helps and what new transfers it creates. Questions tagged to Red Hat appear first, followed by additional practice from our LLM Inference & Serving track. Company tags indicate preparation relevance, not confirmed interview appearances. Confirm the current interview format with your recruiter.

Compare combined and separated inference phases

Account for the KV handoff. For separate prefill and decode workers, include cache transfer, routing and destination capacity. A faster prefill worker does not help if handoff dominates the request.
Measure the workload that needs it. Replay the same prompt and output-length distribution through both designs. Prefer separation only when the measured latency or throughput gain justifies its transfer and operational costs; include a failed-worker test.

Sources: Red Hat AI Inference: vLLM and llm-d. Reviewed 2026-10-21.

Start with the first company-tagged practice question · Continue through the full topic track. Check each resource’s access level.

LLM Inference & Serving questions tagged to Red Hat

2 questions · 1 unlocked for you

Open full answer →

Additional LLM Inference & Serving practice

These questions cover the same topic and are not tagged to Red Hat. Use them to practise applying the ideas in a different situation.

15 questions · 9 unlocked for you

Open full answer →

Concepts for LLM Inference & Serving preparation

These concept explanations cover the ideas used in the questions. Check each card for its access level.

Foundational

Prefill vs DecodeFollow one prompt through its first prediction and later decode steps, then separate GPU work from the wait a user sees. Learn why batch size and context change the likely bottleneck.

Foundational

The KV CacheThe KV cache saves attention keys and values from processed tokens. Follow one decode step, calculate its memory cost, and see why longer conversations reduce the number of requests a GPU can hold.

Core

Continuous BatchingContinuous batching changes which requests run between model iterations. Learn how a waiting request joins, why an empty slot is not enough, and how to balance throughput against each user’s token latency.

Advanced

🔒 Premium

PagedAttentionFollow a token position through a block table, watch a cache grow, and trace a shared tail through copy-on-write. Separate memory saved from throughput gained.

Advanced

🔒 Premium

Chunked PrefillFollow a long prompt through a shared token budget, including its final partial chunk. Learn why chunking can reduce stream pauses without guaranteeing a fixed token gap or a universal throughput gain.

Advanced

🔒 Premium

Speculative DecodingA draft proposes tokens; the target checks them with a sampling rule that preserves its distribution. Work through the correction after rejection, count accepted drafts separately from emitted tokens, and decide whether the measured round cost earns a speedup.

Advanced

🔒 Premium

Disaggregated Prefill and DecodeSeparate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.

Advanced

🔒 Premium

Prefix Caching and KV ReusePrefix caching reuses compatible attention state for leading tokens. Calculate shared memory and token reuse, then weigh cache identity, block ownership and routing delay before expecting a latency gain.

Explore the full curriculum →

RED HAT LLM INFERENCE & SERVING FAQ

This collection includes practice questions such as "You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?", "vLLM, SGLang or TensorRT-LLM: which engine do you pick for a new deployment, and what would change your mind?", "Why do prefill and decode behave so differently, and why does that matter for the hardware you serve on?". Company tags describe relevance to preparation; they do not confirm that Red Hat asked these questions. Check each question for answer access.

Continue your Red Hat interview preparation

Continue with related questions and the concept curriculum. Free questions are available in every track; check pricing for full access.

See full-access options Full Red Hat interview guide

Independent and not affiliated with Red Hat. All trademarks belong to their owners.

Red Hat LLM Inference & Serving interview questions

Compare combined and separated inference phases

LLM Inference & Serving questions tagged to Red Hat

You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?

vLLM, SGLang or TensorRT-LLM: which engine do you pick for a new deployment, and what would change your mind?

Additional LLM Inference & Serving practice

Why do prefill and decode behave so differently, and why does that matter for the hardware you serve on?

What is the KV cache, and why does it keep growing while a request is being served?

You have eight H100s and Llama 3.1 70B. How many concurrent users can you serve, and what changes the number?

Define TTFT, TPOT and goodput, and tell me how you would measure each one in production.

Compare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.

How does PagedAttention work, and what problem was it solving?

What is the difference between static and continuous batching, and why did it change LLM serving?

A long prompt arrives while sixty users are mid-generation. What happens, and how does chunked prefill fix it?

When does speculative decoding speed up serving, and when does it break even or hurt?

A new replica has to load a 70B model and serve traffic in under a minute. Where do the seconds go, and how do you get there?

Design the fastest serving path for a 70B model with a 200 ms TTFT target at production scale.

When does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?

Your p99 TTFT tripled last night and p50 did not move. Walk me through how you find the cause.

EAGLE, Medusa or a separate draft model: which speculative decoding method do you pick, and why?

nvidia-smi shows 30 percent utilization on our serving fleet. Is that a problem, and what would you look at instead?

Concepts for LLM Inference & Serving preparation

Other topics for Red Hat preparation

Continue your Red Hat interview preparation

Red Hat LLM Inference & Serving interview questions

Compare combined and separated inference phases

LLM Inference & Serving questions tagged to Red Hat

You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?

vLLM, SGLang or TensorRT-LLM: which engine do you pick for a new deployment, and what would change your mind?

Additional LLM Inference & Serving practice

Why do prefill and decode behave so differently, and why does that matter for the hardware you serve on?

What is the KV cache, and why does it keep growing while a request is being served?

You have eight H100s and Llama 3.1 70B. How many concurrent users can you serve, and what changes the number?

Define TTFT, TPOT and goodput, and tell me how you would measure each one in production.

Compare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.

How does PagedAttention work, and what problem was it solving?

What is the difference between static and continuous batching, and why did it change LLM serving?

A long prompt arrives while sixty users are mid-generation. What happens, and how does chunked prefill fix it?

When does speculative decoding speed up serving, and when does it break even or hurt?

A new replica has to load a 70B model and serve traffic in under a minute. Where do the seconds go, and how do you get there?

Design the fastest serving path for a 70B model with a 200 ms TTFT target at production scale.

When does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?

Your p99 TTFT tripled last night and p50 did not move. Walk me through how you find the cause.

EAGLE, Medusa or a separate draft model: which speculative decoding method do you pick, and why?

nvidia-smi shows 30 percent utilization on our serving fleet. Is that a problem, and what would you look at instead?

Concepts for LLM Inference & Serving preparation

Which LLM Inference & Serving questions can I use to prepare for Red Hat?−

How should I study LLM Inference & Serving for a Red Hat interview?+

Does Red Hat hire AI infrastructure engineers?+

What does the Red Hat AI infrastructure interview test?+

Other topics for Red Hat preparation

Continue your Red Hat interview preparation