How should I study LLM Inference & Serving for a Perplexity interview?

Start with a question at your level, use the linked concepts to understand the mechanism, then practise explaining your decision and its limits. Review the company guide's sources and confidence notes, and confirm the current role and interview format with your recruiter.

Does Perplexity hire AI infrastructure engineers?

Yes, for the inference stack, the GPU fleet and the retrieval and data systems behind the product; check the careers page for current titles.

What does the Perplexity AI infrastructure interview test?

No reliable public breakdown of the loop was found, so we do not list rounds. Prepare serving with a retrieval step inside the latency budget, capacity planning for consumer traffic, and cost per query reasoning.

Perplexity LLM Inference & Serving interview questions

Perplexity’s product documentation separates retrieval from model-based responses. Use this boundary when measuring an answer service rather than attributing every delay to GPU decoding. Questions tagged to Perplexity appear first, followed by additional practice from our LLM Inference & Serving track. Company tags indicate preparation relevance, not confirmed interview appearances. Confirm the current interview format with your recruiter.

Measure a retrieval-assisted response

Break time to first token into stages. Track search, context preparation, model queueing and prefill. Group requests by retrieved context length so a changed search result mix is not mistaken for a decoder regression.
Preserve correctness while reducing delay. Compare a smaller context, cached retrieval and more serving capacity with the same evaluation set. Reject a faster configuration if it drops evidence needed for correct answers or returns stale material outside the product’s freshness requirement.

Sources: Perplexity API product boundaries. Reviewed 2026-10-21.

Start with the first company-tagged practice question · Continue through the full topic track. Check each resource’s access level.

LLM Inference & Serving questions tagged to Perplexity

3 questions · 0 unlocked for you

Open full answer →

Additional LLM Inference & Serving practice

These questions cover the same topic and are not tagged to Perplexity. Use them to practise applying the ideas in a different situation.

15 questions · 10 unlocked for you

Open full answer →

Concepts for LLM Inference & Serving preparation

These concept explanations cover the ideas used in the questions. Check each card for its access level.

Foundational

Prefill vs DecodeFollow one prompt through its first prediction and later decode steps, then separate GPU work from the wait a user sees. Learn why batch size and context change the likely bottleneck.

Foundational

The KV CacheThe KV cache saves attention keys and values from processed tokens. Follow one decode step, calculate its memory cost, and see why longer conversations reduce the number of requests a GPU can hold.

Core

Continuous BatchingContinuous batching changes which requests run between model iterations. Learn how a waiting request joins, why an empty slot is not enough, and how to balance throughput against each user’s token latency.

Advanced

🔒 Premium

PagedAttentionFollow a token position through a block table, watch a cache grow, and trace a shared tail through copy-on-write. Separate memory saved from throughput gained.

Advanced

🔒 Premium

Chunked PrefillFollow a long prompt through a shared token budget, including its final partial chunk. Learn why chunking can reduce stream pauses without guaranteeing a fixed token gap or a universal throughput gain.

Advanced

🔒 Premium

Speculative DecodingA draft proposes tokens; the target checks them with a sampling rule that preserves its distribution. Work through the correction after rejection, count accepted drafts separately from emitted tokens, and decide whether the measured round cost earns a speedup.

Advanced

🔒 Premium

Disaggregated Prefill and DecodeSeparate prefill and decode to control phase interference, then budget the KV handoff. A short isolated transfer can still overload a shared path; cancellation stops generation before it necessarily releases transfer buffers.

Advanced

🔒 Premium

Prefix Caching and KV ReusePrefix caching reuses compatible attention state for leading tokens. Calculate shared memory and token reuse, then weigh cache identity, block ownership and routing delay before expecting a latency gain.

Explore the full curriculum →

PERPLEXITY LLM INFERENCE & SERVING FAQ

This collection includes practice questions such as "When does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?", "How do you route requests across replicas to maximize prefix-cache hits without unbalancing the fleet?", "When does offloading the KV cache to CPU memory or NVMe beat recomputing it?". Company tags describe relevance to preparation; they do not confirm that Perplexity asked these questions. Check each question for answer access.

Continue your Perplexity interview preparation

Continue with related questions and the concept curriculum. Free questions are available in every track; check pricing for full access.

See full-access options Full Perplexity interview guide

Independent and not affiliated with Perplexity. All trademarks belong to their owners.

Perplexity LLM Inference & Serving interview questions

Measure a retrieval-assisted response

LLM Inference & Serving questions tagged to Perplexity

When does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?

How do you route requests across replicas to maximize prefix-cache hits without unbalancing the fleet?

When does offloading the KV cache to CPU memory or NVMe beat recomputing it?

Additional LLM Inference & Serving practice

Why do prefill and decode behave so differently, and why does that matter for the hardware you serve on?

What is the KV cache, and why does it keep growing while a request is being served?

You have eight H100s and Llama 3.1 70B. How many concurrent users can you serve, and what changes the number?

Define TTFT, TPOT and goodput, and tell me how you would measure each one in production.

Compare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.

How does PagedAttention work, and what problem was it solving?

You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?

What is the difference between static and continuous batching, and why did it change LLM serving?

A long prompt arrives while sixty users are mid-generation. What happens, and how does chunked prefill fix it?

When does speculative decoding speed up serving, and when does it break even or hurt?

vLLM, SGLang or TensorRT-LLM: which engine do you pick for a new deployment, and what would change your mind?

A new replica has to load a 70B model and serve traffic in under a minute. Where do the seconds go, and how do you get there?

Design the fastest serving path for a 70B model with a 200 ms TTFT target at production scale.

Your p99 TTFT tripled last night and p50 did not move. Walk me through how you find the cause.

EAGLE, Medusa or a separate draft model: which speculative decoding method do you pick, and why?

Concepts for LLM Inference & Serving preparation

Other topics for Perplexity preparation

Continue your Perplexity interview preparation

Perplexity LLM Inference & Serving interview questions

Measure a retrieval-assisted response

LLM Inference & Serving questions tagged to Perplexity

When does splitting prefill and decode onto separate GPU pools pay for itself, and what does the KV transfer cost?

How do you route requests across replicas to maximize prefix-cache hits without unbalancing the fleet?

When does offloading the KV cache to CPU memory or NVMe beat recomputing it?

Additional LLM Inference & Serving practice

Why do prefill and decode behave so differently, and why does that matter for the hardware you serve on?

What is the KV cache, and why does it keep growing while a request is being served?

You have eight H100s and Llama 3.1 70B. How many concurrent users can you serve, and what changes the number?

Define TTFT, TPOT and goodput, and tell me how you would measure each one in production.

Compare the KV cache footprint of multi-head, grouped-query and multi-head latent attention with numbers.

How does PagedAttention work, and what problem was it solving?

You need to quantize a model for serving. Which method do you pick, and what do you measure before shipping it?

What is the difference between static and continuous batching, and why did it change LLM serving?

A long prompt arrives while sixty users are mid-generation. What happens, and how does chunked prefill fix it?

When does speculative decoding speed up serving, and when does it break even or hurt?

vLLM, SGLang or TensorRT-LLM: which engine do you pick for a new deployment, and what would change your mind?

A new replica has to load a 70B model and serve traffic in under a minute. Where do the seconds go, and how do you get there?

Design the fastest serving path for a 70B model with a 200 ms TTFT target at production scale.

Your p99 TTFT tripled last night and p50 did not move. Walk me through how you find the cause.

EAGLE, Medusa or a separate draft model: which speculative decoding method do you pick, and why?

Concepts for LLM Inference & Serving preparation

Which LLM Inference & Serving questions can I use to prepare for Perplexity?−

How should I study LLM Inference & Serving for a Perplexity interview?+

Does Perplexity hire AI infrastructure engineers?+

What does the Perplexity AI infrastructure interview test?+

Other topics for Perplexity preparation

Continue your Perplexity interview preparation