Fireworks AI Infrastructure Engineer interview questions

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Derive a block-level occupancy limit, then compare the extra residency with spill traffic and lost reuse. Use a matched experiment rather than a target percentage.

Quantize a sixteen-value block, count the scale bytes, and use paired quality tests to decide which tensors need higher precision.

Use a profiler summary to form a memory-traffic hypothesis, then check counters, timing and useful work before choosing an optimization.

Count warp sectors for contiguous, strided and masked accesses, then use cache and timing evidence to decide whether changing the mapping helps.

Follow intermediate tensors through separate and fused stages, calculate the traffic and whole-block bounds, and check spills and numerical behavior.

Trace stable row normalization, handle independent strides and fully masked inputs, then separate fusion traffic from measured speed.

Follow an accumulator across K tiles, count reuse at each memory level, and decide whether a tile improves measured GEMM performance.

Follow the running maximum and weighted sum across attention tiles, with executable arithmetic and an honest memory-traffic comparison.

Compare the mechanisms in FlashAttention versions 1, 2 and 3: IO savings, work partitioning, asynchronous dependencies and FP8 accuracy.

Separate cache activity from DRAM traffic, check the access width and active lanes, and use a controlled layout change to test a profiler hypothesis. A few percentages cannot predict the fixed kernel’s runtime.

Use a timeline to establish launch overhead, then trace what graph replay changes. Stable buffers, page-table updates, capture-safe operations and batch buckets determine whether the serving engine can use it safely.

Use cuBLAS or cuBLASLt when its supported operation fits. CUTLASS exposes kernel building blocks for custom layouts, fusion and scheduling, but each choice needs a whole-operation benchmark and an ownership plan.

Derive when operand scales can be applied after a dot product and when they must follow its K blocks. Separate integer zero points, floating-point range, scale granularity and accumulator behavior.

Separate per-expert submission, small-grid scheduling and partially filled tiles. A grouped kernel can share work across experts, but routing, packing and output combination still determine the layer’s cost.

Define what the timer includes, give both implementations the same conditions, and keep the sample distribution. A useful benchmark explains the workload and comparison well enough for someone else to reproduce them.

Longer sequences exercise more state updates, different boundaries and sometimes different kernel paths. Use a two-tile example to check online softmax, then test masking, indexing and precision against a stable reference.

Follow a token into retained attention state, derive bytes per position for a 70B model, and distinguish present cache usage from the room needed to finish a response.

Trace an accepted prefix, a rejection and the correction token; derive emitted progress and compare it with a measured round cost.

Draw query-head sharing, count actual stored state and compare payloads with explicit model geometry and binary units.

Separate weight storage, matrix arithmetic and attention-cache precision. Choose a supported candidate using task quality, memory and service measurements.

Compare a complete phase handoff with a tuned combined pool, then test the queueing reversal and qualified completions. Nominal link speed and fleet size alone cannot decide the split.

Follow adapter-specific token rows through a shared base, calculate stored adapter bytes, and separate the resident set from the active batch.

Separate checkpoint memory, expert assignments and retained MLA state. The active parameter count alone cannot predict per-rank fit or token latency.

Calculate retained cache and prefill work for a stated 128k model shape, then choose scheduling and memory policies from measured limits.

Compare proposal methods by committed output over complete round time, with explicit sampling guarantees and target compatibility.

Build a measured first-token budget and test the smallest serving design that meets it, including cold prompts, queueing and delivery.

Images add preprocessing and encoder work before the language model can answer. Their effect on prompt length and cache memory depends on the model. Budget each stage, and distinguish image-embedding reuse from prefix-cache reuse.

Diagnose a stream that begins quickly but produces useful answer content late. Compare generation, parser and client clocks, then test a reasoning budget without hiding truncated or incorrect responses.

Derive a deployment candidate from the official tensor index, then account for latent-cache replication and the product workload. A worked example shows why enough aggregate HBM can still leave every GPU out of memory.

Find which startup step runs out of memory, count the allocations on that GPU, and change the setting that controls the shortage.

Turn public metadata into a deployment candidate with explicit uncertainty. Inspect architecture, stored formats and engine support, then work a per-rank memory example that explains why an aggregate fit can fail.

Find whether expert placement, small batches or communication makes the deployment slower. Compare equivalent workloads and inspect the worker that holds up each step.

Measure reachable prefix reuse, separate token savings from latency, and compare both engines on the same agent contract and resource budget.

Use the actual tensor index to size Kimi K3, then separate a published hardware starting point from production capacity. Account for mixed precision, hybrid recurrent state, parser behavior, and the workload behind a throughput claim.

Check the bound’s assumptions, locate the critical path, and quantify which optimization can actually improve one user’s decode latency.

Derive a prompt-plus-output contract, calculate per-rank state capacity, and measure the concurrency that meets latency and overload targets.

The answers sound fluent, but the scores dropped. Compare inputs, generation settings, raw output, parsing and scoring to find where the new deployment differs.

Define a reproducible workload and service contract, account for every offered request, and choose a measured operating region with evidence and margin.

Compare actual encoded tensors, per-rank runtime fit, supported kernels and held-out task quality. Four-bit weight-only kernels can accelerate wider arithmetic, and smaller weights need not reduce the GPU count.

A repeated-prompt test measures one reuse pattern. Production gains depend on token-weighted reuse, routing, residency, alignment and the prefill share of latency; a falling hit rate does not uniquely identify eviction.

Long prefills coincide with decode stalls. Compare a tuned combined pool with separate phases, derive transfer and pool capacity, and test whether the improvement survives connector failures.

The nodes joined the cluster, but the first request hangs. Trace how far it got, check worker assignments, and reproduce the blocked operation on the same communication path.

Compare committed output per complete draft-and-verify cycle with ordinary decoding. Derive the acceptance-length model, separate it from engine counters, and test latency, goodput and memory across the actual workload.

Confirm that requests are waiting for usable state capacity. Separate allocated memory, live blocks and evictable prefixes, then compare admission, lower-precision state, prefix sharing and hardware changes with explicit assumptions.

A fourfold ratio is a symptom, not a diagnosis. Trace individual requests, separate waiting from execution and delivery, and test workload, scheduling, cache and capacity hypotheses against the service objective.

Build a week around release gates and dependencies. Budget artifact transfer, validate the exact runtime path, measure the intended workload, and size the canary evidence before promising full rollout.

Price the same workload and service guarantees, including input tokens, idle capacity, failures and operations. Then check whether the algebraic crossover fits the fleet before treating it as a buying decision.

A context-window limit is not a latency or concurrency guarantee. Budget retained state, prefill and shared service time; distinguish cache reuse from eliminating work; choose isolation only when measured interference justifies its cost.

A minor revision can change weights, encoding or response behavior. Compare pinned deployment bundles, evaluate task slices, monitor a representative canary and keep enough capacity to restore the old service.

Apply the project’s offline tuning guidance in context, derive the memory tradeoff, and use workload-specific acceptance criteria instead of universal queue and headroom targets.

Verify the throughput definition and service outcomes, compare resolved configurations and execution traces, then isolate supported changes. Quantify capacity replacement correctly and choose rollback from the full release risk.

Budget growing attention state, bounded recurrent state and retained prefix snapshots separately. Work the short-context crossover, then test whether the cache manager resumes every layer from the same prefix boundary.

Separate pre-routing from a sequential cascade, charge every attempted generation, and measure checker mistakes. A lower escalation rate is not proof of higher quality, and the largest model is not a correctness oracle.

Define whether half the footprint means checkpoint bytes or runtime memory. Choose a supported recipe, measure which tensors it changes, separate calibration from held-out evaluation, and validate quality and serving cost.

Define cold and reused-context objectives, verify the checkpoint's real state layout, and size per-rank memory and transfer demand. Caching and phase separation help only when their compatibility, retention and capacity assumptions hold.

Follow media through fetching, preprocessing, encoding and language-model execution. Derive visual positions from the actual processor output, account for encoder placement and separate the caches and latency stages that text-only monitoring misses.

A partial drop does not rule out the parser. Define success and its denominator, trace failures through encoding, generation, parsing and execution, then replay the relevant boundary with controlled versions.

Test the complete serving bundle with contract fixtures, representative tasks and load-aware outcomes. Preserve paired results, calibrate the scorer and distinguish a relative improvement from meeting an absolute product requirement.

Constrained decoding restricts valid continuations; it does not guarantee completion or correct values. Define success and error contracts, validate supported schemas, and measure compilation, masking and queueing under the actual workload.

Attribute queueing, token work and live state before choosing a cap. Bound new admission, verify recovery, and distinguish fair scheduling, reserved service and physical isolation without promising instant or absolute protection.

Screen exact checkpoints for hard requirements, then compare deployable candidates on held-out product tasks, service limits and total cost. Smaller artifacts, leaderboard scores and token rates each answer only part of the decision.

Measure exposed launch and dispatch delay, actual graph coverage, capture memory and warmup. Batch size influences the result, but does not alone determine whether graphs improve the service.

Group compatible adapters by pinned base, derive their size from target matrices and ranks, and benchmark mixed batches. Catalogue size, CPU residency, GPU slots and active adapters are separate limits; base sharing does not make fifty variants operationally free.

A mutable branch or local model directory can change what a restarted replica loads. Establish the actual artifact identity, separate disk evidence from loaded state, and make rollout and restart behavior reproducible.

Follow a shard index to the tensors a loader actually receives. Diagnose a missing file and a shape mismatch, then check architecture, quantization and request compatibility before changing hardware.

Map a checkpoint format to compute capability, then say which part of the speedup survives on each generation. The same file can load on two GPUs and take two different execution paths.

Turn a benchmark number into a bill that survives real utilization, then write the serving contract the gateway enforces and the evaluation the tuned configuration still owes.

Compute cost from a matched bill and token count. Show when an activity factor belongs in the denominator and when it would double-count idle time.

Turn FLOPs and HBM bytes into resource bounds, then compare them with elapsed time. A tighter memory bound suggests an investigation rather than proving the measured bottleneck.

Work through projection and attention FLOPs for an 8,192-token prompt. Separate a peak-based lower bound, an assumed-rate estimate and observed TTFT.

Calculate FP8 savings by separating weights, KV cache and other step costs. Compare the same batch first, then check whether larger batches meet latency and quality targets.

Estimate a TP8 decode step from weights, cache reads and explicitly assumed collective times, then replace the serial worksheet with a measured critical path.

Define what a throughput claim counts, then compare its required arithmetic and memory traffic with a feasible deployment and matched latency targets.

Derive adapter bytes from the target matrices, share compatible base weights, and keep each request's cache identity separate. A worked pool design avoids applying the mixed-adapter cost twice.

Work a 200 ms request budget, distinguish stage allowances from percentiles, and price the capacity needed to meet the measured latency contract.

Follow a video job from prompt to downloadable artifact. Work the latent-shape arithmetic, compare batching with actual delivery times, and design bounded queues, cancellation and safe retries.

Compare the output of a speculative round with drafting and verification cost. Work the break-even condition, then design a measured rollout with memory and correctness checks.

Size both serving pools from workload-matched rates, then check memory, shared transfer paths and surviving capacity. Follow admission and buffer ownership through failure and scaling.

Size a hypothetical trillion-parameter MoE from its weight placement, attention-state layout and expert traffic. Use rack specifications as bounds, then measure the serving workload before promising throughput.

Connect a performance change to its measured mechanism. Compare fair baselines and calculate how a local speedup affects the whole workload.

Build a learning habit around questions, source evidence and practical tests. Explain a real example without claiming work you did not perform.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Follow the weight bytes through a small decode batch and a long prefill. A scoped roofline calculation explains the difference without pretending to predict actual latency.

Derive resident-block limits and distinguish resident warps from ready work before trading registers for more occupancy.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Compare raw weight sizes at 16, 8 and 4 bits, then budget KV cache and runtime memory for Llama 3.1 70B. Follow the worked per-GPU example.