Modal AI Infrastructure Engineer interview questions

Size adapter and full-parameter jobs before comparing RTX PRO 6000 and H100. Test memory headroom, MIG packing, software support and real communication costs.

Forecast demand across startup delay, track replicas already being created, and drain slowly enough to avoid repeating the same scale decision.

Trace a model startup through storage, runtime setup and device loading. Transfer floors and dependencies determine whether a minute is plausible.

Thirty percent GPU utilization means kernels ran during about thirty percent of the sampled time. Find out what limits those kernels, what happens between them, and whether requests are meeting their latency targets.

Find which startup step runs out of memory, count the allocations on that GPU, and change the setting that controls the shortage.

Turn public metadata into a deployment candidate with explicit uncertainty. Inspect architecture, stored formats and engine support, then work a per-rank memory example that explains why an aggregate fit can fail.

Measure reachable prefix reuse, separate token savings from latency, and compare both engines on the same agent contract and resource budget.

Derive a prompt-plus-output contract, calculate per-rank state capacity, and measure the concurrency that meets latency and overload targets.

The answers sound fluent, but the scores dropped. Compare inputs, generation settings, raw output, parsing and scoring to find where the new deployment differs.

Define a reproducible workload and service contract, account for every offered request, and choose a measured operating region with evidence and margin.

A repeated-prompt test measures one reuse pattern. Production gains depend on token-weighted reuse, routing, residency, alignment and the prefill share of latency; a falling hit rate does not uniquely identify eviction.

The nodes joined the cluster, but the first request hangs. Trace how far it got, check worker assignments, and reproduce the blocked operation on the same communication path.

The output forecast is a starting point. Match the prompt and state workload, derive replica counts from an SLO-tested rate, check simultaneous loss scenarios and keep the cost denominator consistent with the forecast.

Confirm that requests are waiting for usable state capacity. Separate allocated memory, live blocks and evictable prefixes, then compare admission, lower-precision state, prefix sharing and hardware changes with explicit assumptions.

A fourfold ratio is a symptom, not a diagnosis. Trace individual requests, separate waiting from execution and delivery, and test workload, scheduling, cache and capacity hypotheses against the service objective.

Build a week around release gates and dependencies. Budget artifact transfer, validate the exact runtime path, measure the intended workload, and size the canary evidence before promising full rollout.

Price the same workload and service guarantees, including input tokens, idle capacity, failures and operations. Then check whether the algebraic crossover fits the fleet before treating it as a buying decision.

An apparent outlier can reflect workload mix, queueing, runtime state or a degraded execution path. Normalize the comparison, run a matched replay and use critical-path arithmetic to test possible causes.

A context-window limit is not a latency or concurrency guarantee. Budget retained state, prefill and shared service time; distinguish cache reuse from eliminating work; choose isolation only when measured interference justifies its cost.

A minor revision can change weights, encoding or response behavior. Compare pinned deployment bundles, evaluate task slices, monitor a representative canary and keep enough capacity to restore the old service.

Design a shared catalogue with explicit service tiers, reproducible deployments and bounded cold starts. Residency depends on burst behavior and resource reuse; placement depends on GPU groups and interference as well as memory.

Apply the project’s offline tuning guidance in context, derive the memory tradeoff, and use workload-specific acceptance criteria instead of universal queue and headroom targets.

Verify the throughput definition and service outcomes, compare resolved configurations and execution traces, then isolate supported changes. Quantify capacity replacement correctly and choose rollback from the full release risk.

Separate pre-routing from a sequential cascade, charge every attempted generation, and measure checker mistakes. A lower escalation rate is not proof of higher quality, and the largest model is not a correctness oracle.

Follow media through fetching, preprocessing, encoding and language-model execution. Derive visual positions from the actual processor output, account for encoder placement and separate the caches and latency stages that text-only monitoring misses.

A partial drop does not rule out the parser. Define success and its denominator, trace failures through encoding, generation, parsing and execution, then replay the relevant boundary with controlled versions.

Constrained decoding restricts valid continuations; it does not guarantee completion or correct values. Define success and error contracts, validate supported schemas, and measure compilation, masking and queueing under the actual workload.

Attribute queueing, token work and live state before choosing a cap. Bound new admission, verify recovery, and distinguish fair scheduling, reserved service and physical isolation without promising instant or absolute protection.

Screen exact checkpoints for hard requirements, then compare deployable candidates on held-out product tasks, service limits and total cost. Smaller artifacts, leaderboard scores and token rates each answer only part of the decision.

Group compatible adapters by pinned base, derive their size from target matrices and ranks, and benchmark mixed batches. Catalogue size, CPU residency, GPU slots and active adapters are separate limits; base sharing does not make fifty variants operationally free.

A mutable branch or local model directory can change what a restarted replica loads. Establish the actual artifact identity, separate disk evidence from loaded state, and make rollout and restart behavior reproducible.

A measured run shows where the time goes in a small batch job, then why every result parsed but none was deliverable. The answer turns on startup cost, co-tenant latency and a per-item contract.

Walk a single-GPU vLLM Deployment from pod scheduling to the first token, then show the three settings that turn a working manifest into restart loops and stuck rollouts.

Follow a four-GPU request from advertised capacity to concrete devices, and see why free GPUs across nodes cannot satisfy one pod.

Define warm, cached and truly cold paths, then size transfer budgets and warm capacity without promising universal sub-second restores.

Choose a sandbox from its threat model and supported workload, then calculate density from total session memory and CPU demand. GPU access needs its own isolation and compatibility review.

Measure scheduling, image transfer, unpacking and model initialization separately. A missing-device error needs its own checks for allocation, container access and CUDA compatibility.

Page when material impact or risk needs timely human action. Verify that automation actually recovered the workload, group related events, and use measured responder load to improve the policy.

Page when timely human action matters. Use budget burn, careful grouping, bounded maintenance rules, and regular testing to keep those pages useful.

Eight ready slots meet a twelve-request burst. See why starting instances do not count as ready capacity, then design caches, isolation, admission and a measurable startup contract.

A 60-second demand ramp meets a 70-second startup delay. Compare the rate and ready-capacity curves, then design a controller that tracks pending launches without a blind cooldown.

Separate session state, model calls and tool execution. Calculate growing context, budget isolated workspaces and recover uncertain actions without duplicating side effects.

Build a byte-budget cache with reference-counted pins and admission that preserves existing entries when it fails.

Reconstruct GPU-hour intervals, clip them to a period and preserve provisional usage and conflicting-event evidence.

Tell an incident through the decisions you made, the evidence available at the time and the cost of recovery. Keep your role clear and distinguish mitigation from finding the root cause.

Compare net time saved, maintenance and failure risk before automating. Work a payback example and explain why concurrency limits alone do not bound repeated damage.

Show a cost reduction with comparable service evidence, realized savings and a feasible recovery plan. Distinguish a quiet trial from proof of unchanged reliability.

Coordinate simultaneous incidents with the people available. Contain urgent harm, assign next actions and revisit priorities as evidence changes.

Explain an isolation finding through the intended boundary, authorized evidence, designated reporting process and checks for both denied and permitted access.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Follow the weight bytes through a small decode batch and a long prefill. A scoped roofline calculation explains the difference without pretending to predict actual latency.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Follow a token into retained attention state, derive bytes per position for a 70B model, and distinguish present cache usage from the room needed to finish a response.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.

Work from each GPU's cache budget, reserve room for generated tokens, then check latency separately. A limiting-rank example shows why fitting today's prompts is not enough to admit tomorrow's work.

Separate quota, borrowing, reclaim latency and fairness, with a demand-conserving example of when pooling improves utilization.