Google Cloud and TPU AI Infrastructure Engineer interview questions

Trace a grid into blocks and warps, then distinguish resident work from ready work. Derive an occupancy limit without treating it as a speed target.

Compare useful parallel work, data movement and reuse. A synthetic CPU/GPU crossover shows why a faster kernel can still lose the end-to-end request.

Carry compute and memory reasoning to TPU v6e, correct all-gather ownership arithmetic, and learn the compilation, sharding and custom-kernel workflow.

Trace a two-by-two output-stationary array cycle by cycle. Derive its fill and drain cost without turning a toy schedule into a TPU benchmark.

Count gradient traffic from a named dtype, separate initialization and buffer broadcasts, and measure the reduction tail instead of assuming it is hidden.

Compute MFU from a declared global token count and time interval. Check the accounting before judging the percentage or choosing an optimization.

Trace a chunk, derive both traffic phases, and state the endpoint model behind bandwidth optimality. A worked example shows why nearly constant bytes do not imply constant completion time.

Trace a causal query across local and remote KV blocks. Separate sequence and context sharding, count one concrete tensor, and avoid dividing memory twice.

Follow one token to two expert ranks and back, then count logical assignments separately from physical network transfers.

A global-array program lets the compiler derive communication from sharding choices. Work through a small matrix example, then separate that programming model from TPU topology, compilation and Pathways recovery.

Three routes with different costs: keep PyTorch through XLA, bet on the native backend, or adopt the JAX reference trainer. Choose from the model's custom code and the timeline, then prove parity before scaling.

Calculate retained cache and prefill work for a stated 128k model shape, then choose scheduling and memory policies from measured limits.

Images add preprocessing and encoder work before the language model can answer. Their effect on prompt length and cache memory depends on the model. Budget each stage, and distinguish image-embedding reuse from prefix-cache reuse.

Work two audio timelines: input accumulation before transcription and playback starvation after synthesis. Compare real-time factor, response delay and safe concurrent capacity without mixing their units.

Validate the exact model, accelerator generation and serving features. Separate packed storage from kernel execution, derive the real memory fallback, and compare qualified throughput and total migration cost.

Same model, same engine name, different execution model. Trace what moves into compilation, how shape buckets trade compile time for padding, and how per-device memory decides the slice.

Separate resident parameters, active work and the union of experts touched by a batch. Compare toy dense and MoE budgets without promising a fixed speedup.

Track actual values across ranks, then count bytes with an explicit buffer definition. Reduction combines values; gathering and all-to-all change ownership.

DRA gives the scheduler device claims and attributes to work with. Compare that with extended-resource counts, then check the deployed driver and controller support before promising sharing, partitioning or gang placement.

Size notebook capacity from concurrent demand, memory needs and burst tolerance. Choose a sharing mode for the trust model, and make idle release explicit about which state can actually be restored.

Promise measurable job outcomes: time to useful work, recoverable progress and successful restoration. Define the population and exclusions before choosing targets, and give each error budget an agreed operating policy.

Give every health-state transition a reason, owner and completion condition. Calibrate intervention against missed-fault risk and disruption cost, and prevent a bad signal source from removing the whole fleet.

A successful operation can still produce a wrong value. Explain what comparisons can detect, what they miss, and how to investigate without mistaking ordinary numerical variation for corruption.

Build an incident-review process that leads to verified changes. Set review thresholds, budget the work and interpret recurrence without mistaking a metric for proof.

Remove the largest region from a concrete capacity plan. Check the surviving budget, tenant eligibility, cold prefixes and retry semantics before claiming the service can fail over.

Follow a video job from prompt to downloadable artifact. Work the latent-shape arithmetic, compare batching with actual delivery times, and design bounded queues, cancellation and safe retries.

Simulate both phases of ring all-reduce and verify values, preserved inputs and each rank’s payload bytes.

Write and verify tiled NumPy matmul, derive its logical footprint and distinguish a cache model from the causes of measured runtime.

Explain a reliability concern with evidence, a specific proposal and an honest outcome. Show how you weighed the cost of delaying a launch and what changed afterward.

Clarify what the 256-GPU request must accomplish, then compare it with real placement and quota constraints. Offer a feasible alternative and explain who can decide when teams’ priorities conflict.

Explain the reason for an unpopular migration, choose informative rollout cohorts and describe recovery, communication and transition cost without inventing an incident.

Connect your view on AI safety to systems you could build: model access, audit evidence, evaluation capacity and a tested way to stop or reverse a release. Explain the costs and limits of each control.

Compare net time saved, maintenance and failure risk before automating. Work a payback example and explain why concurrency limits alone do not bound repeated damage.

Separate a disagreement about measurements from a real capacity shortage. Work a reclaim example, expose the missing reserve and escalate an authorized decision.

Handle a request to skip a safety check by identifying the required gate, finding approved alternatives and making exception authority explicit.

Use an evidence-based timeline, specific contributing conditions and verified follow-up to make a postmortem useful.

Support a new responder without turning an outage into an exam. Choose an appropriate role, explain handoffs and review the decisions afterward.

Turn a design disagreement into a concrete comparison. Use a gang-scheduling example to explain evidence, decision ownership and responsible follow-through.

Describe senior and staff responsibility through scope, technical judgment, influence and durable results, while recognizing that company rubrics differ.

Explain a career move using the actual role, team and your priorities. Test assumptions about ownership, hardware access and support.

Connect a researcher tool to the workflow it improved. Measure the whole loop, meaningful adoption and effects on other users.

Follow one request from API admission to streamed output, then zoom into the token-generation loop. See where the cache lives, who schedules work, and how workload measurements turn the drawing into a capacity plan.

Count two matrix shapes to derive the H100 ridge, throughput ceilings and time floors. Compare an illustrative measurement with its bound before deciding what to optimize.

Trace GPU credits through reservation, allocation, metering and settlement, with crash-safe ownership and a separate fairness policy.

Separate storage scope from physical backing, then follow a warp through adjacent and strided loads. Count useful bytes and requested sectors before inferring HBM traffic or speed.

Derive the weight-only shortcut, then add KV traffic and feasible eight-GPU placement. A worked latency floor shows why batch size alone cannot establish the decode bottleneck.

Normalize link directions, count collective bytes, and see why a gradient reduction can outlast its real overlap window.

Track request admission through a concrete page-reservation example, then connect scheduling, growth and stream cleanup to measured service limits.

Derive a 10,240-GPU layout from model work, then account for collective traffic, durable checkpoints, power and replacements. A nested diagram connects tensor, pipeline and data parallel groups to the fleet.