The Kernel Take-Home: Measure, Optimize and Validate Under a Deadline

For a kernel optimization exercise, establish a correct baseline, measure the workload, identify the limiting mechanism and validate each change. The deliverable is both a working implementation and evidence that it improves the requested operation. A large speedup without the same inputs, timing boundary and correctness conditions is not a useful comparison.

Anthropic’s original performance take-home is one public practice resource. It uses a simulator, not a GPU benchmark, and its historical score is not a universal hiring threshold. Read the current assessment instructions for duration, tools and allowed changes.

Establish the contract before the timer

List the operation, input shapes, layouts, dtype and allowed numerical error. Identify whether the evaluator includes allocation, transfers, compilation or just warmed execution. Those boundaries change what an optimization must improve.

Keep a simple reference and test the supported edge cases before modifying the implementation. Include a small input, a complete tile and a partial tile. For floating-point results, use a justified tolerance and check non-finite outputs explicitly. Do not treat a faster but incomplete output as progress.

NVIDIA’s CUDA best practices guide grounds the profiling and memory reasoning. The GPU performance learning path connects correctness and measurement to practical projects, with access tiers shown on their pages.

Derive a lower bound, then investigate the gap

For a simple operation, compare required FLOPs divided by peak compute with required bytes divided by memory bandwidth. The larger term is an optimistic lower bound under that model. It excludes effects such as launch overhead, dependencies and imperfect scheduling. Use matching precision and dense or sparse peak conventions.

Suppose an illustrative kernel reaches only 8% of the theoretical bandwidth bound. That does not prove a data-movement fix is the answer. There may be too little independent work, a dependency chain, a slow instruction or gaps between launches. A timeline and relevant counters help distinguish these causes. The roofline calculator supplies the arithmetic; it cannot diagnose a trace for you.

At 70% of a fixed bandwidth ceiling, improving utilization alone offers at most about 1/0.7 = 1.43 times that bandwidth under the same traffic assumption. Reducing bytes changes the problem and can move the bound. Keep those two claims separate.

Choose a change that follows from the observation

If accesses cause unnecessary transactions, investigate layout and coalescing. If intermediates are repeatedly written and read, fusion may reduce traffic. If data is reused, a different tile may help. Each change has a cost: fusion can increase register pressure, and staging can introduce synchronization or reduce resident work.

For matrix operations, verify that the implementation uses the intended arithmetic path and that its shape and precision are supported. High theoretical tensor throughput is irrelevant if the kernel performs the work elsewhere. More occupancy is useful only when it improves the limiting behavior; it is not a goal independent of elapsed time.

For a small operation dominated by launches or synchronization, investigate those costs directly. The kernel questions and GPU architecture questions work through the mechanisms. Keep one hypothesis per experiment so a regression remains explainable.

For example, a vector add of one million FP32 elements logically reads two four-megabyte inputs and writes a four-megabyte output: 12 MB of useful traffic. At a hypothetical 1 TB/s, the traffic-only lower bound is 12 microseconds. Allocation, transfers between host and device, launch cost and measurement overhead can make end-to-end time much larger. State whether those costs are inside the assessment’s timing boundary. If two versions count different work, their speedup ratio does not answer the same question. The numbers here describe a bound, not a benchmark we ran.

Use an explicit practice budget

For an illustrative four-hour rehearsal, allow about 45 minutes for the contract, baseline, correctness and initial profile. Spend the next portion testing the most promising change. Reserve the final hour for boundary cases, repeated measurements and a clean submission rerun. Adjust the proportions to the actual task; this is not an employer grading scheme.

Write notes throughout. Record the version, shape, timing method and result after each experiment. If the first idea fails, use the evidence to choose another or submit the correct baseline with an honest explanation. Do not spend the last minutes hiding unsupported inputs or removing failing tests.

Report what another engineer can verify

A short report should identify the baseline, bottleneck hypothesis, changes, correctness evidence and timing results. Include variation across repetitions and the shapes where the change regressed. Explain whether the result affects the complete operation or only one kernel inside it.

End with the next experiment you would run and why its outcome would change your decision. A simulator result establishes behavior in that simulator; a measured GPU result applies to its stated environment. The formula sheet is useful for checking units, and the take-home preparation guide for Anthropic distinguishes its public historical resource from current hiring instructions.