TL;DR: MIG partitions supported GPU hardware into configured profiles. Time-slicing lets more clients share a device without promising each a memory partition or fixed throughput. MPS allows concurrent execution from multiple processes, with controls that depend on GPU generation, software version and configuration. Choose from the isolation and performance contract, then measure the workload.
How to approach it
Ask how much memory each workload needs at peak, whether tenants trust each other, and what latency they require. Compare supported configurations on the actual GPU and driver; do not translate a utilization percentage directly into a safe sharing ratio.
A strong answer
Three ways to share one GPU are compared here. MIG (Multi-Instance GPU) carves a supported GPU into isolated hardware slices, each with its own memory. Time-slicing lets several processes take turns on the whole GPU. MPS (Multi-Process Service) lets several processes run kernels on the GPU at the same time. Six notebooks each report low average utilization. It is tempting to put them on one GPU and call the unused percentage recovered. But their users may all start a large cell at the same time, and each process can retain memory while idle. MIG, MPS and Time-Slicing gives the vocabulary; the engineering decision needs a resource budget and a contention test.
MIG's user guide defines the available profiles and placement rules for each supported GPU. A profile allocates specified compute and memory resources. Check its usable capacity with the application, including runtime overhead and temporary allocations. Profile names and legal combinations vary; counting nominal gigabytes is not enough to prove two profiles can coexist.
MIG improves isolation between GPU instances, but it does not create independent servers. CPU, storage, network, power and the physical device can remain shared failure or contention domains. Hardware partitioning also does not remove the need for container and tenant security controls.
NVIDIA's time-slicing configuration increases advertised shared resources. A replica is a scheduling permission to share, not a guaranteed fraction of memory or execution time. Requesting more replicas does not inherently buy a proportional speedup. Use it where clients can tolerate interference and the memory behavior is controlled.
MPS architecture is also easy to oversimplify. Modern clients have separate GPU address spaces; they do not all become one application context. Supported execution and memory controls differ across versions and modes, including newer partitioning capabilities. Verify the deployed version's guarantees. A compute limit is not a throughput guarantee, and shared-device faults can still affect several clients.
Consider this deliberately simplified capacity check for a hypothetical device with 80 GB usable memory:
Four clients, each with a measured peak of 18 GB: 4 × 18 = 72 GB
Reserved device/runtime headroom: 8 GB
Total planned demand: 72 + 8 = 80 GB, leaving no extra margin
A fifth identical client: 5 × 18 + 8 = 98 GB → does not fit
Sanity: low SM utilization does not reduce allocated model memory.
That example is a whole-device budget, not a claim about any MIG profile's capacity. A safe deployment also allows for measurement error and changing inputs, so exact equality is a reason to reduce density or establish a tighter bound.
Average activity also hides overlap. Suppose four clients each run 10% of the time, independently of one another. The expected number running at any moment is only 0.4, but two or more still run together 5.23% of the time: 1 − 0.9⁴ − 4×0.1×0.9³ = 5.23%. If a class starts a notebook exercise together, the independence assumption fails and overlap can be much more frequent. Test these simultaneous peaks, including their memory use and latency, before choosing how many clients to admit.
Start with a supported MIG profile when its resource isolation and tested performance meet the requirement. Use time-slicing for tolerant shared work; consider MPS when concurrent execution improves the measured workload and its controls meet the contract. Reverse any density increase that violates peak memory, tail latency or recovery behavior. A dedicated GPU remains a valid outcome.
What interviewers probe next
- "Can idle MIG memory be borrowed by another profile?" Not as ordinary free memory outside that profile; reconfiguration has its own constraints.
- "Does 25% compute mean 25% throughput?" No. Bottlenecks and co-runners determine throughput.
- "What should the benchmark include?" Concurrent peaks, adversarial timing and recovery, as well as isolated speed.
Common mistakes
- Calling an advertised slot a memory reservation.
- Treating every MPS release as having the same controls.
- Assuming independent client arrivals during a coordinated workload.
Key takeaways
- Four 18 GB clients plus 8 GB headroom already consume an 80 GB budget.
- Four independent 10%-active clients overlap in pairs or more 5.23% of the time.
- State the sharing contract before quoting its density.