01How much GPU memory does it take to run Llama 3.1 70B?▼easy★ EssentialNew4 repliesunlockedThe first number every serving interview asks for, derived from parameter count and bytes per parameter, in three precisions, with the part people forget to add.Open full answer →
14Design the registry and deploy pipeline for model weights: 141 GB artifacts, versioning, distribution to a thousand nodes, integrity.▼mediumNewDatabricksHugging Face4 replies○ sign inA 141 GB artifact is not a Docker image, and shipping it like one costs an hour per node and a registry that melts under a rollout. Content-addressed chunks, a manifest per version, a fan-out that turns one origin into a thousand nodes in minutes, integrity that fails closed, and promotion tied to evaluation.Open full answer →