Amazon SageMaker HyperPod now supports model caching for faster inference autoscaling and reduced cold starts

Summary

• Adds model caching to SageMaker HyperPod inference, pre‑loading model weights and container images to cut cold‑start times from minutes to seconds.
• Implements a local NVMe weight cache and an image pre‑pull cache, delivering ~60% faster scale‑out and up to 97% reduction in image‑pull latency.
• Feature is GA in all HyperPod regions and enabled via a modelCacheConfig section in InferenceEndpointConfig or JumpStartModel resources.