开发者在Hacker News上发布了DeepSeek-V4-Flash-0731模型在单块AMD MI300X GPU上的生产部署方案[1]。该配置提供了完整的Docker Compose堆栈、补丁修复和优化调优表，针对vLLM官方配置不支持MI300X的问题进行了系统解决[1]。
部署方案的关键突破在于解决了多个技术障碍[1]。其中最重要的是FP8格式兼容性问题——MI300X采用AMD专有的FNUZ E4M3标准（float8e4b8，FP8_MAX=224.0），而非行业标准的OCP E4M3规范，这种差异可能导致缩放因子二倍的错误[1]。此外，方案还修复了MoE路由和CUDA图捕获等关键问题[1]。
在性能表现方面，该部署在2-8个并发流处理下运行，峰值可达64个流[1]。预填充性能方面，在2048令牌调度器预算下达到6,988-7,019 tok/s，而在8192令牌预算下可达7.90-7.99K tok/s[1]。模型加载占用156.67 GiB显存，GPU KV缓存最多可处理262,144个令牌，热高水位标记为204.5/205.8 GB显存占用[1]。该方案未采用权重量化即可在单卡上完整装载该模型[1]。
MI300X GPU具备192GB HBM3显存和5.3TB/s内存带宽，成本约为H100 SXM5的一半[1]。部署使用的技术栈包括vLLM ROCm nightly版本0.26.1rc1.dev229+g124154a88.rocm723和AITER版本0.1.19[1]。该配置还支持DSpark-7推测解码、概率草稿生成和块拒绝机制[1]。
A production configuration for running DeepSeek-V4-Flash-0731 on a single AMD MI300X graphics processor has been documented, featuring a complete Docker Compose stack with patches and optimization tuning to address vLLM's lack of native MI300X support [1]. The deployment resolves critical compatibility issues including FP8 format inconsistencies, MoE routing, and CUDA graph capture, enabling the full 304-billion-parameter model to fit on a single card without weight quantization [1].
The MI300X GPU provides 192GB of HBM3 memory and 5.3TB/s memory bandwidth at approximately half the cost of an H100 SXM5 [1]. In this configuration, model loading consumes 156.67 GiB, leaving GPU KV cache capacity to handle up to 262,144 tokens [1]. A key technical fix addresses the AMD FNUZ E4M3 floating-point format (FP8_MAX=224.0) used by MI300X, which differs from the OCP standard E4M3 and can cause scaling errors by a factor of two [1]. The deployment achieves prefill performance of 6,988–7,019 tokens per second with a 2,048-token scheduler budget, scaling to 7.90–7.99K tokens per second with an 8,192-token budget [1]. The system supports 2–8 concurrent streams with peak capacity reaching 64 streams and incorporates DSpark-7 speculative decoding, probabilistic draft generation, and block rejection mechanisms [1].