Wafer研究团队在AMD MI355X GPU上成功部署了参数量达2.8万亿的大模型Kimi K3[1]。在1024-token输入、400-token输出的基准测试中，MI355X实现了每节点952令牌/秒的吞吐量[1]。相比之下，NVIDIA B300的性能成本为每GPU小时6.00美元，而MI355X仅为2.50美元，在成本效益上优于B300 1.65倍[1]。
在单流延迟指标上，MI355X达到118令牌/秒，低于B200张量并行16部署的249令牌/秒/节点[1]。研究团队通过修复ROCm框架中采样验证器内的top_k_renorm_prob缺陷，使单流性能提升约2.2倍[1]。针对prefill阶段的优化同样显著，利用AITER MLA内核加速，将172k-token冷prefill的处理时间从约51秒降低至约13000令牌/秒的稳定吞吐[1]。
Researchers have successfully deployed the 2.8-trillion-parameter Kimi K3 large language model on AMD's MI355X GPU, demonstrating superior cost efficiency compared to NVIDIA's B300 accelerator.[1] The MI355X achieved a throughput of 952 tokens per second per node in benchmarks using 1024-token inputs and 400-token outputs.[1] With a performance cost of $2.50 per GPU-hour, the MI355X offers 1.65 times better cost-effectiveness than the B300, which costs $6.00 per GPU-hour.[1]
The deployment required over 1.5TB of VRAM to accommodate the model's parameters.[1] Researchers optimized the system by fixing a critical sampling kernel defect in the ROCm framework—specifically a NameError in the top_k_renorm_prob validation function—which improved single-stream performance by approximately 2.2 times.[1] Additionally, they enhanced the prefill algorithm, reducing the time for processing a 172,000-token cold prefill from approximately 51 seconds to a steady state of around 13,000 tokens per second through AITER MLA kernel acceleration.[1] While the MI355X's single-stream latency of 118 tokens per second lags behind the B200's tensor-parallel deployment at 249 tokens per second per node, the overall value proposition demonstrates AMD's competitive potential for large-scale language model inference services.[1]