{"id":85204,"date":"2024-10-12T09:07:00","date_gmt":"2024-10-12T01:07:00","guid":{"rendered":"https:\/\/www.ioiotimes.com\/?p=85204"},"modified":"2024-10-12T10:08:17","modified_gmt":"2024-10-12T02:08:17","slug":"amd-instinct-mi325x%e5%8a%a0%e9%80%9f%e5%99%a8%e6%8f%90%e4%be%9b%e9%a0%98%e5%85%88%e6%a5%ad%e7%95%8c%e7%9a%84ai%e6%95%88%e8%83%bd","status":"publish","type":"post","link":"https:\/\/www.ioiotimes.com\/?p=85204","title":{"rendered":"AMD Instinct MI325X\u52a0\u901f\u5668\u63d0\u4f9b\u9818\u5148\u696d\u754c\u7684AI\u6548\u80fd"},"content":{"rendered":"\n<ul class=\"wp-block-list\">\n<li><strong>\u6700\u65b0\u7684\u52a0\u901f\u5668\u63d0\u4f9b\u9818\u5148\u5e02\u5834\u7684HBM3E\u8a18\u61b6\u9ad4\u5bb9\u91cf\uff0c\u4e26\u7372\u5f97\u6234\u723e\u79d1\u6280\u96c6\u5718\u3001HPE\u3001\u806f\u60f3\u3001Supermicro\u7b49\u5408\u4f5c\u5925\u4f34\u548c\u5ba2\u6236\u7684\u652f\u63f4<\/strong><\/li>\n\n\n\n<li><strong>AMD Pensando Salina DPU\u8f03\u4e0a\u4e00\u4ee3\u7522\u54c1\u63d0\u4f9b2\u500d\u7684\u6548\u80fd\u63d0\u5347\uff0cAMD Pensando Pollara 400\u70ba\u696d\u754c\u9996\u6b3eUEC\u5c31\u7dd2NIC<\/strong><\/li>\n<\/ul>\n\n\n\n<p><\/p>\n\n\n\n<p><a href=\"http:\/\/www.amd.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">AMD<\/a>\uff08NASDAQ: AMD\uff09\u5ba3\u5e03\u63a8\u51fa AMD Instinct\u2122 MI325X \u52a0\u901f\u5668\u3001AMD Pensando\u2122 Pollara 400 NIC \u4ee5\u53ca AMD Pensando Salina DPU \u7b49\u6700\u65b0\u52a0\u901f\u5668\u548c\u7db2\u8def\u89e3\u6c7a\u65b9\u6848\uff0c\u5c07\u70ba\u65b0\u4e00\u4ee3\u4eba\u5de5\u667a\u6167\uff08AI\uff09\u57fa\u790e\u8a2d\u65bd\u63d0\u4f9b\u5927\u898f\u6a21\u652f\u63f4\u3002AMD Instinct MI325X \u52a0\u901f\u5668\u70ba\u751f\u6210\u5f0f AI \u6a21\u578b\u53ca\u8cc7\u6599\u4e2d\u5fc3\u8a2d\u7acb\u5168\u65b0\u6548\u80fd\u6a19\u6e96\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"711\" src=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd01.png\" alt=\"\" class=\"wp-image-85234\" title=\"\" srcset=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd01.png 1024w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd01-300x208.png 300w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd01-768x533.png 768w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd01-590x410.png 590w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">\u25b2AMD Instinct MI325X\u52a0\u901f\u5668\u70ba\u751f\u6210\u5f0fAI\u6a21\u578b\u53ca\u8cc7\u6599\u4e2d\u5fc3\u8a2d\u7acb\u5168\u65b0\u6548\u80fd\u6a19\u6e96<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>AMD Instinct MI325X \u52a0\u901f\u5668\u57fa\u65bc AMD CDNA\u2122 3 \u67b6\u69cb\uff0c\u65e8\u5728\u70ba\u57fa\u790e\u6a21\u578b\u8a13\u7df4\u3001\u5fae\u8abf\u548c\u63a8\u8ad6\u7b49\u8981\u6c42\u56b4\u82db\u7684 AI \u4efb\u52d9\u63d0\u4f9b\u5353\u8d8a\u7684\u6548\u80fd\u548c\u6548\u7387\u3002\u5168\u65b0\u7522\u54c1\u5c07\u5354\u52a9 AMD \u5ba2\u6236\u548c\u5408\u4f5c\u5925\u4f34\u5728\u7cfb\u7d71\u3001\u6a5f\u67b6\u548c\u8cc7\u6599\u4e2d\u5fc3\u5c64\u7d1a\u6253\u9020\u9ad8\u6548\u80fd\u548c\u6700\u4f73\u5316\u7684 AI \u89e3\u6c7a\u65b9\u6848\u3002<\/p>\n\n\n\n<p>AMD \u57f7\u884c\u526f\u7e3d\u88c1\u66a8\u8cc7\u6599\u4e2d\u5fc3\u89e3\u6c7a\u65b9\u6848\u4e8b\u696d\u7fa4\u7e3d\u7d93\u7406 Forrest Norrod \u8868\u793a\uff0cAMD \u6301\u7e8c\u57f7\u884c\u6211\u5011\u7684\u7522\u54c1\u85cd\u5716\uff0c\u70ba\u5ba2\u6236\u63d0\u4f9b\u6240\u9700\u7684\u6548\u80fd\u548c\u9078\u64c7\uff0c\u4ee5\u66f4\u5feb\u7684\u901f\u5ea6\u5c07 AI \u57fa\u790e\u8a2d\u65bd\u5927\u898f\u6a21\u63a8\u5411\u5e02\u5834\u3002\u6191\u85c9\u5168\u65b0 AMD Instinct \u52a0\u901f\u5668\u3001EPYC \u8655\u7406\u5668\u548c AMD Pensando \u7db2\u8def\u5f15\u64ce\u3001\u958b\u653e\u8edf\u9ad4\u7522\u696d\u9ad4\u7cfb\u7684\u6301\u7e8c\u6210\u9577\uff0c\u4ee5\u53ca\u5c07\u9019\u4e00\u5207\u6574\u5408\u81f3\u6700\u4f73\u5316 AI \u57fa\u790e\u8a2d\u65bd\u4e2d\u7684\u80fd\u529b\uff0cAMD \u5c55\u73fe\u5efa\u7f6e\u548c\u90e8\u7f72\u4e16\u754c\u7d1a AI \u89e3\u6c7a\u65b9\u6848\u7684\u95dc\u9375\u5c08\u696d\u77e5\u8b58\u8207\u80fd\u529b\u3002<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>AMD Instinct MI325X<\/strong><strong>\u64f4\u5c55<\/strong><strong>AI<\/strong><strong>\u6548\u80fd\u7684\u9818\u5148\u512a\u52e2<\/strong><strong><\/strong><\/h3>\n\n\n\n<p>AMD Instinct MI325X \u52a0\u901f\u5668\u63d0\u4f9b\u9818\u5148\u696d\u754c\u7684\u8a18\u61b6\u9ad4\u5bb9\u91cf\u53ca\u983b\u5bec\uff0c\u5305\u62ec 256GB HBM3E \u8a18\u61b6\u9ad4\u5bb9\u91cf\u652f\u63f4 6.0TB\/s\uff0c\u63d0\u4f9b\u6bd4 H200 \u9ad8 1.8 \u500d\u7684\u5bb9\u91cf\u548c 1.3 \u500d\u7684\u983b\u5bec<sup>\u8a3b1<\/sup>\uff0c\u4ee5\u53ca 1.3 \u500d\u7684 FP16 \u7406\u8ad6\u5cf0\u503c\u548c FP8 \u904b\u7b97\u6548\u80fd<sup>\u8a3b1<\/sup>\u3002<\/p>\n\n\n\n<p>AMD Instinct MI325X \u52a0\u901f\u5668\u7684\u8a18\u61b6\u9ad4\u548c\u904b\u7b97\u80fd\u529b\u53ef\u8f03 H200 \u63d0\u4f9b\u9ad8\u9054 1.3 \u500d\u7684 Mistral 7B FP16 \u63a8\u8ad6\u6548\u80fd<sup>\u8a3b2<\/sup>\u30011.2 \u500d\u7684 Llama 3.1 70B FP8 \u63a8\u8ad6\u6548\u80fd<sup>\u8a3b3<\/sup>\uff0c\u4ee5\u53ca 1.4 \u500d\u7684 Mixtral 8x7B FP16 \u63a8\u8ad6\u6548\u80fd<sup>\u8a3b4<\/sup>\u3002<\/p>\n\n\n\n<p>AMD Instinct MI325X \u52a0\u901f\u5668\u76ee\u524d\u5982\u671f\u5728 2024 \u5e74\u7b2c 4 \u5b63\u91cf\u7522\u51fa\u8ca8\uff0c\u9810\u8a08\u5c07\u65bc 2025 \u5e74\u7b2c 1 \u5b63\u8d77\uff0c\u7531\u6234\u723e\u79d1\u6280\u96c6\u5718\u3001Eviden\u3001\u6280\u5609\u3001HPE\u3001\u806f\u60f3\u3001\u7f8e\u8d85\u5fae\uff08Supermicro\uff09\u7b49\u5e73\u53f0\u4f9b\u61c9\u5546\u5ee3\u6cdb\u63d0\u4f9b\u3002<\/p>\n\n\n\n<p>AMD \u6301\u7e8c\u5c65\u884c\u5e74\u5ea6\u7522\u54c1\u85cd\u5716\u7684\u7bc0\u594f\uff0c\u9810\u89bd\u4e86\u4e0b\u4e00\u4ee3 AMD Instinct MI350 \u7cfb\u5217\u52a0\u901f\u5668\u3002\u76f8\u5c0d\u65bc AMD CDNA 3 \u67b6\u69cb\u7684\u52a0\u901f\u5668\uff0c\u57fa\u65bc AMD CDNA 4 \u67b6\u69cb\u7684 AMD Instinct MI350 \u7cfb\u5217\u52a0\u901f\u5668\u5c07\u5e36\u4f86 35 \u500d\u63a8\u8ad6\u6548\u80fd\u63d0\u5347<sup>\u8a3b5<\/sup>\u3002<\/p>\n\n\n\n<p>AMD Instinct MI350 \u7cfb\u5217\u5c07\u6301\u7e8c\u978f\u56fa\u8a18\u61b6\u9ad4\u5bb9\u91cf\u7684\u9818\u5148\u5730\u4f4d\uff0c\u6bcf\u52a0\u901f\u5668\u5bb9\u91cf\u9ad8\u9054 288GB HBM3E \u8a18\u61b6\u9ad4\uff0c\u5c07\u5982\u671f\u65bc 2025 \u5e74\u4e0b\u534a\u5e74\u63a8\u51fa\u3002<\/p>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>AMD<\/strong><strong>\u65b0\u4e00\u4ee3<\/strong><strong>AI<\/strong><strong>\u7db2\u8def\u89e3\u6c7a\u65b9\u6848<\/strong><strong><\/strong><\/h3>\n\n\n\n<p>AMD \u6b63\u5728\u904b\u7528\u8d85\u5927\u898f\u6a21\u4f9b\u61c9\u5546\uff08hyperscalers\uff09\u90e8\u7f72\u6700\u5ee3\u6cdb\u7684\u53ef\u7a0b\u5f0f\u5316 DPU \u4f86\u70ba\u65b0\u4e00\u4ee3 AI \u7db2\u8def\u63d0\u4f9b\u52d5\u80fd\u3002AI \u7db2\u8def\u5206\u70ba\u5169\u90e8\u5206\uff1a\u524d\u7aef\uff08\u5411 AI \u53e2\u96c6\u63d0\u4f9b\u8cc7\u6599\u548c\u8cc7\u8a0a\uff09\u548c\u5f8c\u7aef\uff08\u7ba1\u7406\u52a0\u901f\u5668\u548c\u53e2\u96c6\u4e4b\u9593\u7684\u8cc7\u6599\u50b3\u8f38\uff09\uff0c\u5c0d\u65bc\u78ba\u4fdd CPU \u548c\u52a0\u901f\u5668\u5728 AI \u57fa\u790e\u8a2d\u65bd\u4e2d\u9ad8\u6548\u5229\u7528\u81f3\u95dc\u91cd\u8981\u3002<\/p>\n\n\n\n<p>\u70ba\u4e86\u6709\u6548\u7ba1\u7406\u9019\u5169\u500b\u7db2\u8def\u4e26\u63a8\u52d5\u6574\u500b\u7cfb\u7d71\u7684\u9ad8\u6548\u80fd\u3001\u53ef\u64f4\u5c55\u6027\u548c\u6548\u7387\uff0cAMD \u63a8\u51fa\u7528\u65bc\u524d\u7aef\u7684 AMD Pensando\u2122 Salina DPU \u548c\u7528\u65bc\u5f8c\u7aef\u3001\u696d\u754c\u9996\u6b3e UEC \u5c31\u7dd2\u7684 AMD Pensando\u2122 Pollara 400 AI NIC\u3002<\/p>\n\n\n\n<p>AMD Pensando Salina DPU \u662f\u5168\u7403\u6548\u80fd\u6700\u5f37\u5927\u53ef\u7a0b\u5f0f\u5316 DPU \u7684\u7b2c 3 \u4ee3\u7522\u54c1\uff0c\u8207\u524d\u4e00\u4ee3 DPU \u76f8\u6bd4\uff0c\u6548\u80fd\u3001\u983b\u5bec\u548c\u898f\u6a21\u63d0\u5347\u9ad8\u9054 2 \u500d\u3002AMD Pensando Salina DPU \u652f\u63f4 400G \u541e\u5410\u91cf\u4ee5\u5be6\u73fe\u5feb\u901f\u8cc7\u6599\u50b3\u8f38\u901f\u7387\uff0c\u662f AI \u524d\u7aef\u7db2\u8def\u53e2\u96c6\u7684\u95dc\u9375\u5143\u4ef6\uff0c\u70ba\u8cc7\u6599\u9a45\u52d5\u7684 AI \u61c9\u7528\u5e36\u4f86\u6700\u4f73\u5316\u7684\u6548\u80fd\u3001\u6548\u7387\u3001\u5b89\u5168\u6027\u548c\u53ef\u64f4\u5c55\u6027\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"592\" src=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd02.png\" alt=\"\" class=\"wp-image-85236\" title=\"\" srcset=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd02.png 1024w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd02-300x173.png 300w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd02-768x444.png 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">\u25b2AMD Pensando Salina DPU\u8f03\u4e0a\u4e00\u4ee3\u7522\u54c1\u63d0\u4f9b2\u500d\u7684\u6548\u80fd\u63d0\u5347<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>UEC \u5c31\u7dd2\u7684 AMD Pensando Pollara 400 \u7531 AMD P4 \u53ef\u7a0b\u5f0f\u5316\u5f15\u64ce\u63d0\u4f9b\u52d5\u80fd\uff0c\u662f\u696d\u754c\u9996\u6b3e UEC \u5c31\u7dd2\u7684 AI NIC\uff0c\u652f\u63f4\u65b0\u4e00\u4ee3 RDMA \u8edf\u9ad4\uff0c\u4e26\u7531\u958b\u653e\u7684\u7db2\u8def\u7522\u696d\u9ad4\u7cfb\u63d0\u4f9b\u652f\u63f4\u3002AMD Pensando Pollara 400 \u5c0d\u65bc\u5728\u5f8c\u7aef\u7db2\u8def\u4e2d\u63d0\u4f9b\u9818\u5148\u7684\u6548\u80fd\u3001\u53ef\u64f4\u5c55\u6027\u548c\u52a0\u901f\u5668\u9593\u901a\u8a0a\u7684\u6548\u7387\u81f3\u95dc\u91cd\u8981\u3002<\/p>\n\n\n\n<p>AMD Pensando Salina DPU \u548c AMD Pensando Pollara 400 \u65bc 2024 \u5e74\u7b2c 4 \u5b63\u9001\u6a23\uff0c\u4e26\u5c07\u5982\u671f\u5728 2025 \u5e74\u4e0a\u534a\u5e74\u63a8\u51fa\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"557\" src=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd03.png\" alt=\"\" class=\"wp-image-85237\" title=\"\" srcset=\"https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd03.png 1024w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd03-300x163.png 300w, https:\/\/www.ioiotimes.com\/wordpress\/wp-content\/uploads\/2024\/10\/20241012-amd03-768x418.png 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">\u25b2AMD Pensando Pollara 400\u70ba\u696d\u754c\u9996\u6b3eUEC\u5c31\u7dd2NIC<\/figcaption><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>AMD AI<\/strong><strong>\u8edf\u9ad4\u70ba\u751f\u6210\u5f0f<\/strong><strong>AI<\/strong><strong>\u63d0\u4f9b\u5168\u65b0\u529f\u80fd<\/strong><strong><\/strong><\/h3>\n\n\n\n<p>AMD \u6301\u7e8c\u63a8\u9032\u8edf\u9ad4\u529f\u80fd\u548c\u958b\u653e\u7522\u696d\u9ad4\u7cfb\u7684\u767c\u5c55\uff0c\u5728 AMD ROCm\u2122\u958b\u653e\u8edf\u9ad4\u5806\u758a\u4e2d\u63d0\u4f9b\u5f37\u5927\u7684\u5168\u65b0\u7279\u6027\u548c\u529f\u80fd\u3002<\/p>\n\n\n\n<p>\u5728\u958b\u653e\u8edf\u9ad4\u793e\u7fa4\u4e2d\uff0cAMD \u6b63\u63a8\u52d5 PyTorch\u3001Triton\u3001Hugging Face \u7b49\u6700\u70ba\u5ee3\u6cdb\u63a1\u7528\u7684 AI \u6846\u67b6\u3001\u51fd\u5f0f\u5eab\u548c\u6a21\u578b\u5c0d AMD \u904b\u7b97\u5f15\u64ce\u7684\u652f\u63f4\u3002\u9019\u9805\u5de5\u4f5c\u70ba AMD Instinct \u52a0\u901f\u5668\u63d0\u4f9b\u4e86\u5373\u6642\u6548\u80fd\u8207\u652f\u63f4\uff0c\u9069\u7528\u65bc Stable Diffusion 3\u3001Meta Llama 3\u30013.1 \u548c 3.2 \u7b49\u71b1\u9580\u7684\u751f\u6210\u5f0f AI \u6a21\u578b\uff0c\u4ee5\u53ca Hugging Face \u8d85\u904e 100 \u842c\u500b\u6a21\u578b\u3002<\/p>\n\n\n\n<p>\u9664\u4e86\u793e\u7fa4\u4e4b\u5916\uff0cAMD \u6301\u7e8c\u63a8\u9032\u5176 ROCm \u958b\u653e\u8edf\u9ad4\u5806\u758a\uff0c\u5e36\u4f86\u652f\u63f4\u751f\u6210\u5f0f AI \u5de5\u4f5c\u8ca0\u8f09\u8a13\u7df4\u548c\u63a8\u8ad6\u7684\u6700\u65b0\u529f\u80fd\u3002ROCm 6.2 \u73fe\u5728\u5c0d FP8 \u8cc7\u6599\u985e\u578b\u3001Flash Attention 3\u3001Kernel Fusion \u7b49\u95dc\u9375 AI \u529f\u80fd\u63d0\u4f9b\u652f\u63f4\u3002\u6191\u85c9\u9019\u4e9b\u65b0\u589e\u529f\u80fd\uff0cROCm 6.2 \u8f03 ROCm 6.0 \u63d0\u4f9b\u9ad8\u9054 2.4 \u500d\u7684\u63a8\u8ad6\u6548\u80fd\u63d0\u5347<sup>\u8a3b6<\/sup>\u4ee5\u53ca 1.8 \u500d\u7684\u5927\u578b\u8a9e\u8a00\u6a21\u578b\uff08LLM\uff09\u8a13\u7df4\u6548\u80fd\u63d0\u5347<sup>\u8a3b7<\/sup>\u3002<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>  \u3000\u3000<\/p>\n\n\n\n<p><sup>\u8a3b1\uff1aMI325-002\uff1a\u622a\u81f32024\u5e745\u670828\u65e5\uff0cAMD\u6548\u80fd\u5be6\u9a57\u5ba4\u5c0dAMD Instinct\u2122 MI325X GPU\u9032\u884c\u7684\u6e2c\u8a66\u7d50\u679c\u70ba1307.4 TFLOPS\u7406\u8ad6\u5cf0\u503c\u534a\u7cbe\u5ea6\uff08FP16\uff09\u30011307.4 TFLOPS\u7406\u8ad6\u5cf0\u503cBF16\u30012614.9 TFLOPs\u7406\u8ad6\u5cf0\u503cFP8\u30012614.9 TOPS INT8\u6d6e\u9ede\u6548\u80fd\u3002\u5be6\u969b\u6548\u80fd\u6839\u64da\u6700\u7d42\u898f\u683c\u548c\u7cfb\u7d71\u914d\u7f6e\u800c\u6709\u6240\u4e0d\u540c\u3002<br>\u5728Nvidia H200 SXM (141GB) GPU\u4e0a\u767c\u5e03\u7684\u7d50\u679c\uff1a989.4 TFLOPS\u7406\u8ad6\u5cf0\u503c\u534a\u7cbe\u5ea6Tensor\uff08FP16 Tensor\uff09\u3001989.4 TFLOPS\u7406\u8ad6\u5cf0\u503cBF16 Tensor\u30011,978.9 TFLOP\u7406\u8ad6\u5cf0\u503cFP8\u30011,978.9 TOPs\u7406\u8ad6\u5cf0\u503cINT8\u6d6e\u9ede\u6548\u80fd\u3002Nvidia\u4f7f\u7528\u7a00\u758f\u6027\u767c\u5e03BFLOAT16 Tensor Core\u3001FP16 Tensor Core\u3001FP8 Tensor Core\u548cINT8 Tensor Core\u6548\u80fd\u3002\u70ba\u9032\u884c\u6bd4\u8f03\uff0cAMD\u900f\u904e\u9664\u4ee52\u5c07\u9019\u4e9b\u6578\u5b57\u8f49\u63db\u70ba\u975e\u7a00\u758f\u6027\uff0f\u5bc6\u96c6\u3002<br>Nvidia H200\u4f86\u6e90\uff1a<a href=\"https:\/\/nvdam.widen.net\/s\/nb5zzzsjdf\/hpc-datasheet-sc23-h200-datasheet-3002446\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/nvdam.widen.net\/s\/nb5zzzsjdf\/hpc-datasheet-sc23-h200-datasheet-3002446<\/a>\u4ee5\u53ca\u00a0<a href=\"https:\/\/www.anandtech.com\/show\/21136\/nvidia-at-sc23-h200-accelerator-with-hbm3e-and-jupiter-supercomputer-for-2024\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/www.anandtech.com\/show\/21136\/nvidia-at-sc23-h200-accelerator-with-hbm3e-and-jupiter-supercomputer-for-2024<\/a>\u3002\u8acb\u6ce8\u610f\uff1aNvidia H200 GPU\u00a0\u5177\u6709\u8207H100\u7522\u54c1\u76f8\u540c\u7684FLOP\u6548\u80fd<a href=\"https:\/\/resources.nvidia.com\/en-us-%20tensor-core\/\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/resources.nvidia.com\/en-us- tensor-core\/<\/a>\u3002<br>\u8a3b2\uff1aMI325-005\uff1a\u57fa\u65bcAMD\u6548\u80fd\u5be6\u9a57\u5ba4\u65bc2024\u5e749\u670828\u65e5\u5b8c\u6210\u7684\u6e2c\u8a66\uff0c\u8a72\u6e2c\u8a66\u4f7f\u7528FP16\u8cc7\u6599\u985e\u578b\u6e2c\u91cfMistral-7B\u6a21\u578b\u7684\u7e3d\u5ef6\u9072\u3002\u4f7f\u7528128\u500btoken\u7684\u8f38\u5165\u9577\u5ea6\u548c\u00a0128\u500btoken\u7684\u8f38\u51fa\u9577\u5ea6\u5c0dAMD Instinct\u2122 MI325X GPU\u52a0\u901f\u5668\u548cNVIDIA H200 SXM GPU\u52a0\u901f\u5668\u7684\u4ee5\u4e0b\u914d\u7f6e\u9032\u884c\u6e2c\u8a66\u3002<br>1x MI325X\u57281000\u74e6\u7684vLLM\u6548\u80fd\uff1a0.637\u79d2\u5ef6\u9072\uff08\u4ee5\u79d2\u70ba\u55ae\u4f4d\uff09\u5c0d\u6bd41x H200\u5728700\u74e6\u53caTensorRT-LLM\uff1a0.811\u79d2\u5ef6\u9072\uff08\u4ee5\u79d2\u70ba\u55ae\u4f4d\uff09\u3002<br>\u914d\u7f6e\uff1a<br>AMD Instinct\u2122 MI325X\u53c3\u8003\u5e73\u53f0\uff1a1x AMD Ryzen\u2122 9 7950X 16\u6838\u5fc3\u8655\u7406\u5668\u30011x AMD Instinct MI325X (256GiB, 1000\u74e6) GPU\u3001Ubuntu\u00ae 22.04\u3001and ROCm\u2122 6.3\u00a0 pre-release\uff1b<br>\u5c0d\u6bd4NVIDIA H200 HGX\u5e73\u53f0\uff1a\u7f8e\u8d85\u5fae\uff08Supermicro\uff09SuperServer\u642d\u8f092x Intel Xeon\u00ae Platinum 8468\u8655\u7406\u5668\u30018x Nvidia H200 (140GB, 700\u74e6) GPU [\u6e2c\u8a66\u4e2d\u53ea\u9069\u75281 GPU]\u3001Ubuntu 22.04\u3001CUDA 12.6\u3002\u4f3a\u670d\u5668\u88fd\u9020\u5546\u53ef\u80fd\u6703\u6539\u8b8a\u914d\u7f6e\uff0c\u7522\u751f\u4e0d\u540c\u7684\u7d50\u679c\u3002\u6548\u80fd\u53ef\u80fd\u6703\u56e0\u4f7f\u7528\u6700\u65b0\u9a45\u52d5\u7a0b\u5f0f\u548c\u6700\u4f73\u5316\u800c\u6709\u6240\u4e0d\u540c\u3002<br>\u8a3b3\uff1aMI325-006\uff1a\u57fa\u65bcAMD\u6548\u80fd\u5be6\u9a57\u5ba4\u65bc2024\u5e749\u670828\u65e5\u5b8c\u6210\u7684\u6e2c\u8a66\uff0c\u8a72\u6e2c\u8a66\u4f7f\u7528FP8\u8cc7\u6599\u985e\u578b\u6e2c\u91cfLLaMA 3.1-70B\u6a21\u578b\u7684\u7e3d\u9ad4\u5ef6\u9072\u3002\u4f7f\u75282048\u500btoken\u7684\u8f38\u5165\u9577\u5ea6\u548c2048\u500btoken\u7684\u8f38\u51fa\u9577\u5ea6\u5c0dAMD Instinct\u2122 MI325X GPU\u52a0\u901f\u5668\u548cNVIDIA H200 SXM GPU\u52a0\u901f\u5668\u7684\u4ee5\u4e0b\u914d\u7f6e\u9032\u884c\u6e2c\u8a66\u3002<br>1x MI325X\u57281000\u74e6\u7684vLLM\u6548\u80fd\uff1a48.025\u79d2\u5ef6\u9072\uff08\u4ee5\u79d2\u70ba\u55ae\u4f4d\uff09\u5c0d\u6bd41x H200\u5728700\u74e6\u53caTensorRT-LLM\uff1a62.688\u79d2\u5ef6\u9072\uff08\u4ee5\u79d2\u70ba\u55ae\u4f4d\uff09\u3002<br>\u914d\u7f6e\uff1a<br>AMD Instinct\u2122 MI325X\u53c3\u8003\u5e73\u53f0\uff1a1x AMD Ryzen\u2122 9 7950X 16\u6838\u5fc3\u8655\u7406\u5668\u30011x AMD Instinct MI325X (256GiB, 1000\u74e6) GPU\u3001Ubuntu\u00ae 22.04\u3001and ROCm\u2122 6.3\u00a0 pre-release\uff1b<br>\u5c0d\u6bd4NVIDIA H200 HGX\u5e73\u53f0\uff1a\u7f8e\u8d85\u5fae\uff08Supermicro\uff09SuperServer\u642d\u8f092x Intel Xeon\u00ae Platinum 8468\u8655\u7406\u5668\u30018x Nvidia H200 (140GB, 700\u74e6) GPU\u3001Ubuntu 22.04\u3001CUDA 12.6\u3002<br>\u4f3a\u670d\u5668\u88fd\u9020\u5546\u53ef\u80fd\u6703\u6539\u8b8a\u914d\u7f6e\uff0c\u7522\u751f\u4e0d\u540c\u7684\u7d50\u679c\u3002\u6548\u80fd\u53ef\u80fd\u6703\u56e0\u4f7f\u7528\u6700\u65b0\u9a45\u52d5\u7a0b\u5f0f\u548c\u6700\u4f73\u5316\u800c\u6709\u6240\u4e0d\u540c\u3002<br>\u8a3b4\uff1a\u00a0MI325-004\uff1a\u57fa\u65bcAMD\u6548\u80fd\u5be6\u9a57\u5ba4\u65bc2024\u5e749\u670828\u65e5\u5b8c\u6210\u7684\u6e2c\u8a66\uff0c\u4f7f\u7528FP16\u8cc7\u6599\u985e\u578b\u6e2c\u91cfMixtral-8x7B\u6a21\u578b\u7684\u6587\u5b57\u7522\u751f\u541e\u5410\u91cf\u3002\u4f7f\u7528128\u500btoken\u7684\u8f38\u5165\u9577\u5ea6\u548c4096\u500btoken\u7684\u8f38\u51fa\u9577\u5ea6\u5c0dAMD Instinct\u2122 MI325X GPU\u52a0\u901f\u5668\u548cNVIDIA H200 SXM GPU\u52a0\u901f\u5668\u7684\u4ee5\u4e0b\u914d\u7f6e\u9032\u884c\u6e2c\u8a66\u3002<br>1x MI325X\u57281000\u74e6\u7684vLLM\u6548\u80fd\uff1a\u6bcf\u79d24598\u8f38\u51fatoken\u5c0d\u6bd41x H200\u5728700\u74e6\u53caTensorRT-LLM\uff1a\u6bcf\u79d22700.7\u8f38\u51fatoken\u3002<br>\u914d\u7f6e\uff1a<br>AMD Instinct\u2122 MI325X\u53c3\u8003\u5e73\u53f0\uff1a1x AMD Ryzen\u2122 9 7950X\u8655\u7406\u5668\u30011x AMD Instinct MI325X (256GiB, 1000\u74e6) GPU\u3001Ubuntu\u00ae 22.04\u3001and ROCm\u2122 6.3 pre-release\uff1b<br>\u5c0d\u6bd4NVIDIA H200 HGX\u5e73\u53f0\uff1a\u7f8e\u8d85\u5fae\uff08Supermicro\uff09SuperServer\u642d\u8f092x Intel Xeon\u00ae Platinum 8468\u8655\u7406\u5668\u30018x Nvidia H200 (140GB, 700\u74e6) GPU [\u6e2c\u8a66\u4e2d\u53ea\u9069\u75281 GPU]\u3001Ubuntu 22.04\u3001CUDA\u00ae 12.6\u3002<br>\u4f3a\u670d\u5668\u88fd\u9020\u5546\u53ef\u80fd\u6703\u6539\u8b8a\u914d\u7f6e\uff0c\u7522\u751f\u4e0d\u540c\u7684\u7d50\u679c\u3002\u6548\u80fd\u53ef\u80fd\u6703\u56e0\u4f7f\u7528\u6700\u65b0\u9a45\u52d5\u7a0b\u5f0f\u548c\u6700\u4f73\u5316\u800c\u6709\u6240\u4e0d\u540c\u3002<br>\u8a3b5\uff1aCDNA4-03\uff1a\u622a\u81f32024\u5e745\u670831\u65e5\u7684\u63a8\u8ad6\u6548\u80fd\u9810\u6e2c\uff0c\u4f7f\u7528\u57fa\u65bc\u672a\u4f86AMD CDNA 4\u67b6\u69cb\u7684Instinct MI350\u7cfb\u5217\u52a0\u901f\u5668\u7684\u8a2d\u8a08\u5de5\u7a0b\u4f30\u7b97\u4f5c\u70ba\u9810\u8a08AMD CDNA\u2122 4\u6548\u80fd\u3002\u8a55\u4f301.8T GPT MoE\u6a21\u578b\u6642\u5047\u8a2dtoken-to-token\u5ef6\u9072= 70\u6beb\u79d2\u5be6\u6642\uff0c\u7b2c\u4e00\u500btoken\u5ef6\u9072=5\u79d2\uff0c\u8f38\u5165\u5e8f\u5217\u9577\u5ea6=8k\uff0c\u8f38\u51fa\u5e8f\u5217\u9577\u5ea6=256\uff0c\u5047\u8a2d4&#215;8\u6a21\u5f0fMI350\u7cfb\u5217\uff08CDNA 4\uff09\u82078x MI300X\u6bcfGPU\u6548\u80fd\u6bd4\u8f03\u3002\u5be6\u969b\u6548\u80fd\u5c07\u6839\u64da\u591a\u7a2e\u56e0\u7d20\u800c\u6709\u6240\u4e0d\u540c\uff0c\u5305\u62ec\u4f46\u4e0d\u9650\u65bc\u751f\u7522\u6676\u7247\u7684\u6700\u7d42\u898f\u683c\u3001\u7cfb\u7d71\u914d\u7f6e\u4ee5\u53ca\u6240\u4f7f\u7528\u7684\u63a8\u8ad6\u6a21\u578b\u548c\u5c3a\u5bf8\u3002<br>\u8a3b6\uff1aMI300-62\uff1a\u7531AMD\u6548\u80fd\u5be6\u9a57\u5ba4\u622a\u81f32024\u5e749\u670829\u65e5\u9032\u884c\u7684\u6e2c\u8a66\uff0c\u5728\u914d\u50998\u500bAMD Instinct\u2122 MI300X GPU\u4e26\u642d\u914dLlama 3.1-8B\u3001Llama 3.1-70B\u3001\u00a0Mixtral-8x7B\u3001Mixtral-8x22B Qwen 72B\u6a21\u578b\u3002<br>\u63a1\u7528vLLM 0.5.5\u7684ROCm 6.2\u6548\u80fd\u8207\u63a1\u7528vLLM 0.3.3\u7684ROCm 6.0\u6548\u80fd\u9032\u884c\u6bd4\u8f03\uff0c\u4e26\u57281\u81f3256\u7684\u6279\u6b21\u5927\u5c0f\u548c128\u81f32048\u7684\u5e8f\u5217\u9577\u5ea6\u4e0a\u9032\u884c\u6e2c\u8a66\u3002<br>\u914d\u7f6e\uff1a1P AMD EPYC\u2122 9534 CPU\u4f3a\u670d\u5668\uff0c\u914d\u50998\u500bAMD Instinct\u2122 MI300X (192GB\uff0c750\u74e6)GPU\u3001Supermicro AS-8125GS-TNMR2\u3001NPS1\uff08\u6bcf\u63d2\u69fd1\u500bNUMA\uff09\u30011.5 TiB 24 DIMMs\u30014800 mts\u8a18\u61b6\u9ad4\u300164 GiB\/DIMM\u30014x 3.49TB\u7f8e\u51497450\u5132\u5b58\u3001BIOS\u7248\u672c1.8\u3001ROCm 6.2.0-00\u3001vLLM 0.5.5\u3001PyTorch 2.4.0\u3001Ubuntu\u00ae 22.04 LTS\u4ee5\u53caLinux Kernel 5.15.0-119-generic\u3002<br>\u5c0d\u6bd41P AMD EPYC 9534 CPU\u4f3a\u670d\u5668\uff0c\u914d\u50998\u500bAMD Instinct\u2122 MI300X (192GB\uff0c750\u74e6)GPU\u3001Supermicro AS-8125GS-TNMR2\u3001NPS1\uff08\u6bcf\u63d2\u69fd1\u500bNUMA\uff09\u30011.5TiB 24 DIMMs\u30014800 mts\u8a18\u61b6\u9ad4\u300164 GiB\/DIMM\u30014x 3.49TB\u7f8e\u51497450\u5132\u5b58\u3001BIOS\u7248\u672c1.8\u3001ROCm 6.0.0-00\u3001vLLM 0.3.3\u3001PyTorch 2.1.1\u3001Ubuntu 22.04 LTS\u4ee5\u53caLinux Kernel 5.15.0-119-generic\u3002<br>\u4f3a\u670d\u5668\u88fd\u9020\u5546\u53ef\u80fd\u6703\u6539\u8b8a\u914d\u7f6e\uff0c\u5f9e\u800c\u7522\u751f\u4e0d\u540c\u7684\u7d50\u679c\u3002\u6548\u80fd\u53ef\u80fd\u6703\u56e0\u5404\u7a2e\u56e0\u7d20\u800c\u6709\u6240\u4e0d\u540c\uff0c\u5305\u62ec\u4f46\u4e0d\u9650\u65bc\u4e0d\u540c\u7248\u672c\u7684\u914d\u7f6e\u3001vLLM\u548c\u9a45\u52d5\u7a0b\u5f0f\u3002<br>\u8a3b7\uff1aMI300-61\uff1aAMD AI\u7522\u54c1\u7ba1\u7406\u5718\u968a\u5728AMD Instinct\u2122 MI300X GPU\u4e0a\u9032\u884c\u7684\u6e2c\u91cf\uff0c\u7528\u65bc\u6bd4\u8f03LLM\u6548\u80fd\u8207\u622a\u81f32024\u5e749\u670828\u65e5\u5728Llama 3.1-70B\u548cLlama 3.1-405B\u548cvLLM 0.5.5\u4e0a\u555f\u7528\u8207\u95dc\u9589\u6700\u4f73\u5316\u65b9\u6cd5\u7684\u5dee\u7570\u3002<br>\u7cfb\u7d71\u914d\u7f6e\uff1aAMD EPYC 9654 96\u6838\u5fc3\u8655\u7406\u5668\u30018\u500bAMD MI300X\u3001ROCm\u2122 6.1\u3001Linux\u00ae 7ee7e017abe3 5.15.0-116-generic #126-Ubuntu\u00ae SMP Mon Jul 1 10:14:24 UTC 2024 x86_64 x86_64 x86_64 GNU\/Linux\uff0c\u983b\u7387\u63d0\u5347\uff1a\u555f\u7528\u3002<br>\u6548\u80fd\u53ef\u80fd\u56e0\u5404\u56e0\u7d20\u800c\u7570\uff0c\u5305\u62ec\u4f46\u4e0d\u9650\u65bc\u4e0d\u540c\u7248\u672c\u7684\u914d\u7f6e\u3001vLLM\u548c\u9a45\u52d5\u7a0b\u5f0f\u3002<\/sup><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<h4 class=\"wp-block-heading has-text-align-right has-very-light-gray-to-cyan-bluish-gray-gradient-background has-background\">\ud83d\udfe6<strong>\u73fe\u5728\u5c31\u52a0\u5165&nbsp;<a href=\"https:\/\/www.facebook.com\/profile.php?id=100086628162118\" target=\"_blank\" rel=\"noreferrer noopener\">ioioTIMES \u81c9\u66f8\u7c89\u7d72\u5718<\/a>&nbsp;\u66f4\u591a\u4e92\u52d5\u3001\u66f4\u591a\u597d\u5eb7\u650f\u62b5\u52a0!!<\/strong><br>\ud83d\udfe6<strong>\u6211\u5011\u6709<a href=\"https:\/\/today.line.me\/tw\/v2\/publisher\/103117\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">LINE TODAY<\/a>\u983b\u9053\u4e86\uff0c\u5feb\u4f86\u8ffd\u8e2a\u6211\u5011\u5427!!&#8211;\u6700\u65b0\u79d1\u6280\u65b0\u805e \u76e1\u5728\u4f60\u624b<\/strong><\/h4>\n","protected":false},"excerpt":{"rendered":"<p>AMD\uff08NASDAQ: AMD\uff09\u5ba3\u5e03<\/p>\n","protected":false},"author":3,"featured_media":85234,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[13],"tags":[181,11367,11366,580,11365],"class_list":["post-85204","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","tag-amd","tag-amd-pensando-pollara-400","tag-amd-pensando-salina-dpu","tag-focus","tag-instinct-mi325x-2"],"_links":{"self":[{"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/posts\/85204"}],"collection":[{"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=85204"}],"version-history":[{"count":4,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/posts\/85204\/revisions"}],"predecessor-version":[{"id":85238,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/posts\/85204\/revisions\/85238"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=\/wp\/v2\/media\/85234"}],"wp:attachment":[{"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=85204"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=85204"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.ioiotimes.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=85204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}