Conflicting report on MI210/250 performance.
MosaicML updated its blog saying that MI250 is competitive with A100, achieving 80% of their performance. Training LLMs at Scale with AMD MI250 GPUs.
EmbeddedML just wrote that MI210 is competitive with A100 in LLM inferencing. High throughput LLM inference with vLLM and AMD: Achieving LLM inference parity with Nvidia
I know they are talking about 2 different workloads. MosaicML is comparing training performance and EmbeddedML is comparing inference. But the difference is just too big since MI250 uses 2x more silicon than MI210. Does anybody have any idea what is going on?