According to CoinMeta, SemiAnalysis has released a report that dissects the underlying architecture of large model inference services. The report suggests that as MOE models become mainstream, AI inference has evolved from a single computational task into a complex pipeline consisting of pre-filling, intermediate filling, attention decoding, and expert decoding. There are significant differences in the requirements for computing power, memory bandwidth, and network at different stages. In most inference scenarios, memory bandwidth is more economically valuable than capacity. High-bandwidth memory can improve token generation efficiency, while idle HBM only increases costs. The report estimates that by 2027, a single pipeline stage may require approximately 400 to 500GB of local fast memory, but the processed KV cache should be promptly migrated to CPU DRAM and lower-cost network storage to avoid occupying scarce HBM resources. The scheduling layer will become a key component of AI inference infrastructure. The report also compares aggregate and separate solutions, concluding that the choice between the two architectures ultimately depends on whether a future generation of accelerators that combine high computing power and high memory bandwidth will emerge.