How Does the WSE Process Multiple Users at Once?

[𝗔𝗜] How does the WSE process multiple users at once?

Cerebras WSE handles multiple users with a 𝗽𝗶𝗽𝗲𝗹𝗶𝗻𝗲, not big batches.

• 𝗢𝗻𝗲 𝘁𝗼𝗸𝗲𝗻 𝗱𝗼𝗲𝘀𝗻’𝘁 𝘂𝘀𝗲 𝗮𝗹𝗹 𝘁𝗵𝗲 𝗯𝗮𝗻𝗱𝘄𝗶𝗱𝘁𝗵, so the chip fills the leftover capacity with other users.
• 𝗗𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝗹𝗮𝘆𝗲𝗿𝘀 𝗿𝘂𝗻 𝗶𝗻 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝘄𝗮𝗳𝗲𝗿 𝗿𝗲𝗴𝗶𝗼𝗻𝘀, so multiple requests can be in flight at once.
• 𝗨𝘀𝗲𝗿𝘀 𝗺𝗼𝘃𝗲 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗹𝗶𝗸𝗲 𝗰𝗮𝗿𝘀 𝗼𝗻 𝗮 𝗰𝗼𝗻𝘃𝗲𝘆𝗼𝗿 𝗯𝗲𝗹𝘁: one user in Layer 2, another right behind in Layer 1.
• 𝗥𝗲𝘀𝘂𝗹𝘁: the whole wafer stays busy, boosting throughput 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝘁𝗵𝗲 𝗲𝘅𝘁𝗿𝗮 𝗹𝗮𝘁𝗲𝗻𝗰𝘆 of GPU-style batching.

𝗧𝗵𝗲 𝗪𝗦𝗘 𝗸𝗲𝗲𝗽𝘀 𝗺𝗮𝗻𝘆 𝘂𝘀𝗲𝗿𝘀 𝗺𝗼𝘃𝗶𝗻𝗴 𝗮𝘁 𝗼𝗻𝗰𝗲 𝗯𝘆 𝘀𝘁𝗿𝗲𝗮𝗺𝗶𝗻𝗴 𝘁𝗵𝗲𝗺 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗹𝗮𝘆𝗲𝗿 𝗯𝘆 𝗹𝗮𝘆𝗲𝗿.