IBM Blue Waters Monstrous Bandwidth
The Blue Waters supercomputer being built by IBM is a game changer, especially in the area of bandwidth. It has 2+ million threads, 2PB of memory with 8PB/s of aggregate memory bandwidth, and over 32,000 2nd generation x16 PCIe slots with 640TB/s aggregate bandwidth.
I based this information on a article written by Timothy Prickett Morgan at The Register, and a little math about how the bandwidth in the switch/hub chip is dedicated. I'm sure there will be more to say when the system is actually installed in 2011.
| Communication Bandwidth | |||
| per Switch/Hub Chip | per drawer | per supernode | per Blue Waters (512 nodes) |
| 192 GB/s of bandwidth into each Power7 MCM (what IBM called a host connection) | |||
| 336 GB/s of connectivity to the seven other local nodes(MCMs) on the drawer | |||
| 240 GB/s of bandwidth between the nodes in a four-drawer supernode |
1920GB/s |
||
| 320 GB/s dedicated to linking nodes to remote nodes | 2560GB/s | 10240GB/s (10TB/s) | |
| total external inter-node (not including PCIe cards) | 4480GB/s | ||
| 40 GB/s of general purpose I/O bandwidth (PCIe) | 320GB/s | 1280GB/s | 655360GB/s (640TB/s) |
Empty spaces in the chart are values which shouldn't be aggregated
|
Thread Count |
Memory Bandwidth 128GB/s per chip 512GB/s per MCM 4TB/s per drawer 16TB/s per supernode 8PB/s |
| DIMM Count and Memory Size | ||
| Capacity | DIMMs | |
| per DIMM | 8GB | 1 |
| per MCM | 128GB | 16 |
| per drawer | 1TB | 128 |
| per supernode | 4TB | 512 |
| per Blue Waters | 2PB | 262144 |
|
PCIe |
Leave your comment