Responding to my previous post on how Linear-Drive Pluggable Optics (LPO) and Linear Receiver Optics (LRO) can reduce power consumption in optical transceivers, there were several enquiries about Co-Packaged Optics (CPOs). This post will look specifically at CPOs.
CPO technology refers to the integration of optical transceivers directly into the ASIC package of the host chip, with fibres coming out of the package. Fibre optic cables can connect to these through specialized connectors. This is made possible by silicon photonics, which integrates all optical components of the transceiver, except the laser, into a single CMOS Photonic Integrated Circuit (PIC). The PIC is bonded to the Electronic Integrated Circuit (EIC), which contains the drivers and amplifiers of the transceiver and the host ASIC, using advanced packaging techniques.
Advantages of CPOs:
- The integrated transceiver can eliminate Digital Signal Processor (DSP) functions, relying instead on the host ASIC’s SerDes DSP, or the host ASIC might remove the SerDes entirely, connecting to the EIC using parallel die-to-die interfaces. Both techniques result in lower overall power consumption.
- With CPOs, the lossy electrical traces between the ASIC and the front panel optics are bypassed. Thus, the DSPs within the ASICs can be optimized further.
However, there are several challenges:
- Thermal management is complex due to the integration of heat-sensitive optical components near heat-generating ASIC dies.
- The complexity of the ASIC package and challenges in manufacturing and testing translates to much higher costs.
- As the switch radix increases, packing more CPOs inside the package becomes challenging.
This year, discussions at Hot Chips 2024 focused on the Optical Compute Interface (OCI), which refers to the use of CPOs inside GPU or CPU packages to build scale-up systems. This integration presents challenges, especially as GPUs often use 2.5D packaging for High Bandwidth Memory (HBM). Broadcom demonstrated this technology with a test chip that emulates a GPU and an HBM on a silicon interposer with a PIC / EIC mounted directly on the substrate.
At OFC 2024, Intel demonstrated a 4Tbps OCI chiplet with an integrated laser in the package, achieving data rates of 64x32Gbps (2Tbps each way). Conversely, Broadcom’s optical engine chiplet boasts a 6.4Tbps capability (64x100Gbps) with the laser positioned externally as a pluggable module, using fibre to transmit light for modulation inside the ASIC.
These proofs of concept are compelling and, when mature, could enable large-scale systems. But will this transition happen?
I recall Jensen Huang’s talk at GTC 2024, where he highlighted 20% power savings using 200Gbps copper cables (instead of optical cables) for the backplane of the NVL72 chassis to connect the Blackwell GPUs and NVLink switches. OCI may not be power- or cost-efficient for intra-rack links below 200Gbps speeds. However, it may be a good option for link speeds above 200Gbps and for systems spanning multiple racks, where cable lengths can exceed 7 to 8 metres. It will be interesting to see if Nvidia or AMD will adopt OCI in their GPUs or transition first to LPO / LROs, as those do not require complex ASIC packages.
Sharada Yeluri is a Senior Director of Engineering at Juniper Networks, where she is responsible for delivering the Express family of Silicon used in Juniper Networks‘ PTX series routers. She holds 12+ patents in the CPU and networking fields.
Originally posted on LinkedIn.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.