Cloud & High-Performance Computing Application Requirements for Advanced Packaging

Basic drivers

The variety and intensity of processing workloads for cloud, edge and local computing is continually increasing.  This places unprecedented demand on processors, communications and the electronics hardware systems that host these.

The major trends that have a critical impact on the electronics hardware include:

• Increasing processing density along with an increasing variety of processor architectures needed to solve more complex tasks.  Cloud computing has evolved from a single-size-fits-all general-purpose processors to graphical processor units to machine-learning training and inference engines.  The efficiency gains afforded by specialization is just one response to a voracious appetite for increased processing power.  The other response is to simply increase the density of compute power.

• The demand for communication throughput has similarly increased.  This occurs at all levels, from rack to board, from board to packaged device, from packaging to chip, from chip to chip.  Speed increases for long-distance wireless and wireline communication are simply a proxy for similar increases at all levels.

• The power consumption for these processing and communication loads is projected to increase dramatically.  This is a critical parameter due to the resulting increases in operational costs for users and in environmental costs due to carbon emissions.  Higher power consumption also implies higher heat dissipation and a knock-on effect on reliability.

• Design for security will come to the forefront of device architecture design. Concerns include prevention of (a) side-attacks through information leakage at the interposer or via on-board power regulators and (b) denial-of-service (DoS) and side-attacks through chiplet proximity.

Architecture Vision

The combined challenges of increased processing workloads, higher data throughput demands and spiralling energy impacts is driving the following architecture paradigms:

Parallelism. The continued march towards fine-grained parallelism challenges scalability in processor devices.  Memory coherence and interconnect bottlenecks will limit algorithmic performance.  Power efficiency at low workloads will also be a concern.

Process in memory.  Many emerging cloud applications have very heavy demands on memory bandwidth to and from the processor cores.  These include:

1. Deep neural networks (DNNs) and homomorphic encryption (HE) workloads in AI solutions

2. Transaction processing

3. Database applications

4. Search 

This is leading to architectures built around the “processing-in-memory” paradigm where memory and processing are co-designed into the same core. 

Increased use of 3D stacking.  Again, to minimize demands on interconnect and the associated power consumption, 3D stacking will be more extensively used.  However, this will be limited by the challenges of power distribution and heat extraction.

More radical architecture innovations.  For specialized applications, more radical architectures may prove successful.  Low-power, noise-tolerant applications may open up opportunities for analog AI accelerators based on analog multiply-and-accumulate (MAC) units. Others have proposed and prototyped large wafer-scale systems that may find niche uses.

Key Technical Attributes

How do these trends map to critical parameters in PCB and packaging design and their underlying technologies?

Processing

Here the main parameters are as follows:

• The choice of chip technology is a key determinant of processing density and even can impact the ease of implementation of the processor architecture.

• The limits of vertically stacking and the horizontal distribution of stacks or chips distributed horizontally are the other key parameters for the processing density.

• The choice of chip size impacts the efficacy of the workload partition and distribution.

See Table 1 below for some insights into how some of these parameters need to evolve over time.

Table 1: Key attribute needs related to processing for cloud and HPC applications.

Interconnect

The data throughput either into a packaged device or into a chip is a relatively simply function, i.e. the product of the per-lane speed and the number of pins/communication lanes.  The latter is inversely proportional to square of the pin pitch and, of course, directly proportional to the package or chip size. 

See Table 2 below for the expected evolution of these parameters.  Note that connector or bump size is generally about 50% of the connector pitch.  While minimum pitch distance represents a key bottleneck, increasing use of designs with mixed bump pitches also are challenging from a manufacturing perspective.

Table 2: Key attribute needs related to device connectivity for cloud and HPC applications.

It should be noted that at the package level, the desired processor capability no longer is the bottleneck, but the data communication requirements are.  This is because radical decreases in pin pitch are not anticipated and lane speeds are increasing at a relatively sedate pace as compared to demands on communication throughput.

Power

Total power consumption can be broken down into three different elements:

1. Power consumption of processing (including memory usage)

2. Power consumption of communications/interconnect

3. The power needed to drive the thermal management solution

The first two components depend directly on the PCB, packaging and die technologies.  The increases in both are driving changes in the power delivery architecture.

The third drives the choice of the overall thermal management architecture, which in turn has knock-on impacts on both PCBs and packaging.

Table 2: Key attribute needs related to power and thermal management for cloud and HPC applications.

References

1. Semiconductor Research Organization, MAPT: 2023 Interim Report, 2023.

Return to Packaging & Heterogeneous Integration Overview.