comment on this article

Waiting to light up the server

Cloud computing has had a dramatic effect on the cost of high-performance processing.

But there have been knock-on effects from an architecture that assumes workloads can hop around machines whenever spare capacity appears. One of the biggest is a spike in network traffic, with much of the communication shuttling back and forth inside data centres: “east-west” in data-centre parlance rather than the traditional “north-south” of server to client.

The rate of growth in interprocessor traffic is not slowing down and is beginning to have an effect on the way architects look at the multicore devices that go into server blades. Imec researcher Yoojin Ban, said at the VLSI Technology Symposium in Honolulu in mid-June that the throughput demands of multicore processes based on nanometre-scale nodes such as 7nm would demand an optical interconnect.

Ban claimed typical processor designs for data centres based on a 16nm finFET technology demands an I/O bandwidth of more than 10Tb/s. “Below 7nm we need even higher bandwidths: more than 50Tb/s over distances of 1m to 500m. We believe those can only be achieved with single-mode fibre-optic interconnect.”

Experimental machines are showing how fibre-optic interconnect can break apart and re-architect the concept of the server. HP’s The Machine takes the idea to its logical conclusion: make multiple racks work as one virtual server.

To get there, HP aims to make heavy use of photonic interconnects, claiming “photonics destroys distance”. It’s a claim that by not taking into account the speed of light does not scale far. No-one is going to attempt to spread a single server across multiple data centres for that reason. But at the scale of a single data centre, the argument makes more sense: once a photon has been launched into fibre it is far less susceptible to losses over distances of up to a few hundred metres than an electrical signal. Photonic interconnect promises lower power consumption, which is vital for data centre-type designs where their energy usage has become a major concern.

Amir Khosrowshahi, CTO of artificial intelligence (AI) at Intel, said at the Design Automation Conference (DAC) in San Francisco in June, optical networking will be vital for cloud-based machine learning. “The silicon-photonics group [at Intel] was surprised we wanted to talk to them about using their work in machine learning. But if you are a very large-scale operator like Google, you have one rack that is Xeons and another is TPUs [Tensorflow processing units]. These things have to communicate.”

Photonic interconnect means “you don’t necessarily have to put machine-learning IP into your Xeon”, Khosrowshahi added. “What’s most interesting is where the laser is bonded into the device. If it’s done at wafer level, it can really change the nature of your systems.”

Deployment remains a problem

Despite the attractions of photonic interconnect, the ability to deploy it remains a problem even two decades on from Bookham Technology’s forays into using micromachined silicon to cut assembly costs. At the device level, researchers are beginning to overcome silicon’s lack of a direct bandgap, which makes it hard to build electro-optical devices into wafers. Today, at institutions such as Imec, multi-wafer bonding provides the ability to integrate the optical components with silicon-based circuitry.

The integrated-optical devices presented at VLSI by Imec take advantage of silicon’s transparency at wavelengths used for optical communication to build modulators into the core wafer. One such component is an active filter. “You can turn the optical signal on and off with an electrical bias signal and use a heater to fix the filter in the sweetspot,” Ban said. This is combined with germanium photodetectors and silicon PIN photodiodes on a “silicon photonics” die on top of which is mounted an SoC based on GlobalFoundries’ 14nm finFET process. “We have a scaling path to terabytes per second per square millimetre,” Ban said, which will be helped by techniques such as wavelength division multiplexing.

Although wafer-level integration can give silicon the tools needed to talk directly to photonic networks, the interconnect fabric currently presents a bigger problem. One major issue is the way that using silicon imposes a severe constraint on the type of fibre that can be used. Multimode fibre is easier to attach as it requires less accuracy. But silicon can only be made transparent to the infrared wavelengths that are suitable for single-mode fibre, which places more stringent demands on assembly tolerances. Today’s single-mode optical connections need to be based on physical contact, which can lead to damaging amount of force, particularly for the high-density parallel connectors that will be need to maintain good airflow for cooling.

Even with easier assembly techniques, fibre management itself could become a major problem in the dense fabric of a data-centre network where each blade may have multiple fibres going to the top-of-rack switch, even when the optical connections are made from the front or backplanes.

Above: IPSR is working on ways to make it easier to bring photonics to the SoC. Here it is focused on making cable connections to the board

At the 2018 Optical Fibre Conference in Los Angeles, Chengliang Zhang, vice president of China Telecom’s Beijing research institute, showed how telecom operators already have a spaghetti-wiring problem with the reconfigurable optical add-drop multiplexers (ROADMs) used in telecom switching. “In some situations there are hundreds of connections. This is a total nightmare. As the ROADM gets popular, fibre connectivity needs to evolve. ”

Zhang looks forward to replacing the fibre connections with an optical backplane and PCBs able to carry the photons direct from SoCs to the rear connector.

A collaboration between the International Electronics Manufacturing Initiative (iNEMI) and the MIT Microphotonics Center may come up with answers to these interconnect problems. The Integrated Photonic Systems Roadmap (IPSR) is working on ways to make it easier to bring photonics to the SoC and make it far less manually intensive and completed its first phase earlier this summer.

The IPSR’s first two phases focus on making cable connections to the board, with the second phase just getting underway to explore how to make the connectors. The focus of the IPSR group is on expanded beam connectors, which uses tiny lenses to increase the size of the illuminated spot at the joining point almost tenfold to around 80µm. According to 3M senior staff scientist Terry Smith, a larger spot would be even easier to work with but the group is conscious of the need for size reductions to enable parallel connectors. The result is a lower mating force because the fibre does not have to make a solid joint with the connector optics, although it increases losses from internal reflections. The angular alignment also needs to be precise but the R&D team believes this can be handled in the manufacture of the connector itself.

The IPSR group expects to start work on phase two in September. It will develop prototype connectors for the front plane and those that can be mounted on an SoC. Phase three moves onto the more difficult problem of embedding optical waveguides into PCBs. Although some researchers have developed plastic waveguides, Smith says inorganic materials may be needed to support the longer wavelengths needed for silicon photonics.

“Ideally, we want a transceiver module that’s compatible with reflow. When it’s attached electrically, it’s also attached optically. It would eliminate all the labour that might be needed to make the connections,” Smith explained in an iNEMI online seminar. “It’s a very aggressive plan but it’s where optical interconnect needs to go.”

Chris Edwards

Related Downloads

Comment on this article

This material is protected by MA Business copyright See Terms and Conditions. One-off usage is permitted but bulk copying is not. For multiple copies contact the sales team.

What you think about this article:

Add your comments


Your comments/feedback may be edited prior to publishing. Not all entries will be published.
Please view our Terms and Conditions before leaving a comment.

Related Articles