Smart Machines Need Smart Silicon

It seems like even the biggest hyperscale platform developers who have long touted software-defined architectures as the key to computing nirvana are starting to learn a cardinal rule of infrastructure: No matter how much you try to abstract it, basic hardware still matters.

A key example of this is Google’s Tensor Processing Unit (TPU), which the company designed specifically for machine learning and other crucial workflows that were starting to push the limits of available CPUs and GPUs. In fact, the company says that without the TPU, it was looking at doubling its data center footprint in order to support applications like voice recognition and image search. The TPU is custom-designed to work with the TensorFlow software library, generating results 15 to 30 times faster than state-of-the-art Intel Haswell or Nvidia K80 devices.

This may seem like a harbinger of bad times ahead for Intel and Nvidia, but the broader picture is a bit more muddled. At the moment, Google has no plans to license the architecture for general distribution or even to offer TPUs on the Google Cloud. Instead, they will go strictly toward differentiating Google services from those of rival service providers – at least for the time being. Nvidia, for one, says it welcomes the advances that Google has made, and notes that its own accelerators have made dramatic progress since TPUs were introduced in 2015, particularly the newer P40 and P100 devices based on the Pascal architecture.

But since machine learning is all about getting systems to understand and adapt to their work environments, things get really interesting when you combine CPUs, GPUs, TPUs and field programmable gate arrays (FPGAs). As InfoWorld’s Serdar Yegulalp notes, FPGAs can be reprogrammed on the fly, meaning that systems will not only be able to tailor data sets and algorithms to their liking, but silicon-level programming as well. This means designers can produce generic hardware for a wide variety of computing environments that will literally learn on the job and automatically conform to the tasks they are given. All that is needed is a robust set of ML-oriented FPGA development tools and the frameworks to incorporate them into broader ASIC environments.

Even the chips that power most of today’s mobile infrastructure are amping up their machine learning capabilities. ARM Holdings recently unveiled the DynamIQ microarchitecture designed to power ML and other capabilities in heterogeneous, multicore environments. According to All About Circuits, the system utilizes ARM’s long-standing “big.Little” configuration in which a core CPU trades workloads with multiple smaller satellite processors, all while accessing the same memory and appearing to operating systems as a single processor. For complex workloads, the setup can feature up to eight processors per cluster in any combination of big or little ones, with multiple clusters per device all optimized for different tasks and different levels of performance.

This ongoing interest in hardware may come as a surprise to enterprise executives who were led to believe that all things of importance will be handled by software going forward. But this is only true up to a point. Software-defined architectures can do some amazing things, even with complex applications like machine learning and artificial intelligence. But since successful outcomes will increasingly depend on minute advantages in performance and flexibility, the best way to optimize the data environment will be to build strong, albeit automated (or even autonomous), relationships between hardware and higher-level functionality.

To support your smart applications, then, you’re going to need smart silicon.

Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.