The business functions of GPU-accelerated computing are set to increase significantly within the coming years. One of the fastest-growing trends is using generative AI for creating human-like text and all kinds of images.
Driving the explosion of market interest in generative AI are technologies similar to transformer fashions that bring AI to on an everyday basis functions, from conversational textual content to protein structure technology. Visualization and 3D computing are additionally quickly gaining interest, notably within the areas of business simulation and collaboration.
GPUs are poised to turn into a significant driver of efficiency and price financial savings for data analytics, enterprise intelligence, and machine learning, with the acceleration of core functions corresponding to Apache Spark. Finally, AI inference deployments at the edge characterize one of the quickest rising areas for enterprises, driven by the expansion of sensible areas and industrial automation.
A new technology of computing technologies designed to handle these more and more advanced compute demands is emerging. This includes new GPU architectures from NVIDIA, as properly new CPUs from AMD, Intel, and NVIDIA.
Global system manufacturers have created new systems that convey these collectively into highly effective computing platforms designed to address a full vary of accelerated computing workloads. These methods are NVIDIA-Certified to ensure the best efficiency, reliability, and scale for enterprise options, and can be found for buy right now. Visit the Qualified System Catalog to be taught more. This post describes a few of these new technologies, and discusses the finest way for enterprises to take benefit of them.
Accelerate generative AI and enormous language models
Optimized for coaching massive language models and for inference, NVIDIA HGX H100 servers perform up to 4x sooner for AI training and as much as 30x quicker for AI inference in comparability with the previous technology NVIDIA A100 Tensor Core GPUs.* The newest servers, which include the model new technology of CPUs, feature the best efficiency for AI and HPC, as detailed beneath.
* 4-way H100 GPUs with 268 TFLOPs FP64
* 8-way H100 GPUs with 31,664 TFLOPs FP8
* 3.6 TFLOPs FP16 with NVIDIA SHARP in-network compute
* Fourth-generation NVLink with 3x sooner all-reduce communications
* PCIe Gen5 end-to-end for higher information transfer charges from CPU to GPU to community
* three.35 TB/s reminiscence bandwidth per GPU
*Configuration: HGX A100 cluster: HDR IB network. HGX H100 cluster: NDR IB network, GPT-3 16 B 512 (batch 256), GPT-3 16 K (batch 512). All performance numbers are from the NVIDIA H100 GPU Architecture whitepaper.
Figure 1. The NVIDIA HGX H100 significantly outperforms the NVIDIA HGX A100 in real-time inference and coaching throughput in several configurationsAt the NVIDIA GTC 2023 keynote, NVIDIA announced the NVIDIA H100 NVL, an H100 PCIe product with dual connections for NVLink featuring ninety four GB of HBM3 reminiscence. It is ideally suited for large language fashions and delivers 12x the efficiency of NVIDIA HGX A100 for GPT-3.
NVIDIA H100 PCIe GPU configuration includes an NVIDIA AI Enterprise software suite subscription to streamline development and deployment of AI manufacturing workloads. It offers all the capabilities of NVIDIA H100 GPUs in just 350 watts of thermal design power (TDP). This configuration can optionally use the NVLink bridge for connecting as much as two GPUs at 600 GB/s of bandwidth, nearly 5x PCIe Gen5.
Well fitted to mainstream accelerated servers that go into commonplace racks offering lower energy per server, the NVIDIA H100 PCIe GPU offers great efficiency for functions that scale from one to 4 GPUs at a time, including AI inference and HPC purposes.
NVIDIA partners are shipping NVIDIA-Certified servers with H100 PCIe at present. Visit the Qualified System Catalog to study extra. Systems from other partners with each NVIDIA H100 PCIe and NVIDIA HGX H100 are anticipated to be NVIDIA-Certified later this year. Taken together, these new platforms allow enterprises to run the most recent AI and HPC applications with even higher performance and larger scale.
Energy-efficient performance for AI video and inference
The NVIDIA Ada Lovelace L4 Tensor Core GPU delivers common acceleration and vitality effectivity for video, AI, virtual workstations, and graphics purposes within the enterprise, in the cloud, and at the edge. And with the NVIDIA AI platform and full-stack strategy, the L4 GPU is optimized for video and inference at scale for a broad range of AI purposes to deliver one of the best in personalized experiences. To study extra, see Supercharging AI Video and AI Inference Performance with NVIDIA L4 GPUs.
As probably the most efficient NVIDIA accelerator for mainstream, servers equipped with the L4 GPU enable as much as 120x greater AI video efficiency over CPU solutions, while offering 2.7x extra generative AI performance. They present over 4x extra graphics performance compared to the previous era. The NVIDIA L4 GPU is flexible with an energy-efficient, single-slot, low-profile form factor, making it ideal for edge, cloud, and enterprise deployments.
Figure 2. NVIDIA L4 GPU boosts video and AI efficiency over the NVIDIA T4 Tensor Core GPUThe NVIDIA L4 GPU edge use case advantages from its video acceleration with hardware decoders and encoders plus its AI acceleration with Tensor Cores. These are useful in edge video analysis functions for smart cities, factory quality assurance, and retail advertising in good areas. The L4 GPU is uniquely designed to address necessities for AI in HPC edge sensor processing purposes. Its graphics and video efficiency supercharge visualization for scientific applications at the edge instrument.
The NVIDIA L4 GPU is out there in NVIDIA-Certified Systems from NVIDIA partners, together with Advantech, ASUS, Atos, Cisco, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo, QCT, and Supermicro in over one hundred distinctive server fashions.
Next-generation CPUs
Advances in CPU technologies complement the new NVIDIA GPUs. The newest era of CPUs includes the 4th Gen Intel Xeon Scalable processors, also called Sapphire Rapids, in addition to the 4th Generation AMD EPYC processors, also referred to as Genoa. These latest architectures have capabilities that allow enterprises to run the most recent AI functions with even better efficiency and larger scale. This includes high knowledge pace transfer across the system bus and higher data bandwidth from primary reminiscence.
The NVIDIA Grace Hopper Superchip, primarily based on Arm architecture, delivers glorious efficiency and vitality efficiency. Built for giant-scale AI and HPC, Grace Hopper Superchip features NVLink C2C to deliver a CPU plus GPU coherent memory model for accelerated AI.
NVIDIA-Certified Systems for accelerated computing
As every new era of technology brings added sophistication, the necessity for prevalidated solutions to streamline acquisition is bigger than ever. The NVIDIA-Certified Systems program was created specifically to answer this want.
NVIDIA-Certified Systems deliver together NVIDIA GPUs and NVIDIA high-speed, safe networking to methods from main NVIDIA companions in configurations validated for optimum efficiency, reliability, and scale for a various range of workloads.
The tests are based mostly on real-world knowledge and characterize the newest GPU-accelerated applications, including deep studying training with PyTorch and TensorFlow, HPC, information analytics with Apache Spark, and 3D computing with NVIDIA Omniverse.
The certification is constructed entirely on a container-based take a look at suite utilizing Kubernetes for orchestration, making certain that any certified system can be seamlessly integrated into trendy cloud native administration frameworks.
It is necessary to understand the distinction between qualification and NVIDIA certification. A qualified system has undergone thermal, mechanical, power, and signal integrity exams to make sure a specific NVIDIA GPU is totally functional in that server mannequin. A certified system has handed a set of checks to validate its efficiency for a variety of workloads categories, as well as for networking, safety, and administration options. These capabilities become crucial for any enterprise computing resolution.
If you wish to ensure that the system is both supported and optimally designed and configured, select a licensed system.
Enterprise-ready next-generation computing platforms
NVIDIA-Certified Systems from world producers with the model new technology of GPU and CPU technologies are available at present. Visit the Qualified Systems Catalog to see what models can be found from your most well-liked vendor.
