#nvidianvlink

[govindhtech]

NVIDIA NVLink Revolutionizing GPU-Accelerated Computing

For GPU and CPU processors in accelerated systems, NVLink is a high-speed link that drives data and computations to useful outcomes. Once limited to high-performance computers in government research facilities, accelerated computing has become widely available.

AI supercomputers are being used by banks, automakers, factories, hospitals, merchants, and others to handle the increasing amounts of data that they must process and comprehend.

These strong, effective systems are computing superhighways. On their lightning-fast trip to actionable answers, they transport calculations and data via parallel routes.

The resources along the route are CPU and GPU processors, and their onramps are quick connections. NVLink is the industry standard for accelerated computing interconnects.

What is NVLink?

A reliable software protocol creates the high-speed link between GPUs and CPUs known as NVLink, which usually runs on many wire pairs printed on a computer board. It enables lightning-fast data transmission and reception across processors from shared memory pools.Image Credit To NVIDIA

At speeds of up to 900 gigabytes per second (GB/s), NVLink, which is now in its fourth iteration, links host and accelerated processors.

The bandwidth of PCIe Gen 5, the link found in traditional x86 servers, is more than seven times that amount. Additionally, because NVLink uses just 1.3 picojoules per bit for data transfers, it has five times the energy efficiency of PCIe Gen 5.

History of NVLink

With each successive NVIDIA GPU architecture, NVLink has improved in tandem since its initial release as a GPU connection with the NVIDIA P100 GPU.Image Credit To NVIDIA

When NVLink first connected the GPUs and CPUs in two of the most potent supercomputers in the world, Summit and Sierra, in 2018, it gained significant attention in the field of high performance computing.

Installed in Oak Ridge and Lawrence Livermore National Laboratories, the systems are advancing science in areas like drug development and catastrophe forecasting, among others.

Bandwidth Doubles, Then Grows Again

The third-generation NVLink, which came with a dozen interconnects in each NVIDIA A100 Tensor Core GPU, increased the maximum capacity per GPU to 600GB/s in 2020.

AI supercomputers in cloud computing services, business data centers, and HPC laboratories worldwide are powered by the A100.

One NVIDIA H100 Tensor Core GPU now contains eighteen fourth-generation NVLink interconnects. Additionally, the technology has assumed a new, strategic function that will allow for the world’s most sophisticated CPUs and accelerators.

A Chip-to-Chip Link

A board-level connection called NVIDIA NVLink-C2C is used to combine two processors onto a single device, forming a superchip. For instance, the NVIDIA Grace CPU Superchip, a processor designed to provide energy-efficient performance for cloud, corporate, and HPC customers, combines two CPU chips to produce 144 Arm Neoverse V2 cores.

A Grace CPU and a Hopper GPU are additionally joined via NVIDIA NVLink-C2C to form the Grace Hopper Superchip. It combines accelerated computation for the most demanding AI and HPC tasks on a single chip.

One of the first to employ Grace Hopper will be Alps, an AI supercomputer slated for the Swiss National Computing Center. The high-performance supercomputer will tackle huge research challenges in domains ranging from quantum chemistry to astrophysics when it goes online later this year.Image Credit To NVIDIA

Additionally, Grace and Grace Hopper are excellent in reducing energy consumption in demanding cloud computing tasks.

The Grace Hopper processor, for instance, is perfect for recommender systems. In order to provide billions of users with trillions of results every day, these internet economic engines require quick and effective access to large amounts of data.

A potent system-on-chip for automakers that incorporates NVIDIA Hopper, Grace, and Ada Lovelace processors and uses NVLink. NVIDIA DRIVE Thor is a vehicle computer that integrates cognitive features including entertainment, automatic driving, parking, digital instrument cluster, and more into a single architecture.

LEGO Links of Computing

NVLink functions similarly to the socket that is imprinted onto a LEGO component. It serves as the foundation for constructing supersystems to handle the most challenging AI and HPC tasks.

For instance, an NVIDIA DGX system’s eight GPUs’ NVLinks exchange quick, direct connections using NVSwitch chips. When combined, they allow for an NVLink network in which each server’s GPU functions as a single system.

DGX workstations itself may be stacked into modular units of 32 servers to provide even higher performance, forming a strong, effective computing cluster.

Using an NVLink network within the DGX and an NVIDIA Quantum-2 switched InfiniBand fabric between them, users may integrate a modular block of 32 DGX devices into a single AI supercomputer. An NVIDIA DGX H100 SuperPOD, for instance, has 256 H100 GPUs to provide up to an exaflop of the best AI performance.

Users may access cloud-based AI supercomputers, like the one Microsoft Azure is constructing with tens of thousands of A100 and H100 GPUs, to achieve even higher performance. Some of the biggest generative AI models in the world are trained using this service by organizations like OpenAI.

Additionally, it serves as another illustration of the potential of accelerated computing.

Read more on Govidhtech.com

[govindhtech]

GB200 NVL72: Tool for Next-Gen AI and Scientific Discovery

NVIDIA GB200 NVL72

In a rack-scale configuration, GB200 NVL72 links 36 Grace CPUs with 72 Blackwell GPUs. With a 72-GPU NVLink domain that functions as a single huge GPU and provides 30X quicker real-time trillion-parameter LLM inference, the GB200 NVL72 is a liquid-cooled, rack-scale solution.

A crucial part of the NVIDIA GB200 NVL72 is the GB200 Grace Blackwell Superchip, which uses the NVIDIA NVLink-C2C interconnect to link two powerful NVIDIA Blackwell Tensor Core GPUs and an NVIDIA Grace CPU.

Instantaneous LLM Deduction

Together with fifth-generation NVIDIA NVLink, the GB200 NVL72 offers state-of-the-art features including a second-generation Transformer Engine that powers FP4 AI and provides 30X faster real-time LLM inference performance for trillion-parameter language models. A new generation of Tensor Cores, which bring new microscaling formats and offer high accuracy and higher throughput, makes this progress possible. Furthermore, the GB200 NVL72 overcomes communication obstacles by combining liquid cooling and NVLink to build a single, enormous 72-GPU rack.

Large-Scale Instruction

A faster second-generation Transformer Engine with FP8 precision is included in the GB200 NVL72, allowing for an amazing 4X faster training time for big language models at scale. The fifth-generation NVLink, which offers NVIDIA Magnum IOTM software, InfiniBand networking, and 1.8 terabytes per second (TB/s) of GPU-to-GPU connectivity, complements this innovation.

Infrastructure with Low Energy Use

Data centres with liquid-cooled GB200 NVL72 racks use less energy and have a smaller carbon footprint. Large NVLink domain architectures benefit from liquid cooling’s ability to boost computation density, minimise floor space consumption, and enable high-bandwidth, low-latency GPU connection. When compared to NVIDIA H100 air-cooled infrastructure, the GB200 uses less water and offers 25X higher performance at the same power.

Data Entry

For businesses, databases are essential for managing, processing, and evaluating massive amounts of data. GB200 leverages the NVIDIA Blackwell architecture’s high-bandwidth memory performance, NVLink-C2C, and dedicated decompression engines to expedite critical database queries by 18X as compared to CPU and provide a 5X better total cost of ownership.

Although there are many advantages, there can be significant computational and resource costs associated with training and deploying big models. Widespread implementation will depend on computationally, financially, and energy-efficient systems that are designed to provide real-time inference. One such system that is capable of the job is the new NVIDIA GB200 NVL72.

Let’s look at the Mixture of Experts (MoE) models as an example. By utilising pipeline and model parallelism, these models facilitate the training of thousands of GPUs while distributing the computational load across numerous specialists. increasing the effectiveness of the system.

But GPU clusters may be able to make the technical issue manageable thanks to a new level of parallel processing, high-speed memory, and high-performance connections. This is accomplished by the NVIDIA GB200 NVL72 rack-scale architecture, which NVIDIA describes in more detail in the post that follows.

GB200 NVIDIA NVL36 and NVL72

In NVLink domains, the GB200 supports 36 and 72 GPUs. Based on the NVLink Switch System and the MGX reference design, each rack houses 18 computing nodes. With 18 solitary GB200 compute nodes and 36 GPUs in a single rack, it is available in the GB200 NVL36 configuration. With 72 GPUs in one rack and 18 dual GB200 compute nodes, or 72 GPUs in two racks with 18 single GB200 compute nodes, is how the GB200 NVL72 is arranged.

For ease of use, the GB200 NVL72 tightly packs and links the GPUs using a copper cable cartridge. Additionally, it features a liquid cooling system design, which results in 25 times less energy and cost usage.

NVIDIA GB200 NVL72 Features

Architecture by Blackwell

With unmatched speed, efficiency, and scalability, the NVIDIA Blackwell architecture ushers in a new era of computing with revolutionary advances in accelerated computing.

NVIDIA Grace Processor

An innovative processor for AI, cloud, and HPC applications running in contemporary data centres is the NVIDIA Grace CPU. It offers exceptional speed and memory bandwidth at a 2X energy efficiency compared to the top server processors available today.

Fifth-Stage NVIDIA NVLink Technology

For exascale computing and trillion-parameter AI models to reach their full potential, quick, smooth communication between each GPU in a server cluster is necessary. A scale-up link, the fifth iteration of NVLink unlocks faster performance for trillion- and multi-trillion-parameter AI models.

Graphics Processing Unit

As the foundation for distributed AI model training and generative AI performance, the data center’s network is essential to the development and performance of AI. For the best possible application performance, NVIDIA Quantum-X800 InfiniBand, NVIDIA Spectrum-X800 Ethernet, and NVIDIA BlueField-3 DPUs provide effective scalability over hundreds or thousands of Blackwell GPUs.

Nvidia GB200 NVL72 price

The NVIDIA DGX GB200 NVL72 is priced accordingly, being a high-end device aimed at academic institutions and major enterprises. One estimate places the price of a fully loaded system with 72 GB200 Superchips at approximately $3 million USD.

Here’s the reason it costs so much:

Abundant processing power: Equipped with 72 GPUs, it achieves remarkable performance in the 1.44 exaFLOPs of FP4 exaflop range.

Advanced hardware: It includes a liquid cooling system, a unique NVLink switch system for high-speed networking, and 13.5 TB of HBM3e memory.

It is challenging to find a publicly published price for the DGX GB200 NVL72 due to its specialised market. The $3 million estimate, nevertheless, is a reasonable approximation.

Read more on Govindhtech.com