Discover Top Posts Tagged with #nvidiah100

How the New NVIDIA BMC Flaw Allows Remote Hackers to Overheat and Kill Your AI Supercomputer

Read the full report on -

CyberDudeBivash News delivers daily cybersecurity threat intel, CVE alerts, malware trends, and crypto security briefings.

#CyberDudeBivash #ThreatWire #NVIDIAH100 #BMCvulnerability #AIinfrastructure #HardwareHacking #ZeroTrust #DataCenterSecurity #SiliconForensics #CybersecurityExpert

Protein Embeddings For Biological Analysis With AMD GPUs

AMD GPUs Powering AI-Driven Biology: Reprogramming Discovery

Contribution from IPA Therapeutics

Part 2: Protein Embeddings Improve Biology Analysis

The second chapter of the benchmarking series compares AMD Instinct MI300X and NVIDIA H100 GPUs for drug development AI activities.

Part 1: NLP-based biomedical research knowledge extraction. It now examines the protein layer to determine how well both GPUs handle large-scale protein language models (pLLMs), which help understand structure, function, and mutational impacts.

These benchmarks were conducted by ImmunoPrecise Antibodies (IPA) and its AI subsidiary BioStrand, makers of LENSai, an AI-native platform that uses sequencing, structure, and functional reasoning to advance biologics discovery. Every test employed Vultr's high-performance cloud architecture for reproducible, side-by-side comparisons in a production environment.

HYFT biological fingerprinting by LENSai combines conserved sequence, structure, and function into one index. HYFTs were created to solve AI's incapacity to understand biological processes. HYFTs integrate biological reasoning into computational fabric to let AI models reason biologically rather than only calculate.

AMD currently studies protein embeddings to understand binding interactions, mutation consequences, and molecular function. These embeddings underpin structural models and therapeutic target prioritisation. It compares different ESM-2 models and examines 'anchored embeddings' in LENSai and HYFT to show AMD GPUs' performance in a sector where memory capacity and biological accuracy are critical.

Using functional and evolutionary information, Protein Language Models (pLLMs) decode amino acid sequences into manageable vectors, improving biological data processing.

They let researchers ask: How similar is this sequence to druggable targets? Which structure will this unnamed protein adopt? How might a mutation affect binding or function? Embeddings improve immunogenicity screening, antibody identification, and multi-omics interpretation by reducing data for machine learning algorithms.

Benchmarks for ESM-2 Protein Language Model

AMD's throughput and scaling advantages were examined using ESM-2 benchmarks with various model sizes:

AMD GPUs handled larger batches smoothly, reducing costs and improving throughput.

Combined Drug Discovery With HYFT/LENSai

LENSai uses "HYFT anchored embeddings," which pick residues inside conserved HYFT patterns to reduce noise and improve biological signal clarity.

Biologically significant HYFTs are conserved structure or function motifs. LENSai embeds HYFTs to decrease noise and focus on the sequence's most functionally informative parts.

HYFT embedding support:

Estimating structural mutation consequences precisely.

Finding conserved motifs.

Effective semantic search across treatment libraries.

Easy Transition to AMD Graphics Processors

AMD GPU protein embeddings use a single-line Dockerfile update to minimise disruption:

FROM ROCM6.3.1_ubuntu22.04_py3.10_pytorch rocm/pytorch

No major code modifications are needed.

Protein embeddings connect unprocessed sequences to function. The experiment demonstrates that AMD MI300X has the memory headroom and performance for the most advanced protein models.

AMD compares RFdiffusion, a generative model that can imagine and produce new proteins, in the final chapter of this series to push AI-driven design.

Conclusion

AMD MI300X GPUs revolutionise AI-driven biological research, says this article. The MI300X's enormous memory and bandwidth let researchers process complicated biological data faster, boosting genomics and drug discovery. These powerful GPUs enable computational biology AI models and analytics to advance.

#ProteinEmbeddings #AMDInstinctMI300XandNVIDIAH100 #proteinlanguagemodels #AMDInstinctMI300X #NVIDIAH100 #InstinctMI300XandNVIDIAH100 #technology #technews #technologynews #news #govindhtech

AMD Instinct MI300X Vs NVIDIA H100 Which Leads in Biology

NVIDIA H100 vs AMD Instinct MI300X

AMD Instinct GPUs Drive AI-Driven Biology's Future

Contribution from IPA Therapeutics

Part 1: Life Sciences NLP Embeds

This is the first of three articles comparing AMD Instinct MI300X GPUs versus NVIDIA H100 GPUs in drug discovery AI workloads. The benchmarks were performed by ImmunoPrecise Antibodies (IPA) and its AI subsidiary BioStrand, which collaborated with Vultr on the LENSai platform for AI-powered biologics discovery. Fast deployment and hardware design reproducibility were possible with Vultr's high-performance cloud infrastructure. It evaluated these GPUs for real-world medical development tasks like generative protein design and NLP-driven target discovery.

LENSai's HYFT technology biological fingerprinting method encodes conserved sequence, structure, and function into a single index. To overcome AI's inability to understand biological systems, HYFTs were created. HYFTs integrate biological reasoning into computational fabric to let AI models reason biologically rather than only calculate.

In three papers, AMD will test the MI300X GPUs in the LENSai tech stack: generative design using RFdiffusion, protein embedding generation for structure-function inference, and NLP-driven literature mining.

AMD used real-world NLP, protein embedding, and de novo protein design benchmarks to evaluate modern bioinformatics pipelines' raw performance, cost, and deployment practicality.

This first episode focusses on Natural Language Processing (NLP) and how Retrieval-Augmented Generation (RAG) and huge language models speed up early-stage therapy discovery by gleaning practical insights from scientific literature. Most significant lesson? AMD GPUs are cost-effective and fast, which is vital for life science organisations building AI-driven systems.

Natural Language Processing (NLP) efficiently mines enormous volumes of textual data, boosting therapeutic breakthroughs. NLP extracts hidden insights from genomic datasets, clinical reports, and scientific literature. Computational models, which emphasise safety, efficacy, and cost-effectiveness, are similar to NLP-driven large language models (LLMs), which ease drug research analysis and prediction.

Knowledge-aware models can shed light on RAG (Retrieval-Augmented Generation) vector embeddings using semantics rather than language. Since they support biological sequences and structures as well as text, these embeddings help NLP connect life science silos.

LENSai improves vector search by adding a powerful semantic layer that detects sub-sentence units and extracts subject-predicate-object triples to identify biological connections. By documenting molecular interactions between targets, pathways, and compounds, LENSai helps researchers map disease pathways, identify therapeutic targets, and better anticipate drug behaviour. This depth of understanding, often concealed in unstructured biological data, can be discovered and leveraged to hasten discovery and reduce risk and cost before wet lab research begin.

Infrastructure context

It used the AMD Instinct MI300X and NVIDIA H100 GPUs in a flexible, cloud-native environment to provide repeatable benchmarking and fair hardware generation comparisons.

Benchmark NLP Results

It retrieval-augmented generation (RAG) techniques use literature vector embeddings for contextual insights. AMD outperformed in cost and throughput:

MI300X also improved stability under high-concurrency workloads.

Technical Execution: Smooth AMD GPU Switch

Use ROCm PyTorch Docker images to migrate NLP jobs on AMD GPUs easily:

FROM ROCM6.3.1_ubuntu22.04_py3.10_pytorch rocm/pytorch

No Python code changes are needed. The PyTorch device abstraction (torch.device(“cuda”)) ensures compatibility.

These NLP benchmarks show that AMD Instinct MI300X GPUs benefit both technically and financially in one of the simplest AI-assisted drug discovery tiers.

#AMDInstinctMI300X #NVIDIAH100 #AMDInstinctMI300XVsNVIDIAH100 #MI300XVsNVIDIAH100 #AMDMI300XVsNVIDIAH100 #NVIDIAH100GPUs #technology #TechNews #technologynews #news #govindhtech

MLPerf Inference v4.1 For AMD Instinct MI300X Accelerators

Engineering Insights: Introducing AMD Instinct MI300X Accelerators’ MLPerf Results. The full-stack AMD inference platform demonstrated its prowess with the remarkable results AMD Instinct MI300X GPUs, powered by one of the most recent iterations of open-source ROCm, obtained in the MLPerf Inference v4.1 round.

LLaMA2-70B

The first submission concentrated on the well-known LLaMA2-70B type, which is renowned for its excellent performance and adaptability. By outperforming the NVIDIA H100 in Gen AI inference, it established a high standard for what AMD Instinct MI300X accelerators are capable of.

MLPerf Inference

Comprehending MLPerf and Its Relevance to the Industry

Efficient and economical performance is becoming more and more important for inference and training as large language models (LLMs) continue to grow in size and complexity. Robust parallel processing and an optimal software stack are necessary to achieve high-performance LLMs.

This is where the best benchmarking package in the business, MLPerf, comes into play. The open-source AI benchmarks known as MLPerf Inference, which were created by the cross-industry cooperation MLCommons, of which AMD is a founding member, include Gen AI, LLMs, and other models that give exacting, peer-reviewed criteria. Businesses are able to assess the efficacy of AI technology and software by using these benchmarks.

A major accomplishment for AMD, excelling in MLPerf Inference v4.1 demonstrates their dedication to openness and providing standardized data that enables businesses to make wise choices.

An Extensive Analysis of the LLaMA2-70B Benchmark

The AMD LLaMA2-70B model was utilized in their first MLPerf Inference. A major development in LLMs, the LLaMA2-70B model is essential for practical uses such as large-scale inference and natural language processing. A Q&A scenario using 24,576 samples from the OpenORCA dataset, each with up to 1,024 input and output tokens, was included in the MLPerf benchmarking test. Two situations were analyzed by the benchmark to assess inference performance:

In an offline scenario, queries are processed in batches to increase throughput in tokens per second.

Server Scenario: This model tests the hardware’s capacity to provide quick, responsive performance for low-latency workloads by simulating real-time queries with stringent latency limitations (TTFT* < 2s, TPOT* ≤ 200ms).

Performance of AMD Instinct MI300X in MLPerf

With four important entries for the LLaMA2-70B model, the AMD Instinct MI300X demonstrated remarkable performance in its first MLPerf Inference utilizing the Supermicro AS-8125GS-TNMR2 machine. These findings are especially noteworthy since they provide an apples-to-apples comparison with rival AI accelerators, are repeatable, vetted by peer review, and grounded in use cases that are relevant to the industry.

Combination Performance of CPU and GPU

Submission ID 4.1-0002: Two AMD EPYC 9374F (Genoa) CPUs paired with eight AMD Instinct MI300X accelerators in the Available category.

This setup demonstrated the potent synergy between 4th Gen EPYC CPUs (previously codenamed “Genoa”) and AMD Instinct MI300X GPU accelerators for AI workloads, providing performance within 2-3% of NVIDIA DGX H100 with 4th Gen Intel Xeon CPUs in both server and offline environments at FP8 precision.

Previewing Next-Generation CPU Performance

Submission ID 4.1-0070: Two AMD EPYC “Turin” CPUs and eight AMD Instinct MI300X CPUs in the Preview category.

It showcased the performance increases from the next AMD EPYC “Turin” 5th generation CPU when paired with AMD Instinct MI300X GPU accelerators. In the server scenario, it outperformed the NVIDIA DGX H100 with Intel Xeon by a small margin, and it maintained a similar level of performance even offline at FP8 precision.

LLaMA2-70B GPU

Efficiency of a Single GPU

Submission ID 4.1-0001: In the Available category, AMD Instinct MI300X accelerator with AMD EPYC 9374F 4th Gen CPUs (Genoa).

This submission emphasized the AMD Instinct MI300X’s enormous 192 GB memory, which allowed a single GPU to effectively execute the whole LLaMA2-70B model without requiring the network cost that comes with dividing the model over many GPUs at FP8 precision.

The AMD Instinct MI300X has 192 GB of HBM3 memory and a peak memory bandwidth of 5.3 TB/s thanks to its AMD CDNA 3 architecture. The AMD Instinct MI300X can execute and host a whole 70 billion parameter model, such as LLaMA2-70B, on a single GPU with ease because to its large capacity.

The findings in Figure 2 show that the scaling efficiency with the ROCm software stack is almost linear from 1x AMD Instinct MI300X (TP1) to 8x AMD Instinct MI300X (8x TP1), indicating that AMD Instinct MI300X can handle the biggest MLPerf inference model to date.

Outstanding Dell Server Architecture Outcomes Using AMD Instinct MI300X Processors

Submission ID 4.1-0022: Two Intel Xeon Platinum 8460Y+ processors and eight AMD Instinct MI300X accelerators in the Available category.

Along with AMD submissions, Dell used their PowerEdge XE9680 server and LLaMA2-70B to submit their findings, validating the platform-level performance of AMD Instinct accelerators on an 8x AMD Instinct MI300X arrangement. This proposal demonstrates their collaboration and emphasizes how strong it ecosystem is, making them a great option for deployments including both data centers and edge inference. Further information on such outcomes is available here.

Performance Of Engineering Insights

The AMD Instinct MI300X accelerators exhibit great competitive performance due to their high computational power, huge memory capacity with rapid bandwidth, and optimized ROCm software stack. The latter enables effective processing of large AI models such as LLaMA2-70B. A few important elements were pivotal:

Big GPU Memory Capacity

The AMD Instinct MI300X has the most GPU memory that is currently on the market, which enables the whole LLaMA2-70B model to fit into memory while still supporting KV cache. By avoiding model splitting among GPUs, this maximizes inference speed while avoiding network cost.

Batch Sizes: They set the max_num_seqs parameter to 2048 in the offline scenario to optimize throughput, and to 768 in the server scenario to achieve latency requirements. These values are much greater than the 256 default value used in vLLM.

Effective KV cache management is made possible by the vLLM’s paged attention support, which helps prevent memory fragmentation brought on by huge memory AMD Instinct MI300X accelerators.

FP8 Precision

AMD expanded support for the FP8 numerical format throughout the whole inference software stack, using the AMD Instinct MI300X accelerator hardware. They quantized the LLaMA2-70B model weights to FP8 using Quark while maintaining the 99.9% accuracy needed by MLPerf. To further improve speed, it improved the hipBLASLt library, introduced FP8 support to vLLM, and implemented FP8 KV caching.

Software Enhancements

Kernel Optimization: AMD Composable Kernels (CK) based prefill attention, FP8 decode paged attention, and fused kernels such residual add RMS Norm, SwiGLU with FP8 output scaling were among the many profiles and optimizations to carried out.

vLLM Enhancements: The scheduler was improved to optimize both offline and server use cases, allowing for quicker decoding scheduling and better prefill batching.

CPU Enhancement

While GPUs handle the majority of the AI task processing, CPU speed is still quite important. CPUs with fewer cores and higher peak frequencies such as the 32-core EPYC 9374F offer the best performance, particularly in server applications. Performance improvements over the 4th generation EPYC CPUs which were submitted as a preview were seen during testing with the forthcoming “Turin” generation of EPYC CPUs.

LLaMa 3.1 405B

Establishing a Standard for the Biggest Model

The AMD Instinct MI300X GPU accelerators have shown their performance in MLPerf Inference with LLaMA2-70B, and the positive outcomes set a solid precedent for their future efficacy with even bigger models, such as Llama 3.1. They are pleased to provide Day 0 support for AMD Instinct MI300X accelerators with Meta’s new LLaMa 3.1 405B parameter model.

Only a server driven by eight AMD Instinct MI300X GPU accelerators can fit the whole LLaMa 3.1 model, with 405 billion parameters, on a single server utilizing FP16 datatype MI300-7A, owing to the industry-leading memory capacities of the AMD Instinct MI300X platform MI300-25. This lowers expenses and lowers server use. The most ideal way to power the biggest open models on the market right now is with AMD Instinct MI300X accelerators.

Read more on govindhtech.com

#MLPerfInferencev41 #AMDInstinct #MI300XAccelerators #NVIDIAH100 #AMDInstinctMI300X #largelanguagemodels #LLM #news #4thGenEPYCCPU #LLaMa31model #Llama31 #technology #technews #IntelXeon #AMDCDNA #IntelXeonPlatinum #MI300Xaccelerators #govindhtech

Nvidia H100'den 30 kat daha zayıf, ancak 200-400 kat daha ucuz

Yapay zeka için en verimli hızlandırıcılar onbinlerce dolara mal olurken, Çinli Intellifusion şirketi çok ucuz olan çözümünü sundu. Deep Eyes adı verilen yapay zeka chipi, SoC DeepEdge10Max formunda sunuluyor ve müşterilere yalnızca 140 dolara mal olacak. Elbette bu fiyata çok mütevazı bir performanstan bahsediyoruz - 48 TOPS. Bu, modern Intel ve AMD işlemcilerden birkaç kat daha fazladır, ancak Snapdragon X Elite SoC, 75 TOPS'a kadar NPU performansına sahip olacaktır. Bu arada bir PC işlemcisinde bir dizi Copilot AI fonksiyonunu yerel olarak çalıştırmak için en az 40 TOPS performansa sahip bir NPU ünitesine ihtiyacınız olduğunu da hatırlatalım . Karşılaştırma için, aynı INT8 modundaki Nvidia H100'ün neredeyse 4000 TOPS performansı var, yani neredeyse 30 kat daha yüksek, ancak maliyeti 200-400 kat daha fazla. Bununla birlikte, böyle bir karşılaştırma yalnızca farklı yonga seviyelerinin gösterilmesi açısından anlamlıdır. Şirket ayrıca daha sonra 24 TOPS AI motorlu DeepEdge10Pro SoC'yu ve 96 TOPS performansına sahip DeepEdge10Ultra'yı piyasaya sürmeye hazırlanıyor. Tüm bu çözümler şirketin kendi teknolojileridir ve özel bir yapay zeka çipi Intellifusion NNP400T'ye dayanmaktadır. SoC konfigürasyonu 1,8 GHz frekansına sahip 10 çekirdekli RISC-V işlemciyi, 800 MHz frekansına sahip GPU'yu da içeriyor ve tüm bunlar 14 nm işlem teknolojisi kullanılarak üretilecek. Read the full article

#DeepEdge10Ultra #Intellifusion #IntellifusionNNP400 #NvidiaH100

NVIDIA Triton Speeds Oracle Cloud Inference!

Triton Speeds Inference Thomas Park is an enthusiastic rider who understands the need of having multiple gears to keep a quick and seamless ride.

Thus, the software architect chose NVIDIA Triton speeds Inference Server when creating an AI inference platform to provide predictions for Oracle Cloud Infrastructure’s (OCI) Vision AI service. This is due to its ability to swiftly and effectively handle almost any AI model, framework, hardware, and operating mode by shifting up, down, or sideways.

The NVIDIA AI inference platform, according to Park, a competitive cyclist and computer engineer based in Zurich who has worked for four of the biggest cloud service providers in the world, “gives our worldwide cloud services customers tremendous flexibility in how they build and run their AI applications.”

More specifically, for OCI Vision and Document Understanding Service models that were transferred to Triton, Triton speeds decreased inference latency by 51%, enhanced prediction throughput by 76%, and decreased OCI’s total cost of ownership by 10%. According to a blog post made earlier this year by Park and a colleague on Oracle, the services are available worldwide across more than 45 regional data centers.

Computer Vision Quickens Understanding For a wide range of object identification and image classification tasks, customers rely on OCI Vision AI. To avoid making busy truckers wait at toll booths, a U.S.-based transportation agency, for example, utilizes it to automatically determine the number of vehicle axles going by to calculate and bill bridge tolls.

Additionally, Oracle NetSuite a suite of business tools utilized by over 37,000 enterprises globally offers OCI AI. One application for it is in the automation of invoice recognition.

Park’s efforts have led to the adoption of Triton speeds by other OCI services as well.

A Data Service Aware of Triton Speeds Tzvi Keisar, a director of product management for OCI’s Data Science service, which manages machine learning for Oracle’s internal and external users, stated, “We’ve built a Triton-aware AI platform for our customers.”

“We will save customers time by automatically completing the configuration work in the background and launching a Triton-powered inference endpoint for them if they want to use Triton speeds ,” added Keisar.

Additionally, his team intends to facilitate the adoption of the quick and adaptable inference server by its other users even more. Triton speeds is part of NVIDIA AI Enterprise, an OCI Marketplace-available platform that offers all the security and support that enterprises require.

An Enormous SaaS Platform The machine learning foundation for NetSuite and Oracle Fusion software-as-a-service applications is provided by OCI’s Data Science service.

He claimed, “These platforms are enormous, with tens of thousands of users building their work on top of our service.”

A broad range of users, mostly from enterprises in the manufacturing, retail, transportation, and other sectors are included. They are creating and utilizing AI models in almost all sizes and shapes.

One of the group’s initial offerings was inference, and shortly after its launch, Triton speeds caught the team’s attention.

An Unmatched Inference Framework We began testing with Triton speeds after observing its rise in popularity as the best serving framework available, according to Keisar. “They observed very strong performance, and it filled a vacuum in their current offerings, particularly with regard to multi-model inference it’s the most sophisticated and adaptable inferencing framework available today.”

Since its March OCI launch, Triton speeds has drawn interest from numerous Oracle internal teams that want to use it for inference tasks requiring the simultaneous feeding of predictions from several AI models.

He said that Triton speeds performed exceptionally well on several models set up on a single endpoint.

Quickening the Future Going forward, Keisar’s group is testing the NVIDIA TensorRT-LLM program to accelerate inference on the intricate large language models (LLMs) that have piqued the interest of numerous users.

Keisar is a prolific blogger, and his most recent post described innovative quantization methods for using NVIDIA A10 Tensor Core GPUs to run a Llama 2 LLM with an astounding 70 billion parameters.

“The quality of model outputs is still quite good, even at four bits,” he stated. “They found a good balance, and he hasn’t seen anyone else do this yet, but he can’t explain all the math.”

This is just the beginning of more faster efforts to come, after announcements this fall that Oracle is installing the newest NVIDIA H100 Tensor Core GPUs, H200 GPUs, L40S GPUs, and Grace Hopper Superchips.

#nvidiah100

Trending Tags

Recently Viewed Tags

#nvidiah100