Writing About AI
Uvation
Reen Singh is an engineer and a technologist with a diverse background spanning software, hardware, aerospace, defense, and cybersecurity. As CTO at Uvation, he leverages his extensive experience to lead the company’s technological innovation and development.
The Nvidia RTX 5090, built on the next-gen Blackwell architecture, represents a significant leap from the RTX 4090’s Ada Lovelace architecture. This “tectonic shift” in design includes a dramatic increase in transistor count from 76.3 billion to 92 billion, and a substantial boost in CUDA cores from 16,384 to 21,760. These architectural enhancements translate directly into faster parallel processing and greater efficiency for AI workloads. Furthermore, the RTX 5090 introduces 5th Generation Tensor Cores, an upgrade from the 4th Generation in the RTX 4090, promising improved support for mixed-precision computing (like FP8 and FP16), better sparsity acceleration, and smarter matrix multiplication. The Blackwell architecture also incorporates “structured sparsity techniques” to reduce redundant calculations in deep learning models, leading to more efficient processing for tasks like transformer models and large language model (LLM) inference.
The RTX 5090 features groundbreaking memory advancements, being the first consumer GPU to utilise 32GB of GDDR7 VRAM on a 512-bit memory bus. This configuration delivers an astounding 1,792 GB/s of memory bandwidth, a 78% increase over the RTX 4090’s 24GB of GDDR6X VRAM and 384-bit bus, which provides 1,008 GB/s. This significant boost in bandwidth directly addresses data bottlenecks, leading to faster AI training and inference. GDDR7 offers higher signalling rates, improved power efficiency, and greater memory density than GDDR6X, effectively doubling the bandwidth. The larger 32GB VRAM capacity also allows for handling bigger AI models, high-resolution image generation, and complex computational graphs, facilitating larger batch sizes and reducing memory swapping during training. Nvidia has further optimised memory performance through fine-tuned compression algorithms and an expanded L2 cache to reduce VRAM fetch latency.
AI professionals can anticipate substantial performance gains with the RTX 5090. While the RTX 4090 offers 82–100 TFLOPS of FP32 performance, the RTX 5090 is projected to hit 120–140 TFLOPS, a significant increase that could redefine large-scale AI projects. This translates to faster training times, smoother inference, and enhanced efficiency, potentially saving days or weeks of compute time for models like GPT-4. Beyond raw TFLOPS, the 5th Generation Tensor Cores are expected to deliver substantial improvements in sparse matrix operations, crucial for transformer-based models, with projections suggesting up to 30% faster inference speeds over the RTX 4090. These improvements will benefit applications such as reinforcement learning, generative models, and real-time language processing, leading to smoother performance, reduced latency, and fewer computational bottlenecks.
The RTX 5090 has a higher Thermal Design Power (TDP) of 575W, an increase from the RTX 4090’s 450W. This higher power demand necessitates a more robust power supply unit (PSU), with a 1000W PSU recommended for the RTX 5090, compared to 850W for the RTX 4090. Despite this increased power consumption and performance, Nvidia has made a surprising design choice: the RTX 5090 Founders Edition is a sleeker 2-slot card, a reduction from the RTX 4090’s 3-slot configuration. This slimmer profile offers more flexibility for AI-focused workstations, making it easier to stack multiple GPUs for deep learning and machine learning tasks. This reduction in size is achieved through advanced cooling techniques, including enhanced vapor chambers, denser heatsink arrays, and optimised airflow.
Tensor Cores are specialized processing units within Nvidia GPUs designed to accelerate deep learning computations, particularly mixed-precision computing (such as FP8 and FP16) and matrix multiplications. They are a critical component for speeding up AI training and inference in frameworks like PyTorch, TensorFlow, and JAX. The RTX 5090, with its Blackwell architecture, features 5th Generation Tensor Cores, an upgrade from the 4th Generation in the RTX 4090. These new Tensor Cores are expected to bring improved FP8 support, better sparsity acceleration, and smarter matrix multiplication capabilities. This means researchers can achieve faster performance and complete more AI work in less time, fundamentally enhancing the efficiency and speed of AI model development and deployment.
Sparsity-aware acceleration is a key efficiency advancement introduced with Nvidia’s Blackwell architecture in the RTX 5090. Deep learning models often perform redundant calculations, wasting valuable compute cycles. Blackwell’s structured sparsity techniques aim to eliminate this waste without compromising accuracy. In practice, this means transformer models can process data faster, large language model (LLM) inference becomes more efficient, and real-time AI applications experience lower latency. This isn’t just about raw power; it’s about smarter computation, allowing the RTX 5090 to achieve a new era of AI efficiency by ensuring that computing resources are used more effectively.
The choice between the RTX 4090 and RTX 5090 depends on the priorities and specific use cases of the AI professional.
Essentially, the RTX 4090 is the “rock-solid performer for now,” while the RTX 5090 is positioned as the “launchpad for the future” of AI.
When deciding between the RTX 4090 and RTX 5090 for AI development, several trade-offs need to be considered:
We are writing frequenly. Don’t miss that.
