Introduction
A modern GPU may contain thousands of computing units. But how do those units work together to create a three-dimensional image, render a game frame, process video, or run an artificial intelligence model?
In the previous article, we explained why CPUs and GPUs are built around different priorities. A CPU is designed for flexible control and complex decisions, while a GPU is designed to process enormous numbers of similar calculations at the same time.
This article follows one frame from the CPU to the monitor. Along the way, we will look inside the GPU and explain threads, warps, shaders, rasterization, video memory, tensor cores, and ray-tracing hardware.
By the end, the GPU should no longer look like a mysterious box filled with thousands of unrelated cores. It should look like a carefully organized parallel machine.
What Is a GPU?
GPU stands for graphics processing unit.
Graphics processors were created to accelerate the drawing of images. Instead of asking the CPU to calculate every line, triangle, color, and pixel, the computer could send graphics work to specialized hardware.
Over time, the GPU became programmable. Developers could write small programs called shaders to control how vertices, pixels, lighting, and visual effects were processed.
Programmers then discovered that the same hardware could accelerate scientific computing, simulation, video processing, and artificial intelligence.
The modern GPU is therefore more than a graphics chip. It is a parallel processor that performs the same mathematical operations across large amounts of data.
Why Graphics Are Parallel
A three-dimensional scene may contain millions of vertices, triangles, texture samples, and pixels.
Many of these elements can be processed independently.
The position of one vertex can be calculated at the same time as thousands of other vertices. The color of one pixel can be calculated alongside millions of other pixels.
Graphics contains huge numbers of repeated mathematical operations. That is exactly the type of work a GPU is designed to accelerate.
Instead of using a few large and complex cores, the GPU uses many smaller arithmetic units and keeps them busy with large groups of similar tasks.
The Basic Structure of a GPU
When people say GPU, they may mean the graphics card or the processor chip itself.
A graphics card contains the GPU chip, video memory, power-delivery circuits, cooling hardware, display outputs, and usually a PCI Express connection.
Inside the chip are compute units, caches, memory controllers, geometry hardware, rasterization hardware, and display engines.
NVIDIA organizes much of its programmable work inside streaming multiprocessors. AMD uses structures called compute units, although the details vary by generation.
These parts cooperate as a pipeline. Some prepare geometry. Some calculate pixels. Some move data. Some prepare the completed frame for the monitor.
Streaming Multiprocessors and Compute Units
A streaming multiprocessor, or an AMD compute unit, is not one large core. It is a group of execution resources that processes many threads.
Inside are arithmetic units, registers, scheduling hardware, shared memory, and load and store units.
NVIDIA often calls many arithmetic lanes CUDA cores. But a CUDA core is not equivalent to a complete CPU core.
A CPU core has sophisticated control logic, large caches, branch prediction, and the ability to handle a complex instruction stream independently.
A CUDA core is closer to an arithmetic execution lane inside a larger parallel system. It depends on the surrounding streaming multiprocessor for scheduling, registers, memory access, and control.
This is why CUDA core counts and CPU core counts should not be compared directly.
Threads, Warps, and Wavefronts
The basic unit of GPU work is usually called a thread.
One thread might transform one vertex, calculate one pixel, or update one matrix element.
But threads are not normally scheduled one at a time. They are grouped together.
On NVIDIA hardware, a group is called a warp. On AMD hardware, a similar group is called a wavefront.
Threads in the group usually execute the same instruction at the same time, but on different data.
Threads are also organized into larger collections. CUDA calls them thread blocks. Other systems may call them work groups.
Threads inside the same block can often share fast local memory and cooperate.
The GPU launches huge numbers of threads and switches between ready groups to keep its arithmetic units busy.
The Graphics Rendering Pipeline
Now let us follow a frame through the rendering pipeline.
The CPU runs the game logic, processes input, updates physics, manages objects, and decides what should appear in the next frame.
It sends draw commands describing geometry, textures, materials, the camera, lights, and rendering state.
The first major programmable stage is vertex processing.
A vertex shader transforms each vertex from its position inside a model, into the three-dimensional world, and finally into screen space.
The GPU then assembles vertices into geometric primitives, usually triangles.
Next comes rasterization.
Rasterization determines which screen locations are covered by each triangle and creates candidate pixel fragments.
A pixel shader, also called a fragment shader, calculates their appearance. It may read textures, apply lighting, evaluate shadows, and combine material properties.
Depth and visibility tests reject hidden fragments. Visible results are written into the frame buffer.
The display engine reads the completed frame and sends it to the monitor.
This process may happen sixty, one hundred and forty-four, or even hundreds of times every second.
Vertices and Triangles
Most real-time three-dimensional graphics is built from triangles.
A character, vehicle, building, or landscape may appear smooth, but underneath it is a mesh containing many small triangles.
Triangles are useful because three points always define a flat plane. They are simple, predictable, and efficient for hardware to process.
Each corner is a vertex.
A vertex may contain position, color, texture coordinates, surface direction, and other information.
The vertex shader can rotate a model, move it through the world, and project its three-dimensional position onto the two-dimensional screen.
The GPU performs these transformations across large numbers of vertices in parallel.
Rasterization
A triangle is a mathematical shape. A monitor is a grid of pixels.
Rasterization converts the triangle into screen coverage.
The rasterizer determines which pixel locations fall inside the triangle or touch its edges. For every covered location, it creates a fragment.
A fragment is only a candidate contribution to a pixel. It may later be hidden, discarded, blended, or rejected by a depth test.
Rasterization also interpolates values across the triangle.
If the vertices contain different texture coordinates, colors, or surface directions, the rasterizer calculates intermediate values for the fragments between them.
The pixel shader then uses this information to calculate the surface appearance.
Shaders
Shaders are small programs that run on the GPU.
A vertex shader processes vertices and transforms positions.
A pixel shader processes fragments and calculates color, texture detail, lighting, and transparency.
A compute shader is more general. It can process arrays, particles, image filters, simulations, physics, or artificial intelligence workloads.
A very simple shader concept might be:
Read the position.
Apply a transformation matrix.
Read the texture.
Calculate the light level.
Write the final color.
Real shader code can become much more complex, but the main idea is simple.
The same program runs across many vertices or pixels, while each thread uses different input data.
Textures, Lighting, and Shadows
Geometry gives an object its shape. Textures and lighting give it detail.
A texture is an image or data map stored in memory. It can provide color, roughness, metallic properties, surface direction, transparency, and other information.
Lighting calculations consider the direction of the light, the direction of the surface, the camera position, and the material.
Shadows are often created with shadow maps. The scene is first rendered from the light’s point of view. The resulting depth data helps determine whether a surface point is visible to the light.
Reflections may use environment maps, screen-space methods, ray tracing, or a combination of techniques.
Transparency requires blending visible results, often in a carefully controlled order.
These effects require enormous numbers of repeated calculations, making them suitable for GPU processing.
Video Memory and Memory Bandwidth
A fast GPU is useless if it cannot receive data quickly enough.
The chip constantly reads and writes textures, geometry, frame buffers, depth buffers, shader data, and intermediate results.
This is why graphics cards use dedicated high-bandwidth memory.
Many cards use GDDR memory. High-end accelerators may use HBM, meaning high-bandwidth memory.
Memory connects to the GPU through a wide bus. Memory speed and bus width together determine bandwidth, or how much data can move every second.
Modern workloads may require hundreds of gigabytes, or even more than a terabyte, per second.
Registers, shared memory, and caches keep frequently used data close to the arithmetic units. Their purpose is to reduce expensive trips to external memory.
How the GPU Hides Memory Latency
High bandwidth does not eliminate latency. Requested data still takes time to arrive.
A CPU often reduces the delay with large caches, speculation, and out-of-order execution.
A GPU keeps many groups of threads ready.
When one warp or wavefront waits for memory, the scheduler issues instructions from another ready group.
The entire compute unit does not need to stop. It switches to other work.
This is called latency hiding.
A GPU program therefore launches far more threads than the number of physical arithmetic units. The extra threads create a large pool of work that can cover delays.
Branch Divergence
Parallel execution works best when threads in the same warp or wavefront follow the same instructions.
Imagine a shader with an if statement.
Pixels in bright areas follow one path. Pixels in dark areas follow another.
When both types are inside the same execution group, the GPU may process one path first and the other path afterward. Threads that do not belong to the active path wait.
This is called branch divergence.
Efficiency falls because some arithmetic lanes are temporarily inactive.
Modern GPUs handle branching better than earlier designs, but the main principle remains.
Regular and predictable workloads are easier to process efficiently than decision-heavy workloads.
Compute Shaders and General-Purpose GPU Computing
Once GPUs became programmable, developers started using them for more than graphics.
The same hardware could accelerate scientific simulations, image recognition, fluid dynamics, financial models, and large numerical problems.
Compute shaders made general parallel work possible inside graphics systems.
Platforms such as CUDA, OpenCL, and DirectCompute made the GPU available as a general-purpose accelerator.
The workload is divided among many threads. Each thread processes part of the data, and nearby threads may cooperate through shared memory.
The GPU works best when a problem contains a large amount of parallel work and relatively simple control flow.
Why GPUs Are Good at Artificial Intelligence
Modern artificial intelligence relies heavily on matrix multiplication.
A neural network repeatedly multiplies arrays of numbers, adds the results, and applies simple functions.
One essential operation is multiply and accumulate.
Multiply two values. Add the result to an accumulator. Repeat this operation enormous numbers of times.
This maps extremely well to GPU hardware.
Modern GPUs also include specialized matrix units, often called tensor cores or matrix cores.
Artificial intelligence can use lower-precision formats, such as sixteen-bit or eight-bit numbers. Lower precision reduces memory use and allows more operations to run at the same time.
For training and inference, the GPU combines massive parallelism, high memory bandwidth, mature software tools, and specialized matrix hardware.
That is why it became one of the most important processors in modern artificial intelligence.
Ray Tracing and Specialized Hardware
Modern GPUs contain specialized accelerators in addition to general shader units.
Ray-tracing hardware speeds up the search for intersections between rays and geometric objects. This supports more realistic reflections, shadows, and lighting.
Tensor cores accelerate matrix operations and can support artificial-intelligence image reconstruction.
Video encoders and decoders process formats such as H.264, H.265, and AV1 without using the main shader units for every step.
Optical-flow accelerators estimate motion between frames.
Display engines control resolution, timing, color output, and monitor connections.
Specialized hardware improves efficiency by sending certain operations to circuits designed specifically for them.
One Complete Frame: From CPU to Screen
Let us put the entire process together.
The user moves the mouse or presses a key.
The CPU receives the input.
The game engine updates movement, artificial intelligence, physics, animation, and the camera.
The CPU prepares draw commands.
The GPU reads those commands.
Vertex shaders transform the geometry.
The hardware assembles triangles.
The rasterizer converts triangles into fragments.
Pixel shaders calculate textures, lighting, shadows, reflections, and material appearance.
Depth testing removes hidden surfaces.
Blending combines transparent and visible results.
The completed colors are written into the frame buffer in video memory.
The display engine reads the frame.
The image travels through the display cable.
The monitor lights its pixels.
And the player sees the result.
All of this happens in a tiny fraction of a second. Before the viewer can think about the frame, the CPU and GPU are already preparing the next one.
Conclusion
A GPU is not simply a processor with thousands of cores.
It is a highly organized parallel machine designed to keep enormous numbers of arithmetic units busy.
It divides work into threads and groups them into warps or wavefronts.
It transforms vertices, assembles triangles, rasterizes geometry, runs shaders, reads textures, calculates lighting, and writes the frame buffer.
When one thread group waits for memory, the GPU switches to another group to hide the delay.
The same architecture that creates millions of pixels can also process scientific data, simulations, video, and artificial intelligence.
The GPU became powerful not because every computing unit is individually complex, but because the complete machine coordinates massive amounts of parallel work.
That is how thousands of GPU cores create every frame.
And that is why the GPU has become one of the most important processors in modern computing.
The next article in this GPU fundamentals series will examine CUDA cores, tensor cores, and ray-tracing cores, and explain what each type of hardware actually does.