[8] ANGLE Metal BufferPool Integer Truncation
Severity: Medium | Component: ANGLE Metal backend — BufferPool | 4369d90
Rated Medium because the observable effect is silent offset truncation past 4GB leading to GPU-side OOB accesses, but exploitation requires triggering >4GB cumulative allocations in a single BufferPool from WebGL content — feasible on high-memory systems but not trivially reachable, and the corruption is confined to the Metal buffer's virtual address range.
Fix Metal backend integer overflow vulnerabilities in BufferPool. BufferPool uses uint32_t for offset tracking, causing silent truncation beyond 4GB. Change mNextAllocationOffset and mLastFlushOffset to size_t and remove truncating casts. Add ANGLE white box test to test BufferPool class.
Source/ThirdParty/ANGLE/src/libANGLE/renderer/metal/mtl_buffer_pool.h
Source/ThirdParty/ANGLE/src/libANGLE/renderer/metal/mtl_buffer_pool.mm
Source/ThirdParty/ANGLE/src/libANGLE/renderer/metal/mtl_command_buffer.h
Source/ThirdParty/ANGLE/src/libANGLE/renderer/metal/mtl_command_buffer.mm
Integer truncation from size_t to uint32_t in GPU buffer offset tracking, causing silent wraparound beyond the 4GB boundary.
Patch Details
The fix widens mNextAllocationOffset and mLastFlushOffset from uint32_t to size_t in BufferPool, then propagates the wider type through the entire offset pipeline: RenderCommandEncoderShaderStates::bufferOffsets array, all setBuffer/setVertexBuffer/setFragmentBuffer method signatures in RenderCommandEncoder and ComputeCommandEncoder, SetComputeOrVertexBuffer helpers, mLegalizedOffsetedUniformBuffers pair type in ProgramExecutableMtl, vertex array offset in VertexArrayMtl.mm, and the IntermediateCommandStream fetch types for serialized offset commands. All static_cast<uint32_t>(...) truncating casts on offset values are removed. A white-box test allocates near-4GB buffers and verifies the second allocation's offset is not truncated.
Background
ANGLE is WebKit's OpenGL ES to Metal translation layer on Apple platforms, used to implement WebGL. BufferPool is a sub-allocator that carves regions from large Metal buffers — each allocate() call bumps an internal offset (mNextAllocationOffset) forward, and the resulting offset is passed to Metal's setVertexBuffer:offset:atIndex: and similar APIs to tell the GPU where in the buffer to read/write shader data. Metal buffer offsets are NSUInteger (64-bit on modern Apple platforms). Integer truncation occurs when a wider value is stored into a narrower type — for uint32_t, any value above 0xFFFFFFFF (4 GB) silently loses its upper bits.
Analysis
The root cause was systemic: BufferPool tracked allocation offsets as uint32_t, and the entire downstream offset pipeline — command encoder state, command stream serialization, uniform buffer offset storage, vertex buffer offset locals — also used uint32_t. When cumulative allocations within a single buffer exceeded 4 GB, the offset silently wrapped via unsigned integer truncation. The truncated offset was then passed to Metal API calls, causing the GPU to bind buffer data at an incorrect position.
The original code even had a static_cast<size_t>(mNextAllocationOffset) at the output point, widening back to size_t after the internal truncation had already occurred — masking the data loss from downstream callers. This pattern of 'truncation by convention' — where an early architectural choice of uint32_t for offsets propagated through every consumer via explicit static_cast<uint32_t>() casts — illustrates how integer width bugs can become systemic.