← All reports

WebCodecs `VideoDecoderConfig` rotation and flip (new feature)

Component: WebCore WebCodecs | d49592f

Source/WebCore/Modules/webcodecs/WebCodecsVideoDecoder.cpp

+ .rotation = parseVideoFrameRotation(config.rotation),
+ .isMirrored = config.flip
...
+ // Decoders should create frames with the orientation of the configuration, we override for decoders that do not know about orientation.
+ frame = frame->updateOrientation(rotation, isMirrored, VideoFrame::ShouldClone::No);
+ auto presentationSize = frame->presentationSize();
...
+ .displayWidth = static_cast<size_t>(frame->rotatedSize().width()),
+ .displayHeight = static_cast<size_t>(frame->rotatedSize().height()),

Source/WebKit/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in

+ CreateDecoder(..., struct std::optional<WebCore::PlatformVideoColorSpace> colorSpaceOverride, enum:uint16_t WebCore::VideoFrameRotation rotation, bool isMirrored) -> (bool success);

WebCodecs VideoDecoder lets web content configure a decoder and receive VideoFrame objects; in WebKit the decoder often lives in the GPU process, reached from LibWebRTCCodecs over IPC to LibWebRTCCodecsProxy, with frames returning as CVPixelBuffer-backed objects. This commit adds rotation and flip to the VideoDecoderConfig WebIDL (gated by WebCodecsVideoFrameRotationEnabled), plumbs them through LibWebRTCCodecs and the CreateDecoder IPC message into the libwebrtc VPX and GStreamer decoders, and overrides decoded frames' orientation just before exposure. The WebRTC path through LibWebRTCCodecs::createDecoder hardcodes None/false.

WebCodecs decoders now honor config orientation, flipping the WPT orientation tests from FAIL to PASS for AV1, H.264, H.265, VP8 and VP9. The change also adds web-controlled fields to the WebContent-to-GPU decoder-creation IPC.

Narrow: rotation travels as enum:uint16_t, so check how the GPU-process handler treats out-of-range values from a compromised WebContent process, and whether validation is consistent across local, remote, VPX and GStreamer paths. Inspect how 90/270 rotation makes rotatedSize() (display dimensions) diverge from presentationSize() (coded dimensions), and how that interacts with visible rects, canvas drawing, copyTo and texture upload, where width/height swaps can produce size mismatches. Wider: LibWebRTCCodecs::completedDecodingCV and LibWebRTCVPXInternalVideoDecoder::Decoded stamp stored rotation onto frames from decoder callback threads, so check lifetime and thread-safety of those stored values, and verify every caller of the widened createDecoder / createDecoderAndWaitUntilReady / createDecoderInternal signatures passes consistent values.