Unleashing Movie Grain Synthesis on Netflix and Enhancing Visuals for Thousands and thousands
Li-Heng Chen, Andrey Norkin, Liwei Guo, Zhi Li, Agata Opalach and Anush Moorthy
Image this: you’re watching a basic movie, and the refined dance of movie grain provides a layer of authenticity and nostalgia to each scene. This grain, fashioned from tiny particles through the movie’s growth, is greater than only a visible impact. It performs a key function in storytelling by enhancing the movie’s depth and contributing to its realism. Nevertheless, movie grain is as elusive as it’s lovely. Its random nature makes it notoriously tough to compress. Conventional compression algorithms battle to handle it, usually forcing a selection between preserving the grain and decreasing file measurement.
Within the digital age, noise stays a ubiquitous component in video content material. Digital camera sensor noise introduces its personal traits, whereas filmmakers usually add intentional grain throughout post-production to evoke temper or a classic really feel. These parts create a visually wealthy expertise that exams standard compression strategies.
We’re giving members globally a remodeled streaming expertise with the current rollout of AV1 Movie Grain Synthesis (FGS) streams. Whereas FGS has been a part of the AV1 commonplace since its inception, we solely enabled it for a restricted variety of titles throughout our preliminary launch of the AV1 codec in 2021. Now, we’re enabling this modern know-how at scale, leveraging it to protect the inventive integrity of movie grain whereas optimizing knowledge effectivity. On this weblog publish, we’ll discover how FGS revolutionizes video streaming and enhances your viewing expertise.
The AV1 Movie Grain Synthesis software fashions movie grain by two key elements, with mannequin parameters estimated earlier than the encoding of the denoised video:
Movie Grain Sample: an auto-regressive (AR) mannequin is used to copy the sample of movie grain. The important thing parameters are the AR coefficients, which could be estimated from the residual between the supply video and the denoised video, primarily capturing the noise. This mannequin captures the spatial correlation between the grain samples, guaranteeing that the noise traits of the unique content material are precisely preserved. By adjusting the auto-regressive coefficients {ai}, the mannequin can management the grain’s form, making it seem coarser or finer. With these coefficients, a 64×64 noise template is generated, as illustrated within the animation under. To assemble the noise layer throughout playback, random 32×32 patches are extracted from the 64×64 noise template and added to the decoded video.
Movie Grain Depth: a scaling perform is employed to manage the grain’s look beneath various lighting situations. This perform, estimated through the encoding course of, fashions the connection between pixel worth and noise depth utilizing a piecewise linear perform. This enables for exact changes to the grain energy based mostly on video brightness and colour. Consequently, the movie grain energy is tailored to the areas of the image, intently recreating the look of the unique video. The animation under demonstrates how the grain depth is adjusted by the scaling perform:
With these fashions specified by AV1 commonplace, the encoding course of first removes the movie grain from the video. The usual doesn’t mandate a selected methodology for this step, permitting customers to decide on their most popular denoiser. Following the denoising, the video is compressed, and the grain’s sample and depth are estimated and transmitted alongside the compressed video knowledge. Throughout playback, the movie grain is recreated and reintegrated into the video utilizing a block-based methodology. This method is optimized for client units, guaranteeing clean playback and high-quality visuals. For a extra detailed clarification, please discuss with the unique paper.
By combining these elements, the AV1 Movie Grain Synthesis software preserves the inventive integrity of movie grain whereas making the content material “simpler to compress” by denoising the supply video previous to encoding. This course of permits high-quality video streaming, even in content material with heavy grain, leading to vital bitrate financial savings and improved visible high quality.
In our pursuit of premium streaming high quality, enabling AV1 Movie Grain Synthesis has led to vital bitrate discount, permitting us to ship high-quality video with much less knowledge whereas preserving the inventive integrity of movie grain. Beneath, we showcase visible examples highlighting the improved high quality and decreased bitrate, utilizing a body from the Netflix title They Cloned Tyrone:
