Entertainer.newsEntertainer.news
  • Home
  • Celebrity
  • Movies
  • Music
  • Web Series
  • Podcast
  • OTT
  • Television
  • Interviews
  • Awards

Subscribe to Updates

Get the latest Entertainment News and Updates from Entertainer News

What's Hot

Kiss Will Release Three New Box Sets in 2027

November 16, 2026

AMA/Q&A Announcement – Tony Gilroy – Monday 11/16 at 4 PM ET – Oscar-Nominated Filmmaker & Screenwriter – ‘Michael Clayton’, ‘Andor’, ‘Rogue One’, ‘Behemoth!’, ‘Bourne’ Franchise

November 16, 2026

‘Sister Wives’ Could Polygamy Be Dealbreaker Meri Brown’s Men?

November 16, 2026
Facebook Twitter Instagram
Monday, November 16
  • About us
  • Advertise with us
  • Submit Articles
  • Privacy Policy
  • Contact us
Facebook Twitter Tumblr LinkedIn
Entertainer.newsEntertainer.news
Subscribe Login
  • Home
  • Celebrity
  • Movies
  • Music
  • Web Series
  • Podcast
  • OTT
  • Television
  • Interviews
  • Awards
Entertainer.newsEntertainer.news
Home TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads
Web Series

TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads

Team EntertainerBy Team EntertainerOctober 15, 2026Updated:November 3, 2026No Comments12 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr WhatsApp VKontakte Email
Share
Facebook Twitter LinkedIn Pinterest Email


By Oleksii Tkachuk, Rajiv Shringi, Chris Lohfink, Abe Ratnofsky, Kartik Sathyanarayanan

Introduction

In an earlier submit, we launched the TimeSeries Abstraction, our high-throughput, low-latency platform designed to retailer and question temporal occasion knowledge at scale. It has since grow to be the operational spine for Netflix’s largest datasets, which now span a number of petabytes.

Nonetheless, managing such massive datasets in scorching storage, tuned for low-millisecond latency entry, just isn’t low-cost. At Netflix, cost-efficiency is a steady self-discipline somewhat than a brief initiative. This submit shares two options that considerably cut back storage prices by repeatedly transferring older, much less continuously accessed knowledge into cheaper object storage:

  • Answer 1: Makes use of Apache Kafka and Flink to stream knowledge on to S3
  • Answer 2: Leverages Cassandra backups in S3 and pluggable Memtables to learn knowledge straight from S3

We’ll begin with the Kafka/Flink-based design, then describe how we developed to a local Cassandra-based chilly tier.

Challenges

As Netflix’s TimeSeries datasets have expanded, we’ve encountered a number of compounding challenges:

  • Rising retention calls for: Customers typically push for rising knowledge retention to serve a broader vary of use instances, which in flip will increase the storage prices.
  • Escalating prices: Sustaining huge, dense clusters for occasionally accessed historic knowledge is prohibitively costly.
  • Sparse historic entry: Most queries goal latest occasions, however older knowledge should stay available for infrequent audits and investigations.
  • Operational pressure: Excessive storage density on scorching nodes complicates scaling, deployment, and cluster stability.
  • Latency necessities: Even historic knowledge typically requires sub-second question efficiency for inside tooling and evaluation.

This creates a transparent alternative: maintain the newest knowledge on quick storage and migrate older, not often learn knowledge to cheaper storage whereas preserving a constant API. Finished nicely, this may meet consumer and operational wants whereas vastly decreasing our infrastructure spend.

TimeSeries Abstraction

Earlier than we dive into the chilly storage options, here’s a fast recap of how TimeSeries partitions its dataset into a number of tables, every masking a window of time we name Time Slices.

Time Slices and Immutable Home windows

This permits a number of key optimizations, most notably permitting a desk to grow to be immutable as soon as it crosses the write boundary, which lets us safely apply additional optimizations. For extra particulars, confer with our earlier submit.

This slice info is important and is maintained in a devoted metadata desk:

Time Slice Metadata Storage

Creating new Time Slices and sustaining their metadata is a major duty of the TimeSeries management airplane. The TimeSeries knowledge airplane makes use of this info to accurately route the learn and write site visitors to the provisioned tables.

TimeSeries Knowledge and Management Airplane

With that background, let’s have a look at the totally different options we constructed for knowledge tiering.

Answer 1: Streaming with Apache Kafka and Flink

Our first answer adopted a well-known streaming sample utilizing Apache Kafka and Flink, the place we streamed knowledge to S3 in realtime whereas concurrently persisting it to Cassandra.

Streaming with Apache Kafka and Flink

Getting Knowledge Into S3 Effectively

To optimize knowledge streaming to S3, the system hash-partitions Time Collection IDs into buckets. This partitions the stream and lets devoted Flink duties deal with every bucket, finally batching and decreasing the write throughput to S3. Flink staff batch occasions into Parquet information grouped by occasion time inside the hour. These information are nice for high-throughput streaming writes, however not for low-latency reads.

Enter the Compaction job.

Compaction Job to Optimize Reads

The compaction job runs day by day and merges small Parquet information right into a single, optimized day by day Parquet file per namespace.

The job additional optimizes the Parquet information within the following methods:

  • Small Row Teams: Creates small (4–8 MB) row teams to attenuate lookup value. We are able to then use the Parquet Java library to carry out vary reads of row teams from S3. That is deliberately nicely under Parquet’s advisable row group flooring — smaller teams have been higher as a result of our reads are dominated by level/vary lookups somewhat than full scans.
  • Occasion Sorting: Types occasions inside a row group by time, eliminating brute-force scans when querying for a time vary.
  • Bloom Filters: Caches Bloom filters and file metadata in EVCache, permitting the TimeSeries question planner to effectively skip pointless reads from S3.
Compacted and Sorted Parquet File Format

As soon as the compaction job completes, all compacted information per partition are recorded within the TimeSeries management airplane, and the new storage slices can now be truncated!

On a few of our largest datasets, this job sometimes takes about 12 hours, prioritizing decrease compute prices over execution velocity.

Compaction Burndown Per Day

Streaming Chilly Tier Learn

As soon as compacted information are optimized in chilly storage, they’re able to be learn. The TimeSeries knowledge airplane followers out a given question throughout each the cold and hot tiers and merges their consequence. The customers nonetheless use the identical question interface:

TimeSeries Tiered Storage Learn Path

The Chilly Tier Knowledge Airplane makes use of compaction metadata for aggressive planning and pruning, which dramatically reduces the S3 question fan-out. Because of this, retrieving a full day of knowledge requires at most three sequential S3 reads, even within the unusual case the place an ID spans two row teams. This yields a p50 latency of ~150 ms and a p99 latency of ~500 ms per learn.

Chilly Tier Learn Latency in Seconds

On the time of writing, the streaming chilly tier implementation was processing as much as 1,250 reads per second from chilly storage globally:

Chilly Tier Learn Requests Per Second

Why not cease right here?

The streaming chilly tier has delivered actual financial savings, shrinking each Cassandra’s EBS footprint and the Cassandra backup footprint in S3. However the streaming infrastructure itself is dear and provides transferring items that widen the workforce’s on-call scope:

Further Infrastructure and Operational Value

We needed to enhance this additional and requested ourselves:

  • Can we keep away from streaming knowledge into the thing retailer and the related knowledge switch prices?
  • Can we cut back the compute and operational overhead of recent parts?

That led to a extra radical query: Can we learn straight from present Cassandra backups in S3?

Answer 2: Cassandra Chilly Storage for Immutable Knowledge

At Netflix, we again up Cassandra knowledge into S3 at common intervals for catastrophe restoration. The concept for tiering is easy:

Learn straight from these S3 backups as a substitute of paying a separate knowledge switch value to S3.

As soon as TimeSeries marks a scorching Time Slice as prepared for chilly storage, we will customise Cassandra to learn straight from its backups, permitting us to delete the on-disk knowledge copy. This could additionally incur minimal modifications to the TimeSeries learn path!

New TimeSeries Learn Path

To make this viable in manufacturing, we needed to reply a sequence of “How…?” questions.

How Can We Learn the Backups Effectively?

We launched a brand new compaction technique in Cassandra, which we name ReadOnlyCompactionStrategy (ROCS). Right here is the way it works:

As soon as a time slice crosses the user-configured cold-tier boundary, TimeSeries stops accepting writes for that point slice making it immutable. This immutability unlocks vital optimization alternatives:

  • SSTable overlap: Commonplace methods (LCS, TWCS) typically produce SSTables that overlap within the token house, and so a single partition key might exist throughout a number of information.
  • Learn fan‑out: As a result of a learn should contact each SSTable masking the important thing’s token vary, this overlap turns a single point-read into a number of lookups, rising latency throughout all tiers.

ROCS re-compacts all SSTables as soon as, aligning them exactly with token ranges:

ROCS SSTable Boundaries

ROCS applies extra broadly to all TimeSeries use instances, driving tail latency down on all storage tiers by decreasing the SSTable fan-out throughout learn operations:

Tail Latency Enchancment in Reads after ROCS

As soon as the immutable Time Slices have been optimized, it’s time to set off backups.

How Would We Leverage Backups?

Cassandra backs up each node within the cluster, organizing every backup on S3 as a declarative manifest, together with the SSTables backing the keyspace’s tables:

 {bucket}/
└── {prefix}/ ← cluster id (env_backup/hash_app)
└── {token}/ ← node id (ring token)
├── META_V2/ ← manifests, one per snapshot
│ ├── {snapshotInstantMs}.json
│ ├── {snapshotInstantMs}.json
│ └── ...
└── SST_V2/ ← precise SSTable information
└── {backupTimestamp}/ ← snapshot timestamp
└── {keyspace}/
└── {table-with-uuid}/
└── {compression}/
└── {encryption}/
├── {component-file}
└── ...

The backup manifest orchestrates the backup, itemizing the paths to all its SSTables:

manifest.json
├── information
│ ├── appName "cass-cluster"
│ ├── area "us-east-1"
│ ├── snapshotInstantMs 1748908800000
│ ├── backupPathPrefix "prod_backup/1234_cass/token"
└── knowledge[]
└── [0]
├── keyspaceName "ks"
├── columnfamily "tbl-a1b2..."
└── sstables[]
└── [0]
└── parts[]
├── na-1-big-Knowledge.db
├── na-1-big-Index.db
├── na-1-big-Filter.db
├── na-1-big-Statistics.db
├── na-1-big-Abstract.db
├── na-1-big-CompressionInfo.db

Alongside the big Knowledge.db file, backups embody a compact set of index, metadata, and integrity information. This construction permits us to shortly find, validate, and restore particular partitions or knowledge ranges with out scanning your entire dataset. Moreover, for the reason that manifest catalogs each S3 path, we will obtain most of those parts on to the Cassandra node (excluding the Knowledge.db file) the place they occupy negligible house. These native parts allow environment friendly question planning earlier than any knowledge is learn from S3.

How Would Cassandra Learn From Backups?

Cassandra reads sometimes pull knowledge from two sources: the Memtable, which holds latest unflushed writes, and flushed SSTables on sturdy storage. As of Cassandra 4.1, we will leverage pluggable Memtables to redirect reads from “chilly” tables on to the backup storage in S3.

-- create desk for a given Time Slice
CREATE TABLE data_20260616_00_04 (
...
...,
PRIMARY KEY ((time_series_id, time_bucket, event_bucket), event_time, event_id, key)
) WITH CLUSTERING ORDER BY (event_time DESC, event_id DESC, key DESC)
...
AND compaction = ...
AND compression = ...
AND memtable = 'backupmemtable:timestamp=1781953200000' ← Customized memtable
...

The timestamp marks the purpose at which the backup is assured to cowl the total desk, and the desk is assured to be immutable.

Opening a backup Memtable kicks off a state-machine initialization that pulls the index knowledge from S3 onto the node as soon as:

State Machine for Cassandra Backup Memtable Initialization

This course of ensures all SSTable parts are downloaded reliably by means of a number of resilience mechanisms:

  • Retries and backoff: All S3 operations and file-system cache writes use retries with exponential backoff.
  • Useful resource isolation: Initialization runs on a devoted thread pool, remoted from regular Cassandra operations.
  • Partial obtain safety: If a downloaded element’s measurement doesn’t match the manifest, it’s deleted and re-downloaded.
  • DDL change safety: A .timestamp file marks accomplished initialization and ties it to the backup timestamp. If the desk’s backup timestamp modifications, this file is eliminated and initialization restarts for the brand new timestamp.
  • Learn safety: Memtable initialization is asynchronous; reads to an initializing memtable block for at most half the learn RPC timeout.

As soon as initialization has pulled each SSTable element (besides Knowledge.db) from S3 onto the native disk, the Memtable is able to serve reads. Here’s what the learn course of seems like:

Studying from Cassandra Backup Memtables

To reduce latency, the system parallelizes S3 reads throughout requests, SSTables, and particular person chunks. A shared cache deduplicates fetches and prefetches knowledge, so chunks are prepared earlier than they’re wanted.

We additional optimize cluster and desk settings to scale back the variety of S3 calls:

  • Bigger compression chunks: Scale back S3 calls per partition by studying fewer, bigger chunks (accepting some over-fetch on small reads).
  • Learn restore disabled: Keep away from costly, pointless restore site visitors towards immutable, pre‑repaired backup knowledge.
  • Speculative retries disabled: Forestall S3’s greater latency from triggering redundant, overlapping reads.

How Would We Know When It Is Secure To Learn From Backups?

Nonetheless, earlier than the Memtable can level to a backup, we should affirm two issues throughout the entire ring, and we use Cassandra Digital Tables to do that.

First, that ROCS re-compaction succeeded:

cqlsh> SELECT * FROM netflix_views.cluster_view                                                                                                                                                          
WHERE keyspace_name = 'system_views' AND table_name = 'sstable_tasks';

keyspace_name | table_name | host | worth
--------------+---------------+----------+------------------------------------------------
system_views | sstable_tasks | 10.0.0.1 | [{"table_name":"<.>","kind":"compaction", ...}]
system_views | sstable_tasks | 10.0.0.2 | [] <---- no lively compactions
system_views | sstable_tasks | 10.0.0.3 | []

And second, that backups have been totally captured (denoted by uploaded=True):

cqlsh> SELECT * FROM netflix_views.complete_backups                                                                                                                                                      
WHERE keyspace_name = 'my_ks' AND table_name = 'my_table';

keyspace_name | table_name | timestamp | num_tokens | num_uploaded | uploaded
--------------+------------+---------------+------------+--------------+----------
my_ks | my_table | 1765211400000 | 3 | 3 | True
my_ks | my_table | 1765213200000 | 3 | 2 | False

How Would We Automate This Whole Circulate?

The TimeSeries management airplane automates your entire transition as slices cross the immutable boundary:

Automating Time Slice Lifecycle Transitions

Every slice might be in any lifecycle stage at any time with out impacting system availability, and the system stays resilient to delayed backups or lagging compactions.

How Would Cassandra Deal with The Further Load?

Serving knowledge from S3 is way more CPU-intensive than studying from native storage. A sudden spike in learn requests can overwhelm the Cassandra cluster and have an effect on the new path, turning a cost-saving function right into a reliability danger.

To keep away from this, chilly reads can optionally run on a separate Cassandra cluster that factors its backup Memtables on the scorching cluster’s backups:

Remoted Chilly Tier Cassandra Topology

The chilly Cassandra cluster should match the unique cluster in topology however needn’t match it in {hardware}! We are able to run the chilly cluster on a lot smaller, cheaper, storage-light situations.

Actual World Efficiency

Cassandra Chilly Tier was rolled out to essentially the most storage‑intensive TimeSeries Abstraction clusters. It at the moment shops over 15 PB of compressed knowledge in Cassandra chilly storage and has allowed us to chop hooked up storage considerably on all affected clusters:

Storage Discount Throughout Rollout: from 80% disk utilization to 7%

On the identical time, it serves 1000’s of reads per second from chilly tier storage in any given area:

Chilly Tier Learn Requests Per Second

Cassandra Chilly Tier improves p90 latency by ~30% over the earlier chilly storage implementation, because of extra environment friendly S3 entry patterns and smaller learn sizes, whereas retaining p99 latency below ~500 ms. Cassandra already handles bursts of chilly reads gracefully by way of its chunk cache, so this design preserves that very same resilience for the S3-backed chilly tier:

Chilly Tier Learn Latency: blue — p90 latency, brown — p99 latency; latency measured in seconds)

Acknowledgements: Particular because of our gorgeous colleagues who contributed to the TimeSeries Abstraction’s success and collaborated with us on the chilly storage options: Kaidan Fullerton, Matt Lehman, Sumanth Pasupuleti, Carlos Matias Herrero, Nick Pavlakis, Bryan Keller, Francesco Di Chiara


TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Chilly Reads was initially revealed in Netflix TechBlog on Medium, the place persons are persevering with the dialog by highlighting and responding to this story.



Source link

Cassandra Cold Journey KafkaFlink Native Reads Storage Streams Tiered TimeSeries
Share. Facebook Twitter Pinterest LinkedIn Tumblr WhatsApp Email
Previous ArticleBehind the Scenes: Evolving Netflix’s Ads Event Pipeline for Live — Part II
Next Article Measuring the Impact of Personalized Recommendations
Team Entertainer
  • Website

Related Posts

New Releases on Netflix This Week and Top 10 Movies & Series: November 14, 2026

November 15, 2026

Doomsday’? Why Are Fans Convinced He Is Playing Victor von Doom?

November 14, 2026

“Why Would a Nice Man Do That?”

November 14, 2026

Netflix Reveals Title, 2027 Release Window & First Look for ‘Magic: The Gathering: Planeswalker’ Animated Series

November 14, 2026
Recent Posts
  • Kiss Will Release Three New Box Sets in 2027
  • AMA/Q&A Announcement – Tony Gilroy – Monday 11/16 at 4 PM ET – Oscar-Nominated Filmmaker & Screenwriter – ‘Michael Clayton’, ‘Andor’, ‘Rogue One’, ‘Behemoth!’, ‘Bourne’ Franchise
  • ‘Sister Wives’ Could Polygamy Be Dealbreaker Meri Brown’s Men?
  • Gypsy Rose Blanchard Addresses Awful Accusations Against Her In First Live Interview Since Ken Urker’s Death – Watch

Archives

  • November 2026
  • October 2026
  • September 2026
  • August 2026
  • July 2026
  • June 2026
  • May 2026
  • April 2026
  • March 2026
  • February 2026
  • January 2026
  • December 2025
  • November 2025
  • October 2025
  • September 2025
  • August 2025
  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021

Categories

  • Actress
  • Awards
  • Behind the Camera
  • BollyBuzz
  • Celebrity
  • Edit Picks
  • Glam & Style
  • Global Bollywood
  • In the Frame
  • Insta Inspector
  • Interviews
  • Movies
  • Music
  • News
  • News & Gossip
  • News & Gossips
  • OTT
  • Podcast
  • Power & Purpose
  • Press Release
  • Spotlight Stories
  • Spotted!
  • Star Luxe
  • Television
  • Trending
  • Uncategorized
  • Web Series
NAVIGATION
  • About us
  • Advertise with us
  • Submit Articles
  • Privacy Policy
  • Contact us
  • About us
  • Disclaimer
  • Privacy Policy
  • DMCA
  • Cookie Privacy Policy
  • Terms and Conditions
  • Contact us
Copyright © 2026 Entertainer.

Type above and press Enter to search. Press Esc to cancel.

Sign In or Register

Welcome Back!

Login to your account below.

Lost password?