It's a familiar story in IT operations: your backups kick off right on schedule but you glance at the job status and see three days remaining. Nothing has failed. Nothing is off. That's just how long it takes to move data when you're talking about billions of objects.
What does this mean for you?
It means three days without that data, three days of downtime, three days of lost revenue. Three days where the entire company shines a spotlight on you, in a very bad way.
Amazon S3 and Azure Blob have grown into some of the largest data stores most organizations run, holding application data, telemetry, media assets, compliance archives, and providing a landing ground for AI operations. it’s increasingly common to see millions (if not billions of objects) within a bucket. At that size, backup stops being a job you schedule and forget. The bigger the bucket gets, the longer that progress bar runs, and it becomes a race against your own growth curve. Restore turns into an entirely different kind of problem on top of it.
Fortunately, there's help. Rubrik has engineered object storage solutions that can grow with you. These solutions can handle billions of objects for both backup and restore, but also deliver it in a cost-efficient architecture to optimize an organization's total cost of owenership (TCO).
The Bottleneck Behind Data Protection at Scale
AWS S3 backup and Azure Blob backup at scale presents a listing and comparison problem. A billion-object bucket means a lot of round trips just to find out what's there. Comparing all that data against your last backup gets expensive if you're doing it object by object. Moving whatever has changed means running transfers in parallel without tripping rate limits or overwhelming the system you're protecting.
Restore is a different animal. While backup has the possibility to perform bulk reads, restore requires writing back one object at a time. Every object is its own API call, and cloud providers rate-limit those calls the same way they rate-limit everything else. Parallelize as aggressively as you want, the baseline math doesn't change: every object is a transaction, every transaction takes time, and a billion of them add up.
What does change is how much of that time gets wasted on top of the minimum, through poor batching, serialized retries, or workers sitting idle waiting on the same rate limit. That's exactly where most native tools and bolt-on solutions fall short. This is not a flaw in any particular product. It's a fundamental constraint of how object storage APIs work.
Where Native Tools Falter
Native protection options run into this same wall, whether you're relying on the storage layer's own versioning or the cloud provider's dedicated backup service.
S3 versioning keeps prior object states around, but rolling back selectively gets slower and more manual as the version count grows. Azure Blob's soft delete and versioning have hard boundaries of their own: retention tops out at 365 days, and Microsoft's own guidance warns of performance degradation once you're past roughly 1,000 versions or soft-deleted snapshots per blob. Neither protects you if the whole storage account or bucket gets deleted, encrypted or compromised.
The cloud's dedicated backup services don't close the gap either. AWS states a bucket with roughly 5 billion objects and 6 petabytes of data can take about 100 hours—more than 4 days—for the initial backup to complete. AWS's own documentation points to why: that first backup requires a full scan of the bucket to inventory every object before anything gets moved, and only later backups can listen for changes instead of scanning everything again. In other words, the same listing bottleneck we walked through above, confirmed by AWS itself. Azure Backup, while not publishing numbers, states that storage accounts with lots of data or high churn “may take longer”
This is not a knock on the cloud providers. These features were built for preservation and basic recovery, not for reconstructing a business-critical bucket at scale, on a timeline your business can actually plan around.
How Rubrik Solves for Scale
Rubrik solves for these scale issues, providing efficient, immutable, air-gapped backups for even your largest object stores.
Note: Protection at Scale for AWS S3 and Azure Blob is currently generally available, Recovery at Scale for AWS S3 is generally available, and planned for Azure Blob in the near future.
The object inventory gets split into independent segments that can be listed and compared in parallel, so the work scales out instead of queuing up behind a single crawl. For very large buckets, we lean on the cloud provider's own inventory reporting rather than issuing live listing calls against the whole bucket every run, which cuts out a huge amount of redundant work. Only the segments that actually changed get touched, and the data movers pulling those changes are scaled horizontally. So adding more objects means adding more parallel processes rather than a longer single-threaded line. That's true whether the bucket sits in S3 or the container sits in Azure Blob: same architecture, same approach, on both clouds.
Restore gets the same treatment, built around the constraint that every object is its own write. Instead of a single queue grinding through a billion objects one at a time, restore work is broken into segments and dispatched across many parallel workers, each managing its own piece of the object set and its own retries against the cloud provider's rate limits. Progress gets tracked per segment rather than as one monolithic job, so a slowdown or failure in one part of the restore doesn't stall the rest. The result is a restore that scales the same way backup does: add more objects, add more parallel workers, rather than watching a single line get longer. Again, this holds whether you're restoring into an S3 bucket or an Azure Blob container.
Getting backup and restore to scale this way is the foundation. It's what makes it possible to move a billion objects in either direction without the process falling over. Scaling up when performance demands it, and scaling down to keep costs low.
But while scale alone ensures performance, it doesn’t help with determining what objects to restore first. That’s a task for Rubrik Prioritized Recovery.
Rubrik Prioritized Recovery: Built For What Matters Most
Picture your data restoring after an outage, terabyte by terabyte, in whatever order the data happens to come back. Somewhere in that queue sits the config file your application desperately needs to come back online, right next to a folder of photos from last year's holiday party. The restore engine doesn't know the difference. So which one comes back first?
Your RTO depends on the answer to that question. Solving backup and restore at scale gets you the ability to move billions of objects efficiently. On its own, that doesn't get your business back online quickly. RTO isn't measured by when the restore job finishes. It's measured by when your applications can resume working, and those are very different numbers at scale.
That's where Rubrik Prioritized Recovery for S3 and Blob comes in. Instead of treating a bucket or container as one monolithic restore operation, you define which objects matter most using glob patterns to match a simple configuration, whatever your applications need first. Those objects get matched and pushed to the front of the restore queue through their own parallel workers, while everything else continues restoring in the background. If the objects your production app actually needs are a small slice of the bucket, your effective RTO can look nothing like a full restore timeline.
Scale gets you the ability to move a billion objects. Prioritization gets you the right ones back first. Your application is back up while last year's staff party photos are still in line.
Cyber Resilience Built For Scale, Not Just To Check a Box
AWS S3 and Azure Blob object storage have become foundational infrastructure. As bucket and container sizes climb into the billions, the gap between "the backup is complete" and "the business is back online" only widens if you're relying on native tools or bolted-on object support. Closing that gap takes real engineering on both sides, backup that scales without falling over, and restore that gets the right data back first, applied the same way across every cloud you run.
A billion objects doesn't have to mean a billion-object problem. It just means you need protection that was built for that math from the start. You need Rubrik.
Want to learn more, check out some of the following resources:
See for yourself how Rubrik’s Cloud Native Protection works by taking one of our self-guided, hands-on labs on Rubrik Explore.
Watch an on-demand webinar covering recovery for your Cloud object storage
Get your free assessment today to see where your organization stacks up within the Cloud Maturity Model.
Safe Harbor Statement: Any unreleased services or features referenced in this presentation are not currently available and may not be made generally available on time or at all, as may be determined in our sole discretion. Any such referenced services or features do not represent promises to deliver, commitments, or obligations of Rubrik, Inc. and may not be incorporated into any contract. Customers should make their purchase decisions based upon services and features that are currently generally available.