I disagree. Storage is expensive. Think of an old video uploaded 15 years ago with total view count of 1k. You can't just put it to a cheap cold storage. Someday, somebody is going to watch it and you have to retrieve it instantly or that somebody will be disappointed.
Yt for example deletes your 720p after a while and replaces it with a potato.
And if you watch a old not relevant yt and it starts after 10 seconds instead of now, no one really cares.
You can put that old highly encoded potato at your huge and cheap storage system de located somewhere around the globe were it's just cheap (energy).
You can also calculate in the time for a band robot and only store half or the first minute of that potato on your cheap storage and let the robot grab the rest of it.
After all if video is your main thing plenty of weird optimizations start to make sense.
Tape robot time is NOT measured in minutes but hours. Also you wouldn't pull individual files like this out of tape.
Tape is for long term sunk storage, not cold infrequent access like a youtube video.
I know aws glacier has an "expedited retrieval time" of 1-5 minutes, but that is not how typical tape setups work. Frankly I would be very interested in what actually hides behind that product.
It's mostly just bandwidth prioritization, slotting large transfers in when there is excess bandwidth.
You can tell this is the case for at least the flexible retrieval tier, because small objects can be returned in a few minutes, whereas larger requests take hours - if the files were actually on a tape drive somewhere, small requests couldn't be fulfilled dramatically faster than large ones, given that tape has shitty random-access performance.
I believe you, I can see how it would make sense that AWS would create a tier to exploit the spare capacity in S3 disk bandwidth, just like they did for EC2 spare VM capacity with spot instances. Still it doesn't make intuitive sense to me how the performance AND the price can be so far. That's why I'd love a longer write-up if you know of any.
It's also weird that the retrieval gives you a regular fast S3 object you can then access. Given that it's already on that hardware, is a copy even happening?
You can use big disk, but not be able to access all the files on the disk with the same frequency, so you have 20% of the disk dedicated to hot storage, and 80% of it to cold storage. Cold storage access is queued , so the 1-5 minutes can come from there.
Google gave up on tape a while back. Latest Google search indicates it is only used for air gapped backups. I don't OP was suggesting using tape though, especially with technologies such as hybrid SMR.
100% that Google put videos on colder storage. Hot cached videos in memory cached at all possible locations. And cold videos stored compressed in a much cheaper storage container. The difference is maybe 500 to 3000ms.
