As climate change escalates, several things become clear to me:

  1. The only thing capitalists are consistently competent at is fucking over their workers
  2. The industry doesn’t have the impulse control to slow its own demise

Most of the consequences of this are horrific, widespread, and harrowing to consider. This one’s a little different. I wanted to get some opinions from some devops folks who are more knowledgeable about the specifics of their work than I am.

Here’s the basic pitch. As climate change accelerates, freak weather patterns which span continents will become more common, as will infrastructure failure. We know that a key aspect of reliable offsite backups is to have proper separation of dependent systems. So if you think your servers are completely “redundant” because they run on separate VLANs and electrical circuits, but are housed in the same data center, sucks for you when the power goes out for longer than your generators can last. This is a major reason we have offsite backups. Trillions of dollars of data is stored on magnetic tapes all over the world (is this still the standard medium? Idk). But as catastrophic weather and infrastructure failure becomes more common, these separated systems become entangled again. If your offsite backups are on the east coast and your data center is on the west coast, having one building burn down in a wildfire while the other experiences rolling blackouts is not a great situation to have to prepare for.

So I’m curious how behind the curve I am on this, whether there are people at cons talking about this, etc. I assume the big players have contingency plans for this sort of thing, but a lot of companies don’t. Just another example of how capital can’t help but create the conditions of its collapse.

  • Cummunism [they/them, he/him]
    ·
    2 years ago

    afaik, tape storage is probably still the best long term storage. the main issue the storing it, temperature control and whatnot. i know there are "archival" blurays that hold like 100GB though so maybe those are better now. im not sure where most backup sites are, but ive seen some and they are in spots and in buildings that there is almost no way the equipment would suffer. infrastructure could go down but i think in this case the important part is keeping the equipment and data safe. ive not seen a datacenter in a flood zone or something.

    • MendingBenjamin [they/them]
      ·
      2 years ago

      The one data center I used to work at was at one of the highest spots in the tri city area and still had a trench dug around it to divert water away. I believe they also had several towers around the perimeter to act as lightning rods and enough space to build a solar farm if need be. It’s pretty clear the designers of real high end places take natural disasters into account, but I don’t know how resilient the whole thing is

      • Cummunism [they/them, he/him]
        ·
        2 years ago

        there is a cost difference between facilities im sure, but that just means that the actual important data is most likely the most securely backed up. i dont know if i fully understand what sort of catastrophe you mean though. Im thinking of something like a datacenter gets cut off because of a flood, but the equipment and data is still safe somehow, maybe youre thinking of some end of the world shit which at that point it won't really matter.

        • MendingBenjamin [they/them]
          ·
          2 years ago

          Maybe I’m mixing up the redundancy of availability with the redundancy of backups

    • Mardoniush [she/her]
      ·
      2 years ago

      Optimum pure storage is indeed bluray (or for "I hope the archeologists can read this" levels of critical data, record discs)

      • xXthrowawayXx [none/use name]
        ·
        2 years ago

        Have you been keeping up with the cd and laserdisc community? Lots of em have been reporting failures on optical disc media for over a decade now. One of the worst types is delaminating where the substrate is bonded to the plastic, around the edge. It’s most common on recordable media, but certain cd pressing plants are bad for it too.

        Of course laserdisc has bit rot, but they’re falling apart in new and interesting ways now too. Paints and dyes used on the sleeves are messing up the media and they’re big enough around to pick up a warp if not stored either upright or flat.

        What’s the prognosis on blu ray? Does it have any longevity engineered in?

        • Mardoniush [she/her]
          ·
          2 years ago

          Yeah, I've seen that. Modern archival-specific discs are mostly immune to these issues, though they are expensive. Blu rays in general don't have the layer of organic material on (most) of them, but I've seen reports of some Archea getting to them and degrading the metal nitride.

          You also have to store any sort of archival stuff properly of course, regardless of medium. You can throw a tungsten tablet with words carved on it into a damp basement and it'll have a bad time in a century or two.

          Cool, dry, humidity controlled, power independent storage. Preferably somewhere high and geologically stable. No light if possible, red light if needed. Store upright, use inert casing if you can.

          This is for "store and forget" stuff no one might look at for a decade or more, realistically you should be copying and where possible updating formats every 5 years or less.

          Source: work as an occasional (unqualified) archivist for casual stuff, know a professional closely