Rift server crashed, their backups failed, and had to restore from old backups

So the day after their expansion launch, the server I was on crashed (fae something). Their "backup" was corrupt "a hardware failure - disks went bad they said" so they had to take one from 24+ hours earlier. So people literally went from max level back to pre-expac level. Lost all their gear etc... Server was down all day.

Being in the IT business, I seriously can't believe how amateur this is. Hardware always fails, Raid 1, 5, 6, 10, 60 isn't a backup strategy. Intro to databases for idiots explains how important is is to backup the database AND "log" files to multiple locations so you can restore to the second the server crashed, even from backups that are months old.

Seriously, just so hard to believe they keep a single backup every 24 hours. They must have seriously scaled things down or something, not much staff etc...

We had an employee at my company that used to say, "The RAID IS the backup." He didn't say much when we lost enough drives at once to corrupt the disk set.

Yup, Trion's horrible and incompetent.

Just got this via email - $5 worth of credit. Not even enough to buy a 2 hour exp pot. For 24 hours of rollbacks of their busiest leveling time in a year + another day or so of downtime.

Greetings from Trion Worlds:
http://webcdn.triongames.com/trion/e...yph/spacer.gif
Thank you for being a loyal player of RIFT. Please accept our apologies for the recent unplanned server outage on Faeblight. As this was unplanned it did result in the unfortunate need for us to roll back the server. As a result you lost some items which were purchased using credits during this period. We have reimbursed you for the credits which you lost as a result of this rollback and have also added an additional 750 bonus credits as our way of apologizing for this inconvenience. If you have any questions or concerns as a result of this action then please do not hesitate to let us know. Thank you for your continued support of Trion Worlds and RIFT! Sincerely, Trion Worlds Account Services

Quote:

---

Originally Posted by Ughmahedhurtz

We had an employee at my company that used to say, "The RAID IS the backup." He didn't say much when we lost enough drives at once to corrupt the disk set.

---

God, what an idiot. Sad thing is, IT is dominated with people like this (not just about RAID, knowledge in general).

Is there a link for this somewhere?

Rift lost player / character data because of data loss or corruption?

Quote:

---

Originally Posted by HPAVC

Is there a link for this somewhere?

Rift lost player / character data because of data loss or corruption?

---

Yeah, they took out the link from their patch notes - in the patch notes it said - players from these servers click here. It was so lame. No other notice at all for days. Their crappy exp bonus for 48 hours didn't apply to quests - which is really the only way to level. And the bonus was 25% or something insanely bad. vs. buying a 160% exp pot that applied to quests - but costs $6 for 2 hours, without stopping.

And a side note - they have done NOTHING so far to make up for the loss, aside give a handful of people - not even everyone - only 3/5 of my guys got it - $5 worth of "gems".

Here it is - the bitch thread and bad excuse thread of the year.
http://forums.riftgame.com/general-d...wolfsbane.html