Last week's outage set us back as a company and community. When Seedr went down for nearly two full days, we let many of you down right when you needed the site for the weekend.
We sincerely apologize for the disruption this caused in your day. Here's a look at what transpired behind the scenes:
On August 9th around 8:15am EST, we experienced multiple power supply failures and issues with one of our server providers. This caused widespread overload issues that took down our primary database servers entirely.
With the databases offline, our web and API services soon became unavailable as well, resulting in a full outage. Our backup systems kicked in as designed. However, the scale of the outage meant it took significant manual effort and time to fully restore service.
Our team worked around the clock (literally) to retrieve backup data from cold storage, re-provision new servers, run diagnostics, and gradually bring Seedr back online in a stable manner.
We know many of you were frustrated by the nearly 2-day recovery time. Believe us, we shared in that frustration. But we made the deliberate decision to prioritize a careful, incremental restoration over rushing to get Seedr back online. We wanted to avoid quick fixes that might re-introduce instability.
The upside is we did extensive tuning to the system to accelerate the recovery:
- We ran rebuilds on our databases to improve website responsiveness.
- We are preparing the new technical reporting system to answer your questions in an automated, active manner if we ever have technical issues again.
- We validated all our backup systems and they worked perfectly, restoring the website to it's original state.
While we never want an outage to occur again, our team is as energized as ever to deliver the excellent experience users expect from Seedr. This experience has only reaffirmed our commitment to setting the standard for reliability and performance. As the site restarts and serves you all again, we want to thank all our users for your patience and support throughout this ordeal.
We were happy to hear our users offering kind words of encouragement to our team as we worked around the clock to restore service. It means a lot that many of you have walked with us for a decade.
In particular, we want to thank our premium users who make Seedr possible in the first place. Your support enables us to develop new features, expand capacity, and provide the level of service we aspire to.
As compensation to premium users, all paying users active on the 11th of August will receive significant additional features for free with the release of our v2 platform. The upgrades will be valid for 3 months. More details will be provided soon.
We don't take your loyalty for granted. If you ever have any feedback, questions, or just want to chat, please reach out to our dedicated customer support team at firstname.lastname@example.org.
If you are currently a free user, please consider supporting our site and getting cool premium features, and earlier access to our V2 platform by Upgrading To Premium
We're here to help however we can. Thank you again for being part of the Seedr community. We can't wait to keep improving and deliver the excellent, seamless experience you expect and deserve.
The Seedr Team