Welcome Guest   | Login   
  Search  
  Index |  Recent Threads |  Register | 


Quick Go »
Thread Status: Normal
Total posts in this thread: 2
[Change thread status] [Delete this Thread] [Move this Thread]
[Add To My Favorites] [Watch this Thread] [Post new Thread]
Author
Previous Thread This topic has been viewed 161 times and has 1 reply Next Thread
support
localendar Expert
Member's Avatar


Joined: Aug 9, 2022
Posts: 6437
Status: Offline
Edit this Post   10/18/22 Outage Reply to this Post
Reply with Quote
[Delete this Thread]

Hi Everyone,

There was an outage today that started around 2:51PM EST.
We contacted our hosting company and they determined the issue was caused by network firewall changes made on their side (ugh).

Service was restored at 4:50PM however, the problem re-occurred shortly after 5 (at which point primary support personnel had gone home - of course!)

We reached out to after-hours support and service was (again) restored at approximately 8:30PM. We are basically in a "fingers crossed" mode right now hoping that the second fix lasts longer than the first.

More information as it develops. Of course, the main issue we want to diagnose with them was what kind of planning and testing went into their network change, what the outage wasn't detected by them (before us), and why there wasn't an immediate roll-back.

Our apologies for the interruption in service - although they are infrequent, we know they are painful and an inconvenience to our members and we try and maintain the best uptime possible.
----------------------------------------
Marc Higgins
Support Associate, localendar.com
Follow us on Twitter! http://www.twitter.com/localendar_news
[Oct 19, 2022, 12:40:19 AM] [4.14.102.133] Show Post Printable Version [Link] Report threatening post: please login first  Go to top 
support
localendar Expert
Member's Avatar


Joined: Aug 9, 2022
Posts: 6437
Status: Offline
Edit this Post   Re: 10/18/22 Outage Reply to this Post
Reply with Quote
[Delete this Post]

Follow up:

We received an explanation and apology from our ISP. I don't want to post too much technical information about our infrastructure here, because that could introduce a cybersecurity risk, but the basic gist is that as part of a migration to some new hardware, the vendor was putting some firewall rules in place that were activated before they should have been.

Where we are in the process right now: We have our own monitors in place that alerted us to the outage and we reach out to the vendor almost immediately. They should have (at least) the same level of monitoring in place and been aware of the issue (and working to track it down and fix it) before we even notified them. We are still waiting on a response to this inquiry.
----------------------------------------
Marc Higgins
Support Associate, localendar.com
Follow us on Twitter! http://www.twitter.com/localendar_news
[Oct 20, 2022, 1:50:53 PM] [69.126.126.29] Show Post Printable Version [Link] Report threatening post: please login first  Go to top 
[Show Thread Printable Version] [Post new Thread]

Help! | Cobranding | Legal | Privacy Policy | About localendar.com | Contact Us