11-07-2004
There is an ongoing network problem with one of our upstream providers having to do with
communicating with one of the major Internet backbone providers. This will
cause some Internet sites to be unreachable at times. Other Internet sites
reachable via another backbone provider are unaffected.
We are currently working with technicians from our upstream provider to reroute Internet traffic away from the affected networks to restore access to the previously unreachable Internet sites.
Any additional updates will be posted as soon as we have them. We apologize for the inconvenience and are working to resolve this issue.
11-08-2004: 12:30 am EST
This outage is still occuring. At this time approx 3/5 of the Internet is solidly reachable.
The remaining 2/5 is intermitently reachable. We have spent the evening working with
various upstream providers trying to find a solution. Currently we are bringing in
hardware manufacture technical support to make sure this isn't a hardware issue.
We will be continuing to work this issue until it is fully resolved.
11-08-2004: 4:45 am EST
A few minutes ago, this situation stabilized. At this time we are not sure what the root
cause of the trouble was. We still have open tickets with Cisco trying to find the source
of this outage so we can prevent it in the future.
Please accept our appologies for the outage.
Date: Mon, 8 Nov 2004 09:32:33 -0500 Subject: [SUPPORT] Widomaker Service Outage 11/07/04 Yesterday, 11/7/04, at approx 11 am, Widomaker customers started experiencing difficulty hitting significant portions of the Internet. This outage was resolved at 4:32 am on 11/8/04. During this period, approximately 3/5 of the Internet was intermittently inaccessible. At times, you would be able to get to a certain website just fine. 30 seconds later, you could get to that same website, albeit at a very slow crawl. 30 seconds later, the website was completely inaccessible. 30 seconds later, all would be fine again. During this time, the remaining 2/5 of the Internet was reachable as if nothing was wrong. Depending on where on the Internet you were trying to go, you may or may not have seen this outage. Some of the more popular websites that were unavailable through us were www.cnn.com, www.microsoft.com and www.yahoo.com. Given the on again, off again nature of this problem, troubleshooting was rather difficult. At times we had technicians from 3 upstream providers, 2 telephone companies and 2 equipment manufactures all looking at this issue. Regrettably, this outage "healed" itself without any overt action by any of us. Due to this, I can not be certain that this type of outage will not occur again. Our current working theory is that we have triggered an obscure bug in Cisco's (our equipment manufacture) routing software. We continue to have open tickets with Cisco in chasing this down. Please accept our apologies for this outage. Here at Widomaker we strive to ensure that you, our customer, are as isolated as possible from these types of outages. Please rest assured that once we find the cause of this, we will make every effort to ensure it does not reoccur. Benjamin F. Loyall President Widomaker Communication Services, Inc.