W3C Systems Team

Archive: W3C System Status

26 March 2008

Mon 31 Mar 2008 14:00Z, There have been no further errors logged for the Zakim telephone lines since 26 Mar 21:30Z. No satisfactory explanation has been found for the 12 minute outage that day. Extra monitoring is being done.

Wed Mar 26 13:30 UTC 2008, Zakim T1 lines are back. We have not identified the cause of the outage. The construction folk claim they are not the cause. Further investigation will be done if an outage recurs.

Wed Mar 26 13:15 UTC 2008, Zakim teleconference bridge lines are out, possibly due to construction outside the Stata Center.

19 January 2008

Sat Jan 19 18:04:03 UTC 2008, Cooling emergency in MIT machine room resolved.

Sat Jan 19 17:16:18 UTC 2008, Cooling emergency in MIT machine room, systems are being shut down.

21 September 2007

Fri Sep 21 15:47:32 UTC 2007, www.w3.org experienced a huge server resource demand.

17 September 2007

Tue Sep 18 07:42:45 UTC 2007, The lists.w3.org hardware migration is complete and list services have been resumed.

Tue Sep 18 04:43:09 UTC 2007, The lists.w3.org hardware migration continues and is expected to end soon. Queued mail will be delivered upon resumption of service, which is expected in the hour.

Mon Sep 17 19:11:20 UTC 2007, Beginning at 0100 UTC, 18 September 2007, we will be migrating lists.w3.org to new hardware. Mailing lists will still be served statically and sent mail will be queued for delivery upon service restoration. The migration is expected to take about 2 hours.

6 September 2007

Thu Sep 6 21:02:21 GMT 2007, www.w3.org was experiencing service delays. We hope to have that addressed at present.

Thu Sep 6 15:26:19 UTC 2007, www.w3.org is currently experiencing intermittent service delays. We are working diligently to resolve all outstanding issues.

Thu Sep 6 21:02:21 UTC 2007, full www.w3.org service restored at present.

29 June 2007

Fri Jun 29 21:43:43 UTC 2007. There seems to be a database issue that may be interfering with access to the Member and Team list archives on lists.w3.org. We are examining the issue now and will work toward a speedy resolution.

Fri Jun 29 22:07:52 UTC 2007. The underlying database issues have been resolved and Member and Team access is fully restored.

25 June 2007

On Monday 25 June 2007 at 13:00 GMT the W3C Systems team will perform a major upgrade of W3C main database server. This intervention is scheduled for 2 hours. Applications that rely on this service (eg: DBWG, WBS, Blogs...) will be unavailable during this outage.

25 June 15:50 GMT. This outage is taking much longer than expected, we will update this page when we have a better estimate on time to resolution.

25 June 16:45 GMT. Unfortunately we suffered a hardware failure during this upgrade process.
We are now moving this service to another server while using the latest data from the old hard drive.
Our sincere apologies for this additional downtime.

25 June around 19:00 GMT. All data backed services should be operational again.

26-28 March 2007

Wed, 28 March 1755Z. Zakim is back in service, though not at full capacity yet.

Wed, 28 March 1700Z. Zakim is rebooting; expected back in service shortly.

Wed, 28 March 1500Z. Repair parts have arrived and are being installed. Optimism is appropriate.

Tue, 27 March 2007 1200Z. During the night of Monday 26 March we had a power supply failure in the Zakim teleconference bridge hardware.

The bridge is currently down and callers hear a busy signal. Repair parts are expected to arrive on Wednesday, 28 March. Further status will be reported when available.

Chairs and those with W3C Member authentication are encouraged to consult Zakim Teleconference Bridge Status for current status and details on an interim alternative for scheduling teleconferences.

We apologize for the inconvenience.

17 November 2006:Friday, 17 November 2006. Previous mail delivery issues with lists.w3.org and the W3C mailhubs, have been reolved.

14-17 November 2006: Tuesday - Friday, 14-17 November 2006, lists.w3.org, and the majority of W3C mailhubs, have experienced significant mail delivery delays due to a combination of factors that is being aggressively examined at this time.
We will apprises all Members and Team staff upon confirmed resolution of all outstanding issues. If you have urgent e-mail pertaining to W3C business, publishing, or activity issues, be apprised of potential delays in response (delay times vary widely).

22 September 2006: Friday, 22 September 2006, MIT, where we host much of W3C's global infrastructure, is experiencing networking issues. Mail, dns and our main website (www.w3.org, less some services) will remain operational in the meantime. MIT networking administrators are aware of the problem and working towards correcting it.

5 September 2006: On Tuesday 5 September 2006, W3C experienced a database service failure that has since been repaired. Please report any service updates that may need our attention if you experienced service failure in any transactions during the brief period of downtime (approximately 1550-1700 UTC).

21 August 2006: On Monday 21 August 2006, W3C again experienced problems with the Zakim teleconferencing bridge. We believe the trouble has been identified and repaired.

21 August 2006: On Monday 21 August 2006, W3C again is experiencing a component failure on the Zakim teleconferencing bridge. Callers to Zakim will not hear the usual greeting and prompt for conference code. It does appear, however, that the code will be accepted if entered without the voice prompt and conferences may proceed. We are investigating the hardware issue.

15 August 2006: On Tuesday 15 August 2006, W3C began experiencing a component failure on the Zakim teleconferencing bridge. The impact is that all currently scheduled teleconferences cannot be conducted until the failure is resolved.

Urgent efforts are being made to get the bridge back online as soon as possible. We will update the status as updates become available. Chairs will receive updates via the system-notices mailing list.

We apologize for the inconvenience.

3 August 2006: On Thursday 3 August 2006, W3C began experiencing mirror overloads, slowing down access to the Web site. The W3C Systems team is in the process of resolving the traffic issues. In the meantime, access to the W3C Web site may include delays and timeouts. We apologize for the inconvenience.

14 February 2006: On Tuesday, 14 February 2006, starting around 5pm EST we will be adding some needed hardware to our existing servers.
There will be an interruption to accessing our CVS repositories cvs.w3.org:/w3ccvs, subsequently jigedit.w3.org, and dev.w3.org:/sources/public and our database. The CVS interruption will likely be more pronounced of the two, perhaps an hour or two in duration. The database interruption will affect numerous services including changing acls, access to authentication required lists, WBS and other data backed services.

30 January 2005: Some scheduled, intentionally non-interruptive work was performed by our CSAIL network managers Sunday, 30 Jan 2005:1700-2000EST, but minor problems were encountered, and some disruptions seem to have occurred. All services have since been restored.

7 August 2004: The server, lists.w3.org, was moved to new hardware.

25 May 2004: The public interface (pserver) to W3C CVS source repository is temporarily closed. The Web interface remains available.

13 March 2004: W3C systems @MIT will be moved during the weekend of 13-15 March 2004. Many services will be effected for Public, Member, and Staff users of the site. Please see the announcement for details.

27 January 2004: W3C's mailing lists may be slow due to a new mass-mailing virus known as W32/Novarg.A, W32/Shimg, or W32/Mydoom.
For more information about this virus see the CERT Incident Note IN-2004-01.

3 September 2003: Some of W3C's mailing list archives may be slow to update due to problems introduced during a recent upgrade of our mail software. As the mail archives are rebuilt, Web Accessibility Initiative (WAI) improvements are being implemented.

26 August 2003: The Archive Approval system is offline subsequent to mail infrastructure changes. Queued mail pending approval will be kept and expiration dates will be extended if necessary to allow adequate response time for all posts.

25 August 2003: W3C email and mailing list services have returned to normal. There are a few remaining cases of 1-2 hour delivery delays.

23 August 2003: All new and backlog email is expected to reach its destinations within 24 hours or so. In some cases, email may be delivered out of chronological order.

22 August 2003: Email delivery to W3C mailing lists remains delayed. Archives are functioning but lists.w3.org is under heavy load and visitors may see occasional connection failures.

21 August 2003: W3C email and mailing lists archives are slowed due to inbound email computer worms. The W3C Systems Team is working to restore service. We apologize for the inconvenience and expect normal delivery to resume soon.

28 December 2002: MIT scheduled power outage

9-10 August 2002: MIT scheduled power outage

31 May - 1 June 2002: MIT scheduled power outage

21-23 December 2001: MIT scheduled power outage

7-8 August 2001: MIT power outage

31 March 2001: MIT scheduled 7 April power outage cancelled


System notices archive (Member-only link)


Webmaster
Last updated: $Date: 2008/03/31 14:10:12 $