Twitter Status

We’re temporarily lowering the API rate-limit to 75 from 175; we do plan to reestablish the higher API rate-limit later today.

Update (9:35 AM PST, 16:35 UTC): We’ve temporarily disabled integrated search.

Update (9:50 AM PST, 16:50 UTC): We’ve reenabled integrated search.

Update (10:20 AM PST, 17:20 UTC): The API rate-limit has been restored to 175.

Update (13:09 PST, 20:09 UTC): Users may be experiencing some delay with their timelines; we’re aware of this and working to resolve this.

Update (13:45 PST, 20:45 UTC): Timelines are no longer delayed.

Update (20:55 PST, 03:45 UTC): Lists are temporarily disabled.

Update (21:22 PST, 04:22 UTC): Users will temporarily not see Tweet counts on their home pages; they remain on the profile pages. SMS delivery of tweets was disabled for approximately 30 minutes but should now be restored.

Update (22:05 PST, 05:05 UTC):Lists and Tweet counts are now back.

We’ll be making adjustments to some site features throughout the day in an attempt to maximize site availability and stability.

Update (7:25 AM PST, 14:25 UTC): RSS and Atom feeds are currently disabled.

Update (8:05 AM PST, 15:05 UTC): API is temporarily rate-limited to 175.

Update (16:05 PM PST, 23:05 UTC): RSS and Atom feeds are re-enabled.

Dealing with high whales (errors) as the result of high load. Ops and engineering are responding. Also go #usa!

We’re seeing periodic high rates of errors on Twitter.com.

We’re seeing periodic high rates of errors on Twitter.com.

We’re seeing periodic high rates of errors on Twitter.com.

Update Jun 21 17:42 UTC: The rate of errors has been reduced to a moderate level.

We’re seeing periodic moderate rates of errors on Twitter.com.

22:20 UTC - 3:20 PDT   Twitter is unavailable. Engineering and ops are responding.

22:59 UTC - 4:00 PDT Site is recovering.

We’re seeing  periodic high rates of errors on Twitter.com. This is the result of testing and maintenance. We expect our maintenance to end at approximately @ Wed Jun 16 14:30 UTC

You can read more here.

MAINTENANCE ACTION:
—————————
We’ll be working with our network provider to perform some tests and maintenance. During this time you can expect a high rate of errors (whales)

DATE/TIME WINDOW:
—————————
 June 16th @ 13:00-14:30 UTC (6-7:30AM PDT)

AFFECTED RESOURCES:

———————————

Twitter.com and Api.twitter.com

Update: Wed Jun 16 17:08:12 UTC 2010: We’re extending what is more accurately described as a window of observation. We expect to close this window of observation at 18:00 UTC. 

Trends have been temporarily disabled. We’ll re-enable them in approximately in two hours.

Update 2:38PM PDT: Trends have been re-enabled.

Users may temporarily experience missing or duplicate tweets from their timelines. This is the result of the Failed Enhancement of Our Timeline Cache last night.

Normal timelines can restored by repeated refreshing your browser.

We’re currently experiencing a high rate of errors (whales) on Twitter.com.

Our infrastructure and operations engineers are responding to the incident.

Update 9:22 AM PDT: We’ve temporarily turned off our Twitter Places (Geo) feature to reduce the high rate of errors.

Update 2:39 PDT:  Twitter Places has been re-enabled.

Please Note: Users may temporarily experience missing or duplicate tweets from their timelines. Normal timelines can restored by refreshing your browser.

Update: We’re currently experiencing site availability issues resulting from the failed enhancement of a new approach to timeline caching. Our infrastructure and operations engineers are currently working to resolve this. We’ll update you soon with an ETA.

Update: 10:17 PM PDT:  Users may temporarily experience missing tweets from their timelines. They will be restored shortly.

Update: 11:51 PM PDT: We’re  recovering from the site outage. Remember, users may temporarily experience missing or duplicate tweets from their timelines. Normal timelines will be restored shortly.

Update: June 15th 04:11 AM: The site is fully recovered.

Please Note: Users may temporarily experience missing or duplicate tweets from their timelines. Normal timelines can restored by refreshing your browser.

Some users are noting inconsistencies in tweet counts. This is purely a display issue and the tweets are not lost or missing. We are working to fix this underlying problem with counts.

Users are seeing an elevated number of errors on twitter.com and we’re working to recover from this problem.

We’re seeing large bursts of errors every twenty minutes. We’re currently investigating.

Update: 08:50 AM PDT:  Our sidebar trends feature is exerting an elevated load on one of our databases causing the bursts of errors. As such, we’ve temporarily disabled sidebar trends and the site has fully recovered. We don’t yet have an estimated time when sidebar trends will be re-enabled.

Update: 1:00 PM PDT: We’ve re-enabled side bar trends.

We are experiencing site-wide availability issues due to scheduled network maintenance. We’re currently working to address the issues. We’ll update with more information as it is available.

Update (3:30 AM PDT): We are now recovering from a brief outage related to network maintenance, though we do not yet have an ETA on full recovery.

Update (4:00 AM PDT): We’re on our way to complete recovery and expect to be fully recovered soon.

We’re seeing periodic high rates of errors on Twitter.com. We’re currently working to resolve this.

At 6:39 AM PDT / 13:39 UTC the site is returning an elevated rate of errors and timelines are sporadically backing up and recovering. Teams are responding to this incident.

Update: 7:00 AM PDT / 14:00 UTC. Incident resolved. The system is functioning normally.

Update: 2:05 PM PDT / 21:06 UTC. The site is returning an elevated rate of errors and timeline are sporadically backing up and recovering. Teams are responding to the incident.

We are experiencing site availability issues — additional latency and errors — this morning. We’re working to address these issues.

Update: 9:07 PDT / 16:07 UTC: Users may notice that their timelines are updating slowly. All tweets will eventually be delivered.

Update: 9:32 AM PDT / 16:32 UTC: To help stabilize the site we’re turning off a set of features including but not limited to: Twitter Search, hovercards, web sidebar data (trends, friends counts, etc), and profile image uploads.

Update 9:57 AM PDT/ 16:32 UTC: Timelines are now current. Various other features remain disabled.

Update 10:34 AM PDT / 17:34 UTC: All site features have been re-enabled. Latency and error rates are recovering quickly.

Update 12:29 PM PDT / 19:29 UTC: We’ve identified the cause of today’s incident as an error with networking equipment. This networking error prevented us from serving at full capacity.

Update 2:25 PM PDT / 21:25 UTC: We’re experiencing a recurrence of this morning’s networking issue. We’re investigating.

We had a system-side issue this morning starting around 0800 Pacific that resulted in  twitter.com experiencing two 30 minute back-to-back incidents over roughly one hour.  The problem has largely been addressed, but we aren’t quite to 100 percent yet.