Site streams

Overview

Site Streams allows services, such as web sites or mobile push services, to receive real-time updates for a large number of users. Events may be streamed for any user who has granted OAuth access to your application. Desktop applications or applications with few users should use user streams.

Site Streams is currently in a limited beta. Access is restricted to whitelisted accounts.

Endpoint

Site streams messages

Types of messages

Site Streams deliver the same types of messages streamed to User Streams, with an additional wrapper indicating the user the message is targeted to. See streaming message types for information.

Data from accounts the user follows

The with parameter controls the types of messages received. The default for Site Streams is with=user, which gives data about the user but not messages from the user’s followings.

Protected data

Site Streams may deliver both public and private data in a single stream. Applications must be meticulous about what data they display:

  • Data should be restricted by default, and only anonymously accessible if you can verify it to be public. This is essential to prevent new private message types or similar changes from being shared by default.
  • Private data must only be displayed to users who present the corresponding authentication token.
  • Do not expose or otherwise draw attention to status deletions or unfollows to users, other than to update displays.

These rules are covered in the Developer Rules of the Road. Violations will result in immediate suspension from Site Streams and possibly all Twitter APIs.

Connecting

Site Streams only accepts OAuth-signed requests from the owner of the application which is connecting. Use the “Your Access Token” feature of your application settings page to quickly obtain a valid access token.

When connecting, specify a list of users to include in the stream using the follow parameter. Note that you will only receive messages for users who have approved (and not revoked) OAuth access to your application at some point in the past.

Minimize the number of reconnections you make by modifying open streams with control streams.

Streams with large numbers of users may consume significant (> 1 Mbit/sec) bandwidth. Follow best practices for processing streaming data and keep in mind that slow reading applications will automatically be disconnected.

Limits

The following limitations must be respected during the beta period. These limits will most likely change when Site Streams comes out of beta:

  • Limit the follow count to 100 users per stream while connecting. Use Control Streams to add up to 1,000 users per connection. See Control Streams for Site Streams.
  • Open no more than 25 new connections per second and exponentially back-off on errors.
  • If you intend to open more than roughly a thousand connections, you should coordinate your testing and launch with the Twitter Platform team.

Best Practices

Be familiar with User Streams

Read the user streams documentation. Site Streams are similar to User Streams—other than multiplexing many User Streams over a connection, almost all of the guidelines outlined also apply to Site Streams.

Modifying streams

As new users authorize your application, you will need to add them to your streams. The best way to do this is to use control streams.

If you are unable to use control streams, open a series of small streams for recently added users. Once you have a sufficient number of small streams open, create a larger stream that contains all recent users and disconnect the small streams.

Do not attempt to open all connections at once. Implement a cap on the number of connections you make per second. The number of connections your application establishes (even across multiple hosts) must be less than 25.

Applying for access

Site Streams access is not given to every applicant. Applicants will be selected depending on availability & current requirements.

Site Streams does not support any of the filter parameters such as track or keywords, so if your application requires these capabilities, Site Streams may not be the right fit. Site Streams access does not provide a greater percentage of the firehose than the self-serve options available today.

There is currently no estimated date for Site Streams to exit beta.

Understanding the above, the steps to apply for access are:

  1. Read all of the streaming API documentation and understand best practices around establishing and maintaining streaming connections.
  2. Follow @twitterapi.
  3. Follow the Twitter Developer Blog. Announcements of API changes and release plans will be posted there.
  4. Implement a single-user proof of concept using user streams, taking into consideration queueing, processing, REST fallback support, and scaling requirements for expanding to many users.
  5. Apply for beta access to the Site Streams API by filling out the form at http://bit.ly/sitestreams. If approved, you will receive an email with further instructions.
  6. If approved, please provide feedback! Feedback, bug reports, and feature requests should be directed to Discussions.