Tweet updates

Introduction

This document outlines developer guidance around changes to Tweets that have been put in place to allow people to express even more on the platform.

Recent revisions:

  • On September 26, 2017, Twitter announced an experiment enabling Tweets with more characters, available to a small group of users. For JSON received from enterprise products, these Tweets are encoded in JSON as an extended Tweet (see below for more information). See the bottom of this page for a sample JSON payload from the enterprise 30-day Search API. 
  • Clarification on .@ syntax
  • Additional JSON sample payloads
  • Additional information added on the compose options (specifically the attachment_url, auto_populate_reply_metadata and exclude_reply_user_ids parameters)
  • Additional information on error codes associated with the new scenarios
     

Overview

The following chart and sample JSON demonstrate the differences between Tweet JSON objects in the various API endpoints (standard, premium and enterprise). If the formatting is difficult to read, please see the documentation and samples on our corresponding Github repo. Additional samples are available on Github and also via the Gnip support documentation.

Mode Objective Availability Details Example Tweets (JSON)
Compatibility Tweet payload will work with all existing integrations, regardless of the content of the Tweet Default for all standard REST APIs

text field is truncated to 140, as needed

truncated field is true

any entities only include those in the available 140 text range
Compatibility with additional extended_tweet in payload Maintain backward compatibility with a non-breaking, payload addition Default for all enterprise and premium APIs, as well as standard streaming APIs.

Includes new extended_tweet field in payload, containing:

  • full_text
  • display_text_range
  • entities
  • extended_entities
Extended Tweet payload contains all information to render Tweets that contain more than 140 characters

Standard REST APIs only: add the below parameters to any endpoint:

tweet_mode=extended

full_text replaces text

truncated field is false

display_text_range delineates different sections of body for mentions/tweets/entities

Developer guidance

To provide the best experience for your users, be sure you’ve built your applications so that they do not include hard-coded length assumptions for the JSON payload or the Tweet display. 

We’ve designed the Tweet payload and most recent experiments to minimize impact to your systems, so that they’ll continue to function without breaking. However, if you have chosen to hard-code assumptions about the lengths of Twitter objects or the values inside them, you may only receive partial JSON payloads. In this case, your displays may not properly show a full Tweet, or you may miss some of the content of the extended entities.
 

Previous updates:

Simplified replies and media attachments

We've simplified the way that replies work on Twitter by moving some of the “scaffolding” of Tweets into display elements so that they no longer count towards the text character limit within the Tweet.

  • Simplified replies: @names that auto-populate at the start of a reply Tweet will not count towards the character limit (but new non-reply Tweets starting with a @mention will count, as will @mentions added explicitly by the user in the text body of the Tweet).
  • Media attachments: A URL at the end of Tweets generated from attaching photos, a video, GIF, poll, Quote Tweet, or DM deep link will also not count towards the character limit (URLs typed or pasted inside the Tweet will be counted towards the character limit as they do today).

This set of changes introduces new limits around the numbers of specific elements that may be included as part of a Tweet (specifically, mentions).

Compatibility, and what this means for developers

Backward and forward compatibility for third-party clients and other API users are primary considerations.

There are a number of areas that are impacted by these changes:

  • the standard REST and Streaming APIs
  • the Ads API
  • the enterprise and premium data products
  • display products, such as Twitter Kit for embedded Tweets and timelines displayed on iOS, Android, and Web.
     

Tweet object changes

The following things change within Tweet payloads:

The displayed text in a Tweet does not exceed the Tweet text length limit, but - when usernames or attachment URLs are included at the appropriate points in the Tweet - the text content of the overall Tweet JSON object will be able to exceed the Tweet text length limit. Developers must avoid hard-coding length assumptions into their applications.

The text shall be logically divided into three regions:

A hidden prefix region that may contain one or more space-separated @mentions which shall not be rendered as part of the display text, but must instead be rendered as metadata.

A display text region, which matches the Tweet text length limit.

A hidden suffix region that may contain one attachment URL which shall not be rendered as part of the display text, but must instead be rendered as metadata. This region is limited to containing a single URL entity that identifies an attachment resource: currently, one to four photos, a GIF, video, poll, Quote Tweet, or DM deep link.

  • Note: URLs for Quote Tweet or DM deep links that are typed or pasted into a Tweet will still count against the character limit. The new attachment_url parameter on the POST statuses/update endpoint will enable valid link formats to be attached to a Tweet. They will not count against the character limit when this method is used.

If the text contains a hidden prefix or suffix region, then the Tweet object shall contain values to identify the start and end indices of the region of the text to be displayed as the Tweet text.

Example payloads are provided in the appendix.
 

Rendering modes

There are two modes for rendering Tweet JSON objects to API clients: compatibility mode and extended mode. Compatibility mode is the default mode for the standard REST and Streaming APIs and Gnip products, and is designed to not break existing clients.

REST API clients may opt into the extended mode via a request parameter.

In the future, an additional announcement will be made when the time is right to make a change for the rendering mode to default to extended mode.
 

Compatibility mode JSON rendering

In compatibility mode, Classic Tweets will be rendered exactly as today.

For Extended Tweets in compatibility mode, the following will be true:

  1. The existing text field will contain a truncated version of the Tweet text, followed by an ellipsis character, a space, and a shortened self-permalink URL. The total length of this text value shall not exceed 140 characters.
  2. The existing truncated field will be set to true.
  3. The existing entity fields (mentions, urls, media, etc.), will only contain entities that are fully contained within the text value. The from and to indices within each entity must be a valid code point index within the text value. The truncation point will avoid truncating mid-entity. A URL entity for the appended self-permalink will be appended to the list of entities.
  4. The payload may contain a new dictionary field named extended_tweet (this is specific to the Streaming and Gnip APIs). This will contain the following sub-fields:
    • full_text: contains the full, untruncated Tweet text.
    • display_text_range: an array of two unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the Tweet.
    • entities/extended_entities, etc.: The full set of entities.
  5. If the Tweet contains a Quote Tweet permalink URL, then the resulting embedded Quoted Tweet, if any, will still be included even if the permalink URL is not included in the truncated text.
  6. If the Tweet contains a URL entity that results in an attached card, then the card will still be included even if the original URL entity is not included in the truncated text.
  7. Since native media is only represented via entities, those will be missing from the truncated list of entities, but will be in extended_tweet.entities.
     

Extended mode JSON rendering

In extended mode, the following will be true both for Classic Tweets and Extended Tweets:

  1. The text field is no longer included; instead, the payload will contain a field named full_text, which contains the entire untruncated Tweet text.
  2. The payload shall contain a field named display_text_range, which is an array of two unicode code point indices, identifying the inclusive start and exclusive end of the displayable content of the tweet.
  3. The truncated field will be set to false.
  4. The entity fields will contain all entities, both hidden and displayable.
     

Limits

There are restrictions placed on the content of the text. This is to improve the end-user experience, and to encourage high quality content. Tweets will be rejected at creation time if they exceed the new entity limits, via new API error codes. These restrictions are enforced on all Tweets, regardless of overall character count (this represents a change to the existing methods that support creating new Tweets).

The numbers listed below are intended as initial guidelines.

  • overall Tweet text: limited to 3,000 Unicode code-points, after applying Unicode Normalization Form C.
  • @mentions: a limit of 50 @mentions per Tweet in the hidden region. This is enforced on the server side, so that users cannot exceed this number.
  • existing numbers and sizes of media attachments remain unchanged (up to 4 images represented by a single URL, or 1 GIF, or 1 video). Links that are added to the Tweet in order to link to media attached via the Twitter app or website (aka “native media links”) do not count, but links typed or pasted into the compose box may do so.
     

API changes

Standard REST API endpoints

Composition

The REST API endpoints that create new Tweets (statuses/update) will accept a new boolean parameter when a Tweet is sent as a reply to a conversation: auto_populate_reply_metadata (true to enable, false to disable, false being the default). The existing in_reply_to_status_id must also be set. The leading @mentions will subsequently be looked up from the original Tweet, and added to the new Tweet from there. In cases where the original Tweet has been deleted, the reply will fail. This is a change to existing behaviour, where it has been possible to reply to a deleted Tweet ID.

For older clients that are not updated for the auto_populate_reply_metadata option, mentions will continue to be included in the body of the Tweet and the server will decide on how to render the new Tweet.

The auto_populate_reply_metadata option will append @mentions into the metadata as a reply chain grows, until the limit on @mentions is reached. In order to edit down the list of handles, an additional option, exclude_reply_user_ids, will enable specific IDs (apart from the leading one) to be excluded from a reply. This parameter is an optional, comma-separated list of user ids which will be removed from the server-generated @-mentions prefix.

Note that the leading @mention cannot be removed as it would break the in-reply-to-tweet-id semantics. Attempting to remove it will be silently ignored.

In order for a URL to not be counted in the body of the Tweet, a new attachment_url parameter will be available on statuses/update to allow a client to attach it to the Tweet without explicitly adding it to the Tweet text. This URL must be a Tweet permalink, or DM deep link. Arbitrary, non-Twitter URLs should remain in the Tweet text and will count against the Tweet text length limit. URLs passed to the attachment_url parameter not matching either a Tweet permalink or DM deep link will fail at Tweet creation and cause an exception.

Consumption

Any REST API endpoints that return Tweets will accept a new tweet_mode request parameter.

Valid request values are compat and extended, which give compatibility mode and extended mode, respectively.

The default mode (if no parameter is provided) is compatibility mode, to support older clients and display methods.

Tweets rendered in compatibility mode via the standard REST API will not contain the extended_tweet field. REST API clients that wish to get the full text can instead opt into extended mode.

Error codes

Due to the limitations listed above, new API response and error codes have been introduced. These reflect the new content requirements listed in the Limits section above.

44 attachment_url parameter is invalid. Corresponds with HTTP 400. The URL value provided is not a URL that can be attached to this Tweet.
385 You attempted to reply to a tweet that is deleted or not visible to you. Corresponds with HTTP 403. A reply can only be sent with reference to an existing public Tweet.
386 The Tweet exceeds the number of allowed attachment types. Corresponds with HTTP 403. A Tweet is limited to a single attachment resource (media, Quote Tweet, etc.)

Standard streams

The Streaming API does not provide the same ability to provide query parameters to configure request options. Therefore, the Streaming API renders all Tweets in compatibility mode at this time.

Tweets rendered in compatibility mode for the streaming APIs, unlike for the REST APIs, will include the extended_tweet field for any extended tweet. This is necessary to avoid breaking existing clients by sending text that is longer than they expect in the existing text field, and also to provide the entirety of the data in a single stream. If there is an extended_tweet field, it will also include the ranges described above.

Streaming API consumers should update their code to first check for the presence of the extended_tweet dictionary, and use that in preference to the truncated data as is applicable for their use case. When extended_tweet is not present, they must fall back to using the existing fields.

In the future, a date for a switchover to extended mode will be announced, after which time apps should be able to process the newer Tweet payloads.

Enterprise and premium (REST and Streaming APIs)

In the case of enterprise and premium products, both the REST and streaming endpoints follow a similar pattern to the standard Streaming API, and the current versions of the data products APIs will render Tweets in compatibility mode, with the extended_tweet field.

The impact of this change was designed to be a minimal, additive and opt-in, non-breaking change. Gnip customers have to make a code change to “opt-in” to utilize the new additive fields when present. They may also want to prepare for the impacts of increased payload sizes, including storage and bandwidth implications.

In addition to payload changes, Gnip operators and enrichments will begin to analyze the longer text and entities as opposed to the truncated version.

Tweet display on Web, iOS, Android

Web

Twitter’s web embed products are powered by our widgets.js JavaScript library which are automatically updated to support new Tweet display formats without additional configuration needed from publishers.

iOS, Android

The current version of the Twitter Kit libraries support the extended Tweet formats.

Tweet Web Intent, Twitter Kit Tweet Composer

We currently have no planned changes to the Tweet web intent, or to the Tweet Composer functionality included in Twitter Kit.

 

Sample Tweet with 280 characters

  {
  "created_at": "Tue Sep 26 21:00:22 +0000 2017",
  "id": 912783930431905797,
  "id_str": "912783930431905797",
  "text": "Can\u2019t fit your Tweet into 140 characters? \ud83e\udd14\n\nWe\u2019re trying something new with a small group, and increasing the char\u2026 https:\/\/t.co\/y1rJlHsVB5",
  "source": "<a href=\"http:\/\/twitter.com\" rel=\"nofollow\">Twitter Web Client<\/a>",
  "truncated": true,
  "in_reply_to_status_id": null,
  "in_reply_to_status_id_str": null,
  "in_reply_to_user_id": null,
  "in_reply_to_user_id_str": null,
  "in_reply_to_screen_name": null,
  "user": {
    "id": 783214,
    "id_str": "783214",
    "name": "Twitter",
    "screen_name": "Twitter",
    "location": "San Francisco, CA",
    "url": "https:\/\/blog.twitter.com\/",
    "description": "Your official source for what\u2019s happening.\n\nNeed a hand? Visit https:\/\/support.twitter.com",
    "translator_type": "regular",
    "derived": {
      "locations": [
        {
          "country": "United States",
          "country_code": "US",
          "locality": "San Francisco",
          "region": "California",
          "sub_region": "San Francisco County",
          "full_name": "San Francisco, California, United States",
          "geo": {
            "coordinates": [
              -122.41942,
              37.77493
            ],
            "type": "point"
          }
        }
      ]
    },
    "protected": false,
    "verified": true,
    "followers_count": 61984546,
    "friends_count": 167,
    "listed_count": 90945,
    "favourites_count": 5165,
    "statuses_count": 6148,
    "created_at": "Tue Feb 20 14:35:54 +0000 2007",
    "utc_offset": -25200,
    "time_zone": null,
    "geo_enabled": true,
    "lang": "en",
    "contributors_enabled": false,
    "is_translator": false,
    "profile_background_color": "ACDED6",
    "profile_background_image_url": "http:\/\/pbs.twimg.com\/profile_background_images\/657090062\/l1uqey5sy82r9ijhke1i.png",
    "profile_background_image_url_https": "https:\/\/pbs.twimg.com\/profile_background_images\/657090062\/l1uqey5sy82r9ijhke1i.png",
    "profile_background_tile": true,
    "profile_link_color": "1B95E0",
    "profile_sidebar_border_color": "FFFFFF",
    "profile_sidebar_fill_color": "F6F6F6",
    "profile_text_color": "333333",
    "profile_use_background_image": true,
    "profile_image_url": "http:\/\/pbs.twimg.com\/profile_images\/875087697177567232\/Qfy0kRIP_normal.jpg",
    "profile_image_url_https": "https:\/\/pbs.twimg.com\/profile_images\/875087697177567232\/Qfy0kRIP_normal.jpg",
    "profile_banner_url": "https:\/\/pbs.twimg.com\/profile_banners\/783214\/1503522865",
    "default_profile": false,
    "default_profile_image": false,
    "following": null,
    "follow_request_sent": null,
    "notifications": null
  },
  "geo": null,
  "coordinates": null,
  "place": null,
  "contributors": null,
  "is_quote_status": false,
  "extended_tweet": {
    "full_text": "Can\u2019t fit your Tweet into 140 characters? \ud83e\udd14\n\nWe\u2019re trying something new with a small group, and increasing the character limit to 280! Excited about the possibilities? Read our blog to find out how it all adds up. \ud83d\udc47\nhttps:\/\/t.co\/C6hjsB9nbL",
    "display_text_range": [
      0,
      239
    ],
    "entities": {
      "hashtags": [
        
      ],
      "urls": [
        {
          "url": "https:\/\/t.co\/C6hjsB9nbL",
          "expanded_url": "https:\/\/cards.twitter.com\/cards\/gsby\/4ubsj",
          "display_url": "cards.twitter.com\/cards\/gsby\/4ub\u2026",
          "unwound": {
            "url": "https:\/\/cards.twitter.com\/cards\/gsby\/4ubsj",
            "status": 200,
            "title": "Giving you more characters to express yourself",
            "description": null
          },
          "indices": [
            216,
            239
          ]
        }
      ],
      "user_mentions": [
        
      ],
      "symbols": [
        
      ]
    }
  },
  "quote_count": 0,
  "reply_count": 16027,
  "retweet_count": 47906,
  "favorite_count": 78829,
  "entities": {
    "hashtags": [
      
    ],
    "urls": [
      {
        "url": "https:\/\/t.co\/y1rJlHsVB5",
        "expanded_url": "https:\/\/twitter.com\/i\/web\/status\/912783930431905797",
        "display_url": "twitter.com\/i\/web\/status\/9\u2026",
        "indices": [
          117,
          140
        ]
      }
    ],
    "user_mentions": [
      
    ],
    "symbols": [
      
    ]
  },
  "favorited": false,
  "retweeted": false,
  "possibly_sensitive": false,
  "filter_level": "low",
  "lang": "en",
  "matching_rules": [
    {
      "tag": null
    }
  ]
}