Results from Engineering for2011

SpiderDuck: Twitter's Real-time URL Fetcher

Monday, November 14, 2011 | By Twitter Engineering (@TwitterEng) [18:44 UTC]

Tweets often contain URLs or links to a variety of content on the web, including images, videos, news articles and blog posts. SpiderDuck is a service at Twitter that fetches all URLs shared in Tweets in real-time, parses the downloaded content to extract metadata of interest and makes that metadata available for other Twitter services to consume within seconds.

Several teams at Twitter need to access the linked content, typically in real-time, to improve Twitter products. For example:Read more…

Twitter’s mobile web app delivers performance

Wednesday, September 14, 2011 | By Twitter Engineering (@TwitterEng) [17:43 UTC]

As the number of people using Twitter has grown, we’ve wanted to make sure that we deliver the best possible experience to users, regardless of platform or device. Since twitter.com is not optimized for smaller screens or touch interactions familiar to many smart phones, we decided to build a cross-platform web application that felt native in its responsiveness and speed for those who prefer accessing Twitter on their phone’s or the tablet’s browser.Read more…

Finagle: A Protocol-Agnostic RPC System

Friday, August 19, 2011 | By Twitter Engineering (@TwitterEng) [22:55 UTC]

Topics:

Finagle is a protocol-agnostic, asynchronous RPC system for the JVM that makes it easy to build robust clients and servers in Java, Scala, or any JVM-hosted language.Read more…

A Storm is coming: more details and plans for release

Thursday, August 4, 2011 | By Twitter Engineering (@TwitterEng) [19:40 UTC]

We’ve received a lot of questions about what’s going to happen to Storm now that BackType has been acquired by Twitter. I’m pleased to announce that I will be releasing Storm at Strange Loop on September 19th! Check out the session info for more details.Read more…

Fast Core Animation UI for the Mac

Friday, July 1, 2011 | By Twitter Engineering (@TwitterEng) [16:55 UTC]

Starting today, Twitter is offering TwUI as an open-source framework (https://github.com/twitter/twui) for developing interfaces on the Mac.Read more…

Join the Flock!

Wednesday, June 22, 2011 | By michael abbott (@mabb0tt) [20:30 UTC]

Twitter’s engineering team is growing quickly. Two-thirds of our engineers were hired in the last 12 months. Those engineers joined us from cities and countries around the world and from companies of various sizes.Read more…

The Engineering Behind Twitter’s New Search Experience

Tuesday, May 31, 2011 | By Twitter Search (@twittersearch) [21:57 UTC]

Today, Twitter launched a personalized search experience to help our users find the most relevant Tweets, images, and videos. To build this product, our infrastructure needed to support two major features: relevance-filtering of search results and the identification of relevant images and photos.Read more…

Faster Ruby: Kiji Update

Thursday, May 19, 2011 | By attila szegedi (@AttilaSzegedi) [20:24 UTC]

In March 2011, we shared Kiji, an improved Ruby runtime. The initial performance gains were relatively modest, but laid the foundation for future improvements. We continued the work and now have some excellent results.Read more…

Twitter Search is Now 3x Faster

Wednesday, April 6, 2011 | By Krishna Gade (@krishnagade) [18:30 UTC]

In the spring of 2010, the search team at Twitter started to rewrite our search engine in order to serve our ever-growing traffic, improve the end-user latency and availability of our service, and enable rapid development of new search features. As part of the effort, we launched a new real-time search engine, changing our back-end from MySQL to a real-time version of Lucene.Read more…

Improving Browser Security with CSP

Tuesday, March 22, 2011 | By Mark Percival (@mdp) [17:11 UTC]

If you are using Firefox 4, you now have an extra layer of security when accessing mobile.twitter.com.Read more…

The Twitter Engineering Blog