The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Results from Engineering forAugust 2012

Joining the Linux Foundation

Monday, August 27, 2012 | By Chris Aniszczyk (@cra) [19:10 UTC]
Topics:

Today Twitter officially joins the Linux Foundation (@linuxfoundation), the nonprofit consortium dedicated to protecting and promoting the growth of the Linux operating system.Read more…

How we spent our Summer of Code

Friday, August 24, 2012 | By Chris Aniszczyk (@cra) [19:06 UTC]
Topics:

For the first time, Twitter participated in the Google Summer of Code (GSoC) and we want to share news on the resulting open source activities. Unlike many GSoC participating organizations that focus on a single ecosystem, we have a variety of projects spanning multiple programming languages and communities.Read more…

Crowdsourced data analysis with Clockwork Raven

Thursday, August 16, 2012 | By Chris Aniszczyk (@cra) [16:59 UTC]
Topics:

Today, we’re excited to open source Clockwork Raven, a web application that allows users to easily submit data to Mechanical Turk for manual review and then analyze that data. Clockwork Raven steps in to do what algorithms cannot: it sends your data analysis tasks to real people and gets fast, cheap and accurate results. We use Clockwork Raven to gather tens of thousands of judgments from Mechanical Turk users every week.Read more…

Visualizing Hadoop with HDFS-DU

Tuesday, August 7, 2012 | By Chris Aniszczyk (@cra) [18:06 UTC]
Topics:

We are a heavy adopter of Apache Hadoop with a large set of data that resides in its clusters, so it’s important for us to understand how these resources are utilized. At our July Hack Week, we experimented with developing HDFS-DU to provide us an interactive visualization of the underlying Hadoop Distributed File System (HDFS).Read more…

Trident: a high-level abstraction for realtime computation

Thursday, August 2, 2012 | By Twitter (@twitter) [17:00 UTC]
Topics:

Trident is a new high-level abstraction for doing realtime computing on top of Twitter Storm, available in Storm 0.8.0 (released today). It allows you to seamlessly mix high throughput (millions of messages per second), stateful stream processing with low latency distributed querying.Read more…

TwitterCLDR: Improving Internationalization Support in Ruby

Wednesday, August 1, 2012 | By Chris Aniszczyk (@cra) [18:43 UTC]
Topics:

We recently open sourced TwitterCLDR under the Apache Public License 2.0. TwitterCLDR is an “ICU level” internationalization library for Ruby that supports dates, times, numbers, currencies, world languages, sorting, text normalization, time spans, plurals, and unicode code point data. By sharing our code with the community we hope to collaborate together and improve internationalization support for websites all over the world.Read more…