The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Results from Engineering forMarch 2012

Security Open House March 29

Thursday, March 22, 2012 | By Twitter Engineering (@TwitterEng) [21:40 UTC]

The past few months have been busy for the Twitter security team: we’ve turned on HTTPS by default for everyone, added great engineers from Whisper Systems and Dasient, and had some stimulating internal discussions about how we can continue to better protect users. We want to share what we’ve been up to and discuss the world of online security, so we’ll be hosting a Security Open House on March 29 here at Twitter HQ. We’ve got a great lineup of speakers to get the conversations going:Read more…

Cassovary: A Big Graph-Processing Library

Thursday, March 8, 2012 | By Twitter Engineering (@TwitterEng) [19:18 UTC]

We are open sourcing Cassovary, a big graph-processing library for the Java Virtual Machine (JVM) written in Scala. Cassovary is designed from the ground up to efficiently handle graphs with billions of edges. It comes with some common node and graph data structures and traversal algorithms. A typical usage is to do large-scale graph mining and analysis.Read more…

Generating Recommendations with MapReduce and Scalding

Friday, March 2, 2012 | By Twitter Engineering (@TwitterEng) [21:54 UTC]

Scalding is an in-house MapReduce framework that Twitter recently open-sourced. Like Pig, it provides an abstraction on top of MapReduce that makes it easy to write big data jobs in a syntax that’s simple and concise. Unlike Pig, Scalding is written in pure Scala — which means all the power of Scala and the JVM is already built-in. No more UDFs, folks!Read more…