Ted Leung on the Air

Solid 2014

June 1st, 2014

Two weeks ago, I had the chance to go to the inaugural O’Reilly Solid conference. Solid was billed as the Software / Hardware / Everywhere conference. The “Internet of Things” is a big buzzword right now, and I am glad that O’Reilly decided not to do an “Internet of Things” conference. There were plenty of “Internet of Things” exhibits and talks, to be sure, but the theme of the conference was bigger than that, and I think rightly so.

There is a lot going on in the hardware space, and Renee DiResta from O’Reilly Alphatech Ventures gave a good talk on hardware trends. The 2014 version of Mary Meeker’s Internet Trends report just came out, and I wonder if DiResta’s hardware trends will someday rise to similar stature. One of my favorite quotes from her talk: “software is eating the world, hardware gives it teeth”.

Changing the dynamics of working with hardware

The most important thing that is happening in hardware is that there is a lot of energy going into making it easy to design and fabricate physical things. This is true for electronic hardware but also for any product that has a physical embodiment.

Nadya Peek from the MIT Center for Bits and Atoms gave a talk on machine tools that can build machine tools. This is important because building and setting up machine tools is one of the critical path tasks in manufacturing physical objects. These tools could help cut the cycle time and cost for tooling. Also, in the spirit of the era, the designs for her machines are open source at http://mtm.cba.mit.edu/

Carl Bass, the CEO of Autodesk talked about how Autodesk took a step back and reimagined Autocad in the the modern world, against three principles:

“Infinite computing” – treat computing power as if it were super cheap
“Cloud based” – for collaboration and for delivery – you can get Autodesk Fusion 360 for $25/mo
“Computational design” – put computing power to work to do parallel search / exploration of designs

The bottom line here is that anyone with $25/mo can have access to start of the art 3d CAD/CAM tools, based on the AutoCAD platform used by designers and engineers everywhere.

Microsoft did a live coding demo where they using an Intel Quark board to build a sound pressure meter. This was coupled with some very nice integration with Visual Studio. I’m not a big Microsoft/Intel fan for this space, but I think that this showed the potential for improving the toolchain and round trip experience for platforms like Arduino.

Julie bought our 11 year old a set of Littlebits as a way to get her some knowledge about electronics. She is a very hands on kind of learner and Littlebits seemed like a good vehicle for her. I didn’t do much homework on this because Julie did most of the research. So my impression of Littlebits was that it was sort of aimed at a kids education kind of space. Some of their marketing like “Lego for the iPad generation”, also gives that impression. Ayah Bdeir, the CEO of gave a great talk on LIttlebits and the vision of where it is going. The notion of making electronics very accessible, to the point where it can be viewed as just another material in someone’s creative toolbox, really resonated with the theme of Solid, and with our own angle on Littlebits. Littlebits is a great modular hardware platform that makes it easy to prototype electronics very rapidly, and it’s the first step in a longer journey of “materializing” electronics. It was a plus to be able to stop by the Littlebits booth and thank Ayah for Littlebits and the fact that I had to explain PWM to my 11 year old.

Vision

There were several talks which stood out because they painted a good picture of some things which are further out, but which appear to be obtainable given consistent application of product improvement and sufficient resources.

Neil Gershenfeld, the director of the MIT Center for Bits and Atoms, talked about fabrication as digital process. The key idea is that at the moment all the knowledge about fabrication is in the machinery doing the fabrication, not in the material being fabricated. He talked about what becomes possible when physical fabrication become more akin to programming.

Hiroshi Ishii from the MIT Tangible Media group talked about Tangible Bits (user interfaces based on direct manipulation of physical embodiments of bits) and Radical Atoms (materials that can change form based on digital information)

Ivan Poupyrev is now at Google but was a part of the Disney Research labs. He demonstrated several inventions from his time at Disney. Touche, which can turn lots of ordinary surface into touch sensitive input devices. He demonstrated a way to generate minute amounts of power locally without batteries, via rubbing pieces of paper together (as in turning the pages of a book). His final demo, Aireal, is a way of using puffs of air to create invisible haptic displays. The overall theme for all of these inventions was “How can we make the whole world interactive”?

Beth Comstock is the Chief Marketing Officer at GE. She talked about the fact that GE makes a huge range of machines of all kinds, and has experience with all kinds of materials and electronics. GE is looking ahead to taking those machines and enhancing them and the data that they produce via software. GE is a physical first company that is in the process of becoming digital, and leading us into a Brilliant Age. In this age:

We’re going to have to learn to speak industrial machine – we need to be able to deal with the immense amount of data generated by sophisticated machines
The Selfish Machine – machines will use data to introspect about their own performance and operation and alter their behavior, ask humans for help, or provide input for the design of the next generation
The Selfless Machine – machines will be able to exchange data with each other to coordinate with other machines
Machine Knock Down that Wall – machines will impact the process of making machines – rapid iteration of hardware design, open innovation processes

Robotics

In some ways, robotics is one of the culiminations of the software / hardware / everywhere mantra. There were lots of talks and exhibits on robotics at Solid. Here are 2 that stood out.

Rod Brooks is the founder of Rethink Robotics and a long time professor of robotics at MIT. I did some robotics as a grad student. Rod’s talk was a great way to get a small glimpse at the current state of the art. His company is doing some really interesting work in making industrial robots more flexible and easier to work with. One of the interesting points in his talk was that they are now working on ways to build robotic arms out of cheaper materials/hardware by leveraging more sophisticated software to compensate for the flaws of cheaper materials. At the end of his talk, he outlined for challenges for robotics:

Visual object recognition on a par with a 2 year old
The language capabilities (not vocabulary) of a 4 year old – noisy environments, grammer
Dexterity of a 6 year old
The social understanding (model of the world) of an 8 year old.

The other talk was Carin Meier’s robotic dance party. This one was purely selfish. My 13 year old is interested in space/robotics, so when Carin did a version of this keynote at OSCON, I showed my daughter the video. She ended up doing a school project on drones. As part of the way her school does projects, she needed to get an “expert”, so I gave her Carin’s email. That eventually resulted in some tweets. For Solid, Carin expanded the scope of the dancing and added some nice music control via Clojure’s overtone library. It was fun to find her afterwards and thank her in person for helping / inspiring my daughter.

Final Thoughts

The most motivational thing about Solid was a mix of ideas from Astro Teller’s talk about GoogleX, and a quote from Jeff Hammerbacher. Teller said (my paraphrase) “most of the world’s real problems are physical in nature, you can’t just solve them with software”. Hammerbacher is responsible for a quote that went around the internet back in 2011 that goes something like: “The best minds of my generation are thinking about how to make people click ads”. A lot of bright people have gone into software because the malleability of software and the internet means that you can make something and see the impact of it in a very short time. I’m excited to see progress in making hardware development more accessible and rapid. Perhaps that will lead to more bright minds finding ways to solve important physical world problems.

Tags: technology
Posted in misc | No Comments »

Strange Loop 2013

September 21st, 2013

It’s been a while since I have written a post, or been to a conference. I wish that I had time to write more and that I could write about what I am up to. In lieu of that, here is a report on Strange Loop 2013, which is the only conference that I am attending all year.

Emerging Languages Camp

One does not attend Strange Loop without attending Emerging Languages Camp (ELC). I view the camp as a kind of roulette. Of course it’s unlikely that many, or perhaps any of the languages presented in the forum will ever see widespread adoption. That’s not really the point. The camp is about getting language designers and implementors together, and giving them a forum to exchange ideas. Much of what has happened in languages recently is that we’ve been taking old ideas and trying to recast them in a modern context, either because computing platforms are now fast enough to bear the inefficiencies of those ideas, or because the computing community as a whole appears to be of a mind to accept them. ELC is also just a place for people who want to experiment with languages. Here was some of what stood out to me:

Gershwin – an embedding of Forth semantics in Clojure – apparently this can be helpful when one is using the Clojure threading macros all over the place
Noether – a highly stratified approach to language design. Unfortunately this talk dwelt too much on the philosophy and mathematical semantics and ran out of time before covering the details of the actual language. It’s a shame that we didn’t get to see the full content.
Qbrt bytecode – This was an interesting look at a byte code system for a different design space, where the byte code was representing somewhat high level functionality as opposed to machine level instructions.
J – J is cool because it’s basically APL 2.0. But now we also have R, Julia, and other languages. This is a space that has many contenders but no clear leader. My problem with J is that it looks even more write only than Perl.
BODOL – This talk wasn’t interest so much for the language, which the presenter acknowledged was “another toy lisp”, but for the presentation, which involved HTML based slides being displayed inside of Emacs. I felt like I was back in 6.001 watching the construction of a lisp, but the presentation quality was really high, which made for a great talk.

In addition to the talks I had a number of dinner and hotel lobby discussions around ELC related topics. The ELC attendees are a great bunch.

Sessions

Jenny Finkel from Prismatic gave a great overview of how they use machine learning in the product. As a user this was a great explanation of how the system really works. Machine learning in keynotes is fough because it’s not something that everyone has studied so it is hard to get the level right. I definitely enjoyed it. The most useful information in the talk was when she said that coding on an iPad version has begun. I will definitely be using the living daylights out of that when it comes out.

This year there was definitely a theme around making it easier to deal with asynchronous / event-driven systems. I was able to make it to two of the talks – there were several more. Matthew Podwysocki and Donna Malayeri from Microsoft presented on the Rx extensions, which I’ve written plenty about on this blog already. This time they came with some cool Kinect based demos. Nice to see something other than web systems. The other talk that I saw was Rich Hickey’s talk on core.async. As usual, Rich brought a great mix of theoretical concerns around core.async’s CSP based model, while melding it with treal world system building. I consider this to be a hallmark of Rich’s style, and he is one of the very very few people who is really able to fuse theory and practice into a whole. And of course, Clojure is the manifestation of that fusion. I’ve got a bunch of David Nolen’s posts on core.async in tabs in my browser, but just haven’t had the time to sit down and read them. I feel a little more caught up now.

Another talk that I really enjoyed was Martin Odersky’s keynote on “The Trouble with Types“. The beginning of the talk was about the usual comparison between static and dynamic typing, and the end of the talk was about his work on Dotty, which is his new work on types based on “projecting higher kinded functional types onto names in modules”. He hopes that Dotty will eventually become the foundation for a future version of Scala. The interesting part for me happened in the middle of the talk, because that was the part where he admitted to some of the problems with types. Like needing a debugger for your type system because it had become so computationally powerful. The aha moment for me was around his discussion of how orthongonality in the type system design had contributed to the problems that he saw with Scala’s type system. It is a tenet among computer scientist that orthogonality is desirable. It was one of the foundational arguments in the CISC vs RISC computer architecture wars, and it is a badge of honor among language designers to have as much orthogonality as possible. However, orthogonality leads to a potentially large surface area of possibilities and interactions, which users need to master and be aware of, and which implementors need to implement completely and efficiently. On reflection, this should be obvious, but the lights went on for me during the talk.

I stopped into to see the latest installment of Friedman and Byrd’s journey with MiniKanren. I was very interested to see their exploration of the Chomsky hierarchy (of computational complexity). As usual, this is related to my interest in Clojure’s core.logic. They “cheated” a little bit in what they showed, but it was still interesting.

Avi Bryant gave a great talk on the applicability of abstract algebra, well mostly monoids, to the type of streaming calculations that are needed in many analytics systems. He showed how this provides a foundation for solutions like hyperloglog, min-hash, and count-min sketch.

Crista Lopes gave a talk called Exercises in Style which was about styles of programming. She observed that within teams, one can often figure out who wrote a piece of code just by looking at it. Another observation was that in art, students are taught what comprises the various styles and how to produce them. I thought that this was leading to a very promising discussion. She then presented the same program written in 9 different styles that she had observed. The first 3-4 styles (which have numbers and not names yet) were really what I was expecting to see. As we moved to more styles, they started to look like language paradigms, which I think is less interesting. Lopes is working on a book on this topic and has 33 styles that she’s identified so far. I’ll be keeping my eye out for that.

Another theme at Strange Loop this year was diversity. You saw it both in the speakers roster and in the session content. I didn’t get a chance to ask him, but I am pretty sure that Alex Miller made a very concerted effort to invite women speakers, because there was a much higher number of women speakers than last year, and also much higher than any other conference that I can remember. On the content side, there were several sessions. There was a good presentation on the history of women in computing. I definitely learned a bunch of things. The focus was on the early history of computing which was great, but I was disappointed that several prominent women in recent history were omitted. That’s the problem with histories, invariable someone gets left out. Some history is better than no history, especially on this topic. Alex also invited Jen Myers to do one of the keynotes. I’m not sure how to summarize this presentation because there are too many different angles. There was the angle about diversity, there was the angle about boosting education, there was the angle of making something good because we are good as people. So rather than try, I’ll just insert a Ray Bradbury quote that Jen used in her talk. This version is longer than the version in the talk, but it speaks to me.

I think it’s part of the nature of man to start with romance and build to a reality. There’s hardly a scientist or an astronaut I’ve met who wasn’t beholden to some romantic before him who led him to doing something in life.

I think it’s so important to be excited about life. In order to get the facts we have to be excited to go out and get them, and there’s only one way to do that — through romance. We need this thing which makes us sit bolt upright when we are nine or ten and say, ‘I want to go out and devour the world, I want to do these things.’ The only way you start like that is with this kind of thing we are talking about today. We may reject it later, we may give it up, but we move on to other romances then. We find, we push the edge of science forward, and I think we romance on beyond that into the universe ever beyond. We’re talking not about Alpha Centauri. We’re talking of light-years. We have sitting here on the stage a person who has made the film* with the greatest metaphor for the coming billion years. That film is going to romance generations to come and will excite the people to do the work so that we can live forever. That’s what it’s all about. So we start with the small romances that turn out to be of no use. We put these tools aside to get another romantic tool. We want to love life, to be excited by the challenge, to life at the top of our enthusiasm. The process enables us to gather more information. Darwin was the kind of romantic who could stand in the middle of a meadow like a statue for eight hours on end and let the bees buzz in and out of his ear. A fantastic statue standing there in the middle of nature, and all the foxes wandering by and wondering what the hell he was doing there, and they sort of looked at each other and examined the wisdom in each other’s eyes. But this is a romantic man — when you think of any scientist in history, he was a romancer of reality.

Alex has historically done a great job of getting great speakers for Strange Loop, and just just recent stars, but pioneers and old timers. This year we had Chuck Moore, Dan Friedman, and Douglas Hofstader. This year’s closing keynote was Hofstader, whose book “I am a Strange Loop” was the inspiration for the name of the conference. Hofstader’s talk was an exploration of that topic, and was everything that you could have hoped for given Hofstader’s amazing works of literature. What one could not have hoped for, however, was what followed. Alex commissioned David Stutz to do a multimedia performance based on Hofstader’s work. “Thrown for a Loop: A Carnival of Consciousness” was a performance that involved theater, a 5 piece brass quintet, Macintosh driven multimedia (including forays into Emacas and Clojure’s nREPL), and an aerialist. You will have to go and watch the video when it comes out, because I don’t have the words to describe it.

Miscellanea

Thursday night of Strange Loop we were treated to a conference party at the City Museum in St. Louis, which can only be described as part architectural museum and part playground, or as @petrellic put it “A habit rail for adults”. This was one of the most amazing venues that I have ever been to for a conference party. The three hours that we were allotted vanished quickly as we all explored the mysteries of the museums and its paths, trail, tunnels, stairways, and slides.

I’ve always had a little trouble describing what Strange Loop is to my coworkers and friends. I found a tagline, courtesy of @samberan: “Strange Loop is TED for programmers”.

Conference organizers take note: Strange Loop has seriously raised the already high bar on you.

Tags: internet, programming
Posted in internet, programming, programming languages | 2 Comments »

Strange Loop 2012

September 27th, 2012

I think that the most ringing endorsement that I can give Strange Loop is that it has been a very long time since I experienced so much agony when trying to pick which talks to go to during any given block.

Emerging Languages Camp

This year Strange Loop hosted the Emerging Languages Camp (ELC), which previously had been hosted at OSCON. I liked the fact that it was its own event, not yet another track in the OSCON panoply. That, coupled with a very PLT oriented audience this year, made Strange Loop a much better match for ELC than OSCON.

I definitely went into ELC interested in a particular set of talks. There is a lot of buzz around big data, and some of the problems around big data and data management more generally. Also I did my graduate work around implementing “database programming languages”, so there was some academic interest to go along with the practical necessity. There were three talks that fell into that bucket: Bandicoot: code reuse for the relational model, The Reemergence of Datalog, and Julia: A Fast Dynamic Language for Technical Computing.

I found Bandicoot a little disappointing. I think that the mid 90′ work of Buneman’s group at UPenn on Structural Recursion as a Query Language and Comprehension Syntax would be a better basis for a modulary and reusable system for programming relations.

Logic Programming may be making a resurgence via the work on core.logic in Clojure and the influence of Datalog on Cascalog, Datomic and Bloom. The Reemergence of Datalog was tutorial on Datalog for those who had never seen it before, as well as a survey of Datalog usage in those modern day systems.

Julia is a language that sits in the same conceptual space as R, SAS, SPSS, and so forth. The problem with most of those systems is that they were designed by statisticians and not programmers. So while they are great for statistical analysis, they are less good for statistical programming. Julia aims to improve on this, while adding support for distributed compuation and a very high performance implementation. There’s no decisive winner in the technical computing space, and it seems like Julia might have a chance to shine.

There were, of course, some other interesting language talks at ELC.

Dave Herman from Mozilla talked about Rust for the first time (at least to a large group). Rust is being developed as a systems programming language. There are some interesting ideas in it, particularly a very Erlang like concurrency model. At the same time, there were some scary things. Part of what Rust is trying to do is achieve performance, and part of how this happens is via explicit specification of memory/variable lifetimes. Syntactically this is accomplished via punctuation prefixes, and I was wondering if the code was going to look very Perl-ish. servo is browser engine that is being written in Rust, and looking at the source code of a real application will help me to see whether my Perlishness concern is valid.

Elixir: Modern Programming for the Erlang VM looks like a very nice way to program atop BEAM (the Erlang VM). Eliminating the prolog inspired syntax goes a long way, and it appears that Elixir also addresses some of the issues around using strings in Erlang. It wasn’t clear to me that all of the string issues have been addressed, but I was definitely impressed with what I saw.

Strange Loop Talks and Unsessions

I’m going to cover these by themes. I’m not sure these are the actual themes of the conference, but they are the themes that emerged from the talks that I went to.

First, and unsurprisingly, a data theme. The opening keynote, In Memory Databases: the Future is Now! was by Mike Stonebraker. It’s been a long time since I saw Stonebraker speak – I think that the last time was when I was in graduate school. He was basically making the case that transaction processing (TP) is not going away, and that there might be applications for a new generation of TP systems in some of the places where the various NoSQL systems are now being used. Based on that hypothesis/assumption, he then went on to describe the trends in modern systems and how they would lead to different design, much of which is embodied in VoltDB. This was a very controversial talk, at for some people. I considered the trend/system analysis part to be reasonable in a TP setting. I’m not sure that I agree with his views on the applicability of TP, but I’m fairly sure that time will sort all of that out. I think that this is an important point for the NoSQL folks to keep in mind. When the original work on RDBMS was done, it was mocked, called impractical, not useful and so forth. It took many years of research and technology development. I think that we should expect to see something similar with NoSQL, although I have no idea how long that timeline will be.

Nathan Marz’s talk Runaway Complexity in Big Data… and a plan to stop it. was basically making the case for, and explaining the hybrid/combined batch/realtime architecture that he pioneered at BackType, and which is now in production at Twitter. That same architecture led to Cascading and Storm, which are pretty interesting systems. Marz is working on a book with Manning that will go into the details of his approach.

The other interesting data talks revolved around Datomic. Unfortunately, I was unable to attend Rich Hickey’s The Database as a Value, so I didn’t get to hear him speak directly about Datomic. There are several Datomic related videos floating around, so I’ll be catching up on those. I was able to attend the evening unsession Datomic Q&A / Hackfest. This session was at 9pm, and was standing room only. I didn’t have quite enough background on Datomic to follow all of what was said, but I was very interested by what I saw: the time model, the immutability of data which leads to interesting scalability, the use of Datalog. I’m definitely going to be looking into it some more. The one thing that troubles me is that it is not open source. I have no problem with a paid supported version, but it’s hard to make the argument for proprietary system or infrastructure software nowadays.

Another theme, which carried over from ELC was logic programming. I had already heard Friedman and Byrd speak at last fall’s Clojure/conj, and I was curious to see where they have taken miniKanren since then. In their talk, Relational Programming in miniKanren, they demonstrated some of what they showed previously, and then they ran out of material. So on the fly, they decided to implement a type inferencer for simple lambda terms live on stage. Not only were they able to finish it, but since it was a logic program, they were also able to run it in reverse, which was pretty impressive. I was hoping that they might have some additional work on constraints to talk about, but other than disequality constraints, they didn’t discuss anything. Afterwards in Twitter, Alex Payne pointed out that there are some usability issues with miniKanren’s API’s. I think that this is true, but it’s also true that this is a research system. You might look at something like Clojure’s core.logic for a system that’s being implemented for practitioners.

David Nolen did an unsession Core Logic: A Tutorial Reconstruction where he walked the audience through the operation of core.logic, and by extension, miniKanren, since the two systems are closely related. He pointed out that he read parts of “The Reasoned Schemer” 8 times until he understood it enough to implement it, and then he found that he didn’t really understand it until after the implementation was done. There was also a large crowd in this session, and Christopher Petrelli made a video recording on his phone, since InfoQ wasn’t recording the unsessions.

The final talk in the logic programming them was Oleg Kiselyov’s talk Guess lazily! Making a program guess and guess well. Kiselyov has been around for a long time and written or coauthored many important papers related to Scheme and continuations. I’ve be following (off and on) his work for a long time, but this is the first time I was at a conference where he was speaking. I was shocked to find that the room was packed. His talk was about how to defer making the inevitable choices required by non-determinism, especially in the context of logic type systems. His examples were in OCaml, which I had some trouble following, but after Friedman and Byrd the day before, he apparently felt compelled to write a type inferencer that could be run backwards as well. His code was a bit longer than the miniKanren version.

The next theme is what I’d call effective use of functional programming. The first talk was Stuart Sierra’s Functional Design Patterns. This was a very worthwhile talk, which I won’t attempt to summarize since the slides are available. Needless to say, he found a number of examples that could be called design patterns. This was one of the talks where I need to sit down and look at the patterns and think on them for a while. That’s hard to do during the talk (and the conference, really). Some things require pondering, and this is one of them.

The other talk in this category was Graph: composable production systems in Clojure, which described the Prismatic team’s approach to composing systems in Clojure. What they have is an abstraction that allows them to declaratively specify how the parts of the system are connected. For a while it just looked to me like a way to encode a data flow graph in a Clojure abstraction. The aha moment was when he showed how they use Clojure metadata to annotate the arguments or pipe connectors if you will. The graphs can be compiled in a variety of ways including Clojure lazy maps, which present some interesting possibilities. Unfortunately, I had to leave half way through the talk, so I missed the examples of how the apply this abstraction in their system.

Theme number four was programming environments. I hesitate to use the term IDE, because it connotes a class of tools that is loved by some, reviled by others, and when you throw that term around, it seems to limit people’s imagination. I contributed to the Kickstarter for Light Table, so I definitely wanted to attend Chris Granger’s talk Behind the Mirror: The birth of Light Table. Chris gave a philosophical preamble before showing off the current version of Light Table. He demonstrated adding support for Git in a short amount of code, and went on to demonstrate a mode for developing games. He said that they are planning to release version 1 sometime in May, and that Light Table will be open source. I also learned that Kickstarter money is counted as revenue, so they have lost a significant amount of the donations to taxes, which is part of the reason that Kodawa participated in Y Combinator, and is trying to raise some money to get a bigger team.

Not long after the Light Table kickstarter, this video by Bret Victor made the rounds. It went really well with all the buzz about Light Table, and Alex Miller, the organizer of Strange Loop, went out and persuaded Bret to come and talk. Bret’s title was Taking off the Blindfold, and I found this to be a very well motivated talk. In the talk, Bret talked about the kinds of proerties that our programming tools should have. The talk was vey philosophical despite the appearance of a number of toy demos of environment features.

During both of these talks there was a lot of chatter. Some was harking back to the Smalltalk (but sadly, not the Lisp Machine) environments,while some questioned the value of a more visual style of tools (those emacs and vi graybeards). When I first got into computers I read book called “Interactive Programming Environments” and ever since i’ve always been wishing for better tools. I am glad to see some experimentation come back into this space.

Some old friends are busy making hay in the Node.js and Javascript communities, and it probably horrifies theme that I have ClojureScript as a theme, but so be it. I went to two ClojureScript talks. One was David Nolen’s ClojureScript: Better Semantics at Low Prices!, which was really state of the union of ClojureScript. The second was Kevin Lynagh’s Building visual data driven UI’s with ClojureScript. Visualization is becoming more and more important and ClojureScript’s C2 library look really appealing.

It’s fitting that the last them should be Javascript. Well, maybe. I went to two Javascript talks, and both of them were keynotes, so I didn’t actually choose them. But Javascript is so important these days that it really is a theme. In fact, it’s so much of a theme, that I’ve been going to Javascript conferences for the last 2 years. It’s been several years since I saw Lars Bak speak. His talk on Pushing the Limits of Web Browsers was in two parts. Or so I think. I arrived just as he was finishing the first part which seemed like an account of the major things that the V8 team has learned during their amazing journey of speeding up Javascript. The second part of his talk was about Dart. I didn’t know that Bak was the lead of the Dart project, but that doesn’t change how I feel about Dart. I see the language, I understand the rationale, and I just can’t get excited about it.

I’ve been to enough of those Javascript only talks to hear Brendan Eich talk about The State of Javascript. Brendan opened by giving a brief history of how Javascript got to be the way it is, and then launched into a list of the improvement coming in EcmaScript 6 (ES6). That was all well and good, and towards the end, after the ES6 stuff, he threw in some items that were new, like the sweet.js hygienic macro project, and the lljs typed JavaScript project. It seemed like this was a good update for this audience, who seemed unaware of all the goings on over in JavaScript land. From a PLT point of view, I guess that’s understandable, but at the same time, JavaScript is too important to ignore.

Final Thoughts

Strange Loop has grown to over 1000 people, much larger than when I attended in 2010 (I had to miss 2011). I think that Alex Miller is doing a great job of running the conference, and of finding interesting and timely speakers. This was definitely the best conference that I attended this year, and probably the last 2-3 years as well.

If you’re looking for more information on what happened at Strange Loop 2012:

Slides: https://github.com/strangeloop/strangeloop2012/tree/master/slides

Other Strange Loop Coverage: https://github.com/strangeloop/strangeloop2012/wiki/Coverage

Tags: data, functional programming, internet, languages
Posted in internet | No Comments »

JSConf 2012

April 7th, 2012

This year JSConf was in Scottsdale Arizona, which provided some welcome relief from the cold, wet, Seattle winter/spring.

News

One of the biggest pieces of news was that Mozilla gave all attendees a Nexus S smartphone running a developer version of the Boot to Gecko (B2G) phone operating system. When I say developer, I mean, camera support was broken, things were crashing, that sort of thing. These phones were a big hit among the attendees. They contributed to knocking the conference wifi out temporarily, and I saw several groups of people who were working on projects for the phone. My experience at Google I/O had soured me on the idea of giving away free devices. In the case of Google I/O, device giveaways have become an expectation, and there is some proportion of people who sign up for that conference based on the hope of getting a free device. Still, Mozilla is going to need all the help that they can get, and people seemed to take the challenge to heart. I did find it interesting that the Mozilla folks were speaking of B2G as a great feature phone software stack. This is a realistic way of climbing up the stairs in the mobile phone market. It’s hard to imagine a real competitor to iOS and Android, but I’m glad to see an effort in this direction. There’s WebOS, Windows Phone 7, and B2G all using some variant of the open web stack. It seems like there ought to be some collaboration between B2G and WebOS’s Enyo framework.

Talks

There were a bunch of talks on the internals of Javascript Virtual Machines. From a computer science point of view, these talks are interesting. I heard a lot of these kinds of talks at PyCon and during my days at Sun. It seemed that most of the audience appreciated this material, so the selections were good. The part of this that I found disturbing is wrapped up in one of the questions, which was basically, how can we write our code to take advantage of how the VM works. Given the number of VM’s that Javascript code might execute on, this seems like a path fraught with peril.

Also on the language front, there was more representation from functional programming. There was a talk on Roy, and David Nolen gave a talk that was billed as being about Clojurescript, but was really more about having a sense of play regarding all this technical work. Closely related to the functional programming was GPU programming. Jarred Nichols talked about implementing a Javascript interpreter in OpenCL. Stephan Herhut from Intel talked about the RiverTrail parallel extensions to Javascript which do data parallel computing using operations taken from functional programming. The extensions compile to OpenCL, which I found interesting. I wonder how many more languages we’ll see compiling to OpenCL or partially compiling to OpenCL.

Paul Irish did a nice presentation on tools which gave a great overview of the state of the practice in the various areas related to web application development. There were several tools that I didn’t know about. The presentation is all HTML5 but has some very nice visuals and animation. I’d love to know the name of the package that he used.

Ever since Node.js came out, I’ve been enamored of the idea that you could share/move some amount of code back and forth between the client and the server, much as code used to move back in the days of NeWS. Yahoo’s Mojito is an investigation in this space. It relies heavily on YUI, which I haven’t used. I’m looking forward to looking into the code and seeing how it all fits together.

The team at Bitovi make a special lunchtime presentation about CanJS, which is another MVC framework for Javascript. CanJS is in the same space as backbone, knockout, and so forth. It’s claims to fame are reduction of certain kinds of memory leaks, size, and speed. From the benchmark slides it seems worth a look.

Keynotes

Dan Ingalls delivered the closing keynote on the first day. I met Dan briefly when I worked at Sun, and I was familiar with his work on the Lively Kernel. The Lively Kernel is the answer to the question “what if we tried to build Squeak Smalltalk in Javascript”. It is much more than a language, it is an environment for building programs and simulations. I’m of two minds about this work. On the one hand, there’s depression that we still haven’t managed to catch up to the work that Ingalls and his contemporaries pioneered 30 years ago, and that today’s practitioners are completely oblivious to this work (a comment on Twitter confused Lively with an advanced version of the NeXT Interface Builder — the causality is reversed). On the other hand, although the Lively Kernel is written in Javascript and runs in a browser, it’s not really connected to today’s world, and so it’s applicability to solving web problems is limited. Nonetheless, Ingalls received a well deserved standing ovation. He is among the pioneers of our field, and as his generation is starting to pass on, it feels good to be able to personally honor them for their contributions to the field.

I have no idea how Chris Williams convinced Rick Falkvinge, the founder of the first (Swedish) Pirate Party to come and speak at JSConf. The topic of his talk was the politics of the net generation. Falkvinge told the story of how he came to found the Pirate party in Sweden, and described the success that the party is having in Europe. He claimed that about every 40 years, we have a new key idea. Apparently the theme for the period which is now ending was sustainability, and the claim is that the theme for the next 40 years will be free speech and openness. He credits this theme with the rise of the various Pirate parties in Europe, pointing to the European protests around ACTA and the US protest around SOPA as additional corroborating evidence. Falkvinge claims that the Pirate party has widened the scope of politics and given young people a way to vote for the issues that they care about. I wish that something similar was happening in American politics.

Hallway

As always, JSConf had a rich hallway/party track. I had lots of great conversations with people on topics including the Javascript API’s for Windows 8, the mismatch between many concurrency models and real shared memory hardware, and proper use and optimization of CSS. I think that facilitating the hallway track is one of the areas where JSConf excels. The venues are always nice, and this year there were hallway conversations, in pools, around campfires, as well as the usual hotel lobbies and restaraunts/bars/lounges. I was also happy to be able to introduce Matthew Podwysocki, who has been doing excellent work on RX.js, and David Nolen, who has been working on Clojurescript. I think that there can be some nice synergy between these two projects, and I’m eager to see if they agree.

The best roundup of JSConf coverage appears to be on Lanyrd.

Tags: dynamic languages, internet, javascript, programming
Posted in internet, programming | No Comments »

Strata 2012

March 5th, 2012

Here’s a roundup of last week’s Strata conference.

Jumpstart

This year, the O’Reilly team introduced a new tutorial day track, called “Jumpstart”. This track was more oriented towards the business side of big data, and I think that the word MBA actually appeared in the marketing. I think that the track was a success, and was very appropriate. The effect of the next generation of data oriented technologies and applications is going to be very significant, and will have a big impact on the way that business operate. It’s very important that technologists and business people work closely in order to produce the best results.

There were two talks that stood out for me. The first was Avinash Kaushik’s What Marketers can learn from Analysis. Kaushik is a very entertaining and dynamic speaker, and he has had a lot of experience working to help companies use analytics effectively. In his world, processing and storage is 10% of what you need, and analysts – humans are the other 90%. In other words, technology is not nearly as important as having people who can ask the right questions and verify hypotheses experimentally. And even good analysis is not enough. Organizations must be able to act on the results of analysis. I have been (and will continue to be) interested in the ability to use data as quickly as it is collected. Some people call this a “real-time” data capability, although in computer science terms, this is a misnomer. One of the best quotes from Kaushik’s talk was “If you do not have the capacity to take real time action, why do we need real time data?”. Without the ability to act, all the data collection and analysis in the world is fruitless. Kaushik’s claim was that we must remove all humans from the process in order to achieve this. Back to analysis, Kaushik feels that the three key skills of data analysis are: the scientific method, design of experiments, and statistical analysis.

The second talk was 3 Skills of a Data Driven CEO by Diego Saenz. I liked his notion that a company’s data is a raw material, just like any other raw material that might be used by a company. Raw materials must be collected, mined, purifed, and transformed before they can turn into a product, and so with a company’s data. The most important information that I got out of this talk was the case study that he presented on the Bob McDonald, the CEO of Proctor and Gamble. P&G has built a business wide real time information system called Business Sphere. One manifestation of Business Sphere is a pair of 8 foot high video screens that sit in the conference room used by the CEO for his regular staff meeting. Real time data on any aspect of the company’s operations can be displayed on these screens, discussed and acted upon at the CEO staff level. Also of note is that a data analyst attends the CEO staff meeting in order to facilitate discussion and questions about the data. I remember back in the 2000’s when Cisco talked about how they could close their books in a day. Now we have the worlds largest consumer products company with a real time data dashboard in the CEO’s conference room. The bar is being raised on all companies in all industries.

Talks

I felt that the talks In the regular conference were weaker than last year. Part of that may be due to my talk selection – there were lots of tracks, and in some cases it was hard to figure out which talks to pick. I tend to seek out unusual content, which means more risk in terms of a “quality” talk. The advent of the O’Reilly all access path has taken some of the risk out, since that pass gives you access to the full video archive of the entire conference. The topic of video archives is probably content for another blog post. I know that there are some talks that I missed that I want to watch the videos for, but apparently, I’ll need to wait several weeks. It will be interesting to contrast that with this week’s mostly volunteer run PyCon, which has a great track record of getting all their videos up on the web during the conference, for no fee.

Talks which were easy to remember included Sam Shah’s Collaborative Filtering with MapReduce, which included a description of how to implement collaborative filtering on Hadoop, but more importantly discussed many of the issues around building a production worthy version of such a system. It’s one thing the implement a core algorithm. It’s another to have all the rest of the infrastructure so that the algorithm can be used for production tasks.

A large portion of the data the people are interested in analyzing is coming from social networks. I attended Marcel Salathé’s Understanding Social Contagion in the hopes of gaining some greater insight into virality. Salathé works at an infectious disease center and he spent a long time comparing biological contagion with internet virality. I didn’t find this to be particularly enlightening. However, in the last third of the talk, he started talking about some of the experimental work that his group had done, which was a little more interesting. The code for his system is available on github.

I really enjoyed DJ Patil’s talk Data Jujitsu: The Art of Turning Data into Product. According to Patil, data jujitsu is using data elements in an iterative way to solve otherwise impossible data problems. A lot of his advice had to do with starting small and simple, and moving problems to where they were easiest to solve, particularly in conjunction with human input. As an example, he discussed the problem of entity resolution in one of the LinkedIn products, and described how they moved the problem from the server side, where it was hard, to the client side, where it was easy if you asked the user a simple question. The style he discussed was iterative, opportunistic, and “lazy”.

Jeremy Howard from Kaggle talked about From Predictive Modelling to Optimization: The Next Frontier. Many companies are now building a lifetime value model of a customer, and some companies are even starting to build predictive models. Howard’s claim was that the next steps in the progression are take these models and use them to build simulations. Once we have simulations, we can then use optimization algorithms on the inputs to the simulation, and optimize the results in the direction

Keynotes

Last year, I was pretty unhappy with a number of the keynotes, which were basically vendor pitches. This year things were much better, although there were one or two offenders. Microsoft was NOT one of the offenders. Dave Campbell’s Do We Have The Tools We Need To Navigate The New World Of Data? was one of the better Microsoft keynotes that I’ve seen at an O’Reilly event (i.e. out of the Microsoft ecosystem). The talk included good non-Microsoft specific discussion of the problems, references to academic papers (each with at least one Microsoft author), and a friendly, collegial, non-patronizing tone. I hope that we’ll see more of this from Redmond.

Avinash Kaushik had a keynote spot, and one of the most entertaining, but insightful slides was an infamous quote from Donald Rumsfeld

[T]here are known knowns; there are things we know we know.

We also know there are known unknowns; that is to say we know there are some things we do not know.

But there are also unknown unknowns – there are things we do not know we don’t know.

Kaushik was very keen on “unknown unknowns”. These are the kind of things that we are looking to find, and which analytics and big data techniques might actually help discover. He demonstrating a way of sorting data which leaves out the extremes, and leaves the rest of the data, which is likely where the unknown unknowns are hiding.

I’ve been a fan of Hal Varian ever since I read his book “Information Rules: A Strategic Guide to the Network Economy” back during the dot-com boom. One the one hand, his talk Using Google Data for Short-term Economic Forecasting, was basically a commercial for Google Insights for Search. On the other hand, the way that he used it and showed how it was pretty decent for economic data was interesting. There were several talks that included the use of Google Insights for Search. It’s a tool that I’ve never paid much attention to, but I think that I’m going to rectify that.

The App

This is the first O’Reilly conference I’ve attended where they had a mobile app. There were iPad, iPhone, and Android versions. I only installed the iPad version, and I really liked it. I used it a lot when I was sitting in sessions to retrieve information about speakers, leave ratings and so forth. I’d love to see links to supplemental materials appear there. I also liked the fact that the app synced to the O’Reilly site, so that my personal schedule was reflected there. I didn’t like the fact that the app synced to the O’Reilly website because the WiFi at the conference was slow, and I often found myself waiting for those updates to finish before I could use the app. The other interesting thing was that I preferred the daily paper schedule when I was walking the hall between sessions. Part of this was due to having to wait for those updates, but part of it was that there was no view in the app that corresponded to the grid/track view of the paper schedule. More work to do here, but a great start.

Final thoughts

This year’s attendance was over 2300, up from 1400 last year, and I saw badges from all sorts of companies. It is apparent to me that the use of data and analytics being discussed at Strata is going to be the new normal for business.

Tags: data, internet
Posted in internet | No Comments »

Blogaversary 2012

January 9th, 2012

Twitter, Google Plus, Facebook, Flickr, 500px. pinterest on the horizon. All that’s missing is a partridge in a pear tree, and sometimes that appears to be there as well. The world of online communication and connection is pretty different that it was nine years ago, when I slapped a bunch of Python scripts onto a server running in a closet in my house. For all the other forms that have emerged, blogging still has a warm place in my heart. I’ve used this blog to host long (some might say too long) and short form content over the years. I never really mastered the short form content, so it was easy for that to migrate off into Twitter land. Some of the more personal stuff that I used to write has also gone to Twitter, or to a lesser degree Facebook, but there hasn’t been a lot of time for that, so most of it has gone by the wayside. What remains is the much longer form content. Unfortunately, firing up Ecto and writing a few dozen paragraphs doesn’t come easily, and getting over some of that inertia is what’s kept more content from appearing here.

Despite my initial misgivings, I like Google Plus. I like the ability to write short to medium content, as well as really nice picture support. I like how easy it is for conversations to start. I’m probably going to put a little more energy into being there. Will there be a 10th anniversary blog post in January 2013? I’m not sure. I’d like there to be, and I plan to keep on posting, but at the same time, there are only so many hours in a day.

Tags: blogging
Posted in blogging | 2 Comments »

2011 in Photography

December 29th, 2011

I’ve been dreading writing the photography roundup post this year, because I haven’t taken a lot of photographs. I’ve only a few months worth of photographs on Flickr, which makes a month by month roundup pretty tough to do. We’ve had an enormous amount of stuff going on schedule wise this year, and between all of that activity, and me getting fussier about my pictures, 2011 saw a precipitous decline in the average number of pictures that I made during a given month.

I’m still doing some dance performance work,

and on the right occasions, I am going to interesting places that yield interesting pictures

The highlight of the year for me photographically was a project that I did with one of the seniors at my daughters’ dance studio

I think that the other notable thing for me and photography in 2011 is a move away from Flickr and towards Google+, Facebook, and 500px. This is definitely a bittersweet thing for me. Flickr is pretty much responsible for getting me back into photography and putting me on a good track of growth. At the same time, I see Flickr weakening in various ways. I do a limited amount of portrait/dance shooting in my local area. For this kind of work, Facebook is pretty much the place where people might see my work. When Google+ burst onto the scene earlier this year, it was clear that it was really pretty decent for photography, and a lot of professional and advanced amateur photographers have taken to the service. In particular, Google+’s hangouts feature is great for photography growth. There are amazing photographs and photographers on Flickr, but if you look at 500px, the level of photography being displayed there is pretty amazing. I’m going to keep using Flickr, I think, but I’m going to be shifting more of my energy to Google+ and 500px in 2012.

One of the big happenings in 2011 was that we did a remodel of the empty bonus room that’s over our garage. One of the use cases for the design of that remodel was as a photo studio. Due to time, I haven’t really been able to get up there much, but I hope to spend some time working there come the new year, so perhaps next year’s roundup won’t be so lean.

Tags: photography
Posted in photographs, photography | No Comments »

Book Review: Head First HTML5 Programming

November 20th, 2011

You can’t read about modern web development without hearing something about HTML5. It is a term that covers not only the next version of the HTML markup language, but a broad array of facilities exposed as JavaScript API’s. Eric Freeman and Elisabeth Robson’s new book Head First HTML5 Programming approaches HTML5 from this point of view, and uses the friendly, pictorial style of their highly successful Head First Design Patterns.

You’ll need an understanding of HTML and CSS before you dive in. This book assumes that you have those technologies under your belt, and starts right in with JavaScript. The first four chapters of the book are a good introduction to JavaScript, the web browser Document Object Model (DOM), event handling and objects. This ensures that you’ll have the level of JavaScript knowledge necessary for the chapters to come. And you will need it. In the remaining six chapters, the book covers the use of Geolocation, AJAX with JSON, including JSONP, the Canvas tag, the video tag, the various Web Storage API’s, and Web Workers. Each chapter teaches you about its subject material by building a decent size application in JavaScript. I think that seeing how an application will use a feature is the best way to really learn about it.

If you’ve been wondering about the new features in HTML5, this book is a good introduction. The authors have stuck to the parts of HTML5 that are pretty well defined, and stayed away from those parts of the standard that are still changing. The Head First style presents you with the material from many different angles, which helps to make sure that your brain holds onto it.

One of my daughters is interested in web programming. She needs just a little more CSS learning and then she’ll be ready to step up to dynamic HTML / HTML5. When she is, I’ll be handing her a copy of this book.

Disclosure: Both the authors of Head First HTML5 are friends, and Eric Freeman used to be my boss at Disney.

Tags: books
Posted in books | No Comments »

Clojure Conj 2011

November 15th, 2011

Last week I was in Raleigh, attending the second Clojure/Conj. The last time that I attended a Lisp conference was the 1986 ACM Conference on Lisp and Functional Programming. I am a Lisp guy. I took the famed “Structure and Interpretation of Computer Programs” course from Sussman and Abelson. I spent some time doing undergraduate research on Symbolics Lisp Machines. When Apple invested some energy into Dylan, I hoped that I’d be able to use a Lisp on a personal computer. Java pretty much ruined that. Over the years, I pretty much gave up on the idea of being able to use Lisp for my day to day work. So much so, that when I first heard Rich Hickey talk about Clojure, my reaction going in was unenthusiastic. By the end of Rich’s talk, he had my attention. Clojure has been doing some growing up since then, and I really wanted to attend last year’s Clojure/Conj, but wasn’t able to.

Almost all of my conversations at the conference involved the questions, “Why are you at Clojure/Conj” and “How did you get interested in Clojure”. I’ve answered the second question in the previous paragraph. The question of “why” boils down to three themes: Clojure itself, Data, and Clojurescript. I’m going to use these threes theme to report on the conference talks.

Clojure itself
Clojure is a Lisp dialect that runs on the JVM and has great interoperability with existing Java code. It has great support for functional programming, as well as several innovative features for dealing with concurrency.

Stuart Sierra started off with a talk that pointed out areas where people could learn beyond the books and online exercises that are available. In each of those areas, he also proposed projects that people could work on. One of the things that stood out for me was his use of the Clojure reader to deal with Java Resources. I always found Resources to be annoying, and the use of the Reader is a clever way to make them more palatable and useful.

Clojail is a system for executing Clojure code in a sandbox. The system is quite flexible and the applications aren’t just limited to security. I can imagine using clojail to implement something like the Sponsors described in the original Actor model. Anthony Grimes, one of the committers for clojail gave the presentation. He is 17 years old.

One thing that made me happy was to see the bridge building between the Scala and Clojure communities. Phil Bagwell, who pioneered many of the persistent data structures in Clojure is now at Typesafe, the Scala company. He came and gave a nice talk about Scala’s parallel collection classes. Perhaps these classes will one day find their way into Clojure Daniel Spiewak gave a very solid presentation on the computer science behind the persistent data structures in Clojure.

At many conferences a talk like Clojure on Android would be at the higher end. The technical level of the talks at the Conj was high enough to make the task of getting Clojure on Android seem mundane. This is to take nothing away from the very impressive work that has been done. There are some issues remaining like footprint and startup time, but it looks like some effort is going to happen at the Clojure core team level to make some of this possible. The thought of talking to a REPL running on a phone, or tablet is a tasty one.

Rich Hickey’s keynote reminded me very much of a Guido keynote at PyCon: a discussion of language issues that he was looking at, and a solicitation for discussion. Rich was very careful to say that the stuff he was discussing was not a roadmap, so I’ll repeat that disclaimer. Here are some of the items that stood out to me. Plans to allow multiple builds of Clojure – a regular version, a leaner deployment version, a really lean Android version, a super deluxe development/debugging version and so on. There is discussion about allowing the reader to be extensible, in order to allow new data types to be round tripped. I didn’t follow the history of ClojureScript, so it was useful to see that Rich is pretty committed to this idea, and that bits of technology might even be flowing “backward” from the ClojureScript compiler into Clojure on the JVM. I was also very interested on Rich’s view that the use of a logic system like that in core.logic would be a far better tool than a traditional type system. More on the logic system below.

The last talk of the conference was Sam Aaron’s talk on Overtone, which is a computer music system written in Clojure. The major point was that he used Clojure to define a language for describing computer music, much in the sam way that sheet music describes regular music. There was lots of cool music along the way, including a pretty good simulation of the sound portion of the THX commercial that often plays before movies. The description of that commercial fit in a single projected screen of code.

Data

One thing that I’ve been looking at recently is exploratory environments for working with “federated” data. I’ve grown to dislike the term Big Data, because it’s come to mean almost nothing, however, the ship has already sailed on that one. Most people would be familiar with the idea of sitting down in front of their relational database SQL command prompts, and issuing ad-hoc queries. As the use of varied kinds of storage systems grows, we are losing that kind of interactive relationship with data. Some of the people in the Clojure community have built some interesting data systems, and Clojure is itself amenable to exploratory work with data, between it’s orientation around functional programming, and a development style oriented around a REPL.

David McNeil talked about Revelytix’s federated (among RDBMS and RDF triple store) SPARQL query engine. Their system uses s-expressions to represents the nodes in a graph of stream processing nodes. These expressions are then compiled down to a form that can be executed in parallel using the Java Fork/Join framework. The operators in the s-expessions are mirrors of built in Clojure sequence functions, and can use and be used in Clojure expressions. It’s not hard to imagine extending the set of federatable storage systems.

Heroku’s Mark McGranaghan talked about viewing logs data. What he really meant was viewing log data as akin to a native data type on Clojure and being able to use Clojure’s built in functions on log data in a natural way. Heroku has built a system call Pulse which takes this view. I particularly liked the small functions that he defined for expressing the intervals for recomputing statistics. It’s the cleanest formulation of that kind of thing that I’ve seen, and it’s enabled by his thesis view and Clojure.

Nathan Marz has been doing some great work at BackType and now Twitter. At StrangeLoop he open sourced Storm, a set of general primitives for doing realtime computation. At the Conj, he was talking about Cascalog, which is a Clojure DSL for Hadoop. Both Cascalog and Storm are in use at Twitter. Cascalog is inspired by Datalog and targets the same space as Pig. Cascalog has the full power of Clojure available to it, as well as the power of Datalog. It’s a little unclear to me exactly how much of Datalog is supported, but this is a powerful idea. Imagine combining the best of Cascalog and the Revelytix system. The source code to Marz’s examples is on Github.

Clojure has a logic programming library, core.logic which is based on the miniKanren system developed at Indiana University by Daniel Friedman, William Byrd, and Oleg Kiselyov. Ambrose Bonnaire-Sergeant presented an excellent tutorial on logic programming in general, and miniKanren in particular. David Nolen talked about predicate dispatching, a much more general way of doing method dispatch, and talked about his plans to tie that together with core.logic. The surprise highlight in this area was that Dan Friedman and William Byrd came to the conference and did a BOF on miniKanren and their constraint extensions to miniKanren. The BOF was surprisingly well attended (over 60 people), due in part to Ambrose’s excellent talk earlier that day. A key philosophical point about miniKanren is that there is a straight forward mechanical conversion from a functional program to logical/relational (miniKanren) program. This looks very promising, and it has me thinking about mashups of miniKanren (core.logic) and Datalog (cascalog). Professor Friedman and his students have done some very important work in the Scheme area over the years, and it was a great experience to meet him and spend some time over dinner. After dinner, we were sitting in the hotel lobby, and David Nolen was walking Friedman and Byrd through the implementation of core.logic, which was ported from the Scheme version of miniKanren, and then optimized for Clojure. There was a free flow of ideas back and forth, and it was a great example of a collaboration between academia and practice (it’s hard to say industry because Nolen and company are doing this in their free time). This is one of the things that I’ve always hoped for around open source, and it was nice to see such a concrete example. MiniKanren is described in Byrd’s PhD dissertation, and in the book “The Reasoned Schemer“.

ClojureScript

ClojureScript is a Clojure compiler which emits Javascript, which is then run though Google’s Closure compiler. I’ve been doing some prototyping work using Node.js and HTML/Javascript, so ClojureScript looks kind of interesting, particularly because it is good at some the data intensive stuff that Javascript is so laborious at. There were three ClojureScript sessions. Chris Houser took us on a deep dive of the compiler, Kevin Lynagh show us some basic applications of ClojureScript in the browser, and David Nolen did a BOF where he showed off the browser connected REPL for Javascript. ClojureScript is still in its infancy, but it’s interesting nonetheless. Once David gets the constraint version of core.logic working in ClojureScript, it should get a lot more interesting.

Community

The thing that stood out to me about the Clojure community was the presence of the “young Jedi”, Anthony Grimes, and Ambrose Bonnaire-Sergeant. Both of them were able to attend their first Clojure/Conj (Anthony’s was last year) due to fundraising campaign initiated by Chas Emerick. Anthony is 17, and Ambrose has not yet graduated from college. Both of them are lead developers on highly technical projects within the Clojure community, and both did a great job of speaking in front of 300+ people who were mostly older than them. When I worked at OSAF, I worked with Stuart Parmenter, who started working in open source when he was 14. It’s great to work with these young, very gifted people, and I love seeing the community welcome and make a space for them.

The flip side of this is that like many open source, programming language oriented conferences, there were very few women in attendance. Perhaps the Clojure community could take a page from the very successful work that my friend Sarah Allen has done on RailsBridge.

Learning More

O’Reilly has finally recanted and is doing a Lisp book. Clojure Programming should be done soon, and Manning has Clojure in Action and The Joy of Clojure. If you are looking for an interactive way of learning Clojure, there is Try Clojure. Those looking to sharpen their Clojure skills can look at the Clojure Koans and 4Clojure .

The speaker slides from the Clojure/Conj 2011 are available on GitHub.

Update: corrected the name of Indiana University – thanks to Lindsey Kuper

Update: linked to a more up to date Overtone repository – thanks to Sam Aaron

Tags: Clojure, dynamic languages, functional programming, javascript, languages
Posted in misc | 9 Comments »

Web 2.0 Summit

October 25th, 2011

Last week I attended the Web 2.0 Summit in San Francisco. The theme this years was “The Data Frame”, an attempt to look at the “Points of Control Theme” from last year through the lens of data.

Data Frame talks

Most of the good data frame stuff was in the short “High Order Bit” and “Pivot” talks. The interviews with big company CEO’s are generally of little value, because CEO’s at large companies have been heavily media trained, and it is rare to get them to say anything really interesting.

Genevieve Bell from Intel posed the question “Who is data and if it were a person what would it be like?” Her answers included:

Data keeps it real – it will resist being digitized
Data loves a good relationships – what happens when data is intermediated
Data has a country (context is important)
Data is feral (privacy security,etc )
Data has responsibilities
Data wants to look good
Data doesn’t last forever (and shouldn’t in some cases)

One Kings Lane was one of the startups described by Kleiner Perkins’ Aileen Lee. The interesting thing about their presentation was their realtime dashboard of purchasing activity during one of their flash sales events. You can see the demo at 6:03 in the video from the session.

Mary Meeker has moved from Morgan Stanley to Kleiner Perkins, but her Internet Trends presentation is still a tour de force of statistics and trends. It’s interesting to watch how her list of trends is changing over time.

Alyssa Henry from Amazon talked about AWS from the perspective of S3, and her talk was mostly statistics and customer experiences. One of her closing sentences stuck in my mind: “What would you do if every developer in your organization had access to a supercomputer”. Hilary Mason has talked about how people in sitting at home in their pajamas now have access to big data crunching capability. Alyssa’s remark pushes that idea – pushing the thought that access to supercomputing resources is at the same level as access to a personal computer.

TrialPay is a startup in the online payment space. Their interesting twist is that they will provide payment services free of charge, without a transaction fee. They are willing to do this because they collect the data about the payment, and can then use / sell information about payment behaviors and so on (apparently Visa and Mastercard plan to do something similar).

I am not a fan of talks that are product launches or feature launches on existing products, so I was all set to ignore Susan Wojcicki’s talk on Google Analytics. But then I saw this picture in her slides:

Edward Tufte has made this diagram famous, calling it “probably the best statistical graphic ever drawn”. I remember seeing this graphic in one of his seminars and wondering how to bring this type of visualization to a computer. I appreciated the graphic, but I wasn’t sure how many times one would need to graph death marches. The Google Analytics team found a way to apply this visualization to conversion and visitor falloffs. Sure enough, those visualizations are now in my Google Analytics account. Wojcicki also demonstrated that analytics are now being updated in “real time”. Clearly, there’s no need to view instant feedback from analytics as a future item.

Last year there was a panel on education reform. This year, Salman Khan, the creator of the Khan academy spoke. Philosophically I’m in agreement with what Khan is trying to do – provide a way for every student to attain mastery of a topic before moving on. What was more interesting was that he came with some actual data from a whole school pilot of Khan Academy materials. Their data shows that it is possible for children assigned to a remedial math class to jump to the same level as students in an advanced math class. They have a very nice set of analytic tools that work with their videos, which should lead to a more data based discussion of how to help more kids succeed in learning what they need to learn to be successful in life.

Anne Wojcicki (yes, she and Susan are sisters) talked about the work they are doing at 23andMe. She gave an example of a rare form of Parkinson’s disease, where they were able to assemble a sizable number of people with the genetic predisposition, and present that group to medical researchers who are working on treatments for Parkinsons. It was interesting story of online support groups, gene sequencing, and preventative medicine.

It seems worth pointing out that almost all the talks that I listed in this section were by women.

Inspirational Talks

There were some talks which didn’t fit the data frame theme that well, but I found them interesting or inspirational anyway.

Flipboard CEO Mike McCue made an impassioned plea that we learn when to ignore the data, and build products that have emotion in them. He contrasted the Jaguar XJSS and the Honda Insight as products built with emotion and built on data, respectively. He went on to say that tablets are important because the content becomes the interface. He believes that the future of the web is to be more like print, putting content first, because the content has a soul. Great content is about art, art creates emotion, and emotion defies the data. It was a great, thoughtful talk.

Alison Lewis from Coca Cola talked about their new, high tech, internet connected Freestyle soda machine. A number of futuristic internet scenarios seem to involve soda machines, so it was interesting to hear what actual soda companies are doing in this space. The geek in me thinks that the machine is cool, although I rarely drink soft drinks. I went to the Facebook page for the machine to see what was up, and discovered that the only places in Seattle that had them were places where I would never go to eat.

IBM’s David Barnes talked about IBM’s smart cities initiative, which involves instrumenting the living daylights out of city. Power, water, transportation grid, everything. His main points were:

Cities will have a healthier immune systems. The health web
City buildings will sense and respond like living organisms – water, power, etc systems
Car and city buses will run on empty..
Smarter systems will quench cities thirst and save energy
Cities will respond to a crisis – even before receiving an emergency call

He left us with a challenge to “Look at the organism that is the city. What can we do to improve and create a smarter city?”. I have questions about how long it would take to actually build a smart city or worse, retrofit an existing city, but this is a challenge type of long term project. I’m glad to see that there are companies out there that are still willing to take that big long view.

Final Thoughts

I really liked the short talk formats that were used this year. It forced many of the speakers to really be crisp and interesting, or at least crisp, and I really liked the volume of what got presented. One thing seems true, that from the engineering audience of Strata to the executive audience at Web 2.0, data and data related topics are at the top of everyone’s mind.

And there in addition to ponies and unicorns, be dragons.

Tags: internet
Posted in internet | No Comments »