Actors, hierarchy, and anarchist software.

A few days ago, Steve Waldman of Interfluidity tweeted the following:

“Like in an economic organization, actors naturally form hierarchies.” ideology embedded in software? fact of nature? doc.akka.io/docs/akka/snap…

— Steve Randy Waldman (@interfluidity) February 21, 2013

I responded thusly:

@interfluidity ideology but also practical experience with the fact that it’s super hard to build provably correct decentralized systems. — E. Vigil-McClanahan (@evanmcc) February 21, 2013

Which is true, but at the same time deserves a more detailed unpacking (also more commas).

What this sort of statement typically means in actor systems is that local systems of supervision are hierarchical. Akka’s supervisors look to derive directly from Erlang’s supervisors. In Erlang, an important design principle is to let fail what is going to fail and to restart it cleanly. Supervisors are a generic take on this strategy, with a supervisor starting, watching, and restarting (or reporting) after child actor failures. Supervisors can have their own supervisors, on up the chain to the root supervisor owned directly by the virtual machine.

Note, in the above paragraph, an important distinction from real-world hierarchies. For the most part, Erlang (and, I presume, Akka) supervisors don’t tell their children what to do. Their children know what to do, and supervisors merely mediate when they do it, and clean up after them when they explode. However, in real software, sometimes even this is too much, as a supervisor can represent a serialization point (a concurrency bottleneck) in your software, and you have to decentralize launch and cleanup of critical actor systems.

This leads me to a more important point, which is that in the large, distributed systems tend to be decentralized for both scalability and efficiency reasons. Marshalling all of the data needed to operate a large system correctly into a single decision loop tends to make these systems inflexible, fragile, and slow. As such, distributed systems tend towards decentralized and distributed decision-making systems, such as gossip systems (to globally distribute critical data), leader election (for when centralized decisions need to be made), and most importantly system design that minimizes internal communication and shared state. Riak, the product that I work on, is a reasonable example here. Homogenous nodes with a temporarily elected leader, with all globally important information distributed by gossip. Thousands of actors all working together to make a single system while sending each other as few messages as possible.

So why then supervisors, in the small? My suspicion is time and engineering manpower constraints. Designing these higher level decentralized systems is extremely complex. At Basho, my employer, we spend quite a lot of time getting them right, or as right as we can, and we rely on powerful tools like QuickCheck to help us do it. But if you have to write a formal model for every tiny piece of the system, you’re doomed, at least in today’s fast-moving software industry. So you fall back on informally specified but easier to reason about systems like supervision trees, even though they can end up biting you in the end. When I mentioned supervisors as a concurrency bottleneck above, I was referring to a real-world issue that we encountered, where our get and put FSM supervisors (in a nutshell, these Finite State Machines directly oversee storage and retrieval of data from our disk backends) were becoming overwhelmed in high-load situations, unable to start child actors quickly enough to fully utilize the machine the VM was running on.

So in the end, what does this say about the original query? Hierarchy falls naturally out of untutored designs because we find it easy to think about multi-actor systems this way (also OOP perhaps over-strenuously encourages this sort of thinking, but that’s another post). But in the end, they tend to fall apart, because the parts of the system spend too much time talking to each other to get any actual work done. This is, of course, the logic of the market, or of an anarchist utopia (so many things become isomorphic when viewed from far enough away). I think that the most important lesson to draw from all of this is that designing and understanding highly concurrent systems with many interacting parts (inside or outside of a computer) is a task that lies right at the edge of human cognitive abilities. We should never be entirely comfortable with them, because it’s extremely difficult to tell if they’re correct, so we should always be re-evaluating them, and working on the tooling that allows us to re-evaluate them, to find their defects and correct them.

Be careful out there. Unintended consequences wait everywhere in the tall grass.

Trajectories

I’m sure that this has been noticed before, but I’ve never seen it directly laid out, so here goes:

There exists a trajectory for any good, ‘free’ web service, be it a social network, a search engine, a blogging platform, etc.. It looks something like a parabolic arc. The height of its apex and the speed with which it is reached are different each time, but with enough data, you begin to see similarities.

In the early days, things suck. The site is ugly, it’s slow sometimes, maybe they haven’t got their core mechanics nailed down, or for a social network, the features are good but there is no one there. As things pick up steam, the venture vultures begin circling, more people get hired, and things start to rapidly improved. Things look better, more of your friends are using it, there’s some actual infrastructure money. There is a long, bright period at the top of the arc wherein everything is lovely on the user side. At the company, though, they know already that gravitational rot has already set in. They’ve never charged you anything, because no one ever charges you for anything. They may not even be sure that the product is worth enough to ask you for money. But the desperately need some. The high lifestyle of all their new coders and designers costs money, and the capital men lurking in the background are getting all sweaty in anticipation of their expected liquidity event.

If you and they are lucky, some big company will swoop in and provide the release that everyone is looking for, buying you up and then running you for years with benign neglect while they fold the bit of your tech that they wanted into their own product, or try to steal your user base or generally just figure out some way to make actual money out of you. Often a service can live in this limbo for years, providing use value and pleasure. The ending here tends to be swifter, when the new owner finally decides to shut the service down, or ’re-brand’ (almost always a fatal wound).

When no fairy godparent comes around, though, you enter the dread business of ‘monetization’. For the most part, this comes in two flavors: charging for premium services or starting to sell ads. I have no problems with using premium services to subsidize a free version. It’s a model that I like a lot, although I feel like people aren’t particularly transparent about their business models or flexible in their pricing (you have to take this with a grain of salt, coming from me. I think the Swedish(?) policy of having everyone’s tax bill be public is a great idea). Usually, though, this seems to be the best way to postpone the inevitable decline of a web service, although some of your users will inevitably complain that starting to charge money is actually part of your decline.

Go down the path of advertising, though, and you’ve basically submitted to your fate. It seems easy at first, because ad companies like Google have made it really easy to drop stuff right in. You make a little bit of money, but not really enough to pay anyone. So you have to make more changes. You realize that you’re actually answering to two sets of customers now, and only one of them is paying you. So you make compromises. Over time, you realize that your advertisers are winning every time their desires come into conflict with those of your users. Eventually your users realize this, too. Then they leave, if they have any place to go.

Google is entering this phase right now. They weathered the ownership problems and the external CEO, and any number of other issues that could have sunk them. But inexorably, since it holds the purse strings, the ad-serving part of their business will take over, and will ruin the user experience and usefulness of their service for everyone. Already there are issues: if you run ad-blocking software (which is within your rights. If they don’t want customers who don’t see ads, they’re welcome to turn you away, which is within their rights), or software that keeps them from tracking you elsewhere on the web, it breaks basic search most of the time. At this time, there is no real exit, so you can only chose voice and evasion.

These are hard times. Business models are in violent flux. Many companies are not sure where their next dollar is going to come from. I sympathize with people who’re using ads to patch together a business that they love into something that works. The market isn’t fair or perfect or even good, most of the time. I am not sure that there are good answers. Premium service pricing is only a partial solution, and isn’t going to work for everyone. Independent artists are a particular quandary. It’s time, though, to start looking around for solutions.

Advertorialism.

Attention conservation notice: ~900 words of divisive, under-researched hobby-horse riding.

So I read this piece by Freddie de Boer the other day, and then Rob Horning’s post on Steve Jobs. Something in both bothered me.

Both of them are basically getting at consumerism and capitalism from slightly different left-wing viewpoints (I am assuming that you’ve read both, at this point). I partially agree with both of them. Neither of them gets all the way to the root of their arguments, so maybe I am getting their position subtly wrong.

But I think the thing that they’re getting wrong, both of them, perhaps this entire line of critique, is that the internet that we have cannot possibly be any other way, as a straightforward consequence of where all of the money (or near as makes no difference) on the internet comes from, which is advertisement. What I mean by this is that almost everything interesting that’s happening lately on the commercial web is in the SaaS sphere (I include social networking in this, although the service is a tad nebulous and always changing) and almost all of it that interacts with consumers is funded by advertising rather than payment for the product. Even in app stores, there are ads, although due to the cognitive magic/trickery of encapsulation into ‘apps’, people seem willing to put in a little money up front, although never very much.

Horning seems to address this every so often, but he never seems to take it all the way. The underlying logic of the social deskilling that he sees stemming from Facebook has no root in the logic of what people actually need from something like a social networking platform. In a world where the money comes from somewhere else, Facebook or an entity like it would look and act nothing like it does. As it stands, social deskilling is just a epiphenomenon of Facebook needing information about what you’re doing there, so they can keep you there longer, looking at their pages and the ads displayed on them. One could argue that it isn’t so much a social deskilling as a gamification of online social interaction, with the dual goal of getting more information about the user to sell to advertisers, and to keep them looking at advertisements for longer periods of time.

My critique of de Boer is a little different, because I think that he gets closer, in his closing paragraph. I guess I would say that he reads to much in; he assumes that more people are more deeply engaged with the worthlessness of the current online world than really are (as does Horning). I’ll gladly join him in his crusade to end a world where everyone is a fetishistic consumer/critic, but I think that the number of people who actually aspire to that sort of thing, who construct their selves online, are many fewer than he imagines. Here too, of course, we see the logic of the advertising-funded internet, as innumerable outlets attempt to pull in as many ‘eyes’ as possible with their floods of strategized and seo-optimized ‘content’. Each struggling to establish themselves as a brand, to gain loyal followers (ad-viewers all), rather than follow the logic of their various missions. Trying to be divisive, sticky, intrusive, to keep us looking longer than we would have otherwise.

This is all pretty dreary, I guess. I think that both of these guys are interesting thinkers, and de Boer doesn’t spend a lot of time talking about the internet, so it’s understandable if his insights are a little hazy there. And it isn’t if I come equipped with all of the answers. I mean, I have some proposals, but isn’t as if the US government is going to go around regulating advertisers and taxing marketing budgets and nationalizing Comcast. Nor is anyone going to write a computer virus that installs adblocking on people’s browsers. Although that’s both awesome and doable.

I just mean to highlight the irresistible logic of all of the money on the web currently coming from ads and its consequences. We’re essentially stuck at this stage until we can figure out how to make money doing something else. Google is the highlight here. They’ve brought together thousands of smart people who make daring and great products of genuine utility, and it’s all just a sideline to their real business, which is spying on you for people who want to sell you shoes.

Their critiques are obviously heartfelt (at least de Boer’s. I feel that I am never sure where Horning is coming from, emotionally or contextually), but complaining that ‘the internet’ is vapid or enervating or atomizing or what have you isn’t the point. To some extent, it falls victim to the same kind of end-of-history/there-is-no-alternative thinking both of them inveigh against in other aspects of their political discourses. This is not a surprising thing; the internet is not apart from the world. But the internet is a place where it’s especially problematic, where we’re willing to build an entire world on a pile of shit because the shit-sellers have told us there is nothing else to build on. There is endless analysis to be done on the effects of this, but to me it isn’t important. None of these problems is solvable inside the current framework, and few of them would exist outside of it. Anatomizing the symptoms while ignoring the disease isn’t going to get you anywhere.

Guilt, shame, and fluffy fantasy.

I think that I’ve read Adam Roberts’ review of The Name of the Wind maybe 6 times now, so if you haven’t read that, this isn’t going to make any sense. It may not make any sense anyway, since it’s still a bit hazy, but I wanted to get it out there to work through it.

Something about it has always struck me as off, but I haven’t been able to articulate it up to now. For the most part, he’s correct about the novel and its failings. In no way is it high art, and the sooner Rothfuss finishes this white whale of a series and moves on to something more mature the better for him, and for all of us. But Roberts goes in on Rothfuss’ failure to truly inhabit the medieval mindset he posits is required for this sort of novel: >This could be three pals from any novel set in the 20th or 21st centuryl [sic]; and hundreds and hundreds of similar passages serve only to show the author has not entered into the pre-industrial medieval mindset that his medieval pre-industrial world requires—to, for example, understand the crucial point that not guilt (“I looked as guilty as I felt”) but shame was the key moral dynamic for the period. But to understand that would involve shifting about the psychological portraiture of the entire project; it would have meant writing characters less like, and therefore less appealing to, a 21st-century readership disinclined to make the effort to encounter the properly strange or unusual.

This speaks to a broader state of affairs in which style—the language and form of the novel—is seen as an unimportant adjunct to the “story.” It is not. A bourgeois discursive style constructs a bourgeois world. If it is used to describe a medieval world it necessarily mismatches what it describes, creating a milieu that is only an anachronism, a theme park, or a WoW gaming environment rather than an actual place. This degrades the ability of the book properly to evoke its fictional setting, and therefore denies the book the higher heroic possibilities of its imaginative premise.

I think that this is subtly wrong. Firstly, it is mistaken in assuming that a particular kind of moral technology (for lack of a better word), such as guilt-driven normative self-coercion, necessarily accompanies particular social structures and physical technology levels. But more to the point, it’s mistaken to assume that the supposed mismatch of form and tone has something to do with accessibility. Although writing characters more like his audience surely makes it easier for that audience to relate to them, I think that the fundamental issue is that for books like tNoW, where surely a weighty Moral Lesson is in the offing, is that ante-Guilt characters have nothing, morally, to teach those of us in the post-Guilt world. Men and women in the AG inhabit an different moral universe. Moral lessons taught to and through them are untranslatable, unteachable to us, unless we’re shame-driven atavisms.

So it makes no sense for Rothfuss to do that work, unless, like Tolkien, he’s a big fan of the period and its work. The telling sentence is this one:

But to understand that would involve shifting about the psychological portraiture of the entire project; it would have meant writing characters less like, and therefore less appealing to, a 21st-century readership disinclined to make the effort to encounter the properly strange or unusual.

I would argue here that the aim of Rothfuss’ project here is not actually to expose his readers to the strange or the unusual, and that it’s a mistake to assume that it is (Roberts’ easy ‘kids these days’ condescension wins him no points, either). Rothfuss’ narrow aims are as yet unclear, as there are any number of ways the third novel could resolve all of the issues that have been set up in the first two books, but his broader aims are clear; Kvothe is going to relate to us some important bit of moral knowledge about being an Exceptional Outsider. Hopefully it’ll be more profound than “Get over your first, unrequited love as quickly as possible”, which presumably would have, if learned early enough, prevented most of the series from happening.

Statements of Bias*:

• Adam Roberts: I enjoy his reviews, generally (especially the lighter, quicker ones at his blog). I typically don’t care for his fiction for reasons too involved to get into in a brief statement such as this.
• The Name of the Wind/Patrick Rothfuss: I thought it was entertaining enough, but had to reread it in order to read its follow-up, which isn’t really a good sign. I don’t think that any of its characters are the least bit psychologically realistic, but the manner of the telling makes it a quick and enjoyable read. I know more or less zip about its author.

* I am thinking of making bias statements part of the structure of the blog. I am not sure how useful that would be, but I feel that making bias clear might matter here more than usual.

Upgrading to mainstream Wordpress from an Ubuntu install.

Sorry for the flurry of posts today, I need to get all this stuff out before I forget about the site again for six months.

Originally when I switched over to this host I had decided that I was just going to stick with the apt-provided version of wordpress and just deal with the issues, but it turns out that their modularization is too leaky, and their code churn rate is too high for that to be a viable strategy, what with the low turnover rate of the Ubuntu package. Once I’d fallen too many released behind, all of the AJAX stuff started to break. I couldn’t even reply to comments from the console or post any pictures.

So if you’re looking for how to do this, here is what worked for me in early 2011 (this post will no doubt date rapidly):

1. As root, type # echo wordpress hold | dpkg --set-selections (note that this step is apt cargo cult magic which may fail in the future; I’ll update this if so).
2. Edit /etc/wordpress/wp-config.php, commenting out the line which says define('WP_CORE_UPDATE', false);.
3. Download the latest wordpress tarball from http://wordpress.org/.
4. Unpack it somewhere.
5. Backup /usr/share/wordpress (I used # tar czf backup.tgz /usr/share/wordpress but you may (by which I mean should) want to do something more bulletproof.
6. Do # cp -R /path/to/wp/wordpress/ /usr/share/
7. Look at the output and cleanup as appropriate. I ended up having to nuke /usr/share/workpress/wp-include/js/ and then recopy it from the unpacked tarball.
8. Go back to your dashboard, as you’ll likely have to trigger a database update or something similar.
9. Make sure your site or sites are still working.

Feel free to comment if you have any questions but I am hardly an expert on the Wordpress side. Good luck.

Embassytown, by China Miéville

I also read this recently. No one will be shocked, I suppose, that I didn’t particularly like it.

Quoting myself on twitter:

In the middle of Embassytown right now. Impression is Miéville doing Harrison doing Blish. Less happy with it after the mid-reveal.

And:

Finished Embassytown. Now wishing I hadn’t fought through the middle. “Language” is a dud pivot. Artful awkwardness here just awkward.

I don’t have a ton to add, I suppose, other than a statement of bias, as I thought that Kraken, as fun as it was a times, was kind of a baggy mess and concerned Men’s Business entirely too much, and had some huge disappointments in terms of its female characters.

The treatment of women as important is better here, which is praiseworthy, but doesn’t overcome the novel’s other flaws.

(My) Problems with Post-cyberpunk work:

The earliest and most famous of cyberpunk novels, the endlessly famous Neuromancer concerns the actions of a few marginal, violent people being manipulated by forces larger than they can comprehend. True, they have some agency; indeed they’d be useless as agents for their operators if they did not have some special skills and talents that make them the best tools possible in the situation. Ultimately, though, they are merely tools. Gibson’s interest is, for the most part, on the vast forces at work, rather than the tools themselves. His protagonists (who grow less marginal and less violent as the sprawl series progresses), are more or less a lens through which the world is seen. The culmination of each of these novels is the reveal, in which the tool-protagonist is made aware of the full scope of the drama in which they have played a small part.

This form, which of course has antecedents in older SF, detective and spy novels, and numerous other forms, tends to be what people take away from cyberpunk, second only to its window dressing of augments, street-wise tech-ninjas, and decades-old visual symbols of bad-assery (leather jackets, dark sunglasses or mirrorshades, dusters, ass-kicking boots, that sort of thing).

When Gibson first wrote his novels, deep into the Reagan-Thatcher years, things really did look pretty bad (almost thirty years ago, now). It wasn’t entirely whacked out to picture a world run for the profit of massive corporations who’d suborned the nation-states of the world (one could argue that that is in fact what happened, although it didn’t turn out as badly as it could have). Things really did suck, and it wasn’t insane to imagine marginal people being empowered by some mystical countervailing force being the only way that a little good could be done in the world.

But as time wore on and things got at least a little bit better our noir-loving doomprophets seem to have doubled down on the spit and the muck, and have moved the lens from the physically ineffectual Case to everyone’s favorite sexy ninja badass, Molly Millions.

While there has been some updating of the socio-political concerns that animate these books, their protagonists have gone ever more retrograde, evincing ever more cartoonish moralism and special pleading, while simultaneously growing ever more stimulatingly violent. At times, it seems as if their worlds have to be darkened and violent in order to enable any sort of engagement with their brutal, near-psychotic protagonists. Nyx and Takeshi Kovacs would, if introduced into a future that was anything like the present that we or their authors inhabit they’d swiftly be arrested or killed.

In literature, form and function are intimately linked. Sticking so hard to a genre form birthed in an extremely dark moment limits the effectiveness with which you can navigate the gray time in which we find ourselves now. The easy violence may sell books, and I am loathe to deny anyone their consolatory narratives in hard times, but I feel that these sort of novels are not doing the kind of work that’s pushing the genre forward or doing the sort of adaptive thinking and imaginative investigation which I consider to be the main work of SF.

God’s War, by Kameron Hurley

On Niall Harrison’s recommendation I picked up Hurley’s God’s War and read it over the weekend. As a paid up member of the Post-Cyberpunk clade, it’s a solid piece of work. Violent, entertaining outcasts are jerked around by the powerful, people are killed, scores are settled, bad-assery is done. The world-building is vivid, if not entirely consistent. Its treatment of gender is interesting, and reasonably novel, and it depicts the issues of its two Muslim civilizations as being orthogonal to the fact that they’re Muslim. It’s even well paced.

It isn’t without its flaws, of course. The writing could be better, especially in the neologisms department, and the world-building suffers quite a lot from tech search and replace issue, subbing in ‘bug’ for any number of other terms just to make things fit with the aesthetic, without ever bothering to think of whether these substitutions actually make sense. The vehicles, in particular, suffer from both the neologism issue (‘bakkie’??) and from being powered by and constructed from bug-encrusted handwavium (this wouldn’t be so much of an issue if they weren’t so prominent and often mentioned). The world-building is big on bold, vague strokes and light on telling details, and the visual description could use some real work. Also, for being so many thousands of years into the future, it’s all a bit old-fashioned.

EDIT: Niall points out in comments below that bakkie is South African slang for a pickup truck. So I apologize for that (although it still sounds a bit silly to American ears), but this highlights the visual description issues that I mention. Nowhere that I noticed was a bakkie described in enough detail for me to get that it was anything other than a wheeled vehicle (running on bug spit and unicorn farts).

END EDIT

None of these things are fatal flaws, and are easily overlooked, especially since this is the author’s debut. If you like Richard Morgan (particularly the second two Kovacs books), you’re quite likely to enjoy God’s War.

That said, the more of these I read, the more I wonder why people still bother to write them.

To get at why, I am going to have to delve into spoiler territory, and possibly into some uniquely personal aesthetic preferences, so go read it if you haven’t. It’s only a few bucks online. I’ll write another post detailing why I say that tomorrow, once I’ve had some more time to chew over my objections.

Directions.

I have some political posts almost done, and will post them (both! I promise!) in the coming week, but mostly I am thinking that I should really do something else with this space. I clearly don’t have enough opinions to post daily (even on twitter). I have to keep the machine around, since I am hosting the farm businesses site on the same server, but I am not sure what to do with this domain.

I no longer read enough, nor like enough of what I do read, for this to be a dedicated space for reviews and criticism. My writing output is too low for it to be some place to dump story snippets and chunks of works in progress. I don’t honestly have unique enough opinions on current events, nor do I have enough subject matter expertise for this site to be dedicated to politics. Farm stuff goes at the farm blog.

In all likelihood, it’ll continue to be as it is, as I don’t have the spare time to make it into anything else. Too busy building nursery tables and (if we get some rain) laying fence.

Moving hosts (done)

I am moving this site over to where everything else I run lives.

Not that this site is particularly active, but if you’re looking to access it, it may be up and down over the weekend as I get everything transferred to the new host.

UPDATE: If you can see this, everything should be switched over.  Because I was bored while I was waiting, I fiddled with the layout, and added some TypeKit stuff to try it out.  Mostly I think that it looks a tad better, although if I upgrade to a real TypeKit account, I think I’ll be on the lookout for some better fonts.