Tumbled Logic

National Poetry Day

this is a tale of Robin Hood Airport,
whose staff saw a musing tweet and thought,
“Perhaps this fellow has said something he oughtn’t?”
and saw fit to send the chap to court.

Now what you should understand, and know,
is that a tweet, to friends, that you might blow
an airport to the sky, should you not see your beau,
is clearly a jest, not a portend of woe.

But the CPS didn’t think this would matter,
though there was no threat, just playful chatter;
this fellow’s career is left in tatters.

So let this be a lesson to those who blog and tweet,
that the authorities may turn up the heat,
in the event that your words are similarly downbeat.

But! An appeal was lodged with the aid of a knight,
this fellow, you see, decided to put up a fight,
and so we’ll all know soon if it’s turned out alright.

And if it does, we can let our a cheer,
as once again, we can speak without fear,
(be it on Twitter or the blogosphere),
even if — sometimes — a little cavalier.

A personal tale

I don’t write about me on this blog all that much. Actually, I don’t write about me on here at all. There are lots of reasons for this, which I won’t be going into, but — although I write about what I think, and stuff I’ve been doing — I don’t write about me. Except this post. Suffice to say, I consider this to be an exception, rather than heralding a new era.

For the last eighteen months or so, I’ve been leading a secret double-life. This isn’t some kind of anniversary post, by the way, it’s just that a lot of things have been coming together lately and it seems apt to write this. Anyway: my secret double-life. By day, I build websites. Mostly the back ends of them, and mostly e-commerce sites. I do front-end stuff, too, and advise on accessibility, usability, findability, and other things ending in “ility”. I work for a small firm, so roles are quite flexible. By night, I do something else. It’s probably worth taking a step back for a moment.

About ten years ago, the company I was working for at the time decided (seemingly overnight) to stop building a set-top box which brought the Internet to your TV and video content to your TV via the Internet, primarily on the grounds that the marketplace wasn’t really ready for it, but also because the technology wasn’t really ready either (most people were still on dial-up modems, video codecs were terrible, the web wasn’t especially good when viewed from ten foot away, and TVs were bloody awful at rendering much of the content found on the Web anyway). In hindsight, all of these were very sound reasons for moving on to other things, and although I learned an awful lot while developing the product, my focus quickly shifted away from Internet/TV convergence and on to other things. Even so, I kept half an eye on developments in that area.

Lots of things happened between then and now. The dot com bubble burst, Internet access became more important than it ever was during the boom, the iPod was launched, Joost tried and failed to change the way television was delivered to our eyes, and Web slowly and quietly reinvented itself to be something a lot closer to its original ideals than it ever was in the early part of this century. Sometimes, I blogged about what was happening in the Internet/TV space and mused on how things might succeed or fail, and what was right and wrong about various efforts. Much as I do now, in fact. But that’s all I did.

Until about 18 months ago.

Around that time, it struck me just how far things had come in that near-decade. Many of the big issues of ten years ago of getting content in front of eyeballs via the Internet had been solved in some fashion or another. Not only that, but digital TV was now mainstream. The technologies which make digital TV work are (once you get past the nitty-gritty of actually flinging the signal into the air and sucking it into the back of the TV set) the same technologies which make video on the Internet work. And videos that you’ve ripped from DVD. And stick on your iPod. And so on.

Despite this, there was still a huge gulf between what ordinary people like you and I were doing on the Web and what broadcasters were doing over the air. To make matters worse, the state of the software side of things was terrible: although the encoding tools (such as x264) are brilliant, you need to be an expert to really use them. Dealing with metadata, archiving, scheduling, multiplexing, and interactive applications was even worse; although plenty of things existed, there wasn’t really anything which brought them all together. This didn’t make much sense to me. I couldn’t see any reason why an Internet-based “broadcast chain” couldn’t be built primarily on existing open source software and run on commodity hardware. So, I started to build it.

Along the way, the BBC did some things which caught my attention: they announced a plan to introduce a form of DRM on Freeview HD, and they put forward proposals for Project Canvas (now YouView). At this point, I didn’t really know much about the ins and outs of DVB, and especially not the bits which the Freeview HD DRM plan revolved around.

So, I learned. I learned enough to be able to explain to others what was really being proposed and what the real effects of the plan if approved (which it eventually was). In doing this, I realised that there were some very cool ways in which broadcast and Internet-delivered content could be work together, and the end result In this was that a big part of the “Internet-based broadcast chain” puzzle was solved: how would people actually come to watch the stuff?

Watching TV on your laptop is by no means the be-all and end-all, and I wouldn’t want to bet the future of anything on people doing just that, but if it didn’t really matter to Joe Bloggs whether their TV was squirted down their Internet connection or over the air, or whether it was linear or on-demand, things look a lot more promising.

And so I launched Project Baird. It set out to answer, with extremely limited (i.e., nonexistant) resources, the question of how Internet and broadcast services could be delivered together without being tied to any particular platform, broadcaster, ISP, or anything else. Its goals go a little beyond this, though, and stretch to ways in which “second screen” devices can talk to receivers and to broadcasters’ services and so on.

Fortunately, I’m not the only person working on this. Lots of different people are working on different parts of this overall puzzle, and I’ve spent quite a bit of time collaborating and coordinating efforts in order to bring all of the threads together into something coherent. The RadioDNS project provided the basic service-discovery infrastructure, with bits of DNS-SD thrown in, the NoTube project has been researching what happens when the semantic web, social networks and TV come together and the BBC has been doing lots of cool things with linked data. Meanwhile, I’ve been heckling bits of the BBC as usual and attempting to get DVB people and IETF people to collaborate on things as convergence continues.

This isn’t a “woo — look at me!” post. But, I am proud of what’s been achieved. There have been prototypes and experiments and research. Some of it (in a friendly) sense competing, some of it collaborating. All of it together shows that not only are the ideas behind all of this stuff sound, but they can work with very little in the way of extra effort. A dollop of code here and a bit of extra metadata there, and there’s a whole world of new and exciting applications — everything from smarter TVs which can stream or download programmes from the Internet and provide better programme information through smartphone and tablet applications which know what you’re watching and give you easy ways of exploring people, places and things, right through to toys which can, via Bluetooth or WiFi, react to and trigger events in programmes.

Along the way, I’ve made quite a few friends, and even more acquaintances. Everybody I’ve dealt with in all of this has been downright lovely. These are people who do this stuff for a living, while I bumble along doing it as — pretty much — a hobby. These are people who are exceptionally good at what they do and put up with my questions and idiotic musings.

Something needs to give, though. I’m not sure I can continue leading this double-life. But, I’m not really sure I can stop doing this, either.

Also, I have a headache.

Linked data on second screens

Today I added some code to an existing prototype which I’ve been working towards for a while — using linked data to allow a generic second screen application to give a user the opportunity to explore the subjects related to a programme on TV (or radio). This is a big part of what Project Baird is about.

This prototype is a web page, but it’s mocked up to look like an iPad application — there are two main reasons for this: first, throwing together a web page to do this is a bit easier than building a native app, and second, the protocols to allow a second screen application to ask a hybrid receiver what it’s currently tuned to aren’t exactly nailed down yet.

Instead, there’s a web page, and it’s passed the DVB service identifiers for the “current” channel. What happens next is a bit technical, but Libby Miller from the BBC has written about it on the NoTube blog. In a nutshell, those service identifiers get turned into a domain name. The application then looks for a particular service associated with the domain — what we call a metadata resolver. It then asks the resolver for the canonical URL for the programme which is being aired right now. This gets us the URL which should, in principle, be used to retrieve not just a web page, but data in other formats too (such as RDF/XML, Atom, JSON, and so on).

At the time of writing this, this only works for BBC channels, and in particular only those channels available on Freeview in Bristol. This is the nature of prototyping, though. None of the technologies or techniques here are specific to the BBC in particular, and the actual application has no real specialist knowledge about dealing with BBC channels over anything else (it’s simply the case that the infrastructure has been thrown together to provide the right sort of services for those BBC channels and not for any others — yet).

Once we have this URL, we request — using HTTP — information about it in a machine-readable fashion. This is presented to the user in the form of a title, description and still image. A real app with a nearby hybrid receiver might offer additional options, such as the ability to schedule download of the catch-up version for later viewing, or to start watching the programme on the second screen itself. There’s a mock-up I did of how this could look when fleshed out, from a few months ago:

So far so good: this is a combination of RadioDNS — and its extension TVDNS — in combination with the metadata resolver in order to get back information structured according to the Programmes Ontology. It could be applied, in principle, to just about any broadcaster in the world, without the application having to have any prior knowledge of any of them. That’s the starting point.

What was added today went a little further. I’ve talked in the past about smashing together the semantic web and web applications in order to do cool things on second screens, and today’s addition to the prototype shows that off. It’s a little rough-and-ready, but hopefully I can give enough background in plain English here that the potential should be clear.

At the time of writing this post, the prototype when given BBC Two’s service identifiers looks like this:

In the sidebar, under the heading “Would you like to know more?” is a selection of links. In this case, to Wikipedia. These links are quite ugly-looking, but that’s an issue with my implementation specifically which I haven’t sorted out yet, so my apologies for that. What’s neat about these links is this:

• They’re generated from the programme data
• The application doesn’t know anything special about Wikipedia, or the programmes
• The programme data doesn’t actually contain any Wikipedia links
• Just linking to Wikipedia is one example — they could just as easily launch an application, or link to a different site altogether

Allow me to explain. The programme information contains subjects. These subjects all — in this case — live under http://www.bbc.co.uk/programmes/subjects and all have available RDF/XML. The graphs described by that RDF asserts that, for some of those subjects, the URI under http://www.bbc.co.uk/programmes/subjects is in linked data terms equivalent to a URI under http://dbpedia.org/resource. This is called a “sameAs” assertion. It means that the two documents — the one on bbc.co.uk and the one on dbpedia.org — both describe the same actual thing.

Now, there’s nothing particularly special about the dbpedia.org URIs: the application doesn’t know anything more about those than the bbc.co.uk ones, but what it does know is where there are “sameAs” assertions.

What the application does have is a list. You will know that on computers, there are certain kinds of file which open in certain programs. The program tells the operating system (or web browser) in some way that it handles that particular kind of file.

The list our prototype application has is the equivalent, but for URI “namespaces”. That is, there’s an entry in our list for http://dbpedia.org/resource/, and this indicates that the application (or in this case, website) registered by this entry can handle URIs which start with that prefix — or, to put it another way, occur within that namespace.

Along with the prefix URI, the entry contains a rule about how to translate the URI into something which can be presented to the user. In this case, http://dbpedia.org/resource/Claudia_Winkleman is translated into http://en.wikipedia.org/wiki/Claudia_Winkleman. Just as easily, it could be converted into a special URL which would launch Sophia Teutschler’s Articles.app.

But it’s not at all limited to dbpedia — any application or website could be registered as a handler for any kind of namespace: the subject RDF published by the broadcaster can link to anywhere it pleases. An entry for a person, for example, might link to the IMDb page for them — and if you had an IMDb application installed on your smartphone or tablet, you could be taken straight to it.

In an ideal world, operating systems would provide facilities for registration of handlers for namespaces in the same way that they do for particular kinds of document or URI schemes. I suspect it will be some time before this happens, but in the meantime, there are plenty of other ways for this kind of second screen application to construct their own lists based on automatic discovery and (dare I say it) in places a little prior knowledge.

For consumers, it means that relevant applications and websites are offered automatically based on what’s being watched, using mechanisms which aren’t reliant on one particular broadcaster or consumer electronics manufacturer in order to keep it working and fresh.

For broadcasters, it’s a way to get the additional “multiplatform” content — the microsites, the applications, the extra video clips and so on — in front of audiences without having to find an answer to the somewhat niggling question of why they should launch your application in the first place just because the programme came on (which — let’s face it — unless you’re a serious big-ticket show such as X Factor or Strictly, is pretty unlikely).

For developers, it’s an opportunity to put that linked data to good use, both publishing and consuming. Go crazy. Build apps, build websites. Publish the data. Let it all hang together.

Widget, it’s got a widget…

The third revision of the Apple TV is out (the second revision was a minor bump relatively early-on in its lifespan), Google launched the proper publicity materials for the impending Google TV, the Boxee Box is due to land any day now, Project Canvas has become YouView, and when consumer electronics manufacturers aren’t breathlessly trying to tell you about how 3DTV will save television (not that it’s clear that it actually needs saving), they’re breathlessly trying to tell you how “Connected” devices (what, you thought your TV was plugged into something already? shame on you!) will save television, mostly by way of the marvels of “widgets” and “apps”.

The common theme here is that everybody’s got something to sell, and mostly they don’t seem to much care about whether it’s really got much of a long-term future (and, if you were cynical, you might argue that in the case of the consumer electronics manufacturers at least, products having a relatively short useful lifespan is actually of benefit — at least to them).

There are a few different approaches being taken here, varying from the “TV is just a device for displaying content” (i.e., it’s just a monitor for a small computer which happens to be in a convenient location) — the approach of Apple and Boxee — through “TV is a medium too, but mostly we’ll just provide an easy way to get at The Internet” — the approach of Google and the consumer electronics people — to a more rounded approach, as taken by YouView.

I don’t think there’s a lot wrong with the Apple or Boxee approaches — at least in terms of they are. They don’t really change TV itself any more than a DVD or Blu-Ray player does, but the convenient access to lots of fresh content may well help drive people away from linear broadcast and further towards use of on-demand services. that said, they do still look very much like niche products as compared to the ubiquity of the Freeview box. The one thing the Apple TV does have going for it in its new incarnation is as an accessory for all of the owners of other iOS devices out there, thanks to AirPlay. The Boxee Box, on the other hand, is very much pitched at existing Boxee users who would quite like a dedicated device without the hassle (and expense) of buying a PC for it.

Then there are the “Web on TV” approaches being taken by Google and the device manufacturers. I have a fairly strong belief these are, as they stand, horribly flawed, although Google perhaps stands a better chance of long-term success if it can continue to get Google TV adopted by device manufacturers (supplanting their own efforts) and can respond well to real-world challenges.

Let’s look at “widgets”, currently being touted by set-top box and TV manufacturers alike. I believe widgets in this context are about as useful in real-world settings as the novelty singing Billy Bass. There are several reasons for this, but the death knell is the fact that TVs are very often shared devices. The classes of “widgets” which actually work when there’s more than one person staring at the screen are quite limited — once you move past TV guide stuff (which TVs do anyway) and news/sport alerts, widgets become more of an annoyance than a useful feature. The line seems, not entirely surprisingly, to be drawn depending upon the active/passive boundary: if it’s relatively passive, it may well work; if it requires a lot of interaction, it probably won’t. Perhaps the only exception to this are video-on-demand applications, but it’s a stretch to say that these are “widgets” or that they’re closer to the “active” end of that spectrum (most of the time spent using a video-on-demand application should be watching the video, after all).

Widgets, therefore, strike me as limited value, and as much as Jeff from Marketing thinks the Twitters and Facebooks will sell a new TV or set-top box, it’s really little more than a gimmick — and not a very good one, at that.

Google’s goes much further with the Google TV, but is somewhat schizophrenic with it (which shouldn’t come as much as a surprise to anybody who pays attention to what Google does). On the one hand, it’s a “web on TV” device, complete with wireless QWERTY keyboard — this clearly suffers from the “active” problems of widgets — but on the other hand, Google is keen to talk up the relationships it’s building with content providers, so video-on-demand is evidently a key facet of the proposition. This puts the Google TV in good stead on two fronts: first, by not being restricted to simple “widgets”, the Google TV will appeal to the geek market (as yet-another-Web-device), and second, by pitching on-demand content, it’s going up against the Apple TV and the Boxee Box and pushing into the wider consumer market. The Google TV also has one key advantage over Apple’s and Boxee’s products, and that’s that it’s being built into TVs, which brings it a step closer to ubiquity.

The Google TV also has some promising user interface touches: the “home” screen, for example, aggregates your favourite TV stations, applications and web sites all in one “channel”-based interface not dissimilar to the Wii’s equivalent. Ultimately, though, the proposition as a whole seems limited and confused — general web browsing is a poor experience from 10 foot away with the kinds of input devices the Google TV supports, and doesn’t tend to go down especially well with other people watching the same TV. As a route for providing video-on-demand applications, on the other hand, it’s pretty useful. Similarly, while the level of integration between broadcast TV and websites and standalone applications seems to be absolutely minimal, this is something which could be extended in a future revision of the platform. In a nutshell, the Google TV falls firmly — like many of Google’s products — into the “interesting” and “promising” categories, but isn’t really ground-breaking yet and doesn’t seem to have the hallmarks of a killer product.

Next up is YouView, the proposition formerly known as Project Canvas. YouView is in many ways the complete antithesis of the Google TV, but ends up (in terms of functionality) in a vaguely similar place. YouView is, essentially, an attempt at “extending television to use the Internet as a transmission mechanism, with knobs on”. In other words, it’s what you get if you sit a bunch of TV people down and ask them how they would like to capitalise on the increasing prevalence of Internet connectivity in the home. As such, YouView uses the Internet as a delivery system, and makes use of HTML and Flash for applications (alongside MHEG-5), and can be thought of a “Red Button on Steroids”. Again, there’s been lots of talk of “widgets” and “apps” which give access to well-known Internet services with names ending in “r”, but the real meat is the on-demand/multi-screen applications (which likely isn’t hugely dissimilar from how the Red Button is primarily used today).

Where YouView fails is in its broadcaster-centric approach: where in the linear broadcast world, spectrum and infrastructure costs are the key barrier to entry and create a small number of de facto monopolies, you might be forgiven for thinking that a device designed to use the Internet as an alternative content delivery mechanism and pitched as son-of-Freeview (and so theoretically end up in living rooms across the UK) might act as a leveller, removing those barriers for those content distributors wishing to rely on the Internet to get their wares to the public — the technical aspects of this are not particularly difficult to overcome (even allowing for combinations of both automatic Internet-based service delivery and arbitrary manual subscriptions) — but instead, YouView has opted for a joint-venture-based “gatekeeper” approach, complete with certain financial requirements. To compound matters, although YouView likes to talk about the Internet a lot, the complete specifications for the platform have been distributed only to the UK’s “Digital TV Group” — a broadcasting trade body — and although some specifications have been released to the public, many key aspects remain a mystery if you come from the “Internet” rather than “Broadcasting” world.

In all of this, it’s been pretty easy to point out what doesn’t work, with not a huge amount about what does.

First of all, I do think so-called “connected” (or, to use a better term, “hybrid”) receivers are important. I think there’s a lot of scope and a lot of exciting possibilities when the web and TV (and radio) are brought together. I think we’ll continue to see a steady rise in the take-up of on-demand services on the TV, and applications which focus on that will be the most successful of the lot on the TV itself. After all, while it’s nice to have the flexibility of being able to watch catch-up TV on your laptop, a lot of the time you’d rather watch it on your TV — hence the continued popularity of the Virgin Media and Wii iPlayer offerings.

However, where I see “web” and “broadcasting” meeting isn’t on the TV at all, but on the various other devices (some of which we have now, some of which will appear over the coming years). Increasingly, we have smartphones, computers and — more recently — tablets to hand. Wireless networks for them to connect to are inching closer to being ubiquitous. And, most importantly, “apps” have gone from a selection of crappy cut-down games sold at exorbitant prices by our mobile operators to a billion-pound industry which has very much “gone mainstream”.

At the time of writing this, we’re seeing the tip of the iceberg: production companies and broadcasters are producing various kinds of standalone applications which do all kinds of clever things. But, there’s a limit to how sustainable this is: there’s a limit to how many applications it’s worthwhile installing, switching between them for every programme or channel-hop gets tiresome quickly, there’s minimal (if any) communication between the device and your TV or set-top receiver, and each one works differently and does different kinds of stuff. For second-screen applications to work, all of this needs to be fixed.

The good news is that it can be fixed. This is the Internet’s raison d’être, after all: interoperability and open standards. With a common set of communications protocols between devices for things like “what time do you think it is?”, “what it is we’re watching?”, “play this!” and “how far into the programme are we?”, coupled with standard ways of discovering and delivering applications (which may, of course, be websites), and allowing data communication between the devices of friends and family (be they in the same room or more widely-spread), the potential is there to bring together the Web and TV in a useful and usable fashion.

The applications are — like the Internet itself — nearly endless, but just scratching the surface yields exciting possibilities. Watching Panorama? Jump straight to a list of news and reference articles about the subject of the programme. Watching a documentary about penguins on the Discovery Channel? Learn more about them there and then from a range of different sources of information (including, for example, the BBC’s Wildlife Finder. Watching a film? Have the cast and crew information appear automatically so that you can answer that “ooh! what was that film he was in?” question which always crops up, without having to bother with searching for it. Watching Strictly or X Factor? “Be the judges” with your friends. You get the idea.

Unlike on a TV, where the result of putting lots of “widgets” on a screen are often quite horrendous and almost entirely ill-suited to a shared screen, a second-screen application is something you interact with without interfering with the viewing of anybody else, and isn’t doesn’t suffer from the horrible user interface constraints of having to design for other-side-of-the-room use. By putting these ancillary applications on a second screen, rather than being distractions and gimmicks, they become something useful and fun. By doing it in an open-ended way, you’re not constrained by the limitations of a single vendor or broadcaster’s vision of what a second screen should be able to do, but instead have all of the cool stuff that people inevitably come up with when presented with a neat platform (literally, in the case of touch-screens) at your fingertips.

Of ads and blocking

Let’s start with the stuff we know:

1. Lots of people use ad-blocking software.
2. The number of people using it isn’t dropping (in fact, the opposite).
3. Many sites are paid at least some revenues based on the number of ads served (not everything’s pay-per-click) — and whatever you or I think about the PPV model, it exists, and it helps to fund sites we use
4. Content publishers have been beating the “ad-blocking is killing us!” drum for years, with no clear success.
5. People use adblockers principally because of obnoxious and annoying adverts — most people don’t have a problem with ads in general.

This is not sustainable, clearly.

There are four possible outcomes:

1. Publishers manage to convince people not to use ad-blockers (i.e., the moral argument)
2. People stop using and installing adblockers because they’re no longer necessary
3. Pay-per-view ads die and alternative models are located
4. Sites funded by ads die

To date (1) has failed miserably. I can’t envisage a reason why this would change any time soon. To make matters worse, by the time you’ve got as far as telling people that using an adblocker is bad, they’ve already got it installed (not uncommonly by a helpful friend/relative, at that). Persuading people to actively remove it when the reason for installing it in the first place hasn’t gone away is harder than persuading them not to install it in the first place. To make matters worse, I’d wager very few people outside of the industry can name a site which has gone away because ad revenues have dried up (the reality of this is that many sites have come dangerously close, laid off good staff, and scaled things back in desperate attempts to stay solvent).

Despite the inroads pay-per-click has made, pay-per-view doesn’t look like it will die any time soon, either. In any case, the problem is broadly applicable to pay-per-click ads too — it’s just that the effects are felt more directly with pay-per-view. (3) doesn’t look likely. Moving beyond ads, the industry has failed to produce subscription models which work cross-publisher in any sort of sensible fashion, which makes a paywall unworkable for the vast numbers of smaller sites out there.

(4) is fairly likely, but one everybody wants to avoid.

Thus, we are left with (2): people stop using and installing ad-blocking software.

One way for this to happen is for publishers to convince people to, rather than using ad-blocking software, simply not visit sites which have ‘bad’ ads. In theory, this could be the most effective means of killing off bad ads. There are some spanners in the works, though:

1. It suffers from the same problems as option (1) above.
2. Lots of sites get their traffic from search results. People don’t look at URLs when they’re handing over credit card information, how are you going to convince them to pay attention to destination URLs on a page of search results? And, more importantly, how will people remember which sites to avoid?
3. How on earth will you convince them that this is better than just running adblock software in the (very long) period before the ‘bad’ ads die?

I can’t forsee any way around these practical issues, as nice as the idea is. I wouldn’t want to bet my income on it, put it that way. Attempting to change public behaviour wholesale requires a massive amount of effort, and even then there’s a huge chance of failure.

That leaves only one other option: for the content publishing industry to kill the sorts of ads which drive people to ad-blocking software, from within. There are two reasons for this: first, they have a vested interest in seeing this happen — it’s their revenues on the line, and second, it’s about the only thing left to try which stands any realistic chance of success.

And so, if the content publish industry wishes to preserve its pay-per-view revenue, it only has one choice: collectively apply pressure on the ad brokers and other sites who serve up ‘bad’ ads. As much as the publishers need the ads and their brokers, the brokers need the decent sites to place the ads on. If this smells suspiciously like a form of self-regulation, you’d be right — but it’s something that only self-regulation can solve; it’s not an issue which is worthy of a more onerous regime, and it’s one which works best if everybody pulls together (and there’s nothing like the threat of revenue drying up to bring about a spirit of cooperation).

You might question why the onus should be on the publishers to do this, and there’s no good answer to this question: the onus shouldn’t be on the publishers to do it. However, if the publishers want to preserve their current revenue streams, the only thing they can do which actually stands a realistic prospect of success is working together to make the ‘bad’ ads — which keep ad-blocking software from obsolescence — a thing of the past.

(Addendum: you can substitute “pay-per-impression” for “pay-per-view” above, if you wish, as that’s what’s actually being paid for. There’s then an argument that you could adjust ad-blockers to still load ads, but hide using CSS overrides and that this would be some value of “better” — some ad-blocking solutions do this, in fact. This has two consequences: bad ads get more audacious in attempts to evade the more trivial filters, which serves to drive people back to more effective blocking schemes; and worse, the value of an impression is scaled downward to account for the fact that a not entirely insignificant proportion of people never come close to seeing the ads out of the corners of their eyes).

You should probably ignore this post

Okay, here’s the scenario: I want to demonstrate a certain thing which relies on subject URIs being in a particular namespace and being easily-discoverable.

For example: a clear path (via the RDF) from an episode to a topic in the http://www.bbc.co.uk/nature/ namespace.

Because /programmes doesn’t include /nature URIs in either its episode graphs or its /programmes/subjects graphs, I’m going to take some carbon-copies of certain episode data, preserving all of the referenced URIs except for the topics I want to hook up to /nature. For these, I duplicate whatever’s in /programmes/subjects and insert an owl:sameAs referring to the appropriate /nature topic.

Now, the key questions:

• Is it appropriate to say that my near-duplicate episode graph is sameAs the original?
• Is it appropriate to say that my topic is sameAs the /nature equivalent (if all done right, it should be)?
• What if my topic graph is a stub which contains nothing but a label and references to other places — should they be seeAlso?

Now, to complicate things: there may be situations where there’s no topic reference in the original episode graph to the appropriate /nature topic — say, for example, there’s a reference from the /nature topic to the episode, but for some reason the topic was never created or otherwise wasn’t referenced in /programmes itself — if I insert the additional topic reference (either directly to the /nature topic, or to a stub which links to the /nature topic), do the answers to the above questions change at all?

[The goal here is to be able to point a browser extension or an application or something at an episode page which has RDF and for it to be able to discover the relevant subject URIs, and offer interfaces to those subjects using applications which register handlers for given namespaces — e.g., “Find out more about the Great Penguin using BBC Wildlife Finder-Browser Thingy Two Point Oh”)]

BT confirms it sent customer info to ACS:Law - unencrypted

This piece on TechEye.net confirms what we knew yesterday:

BT has confirmed it sent customer details in unencrypted Excel spreadsheets as email attachments to the legal firm ACS:Law.

BT said this morning it was investigating how this had happened and was still waiting for ACS: Law to let it know if any of its customer details had been compromised by the leak.

When asked if the details were sent unencrypted, a BT spokeswoman told TechEye: “I can confirm that this did happen but has no bearing on the current situation.

and:

In this circumstance our legal department sent data to a firm of solicitors (ACS Law) which reached them safely and we trusted that they would keep the data safe.

and also:

At this time we do not believe any of BT’s customers details have been compromised by this leak, although we are continuing to pressure ACS Law for confirmation of this.

Now, either this BT Spokesperson has a fundamental misunderstanding of how everything she’s talking about works, or this is a smokescreen. As the old one goes, “which are you, lying or ignorant?”

It’s quite clear that the spreadsheets were received by ACS:Law. “Safely” is a different matter. This is not a physical item which can be damaged in transit: this is e-mail, sent to a third-party hosting firm’s servers. BT have literally no way of knowing whether the e-mail was intercepted in transit, and — given that personal information was sent unencrypted via cleartext e-mail — it’s entirely possible that copies were retained on BT’s mail servers also.

Further, “we do not believe any of BT’s customers details have been compromised”: they know for a fact that it is (unless they’re counting customers of their subsidiary PlusNet amongst the people who aren’t BT customers), because otherwise how would anybody have any idea what was in the spreadsheets sent by BT to ACS:Law in order to ask the question in the first place?

Everybody’s scrabbling to blame ACS:Law, when the faults run wider and deeper than just one firm.

What was wrong with ACS:Law’s anti-piracy operations?

As far as I can tell, pretty much every aspect of ACS:Law’s anti-piracy operations was flawed in some horrible way, and I mean that in the very real sense of “utterly broken to the point of the sublime”.

1. Collecting infringing IP addresses. This was carried out — always by a third party, but one with whom ACS:Law kept very close oversight of — in a horribly naive fashion. If you’re curious, the person running the collection system (at least at one stage), quite clearly knew very little about the technical aspects of doing so. TorrentFreak covered this back in April. The leaked e-mails show that although some concerns were raised internally about the standards of evidence and the difficulty of turning any of the information into actual proof, operations continued along the same tack nonetheless. It’s important to note that — at this point, all ACS:Law has to go on are IP addresses and timestamps, along with the information about the files that it is alleged were being downloaded, and it’s even more important to note that the information may very well (and very easily) be wrong — there’s no guarantee whatsoever that the list that ACS:Law possesses is a true reflection of fact and reality.

2. Armed with a list of timestamps, IPs, torrent filenames and hashes, ACS:Law obtained Norwich Pharmacal orders against the ISPs to whom the IPs belonged. These orders compel the ISPs to identify the subscribers (where possible) who were making use of the IPs in the list at the times given. Although ACS:Law does nothing wrong in particular at this stage, the ISPs don’t seem to be fighting the orders at all (I would have thought the fact that there is no actual proof would be grounds to at least attempt it, but I’m no lawyer), and certainly doesn’t appear to be informing subscribers that their information is to be handed over.

3. The ISPs deliver a list of subscriber details — names, addresses, postal codes — to ACS:Law. In BT’s case, as the owner of PlusNet, this is sent as an unprotected Excel Spreadsheet in a cleartext e-mail, albeit with (somewhat laughable) warnings that the information was kept secure.

4. ACS:Law preserves many of these e-mails in their mailboxes, again unencrypted. Moreover, it hosts its e-mail service externally, so all confidential correspondence (including internal correspondence) is stored at a third party external location. I’m given to understand there’s some use of GMail thrown in for good measure. All of the e-mail is stored in such a way that in the event of a Denial of Service attack on ACS:Law’s website, the entire mail spool is leaked: either because security was poor, or because it was accidentally made public (I’m not sure which). In either case, this is something which simply shouldn’t have been possible: confidential correspondence, including the subscriber details, shouldn’t have been directly accessible to the third party operating the website, let alone publicly.

5. Armed with a combination of IP addresses, timestamps, names, addresses, postal codes and alleged download details, ACS:Law sends threatening letters to thousands of people. There are reports abound of people being very confused by the letters because they could not possibly have committed the alleged infringements (and indeed I know one such person myself). In addition, ACS:Law has been written to by several groups and organisations raising serious concerns regarding the threatening tone of the letters.

6. If somebody denies infringement, they are requested to fill in a questionnaire which carries dire warnings that all of the information may be used in court, and is chock full of “have you stopped beating your wife yet?”-style intonations. Those filling in the questionnaire are asked if they are willing to have their computers submitted for forensic analysis — which nobody (innocent or otherwise) in their right mind would — with the clear implication that a “No” answer suggests something to hide. All of this despite the fact that ACS:Law has taken precisely zero people to court over copyright infringement, ever.

7. Alternatively, people do pay up (and in some cases without admitting any liability or involvement whatsoever — they do so purely for a quiet life), despite there being no credible evidence of any sort to compel them to do so.

You might think, at this stage, that this will be the undoing of ACS:Law, and that will be that. You’ll be half-right. While it probably will be the undoing of ACS:Law, the methods which will be used to send “infringement notices” under the terms of the Digital Economy Act are no better: just as flawed in terms of evidence, and while the letters (being sent by the ISPs, rather than some third party trying to make a quick buck) may be worded a little more nicely, there’s still a presumption of guilt and an introduction of unnecessary stress on the basis of very little at all.

What the ACS:law leak tells us

There’s some very interesting stuff in the leaked ACS:law mailboxes. As you’d expect. It also raises more than a few… how can I put this? “concerns”. Not just about ACS:law’s practices, which were the subject of some scrutiny already, but also of certain ISPs, and the Digital Economy Act’s implementation.

First, there’s the fact that a firm of solicitors hosted their internal e-mail externally, and clearly on a host which wasn’t especially gifted in the security department: a DDoS attack doesn’t expose an entire Courier IMAP mail spool, it just stops things working. Either somebody guessed or engineered a particularly important password (which shouldn’t be enough anyway), or the spool was left visible somewhere it shouldn’t have been. To put this in context, if an e-commerce outfit did this, they’d be at severe risk of having their operations shut down by their bank. I struggle to think of a situation where keeping internal communications confidential is more important than at a firm of solicitors.

Next, there’s the behaviour of the ISPs against whom Norwich Pharmacal orders were granted. BT’s Legal Department, at the very least, sent unencrypted Excel spreadsheets containing personal data (including names, addresses, and information about alleged infringements) about PlusNet subscribers in cleartext e-mail to ACS:law. In a somewhat ironic twist, they advised ACS:law to ensure the data once received was kept “secure”.

Amusingly, while Andrew Crossley was being advised internally that the chances of achieving damages in any case against an alleged infringer were pretty minimal, the QC advising Crossley in his dealings with the SRA was strongly suggesting that ACS:law should stop writing letters and instead actually take some people to court to help demonstrate the legitimacy of the operations. Whoops.

And now we get to the real meat of all of this: I know for a fact that ACS:law has sent letters to people who haven’t infringed. Lots of them. I know this, you know this, and ACS:law knows this. Even so, they were paying close attention to the DEAct Code of Conduct machinations and were keen to be collecting all of the right pieces of information so that they could work within its framework. As I’ve argued in the past it’s virtually impossible to prove that somebody has been involved in file-sharing as a passive third party, and you can’t be an active party in it without encouraging or even enabling the illicit activity in the first place. ACS:law’s actions show quite clearly what a mockery of due process and standards of evidence the anti-filesharing provisions of DEAct entail: everything is based upon allegations, and without any actual proof available to back it up, that’s where it stays. Action is taken, costs are run up, not insurmountable amounts of distress caused, all based upon unprovable allegations.

I hope this whole ACS:law debacle exposes to those who didn’t already realise it quite how utterly broken the DEAct provisions are.

PECNs continued

After a bit of to-ing and fro-ing over Twitter, it dawned on me that I missed a trick in my previous write-up of PECNs.

Let’s see that definition again:

“Public Electronic Communications Network” means an Electronic Communications Network provided wholly or mainly for the purpose of making Electronic Communications Services available to members of the public

The part I missed was the distinction between an Electronic Communications Service (ECS) and a Content Service (CS). It’s basically this: an ECS is for direct communication between entities, while a CS is a publishing service. Again, there will be grey areas, but that’s it in a nutshell.

The reason why this is important is because it has a bearing on whether an ISP is a PECN or not.

In days gone by, the main reason for people getting Internet connectivity was e-mail — that is, an ECS.

Nowadays, I’d argue that isn’t true at all. The main reason that people get Internet access, and so the principal purpose of ISPs’ access products, isn’t to make Electronic Communications Services available to members of the public, but to make Content Services available to members of the public — in other words, the Web. Even the e-mail services that we use are more often than not accessed themselves by way of Content Services, because they’re web-based (and rolled into broader propositions such as Google, Yahoo! and Windows Live).

And so, most ISPs’ access products do not fit the definition given earlier, but instead this one:

an Electronic Communications Network provided wholly or mainly for the purpose of making Content Services available to members of the public

If you read through the General Conditions, you’ll see that there isn’t a name attached to the above definition. In fact, it isn’t even mentioned. It isn’t something which has a special name or regulatory regime beyond those applied to Electronic Communications Networks or Content Services in general. If an access product sold to the public fits this second definition, rather than the one near the top of this post, then it can’t be a PECN.