Did Richard Warman lie about "lucy"?

In the recent debate about whether Richard Warman was responsible for the racist Cools-post, one sometimes finds the claim that Warman was caught in a lie about his membership on Freedomsite during the Lemire hearing. For example, one commenter at drdawgsblawg offered this summary

Warman previously denied under oath that he'd signed onto a message board as the infamous "Lucy", then recanted when it became clear moments later that he had indeed used that pseudonym. So, did he "lie" then? I suppose it depends on how finely one likes to split hairs.

When challenged, he linked to a graphic a snippet of the transcript that had been posted to freedominion, which I repost here (the red underline and arrow were added by me):

Based solely on this, one might conclude that Warman had lied. It is always worthwhile, however, to be attentive to how evidence is presented and how that presentation affects one's perception. In this case, note especially the tendentiousness of what I've underlined in red.

Was there some "incontrovertible proof" presented, as was claimed by whoever prepared this picture? No. Let's look at the actual transcripts. Here is a screen capture of the bottom of p. 769 and the top of 770, including the page break. Note the red arrows, which point to what the commentary has replaced.

As you can see, there was no evidence presented at all, much less 'incontrovertible proof', and what the commentary has replaced is nothing more than a page number. What happened? Warman was asked whether he had registered an account, he answered that he had not as far as he remembered. When he was shown an account name that jogged his memory, he immediately corrected himself.

Now, those who dislike Warman or his actions are free to disbelieve him if they choose. But we also know (from Klatt's affidavit) (1) that Warman never posted any messages on the board as "lucy", (2) that he had logged in under this name precisely twice (see here, p. 37 of the pdf), and (3) that both his log-ins occurred within a two-hour period on Nov. 15, 2003. Given that more than three years had passed between these logins in 2003 and his testimony in February, 2007, surely any fair-minded individual would have no difficulty believing that he had simply forgotten about the "lucy" account in the interim.

So. A lie or an honest mistake? Take your pick. But if we're going to call this a lie, what are we to make about the numerous misstatements of Klatt in his sworn testimony?

Edited and rewritten for clarity and concision.

Free speech for me, but not for thee?

A post was made to Freedominion by a long time poster in the early morning hours of last night. I followed the incoming traffic back to FD this morning to see this post:

Moments later, however, the posting disappeared when I reloaded it in another window; deleted apparently.  I took a screen shot of the above, and this (where the posting used to be):

I guess that freedom of expression stuff only counts if you're writing what Free Dominion want to hear.

Update. 10:35. Long-standing FDer, EdS, has now now asked about the missing post here. This should be interesting.

Update 2. 10:42. Connie has given her permission to repost the links.

Update 3. 11:05. EdS has now reposted them. It'll be interesting to see what the reaction is.

One of these things is not like the others: why Warman is innocent

Much of Canada's right-wing blogosphere is aflutter with the idea that human rights lawyer Richard Warman might have posted a racist comment about Canadian senator Anne Cools on freedomsite.com, a white-rights forum. The evidence? Freedomsite's logs preserve data from some visits of "lucy" (a handle Warman is known to have used) and from 90sAREover, who had posted a hasty racist screed against black senator Anne Cools. According to freedomsite's expert witness Bernard Klatt, both the IP and the user-agent data were identical for "lucy" and for "90sAREover". Until now we have had to take Klatt's word on this.

Recently these logs have become available for scrutiny (they were included in Klatt's affidavit, which I've recently posted here). The logs make it clear, however, that there are two important differences between the computer used during Warman's visits and those of 90sAREover (the Cools poster); both of these differences are exculpatory of Warman. Here is the evidence in reverse chronological order (compare what is underlined in red).

The log of Warman's visit from Nov. 23, 2003 (p. 33 of Klatt's affidavit):

The log of Warman's visit from Nov. 15, 2003 (p. 30 of Klatt's affidavit, here):

The log of Warman's visit from Nov. 11, 2003 (p. 29 of Klatt's affidavit, here):

The log of Warman's visit from Oct. 15, 2003 (widely cited, e.g. here):

All of Warman's visits from Oct. 15 — Nov. 23, 2003, show the same IP (66.185.84.204 = wc09.mtnk.rnc.net.cable.rogers.com; see the blue underline) and the same operating system and browser (MSIE 6.0; Windows 98 — see the red underline — which points to a generic version of Explorer 6.0 and Windows 98).

Here is the log of 90sAREover of Sept. 5, 2003 (p. 34 of Klatt's affidavit, here). Here the browser details are different (again underlined in red:

Although both Warman and 90sAREover have the same IP and operating system, 90sAREover's user agent string has two key differences from Warman's computer-setup. First, 90sAREover had a customized version of Explorer 6.0 installed, which had been supplied by Rogers (as is signaled by the phrase "Rogers Hi-Speed Internet"); second, he did not actually use IE 6.0 to post, but instead was using RealOne Player version 2 (hence "R1 1.3": see here). None of Warman's log entries, by contrast, show any sign of RealOne, and for all four of his visits his browser was recorded as a generic version of Explorer, not a Rogers one.

These differences are important and exculpatory and overthrow the technical argument for identifying Warman as "90sAREover", which (given that the IP used, 66.185.84.204, could have been anyone of millions of Rogers customers) has now collapsed. The computer that posted the Cools post was set up differently than the one Warman was using three months later.

(Re-written for clarity and context.)

Klatt affidavit in Richard Warman & CHRC v. Lemire

I've been wondering aloud for the past few weeks (here and here) why it is that although Warman's critics have produced scores and scores of documents (see here, for example), the actual evidence that he was the Cools poster were nowhere to be found. Logs were produced to show that Warman visited the site without logging in on Oct. 15, 2003, and these have been cited as some kind of proof (here and here). But where were the logs of the notorious Cools post? Why was some evidence being fed into the blogosphere, but the most important evidence not?

Someone pointed out to me that the logs were part of the original Klatt affidavit, which I've managed to acquire. So that you can see the whole thing, I've uploaded into scribd.com and embedded it below. There are several serious mistakes in Klatt's testimony that I have already identified, and I hope to return to these soon. For now, however, the logs of the notorious "Cools" post is on p. 35, and these show that the case against Warman is fatally flawed: 90sAREover had a different browser installed than what we know Warman used for all of his posts in October and November. I'll clarify this later today.

Read this doc on Scribd: Klatt affidavit - 22 Aug 2006

Revisiting the lucy-90sAREover identification

Those who have been following the controversy about the Cools post will probably know the following chart. It apparently imitates one that Bernard Klatt presented at the CHRC hearing -- Klatt refers to such a chart during his testimony -- but this version (with my mark-up in red for point-by-point reference) was posted freedominion.ca on January 20 by "Entropy Squared". It is now part of Warman's statement of claim.

The chart was designed to make a case that Richard Warman was responsible for the posting of a racist denunciation of Senator Anne Cools on Sept. 5, 2003. The evidence argues against that conclusion. (Again, what is in red is my mark-up.)

A ~ a and B ~ b. Warman registered at Freedomsite.org using the pseudonym "lucy" and the IP 66.185.84.204, according to Klatt's testimony. Klatt also testified that 90sAREover (who wrote the notorious Cools post) had the same IP. I have shown here, however, that this IP is one of Rogers' web caching servers and could be used by any Rogers' subscriber -- indeed, the post could have been made by anyone with access to a Rogers-connected computer. This could be any one of millions (here).

• Update. Actually, the IP of "90sAREover" is less clear in the logs than Klatt's testimony implies: in the few minutes that 90sAREover spent at the site, his IP can be seen shifting back and forth between 66.185.84.204 and 66.185.84.200 five or six times. See here.

C ~ c. According to Klatt's testimony, the user-agent string recorded in the logs freedomsite.org show that both "lucy" and "90sAREover" used Windows 98. According to W3schools.com, 12.1% of computers were using Windows 98 in Sept. 2003, when the Cools screed was posted. 12% of millions (see previous), however, is still hundreds of thousands.

D ≠ d. During his testimony at the CHRC, Klatt asserted that "lucy" and "90sAREover" used the same browers. This is wrong. Details of a visitor's browser are found in the user-agent string, which is recorded by many sites. To judge from Klatt's testimony (here)*, the user-agent string of 90sAREover included the phrases "Rogers Hi-Speed Internet; (R1 1.3))". The R1 1.3 means that 90sAREover was not using Explorer as his browser, but in fact was using RealOne Player (version 2). The reference to "Rogers Hi-Speed" shows that the Cools poster had a version of Explorer that had been customized by Rogers for its customers (here), even though he was not using it to visit this site. To judge from Klatt's testimony, none of Warman's own visits show either the RealOne Player or the Rogers-customization. This argues against identification. (One wonders why there are not copies of these logs floating around the internet -- is it because they exonerate Warman?)

*Update. Now that the site logs are available for inspection, rather than Klatt's testimony, it is clear that Richard Warman's browser set-up was different than 90sAREover's.

E ≠ e. The IDs and emails are different. This can hardly be used as an argument that they're run by the same individual. Indeed, those trying to argue that they are identical need to explain why Warman registered "lucy" in mid-November (an account that he never posted from) if he already had the 90sAREover accout. If he were "90sAREover", why wouldn't he simply have used that account rather than registering the new one?

F ~ f. It is a matter of public record that "lucyaubrack@yahoo.ca" was Warman. We don't know, however, whether the "rob_m_simpson@hotmail.com" account was anonymous or whether it reflects someone's actual name.

G ~ g. Both "lucy" and "90sAREover" found their way to freedomsite.org, which was relatively obscure. This no more requires that these two were identical, however, than it would for any of the site's others users, or indeed for visitors of other obscure sites (such as the one you are visiting now). In any case, it is worth noting that 90sAREover is member #1331 (affidavit, p. 10), and "lucy" (Warman's handle) is member #1379 (affidavit, p. 10), which implies that at least 50 new members joined up in this period. These surely aren't all the same individual.

H ~ h. The entry seems slightly wrong.* Klatt only mentions a single log-in of "90sAREover", and mentions three log-ins by Warman (two as "lucy" and one as "guest"). (Cf. his summary at p. 1648, where he says that the two accounts logged in "once or twice".) But even if this were correct, it is not helpful. All internet sites attract one-time visitors who don't return. (By way of comparison -- 90% of my traffic at this blog are first-time visitors, and another 8% visit 2-5 times; less than 2% return more than 5 times.) Why would this suggest they are identical?

*Update. Klatt's testimony is less clear than his affidavit (here), which shows that both lucy and 90sAREover did in fact log-in only twice.

I ~ i. The respective usage times differ by a factor of ten. That is hardly an argument for regarding them to be identical.

J ~ j. When they registered both "lucy" and "90sAREover" filled none of the optional boxes. The question is how typical this is. Personally, I never fill in more than I have to, and I suspect that is a common approach to message boards.

K ~ k. "Created for a single purpose" states more than can be known -- the "purpose" (or intention) of one or both of these accounts may have been either grander or more modest than came to pass. Indeed, given that the "lucy" account was never used to post anything at Freedomsite, it's difficult not to assume that its original purpose was something different than what it was actually used for, given that it was never used.

L ~ l. I assume that the dates here are offered for context, not as an argument in favour of identification.

Updated several times for clarity.

90sAREover's browser: RealOne with customized Explorer

Commenter "freemarkets" discovered an important fact when he pointed out (here) that the notorious Cools post was made by someone using RealPlayer, which has a built-in browser.  As it turns out, it was RealOne Player, version 2 (see here). Here is a screen shot in which I use this version of RealOne Player (downloaded from oldversion.com) to go the site useragentstring.com:

The tell-tale sign is the string "R1 1.3" at the end, which signals this specific version of the RealOne player.  

Another noteworthy fact, however, is "MSIE 6.0" earlier in the sting. This stands for Microsoft Internet Explorer 6.0. Explorer wasn't open when I took this screen capture, but these details are reported anyway. Why? As it turns out, although the RealOne browser seems perfectly adequate as a browser, it was not completely independent technologically, but depended on Internet Explorer for some of its functionality. In Jan. 2002 someone had complained to Real, Inc., that his RealOne browser wouldn't work after he had removed Internet Explorer from his system. Here is the answer he received from Real (which he then shared with the world through google groups):

"Please note that it is necessary to install MSIE 5.0x (Internet Explorer) or higher versions of the browser installed on your system for the RealOne Player to function properly. RealOne Player uses some components of Internet Explorer browser. I suggest you install Internet Explorer browser on your computer, then install RealOne Player again on your computer to resolve the problem. Please visit the Microsoft's web site (http://www.microsoft.com) to download the latest version of the MSIE browser."

So, Explorer is needed to run RealOne.

It is probably for this reason that whenever RealOne visited a site, its user agent communicated not only its own details in its user agent string, but also details about the version of Internet Explorer on the host computer. A long list of user agent strings in which we find some version of the RealOne Player can be at botsvsbrowsers.com.

But more can be said about this. As we saw, 90sAREover's user-agent string read something like this: "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))". Just as above, "R1 1.3" refers to the RealOne Player, and MSIE 6.0 refers to the browser that the RealOne used to help it surf the net. But this wasn't just any version of that browser, but one customized by Rogers and distributed to its customers: hence the "Rogers Hi-Speed Internet" tag (here).

The user agent string from Warman's visit shows that he had no such set-up: his version of Explorer was un-customized, and as I've shown (here), this argues against identifying him as the 90sAREover poster.

experimenting with RealOne and Explorer's customization options

When RealPlayer is used as a browser, the user-agent string left behind reveals not only the version that is used to visit a site, but also some details about the version of Internet Explorer that is resident on the surfer's computer.

We have already seen (here) that Microsoft allows customization of user-agent strings. I customized a version of Explorer by inserting "Buckets", surfed over to useragentstring.com, and took this screen shot:

Then I exited Explorer, rebooted, and visited the same site using the RealOne Player.  Here is the screen shot:

Note that the change to the user agent string that I had made for Explorer (inserting 'Buckets') is still there, even though I'm using RealOne, not Explorer.

The presence of "Rogers Hi-speed Internet" in the user agent of 90sAREover, therefore, requires that his browser was one of those customized by Rogers, even if he wasn't using it to directly visit freedomsite.org.

Probably millions, actually.

I showed last week that the notorious Cools post could be made by almost any of the 700,000 or so subscribers that Rogers had in Ontario in 2003 (see here). Big City Liberal reacted by calling the odds one in a million (almost). In fact, a million may understate.

The 700,000 is refers to the number of Rogers' subscribers in Ontario. Many of those subscriptions, however, will have been hooking-up multiple computers, and many of those computers were accessible by multiple people, and not merely to those within their own households. The post in question was made on the evening of Sept. 5, 2003, which was a Friday, which means that as we begin to calculate the theoretical possibilities, we have to include every household whose in-laws were visiting ("can I check my email really quick?"), had had a friend drop by (etc.), or who had an open wifi for war-drivers.

Once all the possibilities are included, we may be approaching a significant fraction of Ontario's population as possible suspects.

More on R1 1.3 as the user agent string in the Cools post of 90sAREover

We have already seen that the user agent string of 90sAREover (who posted a nasty racist screed against Anne Cools on Sept. 5, 2003) was "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))".

The reference to Rogers here is to a special version of Rogers' browser distributed to its subscribers.

In a user agent string, R1 1.3, refers to Real Player, and we can now state definitively what the specific version was: RealOne Player, version 2, and as you can see here (red arrow for the RealOne version; violet arrow for the user agent string):

The difficulty, as commentator freemarkets pointed out in the comments to an earlier thread, is that we can only get the RealOne Player to show up in the user agent string if it is actually the browser that is being used. That is, Real Player versions for Windows, at least since RealOne, have a browser built into them, so that you can surf the web using Real Player.

It seems that the only possible explanation is that the Cools poster 90sAREover was using RealOne as his browser. But an important question remains outstanding. How is it that "Rogers Hi-Speed Internet" found its way into the user agent string. Was there a special Rogers version of RealOne Player?

Where are all the logs?

I have been posting for several weeks on the question of whether, as some have argued, "lucy" (a pseudonym that we know belonged to Richard Warman) was the same person as "90sAREover" (who posted a nasty racist denunciation of Anne Cools), as has been alleged by his enemies. I've already shown that the IP being used, 66.185.84.204, is a web-caching proxy that might be almost anyone in Ontario. And in the last post I pointed out that Klatt's testimony in Warman v. Lemire (see here) about 90sAREover's user agent string on Sept. 5, 2003, seems to imply that it was performed on a different computer than the one that Warman used on October 15.

But all this raises a question that should be uncomfortable for those who have argued that Warman is the Cools poster. Why is it that these logs have not been released? Is it because they are inconsistent with the claim that Warman had written this?

Why Warman is probably innocent, part 2: Rogers Hi-Speed Internet

Whenever you visit a site on the internet, you leave details behind about where you are coming from and information about your computer configuration. These details are included in your user-agent string. (For a tutorial to user-agent strings, see here.) As we have seen, the user-agent string of the Cools poster will have been this:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))"

I have already argued that the "R1 1.3" (blue) means that the Cools poster had Real Player (version 1.3) installed. Commenter "freemarkets", however, points out in the comments of that post that in order to register in a user string, Real Player must be running, which weakens its exculpatory value.

But what about the entry "Rogers Hi-Speed Internet" in red? Clearly it identifies the poster as a Rogers subscriber, but that cannot the whole story, since it is not found in the user agent string of all Rogers' customers. Indeed, most don't. Take, for example, the thirty or so visits the proxy 66.185.84.204 made to the site ArcticCircle.ca in 2002 and 2003 that I have presented here. The user-agent data is listed for these visits, but only four of them include "Rogers Hi-Speed Internet". The same impression arises from the logs here, where over 3000 visits from various Rogers IPs were recorded from mid-2002 to mid-2005, and fewer than 50 have "Rogers Hi-Speed Internet" in their user agent string.

So what is happening here? In fact, this designation is mostly about the browser being used. If you've ever seen a version of Internet Explorer dedicated to a specific company (Yahoo!, Rogers, Bell, etc.), you'll have encountered browser customization, a process Microsoft describes at length here. The relevant section:

Customization Examples for ICPs

To showcase your organization's information and services on the Internet, you might want to consider the following customization options: Add links to your organization's Web sites. For example, if your organization is a radio station, you could add links to Web pages that highlight playlists and composers' biographies. Update the browser with your organization's logo and appropriate graphics. You can add your organization's name to the title bar and your organization's logo seen in Windows Update Setup for Internet Explorer and Internet Tools. In addition, you can replace the browser logo with your organization's logo or other graphics. Track information about your customized browser by using a user agent string, which is a string of characters that a Web browser sends when it visits an Internet site. The custom string that you append to the user agent string enables Web sites to compile statistics about how many of your customers are using your browser to view those sites.

Note especially the red. Custom versions of Explorer were typically signalled in the user agent string. And this is what Rogers did: create a customized version and distribute it their clients. It looked a little different, reminded their subscribers they were (joy!) Rogers' customers, set the default web-page to Rogers' site, etc. Also -- and this is the important point here -- this special customized edition of Rogers left its mark at every web-site they ever visited, with the "Rogers Hi-Speed Internet" tag in the user agent string.

Of course, not all Rogers customers used the special Rogers version. Indeed, the vast majority seem to have opted for the one that came with their machine, or that they downloaded themselves. This group -- who may have been over 90% of Rogers customers -- did not have the "Rogers Hi-Speed" tag in their user-agent string. (Indeed, as you can see here and here, the user agent string might continue even after a change of ISP -- it's about the browser being used, not the ISP.)

So, what does this have to do with our present inquiry?

We have already seen that the Cools-poster (90sAREover) had Real Player installed on his computer and (as "freemarkets points out) had it open when he visited freedomsite.org. We now also know that his visit was completed using a version of Internet Explorer that had been customized by Rogers.

And Richard Warman? His visit to freedomsite.org on Oct. 15, 2003 (a month after the Cools post) left the following log entry (the user agent is underlined in blue):

As you can see, Warman does not have the "Rogers Hi-Speed" tag in his user agent string, therefore he was not using a customized Rogers browser. This presents an obstacle to any attempt to identify Warman as the Cools-poster.

Why Warman is probably innocent

Whenever you visit an internet site, you leave certain details behind about your computer, including what browser you use. These details are called your user agent string; the widget to the right (produced by danasoft) uses your user agent string to give you your own private message. (To learn more, see here.)

We have seen (here),that the notorious Cools post was made on Sept. 5, 2003, by a computer with a user agent that will have looked like this:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))"

As we saw in the last post the details in red here show that the Cools poster, had a computer running Windows 98 and had a version of Microsoft Exporer 6 that had been supplied by Rogers cable (hence the "Rogers"-part of what is in red). The computer also had Real Player (version 1.3) installed and used it to visit the site.

We also know that Warman visited freedomsite.org on October 15, 2003, leaving behind this log entry:

The user agent is underlined in blue. There is an important difference from the Cools poster. What is red in the Cools poster's user agent data is missing from Warman's. Warman, like many Rogers' customers, has no "Rogers Hi-Speed Internet" in his user agent (the version of Explorer on this machine was a generic one, not one supplied by Rogers: see here). Moreover, he did not use Real Player to visit the site (thus no "R1 1.3").*

But this means the user agent data is not identical. So, what do we have to prove Warman wrote the Cools post? An IP shared by almost a million people, and a computer that is differently configured. One might begin to manufacture scenarios in which Warman made both posts, but they're getting into lottery-like odds. At the very least, the forensics require that the posts were made from different machines.

*Note. Commenter "freemarkets" points out (surely correctly) that the Real Player had to be both installed and playing in order for it to register in the user string. See now here.

Edited  and revised for clarity and accuracy.

For more commentary see this more recent post.

What does "R1 1.3" mean in a user agent?

In the last post (here), we saw that Bernard Klatt, Lemire's expert witness in his ongoing CHRC hearing, revealed 90sAREover's user agent data to be this:

"Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))"

Now, Klatt successfully identified the browser (Internet Explorer version 6.0) and Operating System (Windows 98). He was less successful in explaining the latter two items "Rogers Hi-Speed Internet; (R1 1.3))".

Apparently he didn't know User Agent String, a handy-dandy site that takes any user-agent and breaks it up into nice understandable parts. Entering our string yields the following table

Internet Explorer 6.0
Mozilla	It's a Mozilla based browser
4.0	Mozilla Version
compatible	Compatibility flag Indicates that this browser is compatible with a common set of features
MSIE 6.0	Name : MSIE 6.0 Version = 6.0
Windows 98	OS-or-CPU : Windows 98
Rogers Hi-Speed Internet	Rogers Communications Internet provider. Partnered with Yahoo! to offer Rogers-Yahoo! Hi-Speed Internet.
R1 1.3	Using Real Player as a browser
All Internet Explorer user agent strings

Now, especially to be noted is how wide of the mark is Klatt's explanation of R1 1.3. His explanation (p. 1634):

This is clearly wrong, as the useragent.com search shows. "R1 1.3" does not refer to Cisco firmware, but to the fact that the user of this computer had a Real Player installed.

The "Rogers Hi-Speed Internet" is less clear. Not all Rogers' users have it. In the thirty visits from a dozen or so users of 66.185.84.204 here, only a couple have it. (To judge from this, the Rogers-tag here means that the version of Explorer being used was one supplied by Rogers, not a generic version.)

More shortly.

Update. Commenter "freemarkets" points out that the Real Player only shows up in the string if it is being used, not installed.

Clarifications to Rogers Hi-Speed made; quote from Klatt on R113 added.

Klatt on 90sAREover's user agent

Here are pages 1633-5 of Richard Warman and the Canadian Human Rights Tribunal v. Marc Lemire (T1073/5405 vol. 9) of Feb. 8, 2007. In it, Lemire's expert witness, Bernard Klatt, is testifying about the logs of Lemire's site, and trying to make the case that "90sAREover" was identical to "lucy" (a username of Richard Warman). 

In the following passage, Klatt describes a few of the log-entries.  He has already pointed out that "90sAREover" and "lucy" share the same IP: 66.185.84.204, but as we've seen elsewhere, this does not prove they were identical: that IP is a proxy shared by hundreds of thousands of Rogers' customers.  Below Klatt describes other facts found in the entry:

p. 1633

p. 1634

p. 1635

Now, to judge from Klatt's remarks here, the log entry, which will have ended with user-agent data, must have looked something like this:

 "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))"

This is something that we have paralleled in weblogs elsewhere.  This exact same configuration can be found in this log of June 4, 2004, of the Engineering School at the University of Illinois.  The date of that log rather spoils Klatt's theory that this is a firmware update from Cisco.  There is a better explanation that we'll return to in the next post.

Update As you can see from the actual logs (which can be perused in my post on the Klatt affidavit in Warman v. Lemire), the conclusion reached above is correct and 90sAREover's user agent string was follows: "Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Rogers Hi-Speed Internet; (R1 1.3))". Since this is different from the set-up used by Warman in October and November, the posts were made from different computers.

Browsers, systems, proxies, and Warman

Several commentators have pointed out that the 700,000 possible Rogers' customers with access to 66.185.84.204 doesn't take into account that the Cools posting was made by computer using Windows 98 and the browser Internet Explorer 6.0, and this was the same set-up that "lucy" had used three months later.

To review. 66.185.84.204 was one of 42 Rogers caching proxies, which meant that Rogers' traffic was funneled through these computers. They would save copies of files, and if a file was present in the cache, it would be sent back to the user instead of forwarding the request to the site. If the file was not present in the cache, the proxy would forward the request, but rewrite the packet header so that it now looked like the traffic was coming from the proxy, not the original requester. In the packet would also be details about the configuration of the original computer, including its operating system and browser. (The widget to the right collects these details and reports it back.)

Obviously this consideration decreases the size of the pool considerably, since Windows 98 by that stage was getting long in the tooth. To judge by W3 Schools statistics (h/t to Lance at Catprint.ca), only 12% of users world-wide were using Windows 98; the same site (again h/t Lance) reports (here) that 69.7% of users were using IE 6.0. Taken together these imply something like 10% of all internet traffic used this combination.

Two quibbles. First, this was 2003, when the Canadian dollar was at $0.63, which may have produced a drag on hard-ware replacement and thus meant that Canadians lagged a little in the move away from Windows 98. Second, the browser statistics are browser-use for all Operating Systems, not only those using Windows 98, for whom some of the browser options listed there (Mozilla at 6% and Opera at 2%) were probably little used among the old system. Both these considerations should inch the total upwards slightly. But 10% will not be far wrong.

So, 10% of 700,000 Rogers customers in Ontario is 70,000. (Another quibble that may force this upward -- I'm not sure that I am correct to limit this to Ontario customers. I suspect, but cannot demonstrate, that customers in Quebec, New Brunswick, and Newfoundland were also proxied through these three banks)

(Interesting side-note: these old systems are not completely gone. I had two visits yesterday from users who had the Windows 98 and IE 6.0 combination.)

What to look for when recruiting an expert witness

This is a bad sign:

(For the context, see here and here.)

Why there is room for doubt about Warman and the Cools post

Updated at the bottom.

In Sept. 2003, someone posted an ugly racist slur about Senator Anne Cools at the racist freedomsite.com. The poster's IP was recorded by the site's software and subsequently recovered and released by the site's owner, Marc Lemire: 66.185.84.204, which has the hostname wc09.mtnk.rnc.net.cable.rogers.com.

This very hostname reports important information that has been overlooked. First, the "wc" in wc09.mtnk abbreviates web-cache; this IP is web-cache no. 9 of a bank of caching proxy servers in Newkirk Road, Thorn Hill: mtnk presumably abbreviates Metro Newkirk. (Rogers has an office there.) Web-caching proxies are used by Internet companies such as Rogers to save bandwidth -- copies of web-pages are stored on Rogers' proxy, and only if a page is not stored there does traffic get forwarded to a website; when the traffic gets there, however, the IP that is logged is that of the proxy, not of the original requester.

66.185.84.204 is one of 42 such proxies in the Rogers stable, and they were (are?) arranged in three banks of fourteen at Wolfedale in Mississauga (abbreviated to wlfdle), York Mills in Toronto (ym), and Newkirk in Richmond Hill (mtnk) (see here). Rogers routed all its traffic though these servers -- or at least the traffic of all users who used the proxying function. (An old Rogers FAQ, quoted here, encourages its use, but also gives instructions on how to turn it off if it isn't helping.)

That 66.185.84.204 was a web-caching proxy explains a lot. It explains, for example, why it is so easy to find the IP being used in these months: dozens of different users of this IP can be identified for every month, including September, October, and November 2003, when the Cools post was made. For your own curiosity, take your own IP (it is listed in the widget to the right) and google it. Do you see any traces of your own surfing? I can't find any of mine. Why are hundreds of instances of 66.185.84.204 so easy to find? Because as a proxy it was through-putting hundreds or thousands or tens of thousands of times more traffic than an IP assigned to an individual subscriber.

This also explains why we find such a wide variety of individuals using this IP in the months that the Cools post was made. Some of them could read Vietnamese (here and here), Tamil (here), Korean (here), and Danish (here). And they seem to come from a wide variety of locations in Ontario: London (here and here), Waterloo (here), Aurora (here), and (as we know from the present controversy) Ottawa. Again, this is because it's a proxy -- and these are all areas served by Rogers and so any traffic from them might be routed through one of Roger's proxies.

That 66.185.84.204 is a proxy is clear, as is the fact that there seem to be only 42 such proxies to serve all of Rogers' customers (they're listed here).

So, what does this mean about the Cools poster? One might be tempted to divide 42 proxies into the number of Rogers customers to come up with a notional pool in which the Cools poster resided. It seems clear, however, that individual subscribers are not tied to a single proxy, or indeed to a single bank of proxies. I began this investigation after noting that IPs in the 86.185.84.xxx-range often shift quickly back and forth, a phenomenon that I tried to map out with the string-ball to the left (see here). These shifting IPs, however, are two of the banks of web-caching proxies (the wlfdle and mtnk), and the shifting of proxies is the necessary and natural result of Rogers' effort to achieve "load balancing", by which traffic to and from Rogers servers was rerouted to keep traffic moving efficiently (see here).

Although the proxies can change, they do not seem assigned randomly, either. The IPs associated with the prolific editor of numerous Thomas the Tank Engine articles in wikipedia (here, here, and here) seem to suggest that a Rogers subscriber had a "home" proxy; that he might be shifted by Rogers to another proxy for load-balancing, only soon to return to his proxy "home".

So, how many potential Rogers customers might have made that racist Cools post? Probably all of them. As we have seen (here), the proxies are not geographically limited; each seems to be able to serve all areas of the province.

Now, according to this, Rogers had 800,000 internet subscribers in March 2004. The same link states that 90% of cable subscribers are in Ontario, which implies a pool of about 700,000.
The Cools poster could be almost any one of them.

Update. The best previous attempt to explain this matter from a technical perspective was that of Lance at Catprint in the Mash (his work was copied, pasted, and embraced at FreeDominion). He has now withdrawn his explanation in favour of mine here, and in the comments to this post.

Update 2. This post establishes that 66.185.84.204 was a widely used proxy, thereby removing the circumstantial case against Warman. For a discussion of evidence that shows that it is someone else, see Why Warman is probably innocent and Why Warman is probably innocent, part 2: Rogers Hi-Speed Internet.

Update 3. I have now acquired copies of actual logs that Klatt used in his testimony, and these prove that 90sAREover's computer was different from the one Warman used: see here.

Comments are open, but please see my Comment Policy. Comments that fail those standards will be rejected.

What regions does 66.185.84.204 service?

We have already noted that there seems to be little or no regional bias towards the choice of which proxy a Rogers' subscriber might be sent through. This is graphically portrayed by the game of pick-up-sticks to the right (on which, see here).  Whatever the principles that governed the assignment of an individual IP to a specific proxy, they were not strictly geographical.

Another way to look at this is to consider the web-proxy, wc09.mtnk.rnc.net.cable.rogers.com (66.185.84.204), which is found in the following logs proxying for these IPs:

• on Mon Jun 30 20:11:12 2003, for 24.192.3.174 (of region NEWKIRK-2)
  
• on Mon Jun 30 20:11:14 2003, for 24.192.3.174 (of region NEWKIRK-2)
  
• on Thu Jul 10 23:52:54 2003, for 24.112.104.245 (of region KTCH-1)
  
• on Wed Aug 6 23:20:23 2003, for 24.112.104.245 (of region KTCH-1)
  
• on Tue Oct 14 15:06:57 2003, for 63.139.207.87 (region unknown)
  
• on Sat Oct 18 07:37:43 2003, for 24.157.32.153 (of region HNSN-4)
  
• on Sat Nov 8 16:39:15 2003, for 63.138.242.220 (region unknown)
  
• on Wed Dec 24 22:54:27 2003, for 24.102.22.213 (of region SLNT-2)
  
• on Wed Jan 28 14:02:04 2004, for 24.157.32.245 (of region HNSN-4)
  
• on Wed Feb 4 17:36:59 2004, for 63.138.150.211 (region unknown)
  
• on Fri Mar 19 03:32:12 2004, for 24.112.28.241 (of region WFDL-2)
  
• on Sun Apr 4 23:00:52 2004, for 69.193.20.32 (region unknown)
  
• on Mon Jun 14 15:02:39 2004, for 24.100.196.172 (of region WLFDLE-6)
• on Fri Jan 23 12:54:14 2004, for 24.43.79.38 (of region YM-6)
  
• on Thu Jun 24 08:40:44 2004, for 24.103.46.14 (of region MTWH-4)
  
• on Mon Jul 26 14:51:27 2004, for 69.193.70.185 (region unknown)
  
• on Thu Aug 5 13:30:31 2004, for 24.100.186.219 (of region MTKI-2)
  
• on Mon Aug 9 20:35:36 2004, 24.112.177.222 (of region WFDL-5)
  
• on Wed Aug 11 16:47:49 2004, for 24.157.28.249 (of region WLFDLE-3)
  
• on Thu Aug 26 17:02:48 2004, for 24.103.201.85 (of region BLOOR-1)
  
• on Sun Sep 12 02:33:36 2004, for 24.43.82.252 (of region BRNTFD-8)
  
• on Sun Oct 24 21:22:39 2004, for 24.157.65.211 (of region YM-8)
  
• on Fri Nov 5 14:28:05 2004, for 69.196.255.5 (region unknown)
  
• on Sun Nov 7 08:34:04 2004, for 69.193.47.69 (region unknown)

In short, wc09.wtnk~ seems to have been available province wide.

(Localities for the IPs above are based on this database, which seems to have obsolete data in some (but not all) of the ranges relevant to this matter.)

Load-balancing at Rogers

There is an interesting article in the networking magazine Network Computing of 1999 about Rogers' plans for the next few years. Concerned about the challenges that expected increases in traffic would bring, together with the expectation that e-commerce would become more important (making it all the more important that the traffic got through expeditiously), Rogers developed load-balancing technology to prevent server overload.  Some selected quotes:

Ten Alteon Networks ACEswitch 180 server switches balance the traffic flows among Rogers' five Web and cache servers, as well as its VPN and firewall servers. The Gigabit Ethernet server switches redirect traffic when one server goes down or gets jammed with HTTP traffic. The switches also route and handle the packet filtering for Rogers' firewalls, and all of Rogers' servers are connected via Gigabit Ethernet.

………

When a user requests Internet access, the switch directs that request to a proxy server. "It's load-sharing among cache proxies, and if they all aren't available, it then redirects the traffic to the Internet," rather than to the proxy server, Howell says.

This seems to be the explanation for the shifting IPs that we noticed earlier. Those shifts are taking place among various proxy servers.

Rogers proxy-system not regionally limited?

As I mentioned in the last post, a large quantity of data is available in this set of logs, which include several hundred edits that Rogers web-caching servers had forwarded for its customers in Ontario.

Because this log includes the IPs of both the proxy and the individual user, and because the individual IPs can be identified geographically, it is possible for us to see whether there is a tendency for certain centers to send their traffic through certain proxies.  This is what the map posted above explores.

On the left are the proxies that appear in these logs; at the right the localities of the individual IPs.  The colors of the lines are based on the years.  (I hoped that by including the years, some pattern might become apparent.)  Multiple visits in a year by the same individual IP get only a single arrow.

In the end, no pattern is obvious: it seems that any region can send traffic to any proxy.

Roger proxying for Ottawa customers

In the past weeks, we've been trolling through logs looking at Rogers' IPs.  One especially useful set of logs are found here.  These are an especially useful dataset for two reasons. First is its size, with over 100,000 hits between mid-2002 and mid-2005. Second, unlike most logs, this one reports the http_forwarded_for header (on which see here), which means that when a transparent proxy visits, both the proxy's address and the IP of the original user are reported. As a result, over 200 visits from Rogers proxies can be identified in these logs, including most of the proxies identified here.  

But these logs offer another opportunity.  Because they reveal the individual IP behind the proxy, they give some insight into how Roger distributes these proxies.  Consider this entry, from the logs of July, 2003:

Wed Jul 9 17:28:37 2003|wc09.wlfdle.rnc.net.cable.rogers.com|66.185.84.76| http://www.cs.ualberta.ca/~mburo/|35621:24.114.18.212$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MSOCD; AtHome021SI; AtHome0200; .NET CLR 1.0.3705)

Wed Jul 9 17:32:10 2003|wc09.wlfdle.rnc.net.cable.rogers.com|66.185.84.76| http://www.cs.ualberta.ca/~mburo/ggsa/|14811:24.114.18.212$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MSOCD; AtHome021SI; AtHome0200; .NET CLR 1.0.3705)

In this case the web-cache (wc09.wlfdle.etc, which has the IP address 66.185.84.76) is proxying for 24.114.18.212.  Unfortunately, this is no longer a functioning address.  But for some reason some obsolete data remains in this database, and it reports the name as ON-ROG-1-FLFRD-4.  According to this, FLFRD stands for Fallowfield Road in Ottawa. 

Each of the proxy entries in these logs, then, provides us both with the visitor's unique IP address (hidden behind the proxy address that is forwarding its traffic) and a means by which to estimate a geographical point of origin.   Obviously we're especially interested in visits originating in Ottawa.  (Only about two-thirds of the IPs forwarded by Rogers' proxies are in the database mentioned, which means that the following is suggestive only.)

Here is another pair from the Dec. 2003 log:

Wed Dec 24 22:53:37 2003|wc13.mtnk.rnc.net.cable.rogers.com|66.185.84.208| http://www.cs.ualberta.ca/~mburo/log.html|49683:24.102.22.213$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

Wed Dec 24 22:54:27 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/publications.html|11277:24.102.22.213$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

This pair of visits nicely illustrates the kind of shifting that we've seen before (mapped out in a general way here, though obviously now that we know these are proxies we'll have to re-evaluate the causes of this phenomenon).  In this case, the personal IP yields ON-ROG-21-1SLNT-1, which this identifies as St. Laurent Rd. in Ottawa. 

There are seven other entries in these logs that originate in Ottawa:

1. on Sat Feb 7 13:33:55 2004, 24.42.51.190 (resolving to ON-ROG-6-SLNT-6), proxied by wc09.wlfdle.rnc.net.cable.rogers.com (=66.185.84.76);
   
2. on Fri Jun 4 18:39:08 2004, 24.114.145.64 (resolving to ON-ROG-5-FLFRD-5), proxied by wc09.ym.~ (=66.185.85.76);
   
3. on Mon Aug 30 13:30:11 2004, 24.112.0.122 (resolving to ON-ROG-FLFRD-7), proxied by wc09.wlfdle.~ (=66.185.84.76);
   
4. on Wed Oct 6 10:14:56 2004, 24.156.226.105 (resolving to ON-ROG-18-SLNT-7), proxied by wc09.ym.~ (=66.185.85.76);
   
5. on Sun Jan 9 21:10:26 2005, 24.112.88.50 (resolving to ON-ROG-FLFRD-1), proxied by wc05.wlfdle.~ (=66.185.84.72);
   
6. on Mon Jan 31 13:53:08 2005, 24.103.2.52 (resolving to ON-ROG-13-FLFRD-3), proxied by wc13.ym.~ (=66.185.85.80);
   
7. on Sat Feb 5 22:07:10 2005, 24.156.226.105 (resolving to ON-ROG-18-SLNT-7), proxied by wc01.wlfdle.rnc.~ (=66.185.84.68).

Note especially examples 4 and 7 of this list, in which the IP proxied in October 2004 by wc09-ym~ is proxied by wc01.wlfdle~ in February 2005.  Clearly individual nodes are not each assigned to their own proxy.  (We'll return to this in a future post.)

How these proxies are assigned is unclear.  Nevertheless, it is clear that no geographical region is limited to a single proxy or even to a single bank of proxies.  

• SLNT (St. Laurent) is proxied by York Mills (09.ym), Newkirk (09.mtnk, 13.mtnk), and Wolfedale (01.wlfdle, 09.wlfdle, 13.wlfdle)
• FLFRD (Fallowfield Rd) is proxied by York Mills (09.ym, 13.ym), Newkirk (none), and Wolfedale (05.wldle, 09.wlfdle)

This has important implications that I will discuss soon.

Rogers in Canada

Rogers is one of the largest ISPs in Canada. According to its SEC filing of 2004:

At March 31, 2004, it was providing digital cable services to approximately 563,200 subscribers and Internet service to approximately 828,500 subscribers.

Cable has highly-clustered and technologically advanced broadband networks in Ontario, New Brunswick and Newfoundland. Its Ontario cable systems, which serve approximately 90% of our 2.3 million basic cable subscribers, are concentrated in and around three principal clusters (i) the Greater Toronto area, Canada's largest metropolitan centre; (ii) Ottawa, the capital city of Canada, and (iii) the Guelph to London corridor in southern Ontario. Its New Brunswick and Newfoundland cable systems in Atlantic Canada serve the balance of its subscribers.

How many proxies does Rogers have?

As we have seen, wc09.mtnk.rnc.net.cable.rogers.com (=66.185.84.204), which has recently been at the central of controversy, is one of Rogers' web-caching proxy servers. These proxies are named with the prefix wc~ (for 'web caching'), a number, and a geographical reference.

So far, we have seen 14 proxies with the mtnk designation, and 14 with a wlfdle bank. Given the pattern, it only took a moment to identify a third bank of proxies, again 14 in number, this time with the abbreviation ym. This brings the number of identified proxy servers to 42, which are:

IP	hostname	IP	hostname	IP	hostname
66.185.84.68	wc01.wlfdle.~	66.185.84.196	wc01.mtnk.~	66.185.85.68	wc01.ym.~
66.185.84.69	wc02.wlfdle.~	66.185.84.197	wc02.mtnk.~	66.185.85.69	wc02.ym.~
66.185.84.70	wc03.wlfdle.~	66.185.84.198	wc03.mtnk.~	66.185.85.70	wc03.ym.~
66.185.84.71	wc04.wlfdle.~	66.185.84.199	wc04.mtnk.~	66.185.85.71	wc04.ym.~
66.185.84.72	wc05.wlfdle.~	66.185.84.200	wc05.mtnk.~	66.185.85.72	wc05.ym.~
66.185.84.73	wc06.wlfdle.~	66.185.84.201	wc06.mtnk.~	66.185.85.73	wc06.ym.~
66.185.84.74	wc07.wlfdle.~	66.185.84.202	wc07.mtnk.~	66.185.85.74	wc07.ym.~
66.185.84.75	wc08.wlfdle.~	66.185.84.203	wc08.mtnk.~	66.185.85.75	wc08.ym.~
66.185.84.76	wc09.wlfdle.~	66.185.84.204	wc09.mtnk.~	66.185.85.76	wc09.ym.~
66.185.84.77	wc10.wlfdle.~	66.185.84.205	wc10.mtnk.~	66.185.85.77	wc10.ym.~
66.185.84.78	wc11.wlfdle.~	66.185.84.206	wc11.mtnk.~	66.185.85.78	wc11.ym.~
66.185.84.79	wc12.wlfdle.~	66.185.84.207	wc12.mtnk.~	66.185.85.79	wc12.ym.~
66.185.84.80	wc13.wlfdle.~	66.185.84.208	wc13.mtnk.~	66.185.85.80	wc13.ym.~
66.185.84.81	wc14.wlfdle.~	66.185.84.209	wc14.mtnk.~	66.185.85.81	wc14.ym.~

Update.  According to this, mtnk abbreviates "Newkirk"; wlfdle, "Wolfedale"; and ym, "York Mills".

Update 2. Presumably these are located at the Rogers offices at these addresses:

• 3573 Wolfedale Rd, Mississauga
• 855 York Mills Rd, North York
• 244 Newkirk Road, Richmond Hill

proxy servers 9: what Rogers said about their own proxying

We have seen that 66.185.84.204 = wc09.mtnk.rnc.net.cable.rogers.com was functioning as a proxy in 2003 (here and here and here and here and here and here and here).  What does this mean that it was a proxy?

Rogers itself once explained this in its FAQ.  That FAQ has since disappeared, but it was quoted in full on the site of the Residential Broadbant Users Association (here; an archived version from Sept. 2003 can be seen here).  

The Rogers FAQ defines a proxy and describes how it is used:

The proxy is a local HTTP (web) server internal to a regional/main data center, which caches (stores) frequently requested content. It was originally implemented in order to greatly reduce unnecessary network traffic, particularly backbone traffic leading to the @Home Network in the US. When you want to access a website via your proxy, you send a request to your proxy, which then checks to see if any of the related content is stored locally. If it isn't, the proxy will access the remote server and send back the information you originally requested. The new information is then cached on the proxy for a predetermined period of time.

Note that the Rogers FAQ actually tells us that accessing a site through a proxy will send the proxy to the site.  Any logs at the site will capture the proxy's IP.   This is presumably why so much traffic can be identified for these years (see here and here and here) -- because 66.185.84.204 is forwarding numerous users.  Also note that its job is to act as a web cache — indeed, as I conclude here, this is what the "wc" in wc09.mtnk.rnc.net.cable.rogers.com stands for.

WC stands for...

The investigations that produced the ball of string to the right began when I noticed that some IPs seemed to quickly shift while users were using them, and I collected examples where I could find them.

My list of IPs was incomplete, as is clear from the list below, which lists the IPs together with their unique host-name, which fall into two clear series, which start in red.

IP	host-name
66.185.84.72	wc05.wlfdle.rnc.net.cable.rogers.com
66.185.84.73	wc06.wlfdle.rnc.net.cable.rogers.com
66.185.84.74	wc07.wlfdle.rnc.net.cable.rogers.com
66.185.84.75	wc08.wlfdle.rnc.net.cable.rogers.com
66.185.84.76	wc09.wlfdle.rnc.net.cable.rogers.com
66.185.84.77	wc10.wlfdle.rnc.net.cable.rogers.com
66.185.84.78	wc11.wlfdle.rnc.net.cable.rogers.com
66.185.84.79	wc12.wlfdle.rnc.net.cable.rogers.com
66.185.84.80	wc13.wlfdle.rnc.net.cable.rogers.com
66.185.84.81	wc14.wlfdle.rnc.net.cable.rogers.com
66.185.84.196	wc01.mtnk.rnc.net.cable.rogers.com
66.185.84.197	wc02.mtnk.rnc.net.cable.rogers.com
66.185.84.198	wc03.mtnk.rnc.net.cable.rogers.com
66.185.84.199	wc04.mtnk.rnc.net.cable.rogers.com
66.185.84.200	wc05.mtnk.rnc.net.cable.rogers.com
66.185.84.201	wc06.mtnk.rnc.net.cable.rogers.com
66.185.84.202	wc07.mtnk.rnc.net.cable.rogers.com
66.185.84.203	wc08.mtnk.rnc.net.cable.rogers.com
66.185.84.204	wc09.mtnk.rnc.net.cable.rogers.com
66.185.84.205	wc10.mtnk.rnc.net.cable.rogers.com
66.185.84.206	wc11.mtnk.rnc.net.cable.rogers.com
66.185.84.207	wc12.mtnk.rnc.net.cable.rogers.com
66.185.84.208	wc13.mtnk.rnc.net.cable.rogers.com
66.185.84.209	wc14.mtnk.rnc.net.cable.rogers.com

Clearly, the first series of number should have begun at 66.185.84.68 (= wc01.mtnk.rnc.net.cable.rogers.com).

This, however, ignores the more important question: what does "wc" stand for? "Water closet"? "Winston Churchill"? "Workers' Compensation"?

No. It's quite simple. It clear stands for "web cache", which is what these IPs do.

proxy servers 8: yet another two underlying IPs

In this log entry from November 8, 2003, we see 66.185.84.204 proxying for 63.138.242.220:

Sat Nov 8 16:39:15 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/quotes.html|4400:63.138.242.220$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)

And here (from December, 2003), we see it proxying for 24.102.22.213:

Wed Dec 24 22:54:27 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/publications.html|11277:24.102.22.213$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)

proxy servers 7: still two more underlying IPs

We have seen that in 2003 and 2004, 66.185.84.204 was a Roger proxy server, that forwarded traffic for Rogers customers to a variety of site (see here and here and here). Indeed, we have been able to identify the forwarded IPs of some of these:

• on Oct. 14, 2003, 66.185.84.204 forwarded traffic to a page of funny quotations for 63.139.207.87 (here);
• on Oct. 18, 2003, it forwarded to traffic for 24.157.32.153 to a page for an open-source engine for real-time strategy games (here);
• at sometime in May, 2004, this proxy forwarded traffic for 24.150.44.231 (here);

Here is another excerpt from a visitor log from June 2003:

Mon Jun 30 20:11:12 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/|35416:24.192.3.174$|Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; eisa.com)
Mon Jun 30 20:11:14 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/publications.html|9339:24.192.3.174$|Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; eisa.com)

In this case, 66.185.84.204 is forwarding traffic for 24.192.3.164. And in another log from July of 2003, traffic is forwarded for 24.112.104.245:

Thu Jul 10 23:52:54 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/log.html|46319:24.112.104.245$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

The same IP's traffic was forwarded by the same proxy in August (here):

Wed Aug 6 23:20:23 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/|36325:24.112.104.245$|Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)
Wed Aug 6 23:21:39 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/publications.html|9722:24.112.104.245$|Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)

Proxy servers 6: two more underlying IPs

The IP 66.185.84.204 visited many websites in 2003, as evidenced by the many logs in which it can be found (e.g., here for October of that year).  We have already seen (here and here) that this IP was used as a web proxy by Rogers in 2003 -- indeed it is even possible to identify some of the individual IPs that it forwarded traffic for (here).

An interesting visitors log from October 2003 can be seen here.  In it are three interesting entries:

Thu Oct 9 19:50:52 2003|cpe00c0f0219072-cm.cpe.net.cable.rogers.com|24.157.169.36| http://www.cs.ualberta.ca/~mburo/courses/605.RTS/|834:24.157.169.36$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

Tue Oct 14 15:06:57 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/quotes.html|4202:63.139.207.87$|Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)

Sat Oct 18 07:37:43 2003|wc09.mtnk.rnc.net.cable.rogers.com|66.185.84.204| http://www.cs.ualberta.ca/~mburo/orts/orts.html|243:24.157.32.153$|Mozilla/4.0 (compatible; MSIE 5.15; Mac_PowerPC)

In each of the entries, there are two IPs.  In the first, the two IPs are identical, as is the case when there is no proxying at issue.  (See here.)  In the second and third, our old friend wc09.mtnk.rnc.net.cable.rogers.com = 66.185.84.204 appears, again, as a proxy, now for 63.139.207.87 and 24.157.32.153, respectively.

(That the IP appears as a proxy in October 2003, a month at the center of controversy about the use of this IP in the autumn of 2003 is especially important.)