A diagram of Usenet servers and clients. The blue, green, and red dots on the servers represent the groups they carry. Arrows between servers indicate newsgroup group exchanges (feeds). Arrows between clients and servers indicate that a user is subscribed to a certain group and reads or submits articles.
Usenet is a worldwide distributed Internet discussion system. It was developed from the general purpose UUCP architecture of the same name.
Duke University graduate students Tom Truscott and Jim Ellis conceived the idea in 1979 and it was established in 1980.[1] Users read and post messages (called articles or posts, and collectively termed news) to one or more categories, known as newsgroups. Usenet resembles a bulletin board system (BBS) in many respects, and is the precursor to the various Internet forums that are widely used today. Usenet can be superficially regarded as a hybrid between email and web forums. Discussions are threaded, with modern news reader software, as with web forums and BBSes, though posts are stored on the server sequentially.
One notable difference between a BBS or web forum and Usenet is the absence of a central server and dedicated administrator. Usenet is distributed among a large, constantly changing conglomeration of servers that store and forward messages to one another in so-called news feeds. Individual users may read messages from and post messages to a local server operated by their Internet service provider, university, or employer.
Usenet is one of the oldest computer network communications systems still in widespread use. It was conceived in 1979 and publicly established in 1980 at the University of North Carolina at Chapel Hill and Duke University,[1] over a decade before the World Wide Web was developed and the general public got access to the Internet. It was originally built on the "poor man's ARPANET," employing UUCP as its transport protocol to offer mail and file transfers, as well as announcements through the newly developed news software such as A News. The name Usenet emphasized its creators' hope that the USENIX organization would take an active role in its operation.[2]
The articles that users post to Usenet are organized into topical categories called newsgroups, which are themselves logically organized into hierarchies of subjects. For instance, sci.math and sci.physics are within the sci hierarchy, for science. When a user subscribes to a newsgroup, the news client software keeps track of which articles that user has read.[3]
In most newsgroups, the majority of the articles are responses to some other article. The set of articles which can be traced to one single non-reply article is called a thread. Most modern newsreaders display the articles arranged into threads and subthreads.
When a user posts an article, it is initially only available on that user's news server. Each news server talks to one or more other servers (its "newsfeeds") and exchanges articles with them. In this fashion, the article is copied from server to server and (if all goes well) eventually reaches every server in the network. The later peer-to-peer networks operate on a similar principle; but for Usenet it is normally the sender, rather than the receiver, who initiates transfers. Some have noted that this seems an inefficient protocol in the era of abundant high-speed network access. Usenet was designed for the times when networks were much slower, and not always available. Many sites on the original Usenet network would connect only once or twice a day to batch-transfer messages in and out.[4]
Usenet has significant cultural importance in the networked world, having given rise to, or popularized, many widely recognized concepts and terms such as "FAQ" and "spam".[5]
The format and transmission of Usenet articles is similar to that of Internet e-mail messages. The difference between the two is that Usenet articles can be read by any user whose news server carries the group to which the message was posted, as opposed to email messages which have one or more specific recipients.[6]
Today, Usenet has diminished in importance with respect to Internet forums, blogs and mailing lists. Usenet differs from such media in several ways: Usenet requires no personal registration with the group concerned; information need not be stored on a remote server; archives are always available; and reading the messages requires not a mail or web client, but a news client. The groups in alt.binaries are still widely used for data transfer.
Many Internet service providers, and many other Internet sites, operate news servers for their users to access. ISPs that do not operate their own servers directly will often offer their users an account from another provider that specifically operates newsfeeds. In early news implementations, the server and newsreader were a single program suite, running on the same system. Today, one uses separate newsreader client software, a program that resembles an email client but accesses Usenet servers instead.
Not all ISPs run news servers. A news server is one of the most difficult Internet services to administer well because of the large amount of data involved, small customer base (compared to mainstream Internet services such as email and web access), and a disproportionately high volume of customer support incidents (frequently complaining of missing news articles that are not the ISP's fault). Some ISPs outsource news operation to specialist sites, which will usually appear to a user as though the ISP ran the server itself. Many sites carry a restricted newsfeed, with a limited number of newsgroups. Commonly omitted from such a newsfeed are foreign-language newsgroups and the alt.binaries hierarchy which largely carries software, music, videos and images, and accounts for over 99 percent of article data.
There are also Usenet providers that specialize in offering service to users whose ISPs do not carry news, or that carry a restricted feed.
See also news server operation for an overview of how news systems are implemented.
Newsgroups are typically accessed with special client software that connects to a news server. Newsreader clients are available for all major operating systems.[7] Modern mail clients or "communication suites" commonly also have an integrated newsreader. Often, however, these integrated clients are of low quality, compared to standalone newsreaders, and incorrectly implement Usenet protocols, standards and conventions. Many of these integrated clients, for example the one in Microsoft's Outlook Express, are disliked by purists because of their misbehavior.[8]
With the rise of the World Wide Web (WWW), web front-ends (web2news) have become more common. Web front ends have lowered the technical entry barrier requirements to that of one application and no Usenet NNTP server account. There are numerous websites now offering web based gateways to Usenet groups, although some people have begun filtering messages made by some of the web interfaces for one reason or another.[9][10] Google Groups[11] is one such web based front end and web browsers can access Google Groups via news: protocol links directly.[12]
A minority of newsgroups are moderated, meaning that messages submitted by readers are not distributed directly to Usenet, but instead are emailed to the moderators of the newsgroup for approval. The moderator is to receive submitted articles, review them, and inject approved articles so that they can be properly propagated worldwide. Articles approved by a moderator must bear the Approved: header line. Moderators ensure that the messages that readers see in the newsgroup conform to the charter of the newsgroup, though they are not required to follow any such rules or guidelines.[13] Typically, moderators are appointed in the proposal for the newsgroup, and changes of moderators follow a succession plan.[14]
Historically, a mod.* hierarchy existed before Usenet reorganization.[15] Now, moderated newsgroups may appear in any hierarchy.
Usenet newsgroups in the Big-8 hierarchy are created by proposals called a Request for Discussion, or RFD. The RFD is required to have the following information: newsgroup name, checkgroups file entry, and moderated or unmoderated status. If the group is to be moderated, then at least one moderator with a valid email address must be provided. Other information which is beneficial but not required includes: a charter, a rationale, and a moderation policy if the group is to be moderated.[16] Discussion of the new newsgroup proposal follows, and is finished with the members of the Big-8 Management Board making the decision, by vote, to either approve or disapprove the new newsgroup.
Unmoderated newsgroups form the majority of Usenet newsgroups, and messages submitted by readers for unmoderated newsgroups are immediately propagated for everyone to see. Minimal editorial content filtering vs propagation speed form one crux of the Usenet community. One little cited defense of propagation is canceling a propagated message, but few Usenet users use this command and some news readers do not offer cancellation commands, in part because article storage expires in relatively short order anyway.
Creation of moderated newsgroups often becomes a hot subject of controversy, raising issues regarding censorship and the desire of a subset of users to form an intentional community.
Usenet is a set of protocols for generating, storing and retrieving news "articles" (which resemble Internet mail messages) and for exchanging them among a readership which is potentially widely distributed. These protocols most commonly use a flooding algorithm which propagates copies throughout a network of participating servers. Whenever a message reaches a server, that server forwards the message to all its network neighbors that haven't yet seen the article. Only one copy of a message is stored per server, and each server makes it available on demand to the (typically local) readers able to access that server. The collection of Usenet servers has thus a certain peer-to-peer character in that they share resources by exchanging them, the granularity of exchange however is on a different scale than a modern peer-to-peer system and this characteristic excludes the actual users of the system who connect to the news servers with a typical client-server application, much like an email reader.
RFC 850 was the first formal specification of the messages exchanged by Usenet servers. It was superseded by RFC 1036.
In cases where unsuitable content has been posted, Usenet has support for automated removal of a posting from the whole network by creating a cancel message, although due to a lack of authentication and resultant abuse, this capability is frequently disabled. Copyright holders may still request the manual deletion of infringing material using the provisions of World Intellectual Property Organization treaty implementations, such as the United States Online Copyright Infringement Liability Limitation Act.
On the Internet, Usenet is transported via the Network News Transfer Protocol (NNTP) on TCP Port 119 for standard, unprotected connections and on TCP port 563 for SSL encrypted connections which is offered only by a few sites.
The major set of worldwide newsgroups is contained within nine hierarchies, eight of which are operated under consensual guidelines that govern their administration and naming. The current Big Eight are:
- comp.* – computer-related discussions (comp.software, comp.sys.amiga)
- humanities.* – fine arts, literature, and philosophy (humanities.classics, humanities.design.misc)
- misc.* – miscellaneous topics (misc.education, misc.forsale, misc.kids)
- news.* – discussions and announcements about news (meaning Usenet, not current events) (news.groups, news.admin)
- rec.* – recreation and entertainment (rec.music, rec.arts.movies)
- sci.* – science related discussions (sci.psychology, sci.research)
- soc.* – social discussions (soc.college.org, soc.culture.african)
- talk.* – talk about various controversial topics (talk.religion, talk.politics, talk.origins)
See also the Great Renaming.
The alt.* hierarchy is not subject to the procedures controlling groups in the Big Eight, and it is as a result less organized. Groups in the alt.* hierarchy tend to be more specialized or specific—for example, there might be a newsgroup under the Big Eight which contains discussions about children's books, but a group in the alt hierarchy may be dedicated to one specific author of children's books. Binaries are posted in alt.binaries.*, making it the largest of all the hierarchies.
Many other hierarchies of newsgroups are distributed alongside these. Regional and language-specific hierarchies such as japan.*, malta.* and ne.* serve specific countries and regions such as Japan, Malta and New England. Companies and projects administer their own hierarchies to discuss their products and offer community technical support, such as the historical gnu.* hierarchy from the Free Software Foundation. Microsoft closed its newsserver in June 2010, providing support for its products over forums now.[17] Some users prefer to use the term "Usenet" to refer only to the Big Eight hierarchies; others include alt as well. The more general term "netnews" incorporates the entire medium, including private organizational news systems.
Informal sub-hierarchy conventions also exist. *.answers are typically moderated cross-post groups for FAQs. An FAQ would be posted within one group and a cross post to the *.answers group at the head of the hierarchy seen by some as a refining of information in that news group. Some subgroups are recursive—to the point of some silliness in alt.*.
A visual example of the many complex steps required to prepare data to be uploaded to Usenet newsgroups. These steps must be done again in reverse to download data from Usenet.
Usenet was originally created to distribute text content encoded in the 7-bit ASCII character set. With the help of programs that encode 8-bit values into ASCII, it became practical to distribute binary files as content. Binary posts, due to their size and often-dubious copyright status, were in time restricted to specific newsgroups, making it easier for administrators to allow or disallow the traffic.
The oldest widely used encoding method for binary content is uuencode, from the Unix UUCP package. In the late 1980s, Usenet articles were often limited to 60,000 characters, and larger hard limits exist today. Files are therefore commonly split into sections that require reassembly by the reader.
With the header extensions and the Base64 and Quoted-Printable MIME encodings, there was a new generation of binary transport. In practice, MIME has seen increased adoption in text messages, but it is avoided for most binary attachments. Some operating systems with metadata attached to files use specialized encoding formats. For Mac OS, both Binhex and special MIME types are used.
Other lesser known encoding systems that may have been used at one time were BTOA, XX encoding, BOO, and USR encoding.
In an attempt to reduce file transfer times, an informal file encoding known as yEnc was introduced in 2001. It achieves about a 30% reduction in data transferred by assuming that most 8-bit characters can safely be transferred across the network without first encoding into the 7-bit ASCII space.
The standard method of uploading binary content to Usenet is to first archive the files into RAR archives (for large files usually in 15 MB, 50 MB or 100 MB parts) then create Parchive files. Parity files are used to recreate missing data. This is needed often, as not every part of the files reaches a server. These are all then encoded into yEnc and uploaded to the selected binary groups.
This is a list of some of the biggest binary groups. With 1000+ days retention, the (binary) Usenet storage (which binsearch.info indexes) is more than 7
petabytes (7000
terabytes).
[18]
Each newsgroup is generally allocated a certain amount of storage space for post content. When this storage has been filled, each time a new post arrives, old posts are deleted to make room for the new content. If the network bandwidth available to a server is high but the storage allocation is small, it is possible for a huge flood of incoming content to overflow the allocation and push out everything that was in the group before it. If the flood is large enough, the beginning of the flood will begin to be deleted even before the last part of the flood has been posted.
Binary newsgroups are only able to function reliably if there is sufficient storage allocated to a group to allow readers enough time to download all parts of a binary posting before it is flushed out of the group's storage allocation. This was at one time how posting of undesired content was countered; the newsgroup would be flooded with random garbage data posts, of sufficient quantity to push out all the content to be suppressed. This has been compensated by service providers allocating enough storage to retain everything posted each day, including such spam floods, without deleting anything.
The average length of time that posts are able to stay in the group before being deleted is commonly called the retention time. Generally the larger Usenet servers have enough capacity to archive several weeks of binary content even when flooded with new data at the maximum daily speed available. A good binaries service provider must not only accommodate users of fast connections (3 megabit) but also users of slow connections (256 kilobit or less) who need more time to download content over a period of several days or weeks.
Major NSPs do not delete any articles less than a year old, resulting in a retention time of more than 1287 days.[19] This results in more than 9 petabytes (9000 terabytes) of storage. [20]
In part because of such long retention times, as well as growing Internet upload speeds, Usenet is also used by individual users to store backup data in a practice called Usenet backup, or uBackup.[21] While commercial providers offer more easy to use online backup services, storing data on Usenet is free of charge (although access to Usenet itself may not be). The method requires the user to manually select, prepare and upload the data. Because anyone can potentially download the backup files, the data is typically encrypted. After the files are uploaded, the uploader does not have any control over them; the files are automatically copied to all Usenet providers, so there will be multiple copies of it spread over different geographical locations around the world.
While binary newsgroups can be used to distribute completely legal user-created works, open-source software, and public domain material, some binary groups are used to illegally distribute commercial software, copyrighted media, and obscene material.
ISP-operated Usenet servers frequently block access to all alt.binaries.* groups to both reduce network traffic and to avoid related legal issues. Commercial Usenet service providers claim to operate as a telecommunications service, and assert that they are not responsible for the user-posted binary content transferred via their equipment. In the United States, Usenet providers can qualify for protection under the DMCA Safe Harbor regulations, provided that they establish a mechanism to comply with and respond to takedown notices from copyright holders.[22]
Removal of copyrighted content from the entire Usenet network is a nearly impossible task, due to the rapid propagation between servers and the retention done by each server. Petitioning a Usenet provider for removal only removes it from that one server's retention cache, but not any others. It is possible for a special post cancellation message to be distributed to remove it from all servers, but many providers ignore cancel messages by standard policy, because they can be easily falsified and submitted by anyone.[23][24] For a takedown petition to be most effective across the whole network, it would have to be issued to the origin server to which the content has been posted, before it has been propagated to other servers. Removal of the content at this early stage would prevent further propagation, but with modern high speed links, content can be propagated as fast as it arrives, allowing no time for content review and takedown issuance by copyright holders.[25]
Establishing the identity of the person posting illegal content is equally difficult due to the trust-based design of the network. Like SMTP email, servers generally assume the header and origin information in a post is true and accurate. However, as in SMTP email, Usenet post headers are easily falsified so as to obscure the true identity and location of the message source.[26] In this manner, Usenet is significantly different from modern P2P services; most P2P users distributing content are typically immediately identifiable to all other users by their network address, but the origin information for a Usenet posting can be completely obscured and unobtainable once it has propagated past the original server.[27]
Also unlike modern P2P services, the identity of the downloaders is hidden from view. On P2P services a downloader is identifiable to all others by their network address. On Usenet, the downloader connects directly to a server, and only the server knows the address of who is connecting to it. Some Usenet providers do keep usage logs, but not all make this logged information casually available to outside parties such as the RIAA.[28][29]
UUCP/Usenet Logical Map — June 1, 1981 / mods by S. McGeady November 19, 1981
(ucbvax)
+=+===================================+==+
| | | |
| | wivax | |
| | | | |
| | microsoft| uiucdcs | |
| | genradbo | | | | | | (Tektronix)
| | | | | | | purdue | |
| decvax+===+=+====+=+=+ | | | |
| | | | | | | pur-phy | | tekmdp
| | | | | | | | | | |
+@@@@@@cca | | | | | | | | |
| | | | +=pur-ee=+=+=====+===+ | |
| csin | | | | | |
| | +==o===+===================+==+========+=======+====teklabs=+
| | | |
| | | pdp phs grumpy wolfvax |
| | | | | | | |
| | cincy unc=+===+======+========+ |
| | | bio | |
| | | (Misc) | | (Misc) |
| | | sii reed | dukgeri duke34 utzoo |
| | | | | | | | | |
| +====+=+=+==+====++======+==++===duke=+===+=======+==+=========+ |
| | | | | | | | | | u1100s
| bmd70 ucf-cs ucf | andiron | | | | |
| | | | | | |
| red | | | | | pyuxh
| | | | zeppo | | | |
| psupdp---psuvax | | | | | | |
| | | | alice | whuxlb | utah-cs | | houxf
| allegra | | | | | | | | | |
| | | | | | | | | +--chico---+
| +===+=mhtsa====research | /=+=======harpo=+==+ | |
| | | | | | / | | |
| hocsr | | +=+=============+=/ cbosg---+ | |
| ucbopt | | | | | esquire |
| : | | | cbosgd | |
| : | | | | |
| ucbcory | | eagle==+=====+=====+=====+=====+ | |
| : | | | | | | | | | +-uwvax--+
| : | | | mhuxa mhuxh mhuxj mhuxm mhuxv | |
| : | | | | |
| : | | | +----------------------------o--+
| : | | | | |
| ucbcad | | | ihpss mh135a |
| : | | | | | |
| : \--o--o------ihnss----vax135----cornell |
| : | | | | |
+=+==ucbvax==========+===+==+=+======+=======+=+========+=========+
(UCB) : | | | | (Silicon Valley)
ucbarpa cmevax | | menlo70--hao
: | | | |
ucbonyx | | | sri-unix
| ucsfcgl |
| | |
Legend: | | sytek====+========+
------- | | | |
- | / \ + = Uucp sdcsvax=+=======+=+======+ intelqa zehntel
= "Bus" | | |
o jumps sdcarl phonlab sdcattb
: Berknet
@ Arpanet
UUCP/Usenet Logical Map, original by Steven McGeady. Copyright© 1981, 1996
Bruce Jones, Henry Spencer, David Wiseman. Copied with permission from
The Usenet Oldnews Archive: Compilation.[30]
Newsgroup experiments first occurred in 1979. Tom Truscott and Jim Ellis of Duke University came up with the idea as a replacement for a local announcement program, and established a link with nearby University of North Carolina using Bourne shell scripts written by Steve Bellovin. The public release of news was in the form of conventional compiled software, written by Steve Daniel and Truscott.[31] In 1980, Usenet was connected to ARPANET through UC Berkeley which had connections to both Usenet and ARPANET. Michael Horton, the graduate student that set up the connection, began “feeding mailing lists from the ARPANET into Usenet” with the “fa” identifier. As a result, the number of people on Usenet increased dramatically; however, it was still a while longer before Usenet users could contribute to ARPANET.[32] After 32 years, the Usenet news service link at the University of North Carolina at Chapel Hill (news.unc.edu) was finally retired on February 4, 2011.
UUCP networks spread quickly due to the lower costs involved, and the ability to use existing leased lines, X.25 links or even ARPANET connections. By 1983, the number of UUCP hosts had grown to 550, nearly doubling to 940 in 1984.[33]
As the mesh of UUCP hosts rapidly expanded, it became desirable to distinguish the Usenet subset from the overall network. A vote was taken at the 1982 USENIX conference to choose a new name. The name Usenet was retained, but it was established that it only applied to news.[34] The name UUCPNET became the common name for the overall network.
In addition to UUCP, early Usenet traffic was also exchanged with Fidonet and other dial-up BBS networks. Widespread use of Usenet by the BBS community was facilitated by the introduction of UUCP feeds made possible by MS-DOS implementations of UUCP such as UFGATE (UUCP to FidoNet Gateway), FSUUCP and UUPC. The Network News Transfer Protocol, or NNTP, was introduced in 1985 to distribute Usenet articles over TCP/IP as a more flexible alternative to informal Internet transfers of UUCP traffic. Since the Internet boom of the 1990s, almost all Usenet distribution is over NNTP.[35]
Early versions of Usenet used Duke's A News software. At Berkeley an improved version called B News was produced by Matt Glickman and Mark Horton. With a message format that offered compatibility with Internet mail and improved performance, it became the dominant server software. C News, developed by Geoff Collyer and Henry Spencer at the University of Toronto, was comparable to B News in features but offered considerably faster processing. In the early 1990s, InterNetNews by Rich Salz was developed to take advantage of the continuous message flow made possible by NNTP versus the batched store-and-forward design of UUCP. Since that time INN development has continued, and other news server software has also been developed.[36]
Usenet was the initial Internet community and the place for many of the most important public developments in the commercial Internet. It was the place where Tim Berners-Lee announced the launch of the World Wide Web,[37] where Linus Torvalds announced the Linux project,[38] and where Marc Andreessen announced the creation of the Mosaic browser and the introduction of the image tag,[39] which revolutionized the World Wide Web by turning it into a graphical medium.
Many jargon terms now in common use on the Internet originated or were popularized on Usenet.[40] Likewise, many conflicts which later spread to the rest of the Internet, such as the ongoing difficulties over spamming, began on Usenet.[41]
"Usenet is like a herd of performing elephants with diarrhea. Massive, difficult to redirect, awe-inspiring, entertaining, and a source of mind-boggling amounts of excrement when you least expect it."
Sascha Segan of PC Magazine said in 2008 "Usenet has been dying for years[...]" Segan said that some people pointed to the Eternal September in 1993 as the beginning of Usenet's decline. Segan said that the "eye candy" on the World Wide Web and the marketing funds spent by owners of websites convinced Internet users to use profit-making websites instead of Usenet servers. In addition, DejaNews and Google Groups made conversations searchable, and Segan said that this removed the obscurity of previously obscure Internet groups on Usenet. Segan explained that when pornographers and software pirates began putting large files on Usenet, by the late 1990s this caused Usenet disk space and traffic to increase. Internet service providers allocated space to Usenet libraries, and Internet service providers questioned why they needed to host space for pornography and pirated software. Segan said that the hosting of porn and pirated software was "likely when Usenet became truly doomed" and "[i]t's the porn that's putting nails in Usenet's coffin." AOL discontinued Usenet access in 2005. When the State of New York opened an investigation on child pornographers who used Usenet, many ISPs dropped all Usenet access or access to the alt. hierarchy. Segan concluded "It's hard to completely kill off something as totally decentralized as Usenet; as long as two servers agree to share the NNTP protocol, it'll continue on in some fashion. But the Usenet I mourn is long gone[...]"[42] In response, John Biggs of TechCrunch said "Is Usenet dead, as Sascha posits? I don’t think so. As long as there are folks who think a command line is better than a mouse, the original text-only social network will live on." Biggs added that while many Internet service providers terminated access, "the real pros know where to go to get their angst-filled, nit-picking, obsessive fix."[43]
In May 2010, Duke University, whose implementation had kicked off Usenet more than 30 years earlier, decommissioned its Usenet server, citing low usage and rising costs.[44][45]
Over time, the amount of Usenet traffic has steadily increased. As of 2010[update] the number of all text posts made in all Big-8 newsgroups averaged 1,800 new messages every hour, with an average of 25,000 messages per day.[46] However, these averages are minuscule in comparison to the traffic in the binary groups.[47] Much of this traffic increase reflects not an increase in discrete users or newsgroup discussions, but instead the combination of massive automated spamming and an increase in the use of .binaries newsgroups[46] in which large files are often posted publicly. A small sampling of the change (measured in feed size per day) follows:
| Daily Volume |
Date |
Source |
| 4.5 GB |
1996-12 |
Altopia.com |
| 9 GB |
1997-07 |
Altopia.com |
| 12 GB |
1998-01 |
Altopia.com |
| 26 GB |
1999-01 |
Altopia.com |
| 82 GB |
2000-01 |
Altopia.com |
| 181 GB |
2001-01 |
Altopia.com |
| 257 GB |
2002-01 |
Altopia.com |
| 492 GB |
2003-01 |
Altopia.com |
| 969 GB |
2004-01 |
Altopia.com |
| 1.30 TB |
2004-09-30 |
Octanews.net |
| 1.38 TB |
2004-12-31 |
Octanews.net |
| 1.52 TB |
2005-01 |
Altopia.com |
| 1.34 TB |
2005-01-01 |
Octanews.net |
| 1.30 TB |
2005-01-01 |
Newsreader.com |
| 1.81 TB |
2005-02-28 |
Octanews.net |
| 1.87 TB |
2005-03-08 |
Newsreader.com |
| 2.00 TB |
2005-03-11 |
Various sources |
| 2.27 TB |
2006-01 |
Altopia.com |
| 2.95 TB |
2007-01 |
Altopia.com |
| 3.07 TB |
2008-01 |
Altopia.com |
| 3.80 TB |
2008-04-16 |
Newsdemon.com |
| 4.60 TB |
2008-11-01 |
Giganews.com |
| 4.65 TB |
2009-01 |
Altopia.com |
| 6.00 TB |
2009-12 |
Newsdemon.com |
| 5.42 TB |
2010-01 |
Altopia.com |
| 8.00 TB |
2010-09 |
Newsdemon.com |
| 7.52 TB |
2011-01 |
Altopia.com |
| 8.25 TB |
2011-10 |
Thecubenet.com |
| 9.29 TB |
2012-01 |
Altopia.com |
In 2008, Verizon Communications, Time Warner Cable and Sprint Nextel signed an agreement with Attorney General of New York Andrew Cuomo to shut down access to sources of child pornography.[48] Time Warner Cable stopped offering access to Usenet. Verizon reduced its access to the "Big 8" hierarchies. Sprint stopped access to the alt.* hierarchies. AT&T stopped access to the alt.binaries.* hierarchies. Cuomo never specifically named Usenet in his anti-child pornography campaign. David DeJean of PC World said that some worry that the ISPs used Cuomo's campaign as an excuse to end portions of Usenet access, as it is costly for the Internet service providers. In 2008 AOL, which no longer offered Usenet access, and the four providers that responded to the Cuomo campaign were the five largest Internet service providers in the United States; they had more than 50% of the U.S. ISP marketshare.[49] On June 8, 2009, AT&T announced that it would no longer provide access to the Usenet service as of July 15, 2009.[50]
AOL announced that it would discontinue its integrated Usenet service in early 2005, citing the growing popularity of weblogs, chat forums and on-line conferencing.[51] The AOL community had a tremendous role in popularizing Usenet some 11 years earlier.[52]
In August, 2009, Verizon announced that it would discontinue access to Usenet on September 30, 2009.[53][54] JANET(UK) announced it will discontinue Usenet service, effective July 31, 2010, citing Google Groups as an alternative.[55] Microsoft announced that it would discontinue support for its public newsgroups (msnews.microsoft.com) from June 1, 2010, offering web forums as an alternative.[56]
Primary reasons cited for the discontinuance of Usenet service by general ISPs include the decline in volume of actual readers due to competition from blogs, along with cost and liability concerns of increasing proportion of traffic devoted to file-sharing and spam on unused or discontinued groups.[57][58]
Some ISPs did not include pressure from Attorney General of New York Andrew Cuomo's aggressive campaign against child pornography as one of their reasons for dropping Usenet feeds as part of their services.[59] ISPs Cox and Atlantic Communications resisted the 2008 trend but both did eventually drop their respective Usenet feeds in 2010.[60][61][62]
When announcing their Usenet shut down, some of the ISPs claimed that the reason for the shut down was that discussion traffic has shifted away from ISPs toward dedicated Usenet servers accessible via newsreader or via the Web in forums as well as sites that host Usenet newsgroups geared to specific topics via web interface, such as compgroups.net for the comp. groups hierarchy. Those claims conflict with the traffic data analysis reported for that time period showing that traffic in Usenet steadily increased in both number of messages and volume of data.[46]
Public archives of Usenet articles have existed since the early days of Usenet, such as the system created by Kenneth Almquist in late 1982.[63] Distributed archiving of Usenet posts was suggested in November 1982 by Scott Orshan, who proposed that "Every site should keep all the articles it posted, forever."[64] Also in November of that year, Rick Adams responded to a post asking "Has anyone archived netnews, or does anyone plan to?"[65] by stating that he was, "afraid to admit it, but I started archiving most 'useful' newsgroups as of September 18."[66] In June 1982, Gregory G. Woodbury proposed an "automatic access to archives" system that consisted of "automatic answering of fixed-format messages to a special mail recipient on specified machines." [67]
In 1985, two news archiving systems and one RFC were posted to the Internet. The first system, called keepnews, by Mark M. Swenson of The University of Arizona, was described as "a program that attempts to provide a sane way of extracting and keeping information that comes over Usenet." The main advantage of this system was to allow users to mark articles as worthwhile to retain.[68] The second system, YA News Archiver by Chuq Von Rospach, was similar to keepnews, but was "designed to work with much larger archives where the wonderful quadratic search time feature of the Unix ... becomes a real problem."[69] The same Chuq Von Rospach in early 1985 posted a detailed RFC for "archiving and accessing usenet articles with keyword lookup." This RFC described a program that could "generate and maintain an archive of Usenet articles and allow looking up articles based on the article-id, subject lines, or keywords pulled out of the article itself." Also included was C code for the internal data structure of the system.[70]
The desire to have a fulltext search index of archived news articles is not new either, one such request having been made in April 1991 by Alex Martelli who sought to "build some sort of keyword index for [the news archive]."[71] In early May, Mr. Martelli posted a summary of his responses to Usenet, noting that the "most popular suggestion award must definitely go to 'lq-text' package, by Liam Quin, recently posted in alt.sources."[72]
Today, the archiving of Usenet has led to a fear of loss of privacy.[73] An archive simplifies ways to profile people. This has partly been countered with the introduction of the X-No-Archive: Yes header, which is itself controversial.[74]
Main article:
Google GroupsWeb-based archiving of Usenet posts began in 1995 at Deja News with a very large, searchable database. In 2001, this database was acquired by Google.[75]
Google Groups hosts an archive of Usenet posts dating back to May 1981. The earliest posts, which date from May 1981 to June 1991, were donated to Google by the University of Western Ontario with the help of David Wiseman and others,[76] and were originally archived by Henry Spencer at the University of Toronto's Zoology department.[77] The archives for late 1991 through early 1995 were provided by Kent Landfield from the NetNews CD series[78] and Jürgen Christoffel from GMD.[79] The archive of posts from March 1995 onward was originally started by the company DejaNews (later Deja), which was purchased by Google in February 2001. Google began archiving Usenet posts for itself starting in the second week of August 2000. Already during the DejaNews era the archive had become a popular constant in Usenet culture, and remains so today.
|
|
Usenet as a whole has no administrators; each server administrator is free to do whatever pleases him or her as long as the end users and peer servers tolerate and accept it. Nevertheless, there are a few famous administrators:
|
- ^ a b From Usenet to CoWebs: interacting with social information spaces, Christopher Lueg, Danyel Fisher, Springer (2003), ISBN 1-85233-532-7, ISBN 978-1-85233-532-8
- ^ Stephen Daniel, James Ellis, and Tom Truscott (1980). USENET - A General Access UNIX Network. (inside archive as usenet/uprop.n)
- ^ Lehnert, Wendy G.; Kopec, Richard (2007). Web 101. Addison Wesley. p. 291.
- ^ Store And Forward Communication: UUCP and FidoNet. Carnegie Mellon School of Computer Science.
- ^ "USENET Newsgroup Terms - SPAM". http://www.newsdemon.com/usenet_term_spam.php.
- ^ Kozierok, Charles M. (2005). The TCP/IP guide: a comprehensive, illustrated Internet protocols reference. No Starch Press. p. 1401.
- ^ "Open Directory Usenet Clients". Dmoz.org. October 9, 2008. http://www.dmoz.org/Computers/Software/Internet/Clients/Usenet/. Retrieved December 14, 2010.
- ^ Jain, Dominik (July 30, 2006). "OE-QuoteFix Description". http://home.in.tum.de/~jain/software/oe-quotefix/. Retrieved June 4, 2007.
- ^ "Improve-Usenet". October 13, 2008. http://improve-usenet.org.
- ^ "Improve-Usenet Comments". October 13, 2008. Archived from the original on April 26, 2008. http://web.archive.org/web/20080426174022/http://improve-usenet.org/voices1.html. Retrieved June 29, 2009.
- ^ "Google Groups". Groups.google.com. http://groups.google.com/. Retrieved December 14, 2010.
- ^ "News: links to Google Groups". http://mykzilla.blogspot.com/2008/11/news-links-to-google-groups.html.
- ^ "Who can force the moderators to obey the group charter?". Big-8.org. http://www.big-8.org/wiki/Moderated_Newsgroups#Who_can_force_the_moderators_to_obey_the_group_charter.3F. Retrieved December 14, 2010.
- ^ "How does a group change moderators?". Big-8.org. http://www.big-8.org/wiki/Moderated_Newsgroups#How_does_a_group_change_moderators.3F. Retrieved December 14, 2010.
- ^ "Early Usenet Newsgroup Hierarchies". Livinginternet.com. October 25, 1990. http://www.livinginternet.com/u/ui_early.htm. Retrieved December 14, 2010.
- ^ "How to Create a New Big-8 Newsgroup". Big-8.org. July 7, 2010. http://www.big-8.org/wiki/How_to_Create_a_New_Big-8_Newsgroup. Retrieved December 14, 2010.
- ^ Microsoft Responds to the Evolution of Communities, Announcement, undated. Microsoft hitting 'unsubscribe' on newsgroups, CNET, May 4, 2010.
- ^ "Usenet storage is more than 7 petabytes (7000 terabytes)". binsearch.info. https://www.binsearch.info/groupinfo.php. Retrieved February 14, 2012.
- ^ "Giganews FAQ - How long are articles available?". Giganews.com. http://www.giganews.com/faq.html#q0.4. Retrieved February 13, 2012.
- ^ "9 petabyte of usenet storage on giganews.com". giganews.com. http://www.giganews.com/blog/2011/05/announcing-1000-days-retention-prize.html. Retrieved February 14, 2012.
- ^ "usenet backup (uBackup)". Wikihow.com. http://www.wikihow.com/Backup-Your-Data-on-Usenet-%28Ubackup%29. Retrieved February 14, 2012.
- ^ The SuperNews DMCA notifications page shows a typical example of Usenet provider DMCA takedown compliance. http://www.supernews.com/docs/dmca.html
- ^ "Cancel Messages FAQ". Archived from the original on December 12, 2007. http://web.archive.org/web/20071212175002/http://www.killfile.org/faqs/cancel.html. Retrieved June 29, 2009. "...Until authenticated cancels catch on, there are no options to avoid forged cancels and allow unforged ones..."
- ^ Microsoft knowledgebase article stating that many servers ignore cancel messages Support.microsoft.com
- ^ "Microsoft Word - Surmacz.doc" (PDF). http://www.measurement.sk/2005/S1/Surmacz.pdf. Retrieved December 14, 2010.
- ^ ...every part of a Usenet post may be forged apart from the left most portion of the "Path:" header... By-users.co.uk
- ^ "Better living through forgery". http://groups.google.com/group/news.admin.misc/msg/9f6aa173f228ac6a?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Giganews Privacy Policy". Giganews.com. http://www.giganews.com/legal/privacy.html. Retrieved December 14, 2010.
- ^ "Logging Policy". Aioe.org. June 9, 2005. http://aioe.org/index.php?logging-policy. Retrieved December 14, 2010.
- ^ "Quux.org". http://quux.org:70/Archives/usenet-a-news/NET.general/81.12.14_azure.300_net.general,fa.unix-wizards.txt. Retrieved December 14, 2010.
- ^ LaQuey, Tracy (1990). The User's directory of computer networks. Digital Press. p. 386.
- ^ Hauben, Michael and Hauben, Rhonda. “Netizens: On the History and Impact of Usenet and the Internet, On the Early Days of Usenet: The Roots of the Cooperative Online Culture”. First Monday vol. 3 num.August 8, 3 1998
- ^ Haddadi, H. (2006). "Network Traffic Inference Using Sampled Statistics." University College London.
- ^ Horton, Mark (December 11, 1990). "Arachnet". http://communication.ucsd.edu/bjones/Usenet.Hist/Nethist/0111.html. Retrieved June 4, 2007.
- ^ Huston, Geoff (1999). ISP survival guide: strategies for running a competitive ISP. Wiley. p. 439.
- ^ "Unix/Linux news servers". Newsreaders.com. http://www.newsreaders.com/unix/servers.html. Retrieved December 14, 2010.
- ^ Tim Berners-Lee (August 6, 1991). "WorldWideWeb: Summary". alt.hypertext. (Web link). Retrieved June 4, 2007.
- ^ Torvalds, Linus. "What would you like to see most in minix?". comp.os.minix. (Web link). Retrieved September 9, 2006.
- ^ Marc Andreessen (March 15, 1993). "NCSA Mosaic for X 0.10 available.". comp.infosystems.wais, comp.infosystems, alt.hypertext, comp.windows.x comp.infosystems.gopher, comp.infosystems.wais, comp.infosystems, alt.hypertext, comp.windows.x. (Web link). Retrieved June 4, 2007.
- ^ Kaltenbach, Susan (2000-12). "The Evolution of the Online Discourse Community". http://noonuniverse.com/Linked_work/online_discourse.pdf. ""Verb Doubling: Doubling a verb may change its semantics, Soundalike Slang: Punning jargon, The -P convention: A LISPy way to form questions, Overgeneralization: Standard abuses of grammar, Spoken Inarticulations: Sighing and <*sigh*&rt;ing, Anthropomorphization: online components were named "Homunculi," daemons," etc., and there were also "confused" programs. Comparatives: Standard comparatives for design quality""
- ^ Campbell, K.K. (October 1, 1994). "Chatting With Martha Siegel of the Internet's Infamous Canter & Siegel". Electronic Frontier Foundation. Archived from the original on November 25, 2007. http://web.archive.org/web/20071125201904/http://w2.eff.org/legal/cases/Canter_Siegel/c-and-s_summary.article. Retrieved September 24, 2010.
- ^ Sascha Segan (July 31, 2008). "R.I.P Usenet: 1980-2008". PC Magazine: p. 2. http://www.pcmag.com/article2/0,2817,2326849,00.asp. Retrieved May 8, 2011.
- ^ "Reports of Usenet's Death Are Greatly Exaggerated." TechCrunch. August 1, 2008. Retrieved on May 8, 2011.
- ^ Cara Bonnett (May 17, 2010). "A Piece of Internet History". Duke Today. http://news.duke.edu/2010/05/usenet.html. Retrieved May 24, 2010.
- ^ Andrew Orlowski (May 20, 2010). "Usenet's home shuts down today". The Register. http://www.theregister.co.uk/2010/05/20/usenet_duke_server/. Retrieved May 24, 2010.
- ^ a b c "Top 100 text newsgroups by postings". NewsAdmin. http://www.newsadmin.com/top100tmsgs.asp. Retrieved December 14, 2010.
- ^ "Top 100 binary newsgroups by postings". NewsAdmin. http://www.newsadmin.com/top100bmsgs.asp. Retrieved December 14, 2010.
- ^ Rosencrance, Lisa. "3 top ISPs to block access to sources of child porn." Computer World. June 8, 2008. Retrieved on April 30, 2009.
- ^ DeJean, David. "Usenet: Not Dead Yet." PC World. Tuesday October 7, 2008. 2. Retrieved on April 30, 2009.
- ^ "ATT Announces Discontinuation of USENET Newsgroup Services". NewsDemon. June 9, 2009. http://www.newsdemon.com/blog/2009/06/09/att-announces-discontinuation-of-usenet-newsgroup-services/. Retrieved June 18, 2009.
- ^ Hu, Jim. "AOL shutting down newsgroups." CNet. January 25, 2005. Retrieved on May 1, 2009.
- ^ "AOL Pulls Plug on Newsgroup Service". Betanews.com. http://www.betanews.com/article/AOL-Pulls-Plug-on-Newsgroup-Service/1106664611. Retrieved December 14, 2010.
- ^ Bode, Karl. "Verizon To Discontinue Newsgroups September 30". DSLReports. August 31, 2009. Retrieved on October 24, 2009.
- ^ "Verizon Newsgroup Service Has Been Discontinued" Verizon Central Support. Retrieved on October 24, 2009.
- ^ Ukerna.ac.uk
- ^ "Microsoft Responds to the Evolution of Communities". microsoft.com. http://www.microsoft.com/communities/newsgroups/default.mspx#EV. Retrieved September 1, 2011.
- ^ "AOL shutting down newsgroups". cnet.com/. January 25, 2005. http://news.cnet.com/AOL-shutting-down-newsgroups/2100-1032_3-5550036.html. Retrieved September 1, 2011.
- ^ "Verizon To Discontinue Newsgroups". dslreports.com. August 31, 2009. http://www.dslreports.com/shownews/Verizon-To-Discontinue-Newsgroups-September-30-104227. Retrieved September 1, 2011.
- ^ "The Comcast Newsgroups Service Discontinued". dslreports.com. dslreports.com. http://www.dslreports.com/forum/r21120658-The-Comcast-Newsgroups-Service-Discontinued. Retrieved September 1, 2011.
- ^ "Cox to Drop Free Usenet Service June 30th". Zeropaid.com. April 22, 2010. http://www.zeropaid.com/news/88729/cox-to-drop-free-usenet-service-june-30th/. Retrieved September 3, 2011.
- ^ "Cox Discontinues Usenet, Starting In June". Geeknet, Inc. April 21, 2010. http://news.slashdot.org/story/10/04/21/210224/Cox-Discontinues-Usenet-Starting-In-June. Retrieved September 1, 2011.
- ^ "Cox Communications and Atlantic Broadband Discontinue Usenet Access". thundernews.com. April 27, 2010. http://www.thundernews.com/blog/tag/isp/. Retrieved September 1, 2011.
- ^ "How to obtain back news items". http://groups.google.com/group/net.general/msg/ba3ad51d0cedd76b?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Distributed archiving of netnews". http://groups.google.com/group/net.news/msg/e8145eace8aa0529?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Archive of netnews". http://groups.google.com/group/net.news/msg/a9ad42b15688b910?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Re: Archive of netnews". http://groups.google.com/group/net.news/msg/87c9160730b36989?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Automatic access to archives". http://groups.google.com/group/net.general/msg/a377e12ab1a829ff?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "keepnews -- A Usenet news archival system". http://groups.google.com/group/net.sources/msg/737d29fd3af2d5ae?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "YA News Archiver". http://groups.google.com/group/net.sources/msg/d5c0cec804f745dd?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "RFC usenet article archive program with keyword lookup". http://groups.google.com/group/net.news/msg/f26190057834ed87?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Looking for fulltext indexing software for archived news". http://groups.google.com/group/news.software.nn/msg/16381a7e3fd611d4?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ "Summary: search for fulltext indexing software for archived news". http://groups.google.com/group/news.software.nn/msg/f503efa9034f1c1d?dmode=source&output=gplain. Retrieved December 14, 2010.
- ^ Segan, Sascha (January 1, 1970). "R.I.P Usenet: 1980-2008 - Usenet's Decline - Columns by PC Magazine". Pcmag.com. http://www.pcmag.com/article2/0,2817,2326849,00.asp. Retrieved December 14, 2010.
- ^ Strawbridge, Matthew (2006). Netiquette: Internet Etiquette in the Age of the Blog. Software Reference. p. 53.
- ^ Cullen, Drew (February 12, 2001). Google saves Deja.com Usenet service. The Register.
- ^ Wiseman, David. "Magi's NetNews Archive Involvement", csd.uwo.ca
- ^ Mieszkowski, Katharine. "The Geeks Who Saved Usenet", archive.salon.com (January 7, 2002).
- ^ Feldman, Ian. "Usenet on a CD-ROM, no longer a fable", "TidBITS" (February 10, 1992)
- ^ "Google Groups Archive Information" (December 21, 2001)
- Bruce Jones, archiver (1997). USENET History mailing list archive covering 1990–1997. communication.ucsd.edu
- Michael Hauben, Ronda Hauben, and Thomas Truscott (April 27, 1997). Netizens: On the History and Impact of Usenet and the Internet (Perspectives). Wiley-IEEE Computer Society P. ISBN 0-8186-7706-6.
- Bryan Pfaffenberger (December 31, 1994). The USENET Book: Finding, Using, and Surviving Newsgroups on the Internet. Addison Wesley. ISBN 0-201-40978-X.
- Kate Gregory, Jim Mann, Tim Parker, and Noel Estabrook (June 1995). Using Usenet Newsgroups. Que. ISBN 0-7897-0134-0.
- Mark Harrison (July 1995). The USENET Handbook (Nutshell Handbook). O'Reilly. ISBN 1-56592-101-1.
- Henry Spencer, David Lawrence (January 1998). Managing Usenet. O'Reilly. ISBN 1-56592-198-4.
- Don Rittner (June 1997). Rittner's Field Guide to Usenet. MNS Publishing. ISBN 0-937666-50-5.
- Konstan, J., Miller, B., Maltz, D., Herlocker, J., Gordon, L., and Riedl, J. (March 1997). "GroupLens: applying collaborative filtering to Usenet news". Communications of the ACM 40 (3): 77–87. DOI:10.1145/245108.245126. http://portal.acm.org/citation.cfm?doid=245108.245126. Retrieved June 29, 2009.
- Miller, B., Riedl, J., and Konstan, J. (January 1997). "Experiences with GroupLens: Making Usenet useful again". Proceedings of the 1997 Usenix Winter Technical Conference. http://www.grouplens.org/papers/pdf/usenix97.pdf.
- "20 Year Usenet Timeline". Google. http://www.google.com/googlegroups/archive_announce_20.html. Retrieved June 27, 2006.
- "Web 2.0, Meet Usenet 1.0". Linux Magazine. http://www.linux-mag.com/id/2675/. Retrieved February 13, 2007.
- Schwartz, Randal (June 15, 2006). "Web 2.0, Meet Usenet 1.0". http://www.linux-mag.com/id/2675/. Retrieved June 4, 2007.
- Kleiner, Dmytri; Wyrick, Brian (January 29, 2007). "InfoEnclosure 2.0". http://www.metamute.org/InfoEnclosure-2.0. Retrieved June 4, 2007.
- Usenet information, software, and service providers at the Open Directory Project
- Usenet servers at the Open Directory Project
- Public News Servers at the Open Directory Project
- IETF working group USEFOR (USEnet article FORmat), tools.ietf.org
- son-of-1036 (287 KB, historical RFC 1036 bis draft), purl.net
- A-News Archive: Early Usenet news articles: 1981 to 1982., quux.org
- UTZoo Archive: 2,000,000 articles from early 1980s to July 1991
- "Netscan". Archived from the original on June 29, 2009. http://web.archive.org/web/*/http://netscan.research.microsoft.com. Social Accounting Reporting Tool
- Living Internet A comprehensive history of the Internet, including Usenet. livinginternet.com
- Usenet Glossary A comprehensive list of Usenet terminology
- "Big 8 Management Board". Archived from the original on January 30, 2008. http://web.archive.org/web/20080130015435/http://www.big-8.org/. Group creation and maintenance in the Big 8 hierarchies.
- Michael Hauben (Fall 1992). "The Evolution Of Usenet News: The Poor Man's Arpanet". http://www.virtualschool.edu/mon/Internet/HaubenEvolutionNetnews.html. Retrieved May 24, 2010.
- "Giganews Deconstructs Cuomo's Child Porn 'Crackdown'". http://www.dslreports.com/shownews/Giganews-Deconstructs-Cuomos-Child-Porn-Crackdown-98446. Retrieved September 1, 2011.
- "Latest Child Porn Fight Mostly Empty Rhetoric". http://www.dslreports.com/shownews/96203. Retrieved September 1, 2011.
- "N.Y. AG says AOL will curb access to Usenet. It already did". http://news.cnet.com/8301-13578_3-9988278-38.html?hhTest=1%E2%88%82=rss&subj=news&tag=2547-1_3-0-20. Retrieved September 1, 2011.
- "Analysis of Cuomo Crusade shows little results.". http://www.giganews.com/blog/2008/09/clearing-air-usenet-abuse-eliminating.html. Retrieved September 1, 2011.
.svg)
