‌,
Comma
Punctuation apostrophe ( ’ ' ) brackets ( [ ], ( ), { }, ⟨ ⟩ ) colon ( : ) comma ( , ، 、 ) dash ( ‒, –, —, ― ) ellipsis ( …, ..., . . . ) exclamation mark ( ! ) full stop/period ( . ) guillemets ( « » ) hyphen ( ‐ ) hyphen-minus ( - ) question mark ( ? ) quotation marks ( ‘ ’, “ ”, ' ', " " ) semicolon ( ; ) slash‌/stroke‌/solidus ( /,  ⁄  ) Word dividers space ( ) ( ) ( ) interpunct ( · ) General typography ampersand ( & ) at sign ( @ ) asterisk ( * ) backslash ( \ ) bullet ( • ) caret ( ^ ) dagger ( †, ‡ ) degree ( ° ) ditto mark ( 〃 ) inverted exclamation mark ( ¡ ) inverted question mark ( ¿ ) number sign‌/pound‌/hash ( # ) numero sign ( № ) obelus ( ÷ ) ordinal indicator ( º, ª ) percent, per mil ( %, ‰, ‱ ) pilcrow ( ¶ ) prime ( ′, ″, ‴ ) section sign ( § ) tilde ( ~ ) underscore‌/understrike ( _ ) vertical bar‌/broken bar‌/pipe ( ¦, | ) Intellectual property copyright symbol ( © ) registered trademark ( ® ) service mark ( ℠ ) sound recording copyright ( ℗ ) trademark ( ™ ) Currency currency (generic) ( ¤ ) currency (specific) ( ₳ ฿ ₵ ¢ ₡ ₢ ₠ $ ₫ ৳ ₯ € ƒ ₣ ₲ ₴ ₭ ℳ ₥ ₦ ₧ ₱ ₰ £ ₹ ₨ ₪ ₸ ₮ ₩ ¥ ៛ ) Uncommon typography asterism ( ⁂ ) tee ( ⊤ ) up tack ( ⊥ ) index/fist ( ☞ ) therefore sign ( ∴ ) because sign ( ∵ ) interrobang ( ‽ ) irony punctuation ( ؟ ) lozenge ( ◊ ) reference mark ( ※ ) tie ( ⁀ ) Related diacritical marks whitespace characters non-English quotation style ( « », „ ” ) In other scripts Chinese punctuation
Book  · Category  · Portal
This template: view talk edit

The comma, in Unicode U+002C , comma (HTML: ,), is a punctuation mark, and includes several variants in various languages. It has the same shape as an apostrophe or single closing quotation mark in many typefaces, but it differs from them in being placed on the baseline of the text. Some typefaces render it as a small line, slightly curved or straight but inclined from the vertical, or with the appearance of a small, filled-in number 9. It is used to separate parts of a sentence such as clauses, and lists of three or more things.

The comma is used in many contexts and languages, principally for separating things. According to the Oxford English Dictionary, the word comma comes directly from the Greek komma (κόμμα), which means something cut off or a short clause.

Comma variants[link]

Commas are often used as markers in prose and lists; some are used as diacritics when combined with other characters.

History[link]

In the 3rd century BC, Aristophanes of Byzantium invented a system of single dots (distinctiones) that separated verses (colometry), and indicated the amount of breath needed to complete each fragment of text, when reading aloud.^[1] The different lengths were signified by a dot at the bottom, middle, or top of the line. For a short passage (a komma), a media distinctio dot was placed mid-level ( · ). This is the origin of the concept of a comma, although the name came to be used for the mark itself, instead of the clause it separated.

The mark used today is descended from a diagonal slash, or virgula suspensiva ( / ), used from the 13th to 17th centuries to represent a pause, and was first used by Aldus Manutius.^[2][3][4]

Uses in English[link]

In general, the comma is used where ambiguity might otherwise arise, to indicate an interpretation of the text such that the words immediately before and after the comma are less closely or exclusively linked in the associated grammatical structure than they might be otherwise. The comma may be used to perform a number of functions in English writing. It is used in generally similar ways in other languages, particularly European ones, although the rules on comma usage – and their rigidity – vary from language to language.

In lists[link]

Commas are used to separate items in lists, as in They own a cat, a dog, two rabbits, and six mice. In English, a comma may or may not be used before the final conjunction (and, or, nor) in a list of more than two elements. A comma used in such a position is called a serial comma or an Oxford or Harvard comma (after the Oxford University Press and Harvard University Press, both prominent advocates of this style). In some cases, use or omission of such a comma may serve to avoid ambiguity:

Use of serial comma disambiguating:

• I spoke to the boys, Sam and Tom. – could be either the boys and Sam and Tom (I spoke to more than three people) or the boys, who are Sam and Tom (I spoke to two people)
• I spoke to the boys, Sam, and Tom. – must be the boys and Sam and Tom (I spoke to more than three people)

Omission of serial comma disambiguating:

• I thank my mother, Anne Smith, and Thomas. – could be either my mother and Anne Smith and Thomas (three people) or my mother, who is Anne Smith, and Thomas (two people)
• I thank my mother, Anne Smith and Thomas. – The writer is thanking three people: the writer's mother and Anne Smith (who is not the writer's mother) and Thomas.

As a rule of thumb, The Guardian Style Guide^[5] suggests that straightforward lists (he ate ham, eggs and chips) do not need a comma before the final "and", but sometimes it can help the reader (he ate cereal, kippers, bacon, eggs, toast and marmalade, and tea.) The Chicago Manual of Style, and other academic writing guides, require the "serial comma": all lists must have a comma before the "and" prefacing the last item in a series.

If the individual items of a list are long, complex, affixed with description, or themselves contain commas, semicolons may be preferred as separators, and the list may be introduced with a colon.

Separation of clauses[link]

Commas are often used to separate clauses. In English, a comma is generally used to separate a dependent clause from the independent clause if the dependent clause comes first: After I fed the cat, I brushed my clothes. (Compare this with I brushed my clothes after I fed the cat.) A relative clause takes commas if it is non-restrictive, as in I cut down all the trees, which were over six feet tall. (Without the comma, this would mean that only those trees over six feet tall were cut down.)

Some style guides prescribe that two independent clauses joined by a coordinating conjunction (for, and, nor, but, or, yet, so) must be separated by a comma placed before the conjunction.^[6][7] In the following sentences, where the second clause is independent (because it can stand alone as a sentence), the comma is considered by those guides to be necessary:

• Mary walked to the party, but she was unable to walk home.
• Designer clothes are silly, and I can't afford them anyway.
• Don't push that button, or twelve tons of high explosives will go off right under our feet!

But in the following sentences, where the second clause is dependent (because it cannot stand alone as a sentence), those guides prescribe that the comma be omitted:

• Mary walked to the party but was unable to walk home.
• I think designer clothes are silly and can't afford them anyway.
• Don't push that button or set off the twelve tons of high explosives sitting right under our feet.

However, the comma may be omitted if the second independent clause is very short, typically when the second independent clause is an imperative.^[6][7] In the following sentence, it is sometimes considered acceptable to omit the comma, even though the second clause is independent:

• Sit down and shut up.'

In some languages, such as German and Polish, stricter rules apply on comma usage between clauses, with dependent clauses always being set off with commas, and commas being generally proscribed before certain coordinating conjunctions.

The joining of two independent sentences with a comma and no conjunction (as in "It is nearly half past five, we cannot reach town before dark.") is known as a comma splice and is often considered an error in English; in most cases a semicolon should be used instead. A comma splice should not be confused, though, with asyndeton, a literary device used for a specific effect in which coordinating conjunctions are purposely omitted.

Certain adverbs[link]

Commas are always used to set off certain adverbs, including however, in fact, therefore, nevertheless, moreover, furthermore, and still.

• Therefore, a comma would be appropriate in this sentence.
• Nevertheless, I will not use one.

If these adverbs appear in the middle of a sentence, they are ended with a comma, but they are preceded by a semicolon between clauses and a comma within one clause.

• In this sentence, furthermore, commas would also be called for.
• This sentence is similar; however, a semicolon is necessary as well.

Using commas to offset certain adverbs is optional, including then, so, yet, instead, and too (meaning also).

• So, that's it for this rule. or
• So that's it for this rule.

• A comma would be appropriate in this sentence, too. or
• A comma would be appropriate in this sentence too.

Parenthetical phrases[link]

Commas are often used to enclose parenthetical words and phrases within a sentence (i.e., information that is not essential to the meaning of the sentence). Such phrases are both preceded and followed by a comma, unless that would result in a doubling of punctuation marks, or the parenthetical is at the start or end of the sentence. The following are examples of types of parenthetical phrases:

• Introductory phrase: Once upon a time, my father ate a muffin.^[8]
• Interjection: My father ate the muffin, gosh darn it!
• Aside: My father, if you don’t mind my telling you this, ate the muffin.
• Appositive: My father, a jaded and bitter man, ate the muffin.
• Absolute phrase: My father, his eyes flashing with rage, ate the muffin.
• Free modifier: My father, chewing with unbridled fury, ate the muffin.
• Resumptive modifier: My father ate the muffin, a muffin which no man had yet chewed.
• Summative modifier: My father ate the muffin, a feat which no man had attempted.

Between adjectives[link]

A comma is used to separate coordinate adjectives; that is, adjectives that directly and equally modify the following noun. Adjectives are considered coordinate if the meaning would be the same if their order were reversed or if and were placed between them. For example:

• The dull, incessant droning but the cute little cottage.
• The devious lazy red frog suggests there are lazy red frogs (one of which is devious), while the devious, lazy red frog does not carry this connotation.

Before quotes[link]

A comma is used to set off quoted material that is the grammatical object of an active verb of speaking or writing, as in Mr. Kershner says, "You should know how to use a comma." Quotations that follow and support an assertion should be set off by a colon rather than a comma.

In dates[link]

Month, day, year[link]

When a date is written as a month followed by a day followed by a year, a comma separates the day from the year: December 19, 1941. This style is common in American English. The comma is necessary because of the otherwise confusing consecutive numbers, compare December 19 1941. Additionally, most style manuals, including The Chicago Manual of Style^[9] and the AP Stylebook,^[10] recommend that the year be treated as a parenthetical, requiring a second comma after it: "Feb. 14, 1987, was the target date."

If just month and year are given, no commas are used:^[11] "Her daughter April may return in June 2009 for the reunion."

Day month year[link]

When the day precedes the month, the month name separates the numeric day and year, so commas are not necessary to separate them: "On 19 December 1941 the Raid on Alexandria was carried out by Italian manned torpedoes."

In geographical names[link]

Commas are used to separate parts of geographical references, such as city and state (Dallas, Texas) or city and country (Kampala, Uganda). Additionally, most style manuals, including The Chicago Manual of Style^[12] and the AP Stylebook,^[13] recommend that the second element be treated as a parenthetical, requiring a second comma after: "The plane landed in Kampala, Uganda, that evening."^[14]

The United States Postal Service^[15] and Royal Mail^[16] recommend writing addresses without any punctuation.

In numbers[link]

In representing large numbers, English texts usually use commas to separate each group of three digits. This is almost always done for numbers of six or more digits, and often for five or four digits. However, in much of Europe, Southern Africa and Latin America, periods or spaces are used instead; the comma is used as a decimal separator, equivalent to the use in English of the decimal point. In addition, the comma may not be used for this purpose at all in some number systems, e.g. the SI writing style;^[17] a space may be used to separate groups of three digits instead.

In names[link]

Commas are used when writing names that are presented surname first: Smith, John. They are also used before many titles that follow a name: John Smith, Ph.D.

Ellipsis[link]

Commas may be used to indicate that a word has been omitted, as in The cat was white; the dog, brown. (Here the comma replaces was.)

Vocative[link]

Commas are placed before, after, or around a noun or pronoun used independently in speaking to some person, place or thing:

• I hope, John, that you will read this.

Differences between American and British usage[link]

The comma and the quotation mark pairing can be used in several ways. In American English, the comma is commonly included inside a quotation, regardless of whether the comma is part of the original quotation.^[19] For example:

• My mother gave me the nickname "Johnny Boy," which really made me angry.

However, in British English, punctuation is only placed within quotation marks if it is part of what is being quoted or referred to. Thus:

• My mother gave me the nickname "Johnny Boy", which really made me angry.

The use of the serial comma is sometimes perceived as an Americanism, but common practice varies in both American and British English. Barbara Child claims that in American English there is a trend toward a decreased use of the comma (Child, 1992, p. 398). This is reinforced by an article by Robert J. Samuelson in Newsweek.^[20] Lynne Truss says that this is equally true in the UK, where it has been a slow, steady trend for at least a century:

Nowadays… A passage peppered with commas—which in the past would have indicated painstaking and authoritative editorial attention—smacks simply of no backbone. People who put in all the commas betray themselves as moral weaklings with empty lives and out-of-date reference books. (Truss, 2004, p. 97–98)

Use of a comma after the last of a series is also being discontinued, especially in the U.S., even though this may lead to some confusion, as in the quotation: "I thank my parents, the Pope and Sinead O'Conner".^{[citation needed][original research?]}

In his 1963 book "Of Spies and Stratagems", Stanley P. Lovell recalls that, during the Second World War, the British carried the comma over into abbreviations. Specifically, "Special Operations, Executive" was written “S.O.,E.”. Nowadays, even the full stops are frequently discarded.

In other languages[link]

Punctuation has been added to many languages which originally developed without it, including a number of different comma forms.

European languages like German, French, Italian, Spanish, and Portuguese use the same comma as English with similar spacing.

Japanese punctuation commonly uses the '、' (tōten, 読点, U+3001 IDEOGRAPHIC COMMA) in native text and full-width comma '，' (U+FF0C FULLWIDTH COMMA) when mixing Japanese and Latin alphabet characters.

Chinese punctuation normally uses '，' (U+FF0C FULLWIDTH COMMA) but has the "enumeration comma" '、' (simplified Chinese: 顿号; traditional Chinese: 頓號; pinyin: dùnhào; literally "pause mark", U+3001 IDEOGRAPHIC COMMA) for lists. The comma is used to join together clauses that deal with a certain topic or line of thinking. As such, what would appear to an English speaker to be a comma splice is very commonly seen in Chinese writing. Often, the entirety of a long paragraph can consist of clauses joined by commas, with the sole period coming only at the end. Unlike in English, a comma is allowed between a subject and its predicate.

Arabic, Urdu, and Persian languages — written from right to left — use a reversed comma '،' (U+060C ARABIC COMMA) unlike other right-to-left languages like Hebrew.

Korean punctuation uses both commas and interpuncts for lists.

Computing[link]

In the common character encoding systems Unicode and ASCII, character 44 (0x002C) corresponds to the comma symbol. HTML code is ,

In many computer languages commas are used to separate arguments to a function, to separate elements in a list and to perform data designation on multiple variables at once.

In the C programming language the comma symbol is an operator which evaluates its first argument (which may have side-effects) and then returns the value of its evaluated second argument. This is useful in for statements and macros.

In Smalltalk, the comma operator is used to concatenate collections, including strings.

The comma-separated values (CSV) format is very commonly used in exchanging text data between database and spreadsheet formats.

Diacritical usage[link]

The comma is used as a diacritic mark in Romanian under the s (Ș, ș), and under the t (Ț, ț). A cedilla is occasionally used instead of it (notably in the Unicode glyph names), but this is technically incorrect. The symbol d̦ (d with comma below) was used as part of the Romanian transitional alphabet (19th century) to indicate the sounds denoted by the Latin letter z or letters dz, where derived from a Cyrillic ѕ (/dz/). The comma and the cedilla are both derivative of a small cursive z (ʒ) placed below the letter. From this standpoint alone, ș, ț, and d̦ could potentially be regarded as stand-ins for sz, tz, and dz respectively.

In Latvian, the comma is used on the letters ģ, ķ, ļ, ņ, and historically also ŗ, to indicate palatalization. Because the lowercase letter g has a descender, the comma is rotated 180° and placed over the letter. Although their Adobe glyph names are commas, their names in the Unicode Standard are g, k, l, n, and r with a cedilla. They were introduced to the Unicode standard before 1992, and their name cannot be altered. For input Ģ use Alt 290 and Alt 291 sequences, for Ķ use Alt 310 and Alt 311, for Ļ use Alt 315 and Alt 316, for Ņ use Alt 325 and Alt 326.

In the Czech and Slovak languages, the diacritics in the characters ď, ť, and ľ resemble superscript commas, but they are modified carons because they have ascender. Other ascender letters with carons, such as letters ȟ (used in Finnish Romani and Lakota languages) and ǩ (used in Skolt Sami language), did not modify their carons to superscript commas.

See also[link]

• Cedilla
• Ogonek
• Semicolon
• Serial comma
• Decimal separator

References[link]

• This article is based on material taken from the Free On-line Dictionary of Computing prior to 1 November 2008 and incorporated under the "relicensing" terms of the GFDL, version 1.3 or later.

Bibliography[link]

• Barbara Child, Drafting Legal Documents, 2nd Edition, 1992.
• Lynne Truss, Eats, Shoots and Leaves, Gotham Books (2004), ISBN 1-59240-087-6.

External links[link]

• Commas: They're Not Just for English Majors, Anymore
• English comma rules and exercises
• Major Comma Uses
• Notes on Commas
• Comma guidelines – also helpful for non-native speakers
• Grammar, Punctuation, and Capitalization – a comprehensive online guide by NASA
• The Oxford Comma: A Solution – a satirical suggestion to settle the problem of the Oxford Comma once and for all.
• The Quotta and the Quottiod – another satirical compromise between the American and British traditions relating to quotes and commas.
• The Ten Functions of Commas in English

This article contains instructions, advice, or how-to content. The purpose of Wikipedia is to present facts, not to train. Please help improve this article either by rewriting the how-to content or by moving it to Wikiversity or Wikibooks. (July 2011)

Text file created with gedit and viewed with a hex editor
Besides the text objects there are only EOL markers
with the hexadecimal value 0A.

In computing, a newline,^[1] also known as a line break or end-of-line (EOL) marker, is a special character or sequence of characters signifying the end of a line of text. The name comes from the fact that the next character after the newline will appear on a new line—that is, on the next line below the text immediately preceding the newline. The actual codes representing a newline vary across operating systems, which can be a problem when exchanging text files between systems with different newline representations.

There is also some confusion whether newlines terminate or separate lines. If a newline is considered a separator, there will be no newline after the last line of a file. The general convention on most systems is to add a newline even after the last line, i.e. to treat newline as a line terminator. Some programs have problems processing the last line of a file if it is not newline terminated. Conversely, programs that expect newline to be used as a separator will interpret a final newline as starting a new (empty) line.

In text intended primarily to be read by humans using software which implements the word wrap feature, a newline character typically only needs to be stored if a line break is required independent of whether the next word would fit on the same line, such as between paragraphs and in vertical lists. See hard return and soft return.

Representations[link]

Software applications and operating systems usually represent a newline with one or two control characters:

• Systems based on ASCII or a compatible character set use either LF (Line feed, '\n', 0x0A, 10 in decimal) or CR (Carriage return, '\r', 0x0D, 13 in decimal) individually, or CR followed by LF (CR+LF, '\r\n', 0x0D0A). These characters are based on printer commands: The line feed indicated that one line of paper should feed out of the printer thus instructed the printer to advance the paper one line, and a carriage return indicated that the printer carriage should return to the beginning of the current line. Some rare systems, such as QNX before version 4, used the ASCII RS (record separator, 0x1E, 30 in decimal) character as the newline character.
  • LF:    Multics, Unix and Unix-like systems (GNU/Linux, Mac OS X, FreeBSD, AIX, Xenix, etc.), BeOS, Amiga, RISC OS and others.
  • CR+LF: Microsoft Windows, DEC TOPS-10, RT-11 and most other early non-Unix and non-IBM OSes, CP/M, MP/M, DOS (MS-DOS, PC-DOS, etc.), Atari TOS, OS/2, Symbian OS, Palm OS
  • LF+CR: Acorn BBC and RISC OS spooled text output.
  • CR:    Commodore 8-bit machines, Acorn BBC, TRS-80, Apple II family, Mac OS up to version 9 and OS-9
  • RS:    QNX pre-POSIX implementation.
• EBCDIC systems—mainly IBM mainframe systems, including z/OS (OS/390) and i5/OS (OS/400)—use NEL (Next Line, 0x15) as the newline character. Note that EBCDIC also has control characters called CR and LF, but the numerical value of LF (0x25) differs from the one used by ASCII (0x0A). Additionally, there are some EBCDIC variants that also use NEL but assign a different numeric code to the character.
• Operating systems for the CDC 6000 series defined a newline as two or more zero-valued six-bit characters at the end of a 60-bit word. Some configurations also defined a zero-valued character as a colon character, with the result that multiple colons could be interpreted as a newline depending on position.
• ZX80 and ZX81, home computers from Sinclair Research Ltd used a specific non-ASCII character set with code NEWLINE (0x76, 118 decimal) as the newline character.
• OpenVMS uses a record-based file system, which stores text files as one record per line. In most file formats, no line terminators are actually stored, but the Record Management Services facility can transparently add a terminator to each line when it is retrieved by an application. The records themselves could contain the same line terminator characters, which could either be considered a feature or a nuisance depending on the application.
• Fixed line length was used by some early mainframe operating systems. In such a system, an implicit end-of-line was assumed every 80 characters, for example. No newline character was stored. If a file was imported from the outside world, lines shorter than the line length had to be padded with spaces, while lines longer than the line length had to be truncated. This mimicked the use of punched cards, on which each line was stored on a separate card, usually with 80 columns on each card. Many of these systems added an carriage control character to the start of the next record, this could indicate if the next record was a continuation of the line started by the previous record, or a new line, or should overprint the previous line (similar to a CR). Often this was a normal printing character such as '#' that thus could not be used as the first character in a line. Some early line printers interpreted these characters directly in the records sent to them.

Most textual Internet protocols (including HTTP, SMTP, FTP, IRC and many others) mandate the use of ASCII CR+LF (0x0D 0x0A) on the protocol level, but recommend that tolerant applications recognize lone LF as well. In practice, there are many applications that erroneously use the C newline character '\n' instead (see section Newline in programming languages below). This leads to problems when trying to communicate with systems adhering to a stricter interpretation of the standards; one such system is the qmail MTA that actively refuses to accept messages from systems that send bare LF instead of the required CR+LF.^[2]

FTP has a feature to transform newlines between CR+LF and LF only when transferring text files. This must not be used on binary files. Usually binary files and text files are recognised by checking their filename extension.

Unicode[link]

The Unicode standard defines a large number of characters that conforming applications should recognize as line terminators:^[3]

 LF:    Line Feed, U+000A
 VT:    Vertical Tab, U+000B
 FF:    Form Feed, U+000C
 CR:    Carriage Return, U+000D
 CR+LF: CR (U+000D) followed by LF (U+000A)
 NEL:   Next Line, U+0085
 LS:    Line Separator, U+2028
 PS:    Paragraph Separator, U+2029

This may seem overly complicated compared to an approach such as converting all line terminators to a single character, for example LF. However, Unicode was designed to preserve all information when converting a text file from any existing encoding to Unicode and back. Therefore, Unicode should contain characters included in existing encodings. NEL is included in ISO-8859-1^{[citation needed]} and EBCDIC (0x15). The approach taken in the Unicode standard allows round-trip transformation to be information-preserving while still enabling applications to recognize all possible types of line terminators.

Recognizing and using the newline codes greater than 0x7F is not often done. They are multiple bytes in UTF-8 and the code for NEL has been used as the ellipsis ('…') character in Windows-1252. For instance:

• YAML^[4] no longer recognizes them as special in order to be compatible with JSON.
• ECMAScript^[5] accepts LS and PS as line breaks, but considers U+0085 (NEL) white space, not a line break.
• Microsoft Windows 2000 does not treat any of NEL, LS or PS as line-break in the default text editor Notepad
• In Linux, a popular editor "gedit" treats LS and PS as newlines but does not for NEL.

History[link]

ASCII was developed simultaneously by the ISO and the ASA, the predecessor organization to ANSI. During the period of 1963–1968, the ISO draft standards supported the use of either CR+LF or LF alone as a newline, while the ASA drafts supported only CR+LF.

The sequence CR+LF was in common use on many early computer systems that had adopted Teletype machines, typically a Teletype Model 33 ASR, as a console device, because this sequence was required to position those printers at the start of a new line. On these systems, text was often routinely composed to be compatible with these printers, since the concept of device drivers hiding such hardware details from the application was not yet well developed; applications had to talk directly to the Teletype machine and follow its conventions.

Most minicomputer systems from DEC used this convention. CP/M used it as well, to print on the same terminals that minicomputers used. From there MS-DOS (1981) adopted CP/M's CR+LF in order to be compatible, and this convention was inherited by Microsoft's later Windows operating system.

The separation of the two functions concealed the fact that the print head could not return from the far right to the beginning of the next line in one-character time. That is why the sequence was always sent with the CR first. In fact, it was often necessary to send extra characters (extraneous CRs or NULs, which are ignored) to give the print head time to move to the left margin. Even many early video displays required multiple character times to scroll the display.

The Multics operating system began development in 1964 and used LF alone as its newline. Multics used a device driver to translate this character to whatever sequence a printer needed (including extra padding characters), and the single byte was much more convenient for programming. The seemingly more obvious choice of CR was not used, as a plain CR provided the useful function of overprinting one line with another, and thus it was useful to not translate it. Unix followed the Multics practice, and later systems followed Unix.

In programming languages[link]

To facilitate the creation of portable programs, programming languages provide some abstractions to deal with the different types of newline sequences used in different environments.

The C programming language provides the escape sequences '\n' (newline) and '\r' (carriage return). However, these are not required to be equivalent to the ASCII LF and CR control characters. The C standard only guarantees two things:

1. Each of these escape sequences maps to a unique implementation-defined number that can be stored in a single char value.
2. When writing a file in text mode, '\n' is transparently translated to the native newline sequence used by the system, which may be longer than one character. When reading in text mode, the native newline sequence is translated back to '\n'. In binary mode, no translation is performed, and the internal representation produced by '\n' is output directly.

On Unix platforms, where C originated, the native newline sequence is ASCII LF (0x0A), so '\n' was simply defined to be that value. With the internal and external representation being identical, the translation performed in text mode is a no-op, and text mode and binary mode behave the same. This has caused many programmers who developed their software on Unix systems simply to ignore the distinction completely, resulting in code that is not portable to different platforms.

The C library function fgets() is best avoided in binary mode because any file not written with the UNIX newline convention will be misread. Also, in text mode, any file not written with the system's native newline sequence (such as a file created on a UNIX system, then copied to a Windows system) will be misread as well.

Another common problem is the use of '\n' when communicating using an Internet protocol that mandates the use of ASCII CR+LF for ending lines. Writing '\n' to a text mode stream works correctly on Windows systems, but produces only LF on Unix, and something completely different on more exotic systems. Using "\r\n" in binary mode is slightly better.

Many languages, such as C++, Perl,^[6] and Haskell provide the same interpretation of '\n' as C.

Java, PHP,^[7] and Python^[8] provide the '\r\n' sequence (for ASCII CR+LF). In contrast to C, these are guaranteed to represent the values U+000A and U+000D, respectively.

The Java I/O libraries do not transparently translate these into platform-dependent newline sequences on input or output. Instead, they provide functions for writing a full line that automatically add the native newline sequence, and functions for reading lines that accept any of CR, LF, or CR+LF as a line terminator (see BufferedReader.readLine()). The System.getProperty() method can be used to retrieve the underlying line separator.

Example:

  String eol = System.getProperty( "line.separator" );
  String lineColor = "Color: Red" + eol;

Python permits "Universal Newline Support" when opening a file for reading, when importing modules, and when executing a file.^[9]

Some languages have created special variables, constants, and subroutines to facilitate newlines during program execution.

Common problems[link]

The different newline conventions often cause text files that have been transferred between systems of different types to be displayed incorrectly. For example, files originating on Unix or Apple Macintosh systems may appear as a single long line on some Windows programs. Conversely, when viewing a file originating from a Windows computer on a Unix system, the extra CR may be displayed as ^M at the end of each line or as a second line break.

The problem can be hard to spot if some programs handle the foreign newlines properly while others do not. For example, a compiler may fail with obscure syntax errors even though the source file looks correct when displayed on the console or in an editor. On a Unix system, the command cat -v myfile.txt will send the file to stdout (normally the terminal) and make the ^M visible, which can be useful for debugging. Modern text editors generally recognize all flavours of CR / LF newlines and allow the user to convert between the different standards. Web browsers are usually also capable of displaying text files and websites which use different types of newlines.

The File Transfer Protocol can automatically convert newlines in files being transferred between systems with different newline representations when the transfer is done in "ASCII mode". However, transferring binary files in this mode usually has disastrous results: Any occurrence of the newline byte sequence—which does not have line terminator semantics in this context, but is just part of a normal sequence of bytes—will be translated to whatever newline representation the other system uses, effectively corrupting the file. FTP clients often employ some heuristics (for example, inspection of filename extensions) to automatically select either binary or ASCII mode, but in the end it is up to the user to make sure his or her files are transferred in the correct mode. If there is any doubt as to the correct mode, binary mode should be used, as then no files will be altered by FTP, though they may display incorrectly.

Conversion utilities[link]

This article contains instructions, advice, or how-to content. The purpose of Wikipedia is to present facts, not to train. Please help improve this article either by rewriting the how-to content or by moving it to Wikiversity or Wikibooks. (June 2010)

Text editors are often used for converting a text file between different newline formats; most modern editors can read and write files using at least the different ASCII CR/LF conventions. The standard Windows editor Notepad is not one of them (although Wordpad and the MS-DOS Editor are).

Editors are often unsuitable for converting larger files. For larger files (on Windows NT/2000/XP) the following command is often used:

TYPE unix_file | FIND "" /V > dos_file

On many Unix systems, the dos2unix (sometimes named fromdos or d2u) and unix2dos (sometimes named todos or u2d) utilities are used to translate between ASCII CR+LF (DOS/Windows) and LF (Unix) newlines. Different versions of these commands vary slightly in their syntax. However, the tr command is available on virtually every Unix-like system and is used to perform arbitrary replacement operations on single characters. A DOS/Windows text file can be converted to Unix format by simply removing all ASCII CR characters with

tr -d '\r' < inputfile > outputfile

or, if the text has only CR newlines, by converting all CR newlines to LF with

tr '\r' '\n' < inputfile > outputfile

The same tasks are sometimes performed with awk, sed, Tr_(Unix) or in Perl if the platform has a Perl interpreter:

awk '{sub("$","\r\n"); printf("%s",$0);}' inputfile > outputfile  # UNIX to DOS  (adding CRs on Linux and BSD based OS that haven't GNU extensions)
awk '{gsub("\r",""); print;}' inputfile > outputfile              # DOS to UNIX  (removing CRs on Linux and BSD based OS that haven't GNU extensions)
sed -e 's/$/\r/' inputfile > outputfile              # UNIX to DOS  (adding CRs on Linux based OS that use GNU extensions)
sed -e 's/\r$//' inputfile > outputfile              # DOS  to UNIX (removing CRs on Linux based OS that use GNU extensions)
cat inputfile | tr -d "\r" > outputfile              # DOS  to UNIX (removing CRs using tr(1). Not Unicode compliant.)
perl -pe 's/\r?\n|\r/\r\n/g' inputfile > outputfile  # Convert to DOS
perl -pe 's/\r?\n|\r/\n/g'   inputfile > outputfile  # Convert to UNIX
perl -pe 's/\r?\n|\r/\r/g'   inputfile > outputfile  # Convert to old Mac

To identify what type of line breaks a text file contains, the file command can be used. Moreover, the editor Vim can be convenient to make a file compatible with the Windows notepad text editor. For example:

[prompt] > file myfile.txt
myfile.txt: ASCII English text
[prompt] > vim myfile.txt
  within vim :set fileformat=dos
             :wq
[prompt] > file myfile.txt
myfile.txt: ASCII English text, with CRLF line terminators

The following grep commands echo the filename (in this case myfile.txt) to the command line if the file is of the specified style:

grep -PL $'\r\n' myfile.txt # show UNIX style file (LF terminated)
grep -Pl $'\r\n' myfile.txt # show DOS style file (CRLF terminated)

For Debian-based systems, these commands are used:

egrep -L $'\r\n' myfile.txt # show UNIX style file (LF terminated)
egrep -l $'\r\n' myfile.txt # show DOS style file (CRLF terminated)

The above grep commands work under Unix systems or in Cygwin under Windows. Note that these commands make some assumptions about the kinds of files that exist on the system (specifically it's assuming only UNIX and DOS-style files—no Mac OS 9-style files).

This technique is often combined with find to list files recursively. For instance, the following command checks all "regular files" (e.g. it will exclude directories, symbolic links, etc.) to find all UNIX-style files in a directory tree, starting from the current directory (.), and saves the results in file unix_files.txt, overwriting it if the file already exists:

find . -type f -exec grep -PL '\r\n' {} \; > unix_files.txt

This example will find C files and convert them to LF style line endings:

find -name '*.[ch]' -exec fromdos {} \;

The file command also detects the type of EOL used:

file myfile.txt
> myfile.txt: ASCII text, with CRLF line terminators

Other tools permit the user to visualise the EOL characters:

od -a myfile.txt
cat -e myfile.txt
hexdump -c myfile.txt

dos2unix, unix2dos, mac2unix, unix2mac, mac2dos, dos2mac can perform conversions. The flip^[10] command is often used.

See also[link]

• ASA carriage control characters
• C0 and C1 control codes
• Page break
• Line starve

References[link]

External links[link]

• The Unicode reference, see paragraph 5.8 in Chapter 5 of the Unicode 4.0 standard (PDF)
• "The End-of-Line Story"
• The [NEL] Newline Character
• The End of Line Puzzle
• Tofrodos - software for Unix that converts to and from DOS newlines
• ToFroWin: a Windows shell extension that is able to convert multiple files from DOS to UNIX (and vice-versa) line endings right from the context menu.
• "Understanding Newlines" on O'Reilly-Net - an article by Xavier Noria.