The BMP File Format, also known as Bitmap Image File or Device Independent Bitmap (DIB) file format or simply a Bitmap, is a Raster graphics image file format used to store bitmap digital images, independently of the display device (such as a graphics adapter), especially on Microsoft Windows and OS/2 operating systems.

The BMP File Format is capable of storing 2D digital images of arbitrary width, height, and resolution, both monochrome and color, in various color depths, and optionally with data compression, alpha channels, and color profiles.

Device-independent bitmaps and BMP file format

Microsoft has defined a particular representation of color bitmaps of different color depths, as an aid to exchanging bitmaps between devices and applications with a variety of internal representations. They called these device-independent bitmaps or DIBs, and the file format for them is called DIB file format or BMP image file format. According to Microsoft support:

''A device-independent bitmap (DIB) is a format used to define device-independent bitmaps in various color resolutions. The main purpose of DIBs is to allow bitmaps to be moved from one device to another (hence, the device-independent part of the name). A DIB is an external format, in contrast to a device-dependent bitmap, which appears in the system as a bitmap object (created by an application...). A DIB is normally transported in metafiles (usually using the StretchDIBits() function), BMP files, and the Clipboard (CF_DIB data format). ''

The following sections discuss the data stored in the BMP file or DIB in details. This is the standard BMP file format. Some applications create bitmap image files which are not compliant with the current Microsoft documentation. Also, not all fields are used; a value of 0 will be found in these unused fields.

File structure

The Bitmap Image File consists of fixed-size structures (headers) as well as variable-size structures appearing in a predetermined sequence. Many different versions of some of these structures can appear in the file, due to the long evolution of this file format.

Referring to the Diagram 1, the Bitmap Image File is composed of structures in the following order:

! Structure Name	! Optional	! Size	! Purpose	! Comments
Bitmap File Header	No	14 Bytes	To store general information about the Bitmap Image File	Not needed after the file is loaded in memory
DIB Header	No	Fixed-size(however 7 different versions exist)	To store detailed information about the bitmap image and define the pixel format	Immediately follows the Bitmap File Header
Extra bit masks	Yes	3 or 4 DWORDs(12 or 16 Bytes)	To define the pixel format	Present only in case the DIB Header is the BITMAPINFOHEADER
Color Table	Semi-optional	Variable-size	To define colors used by the bitmap image data (Pixel Array)	Mandatory for color depths <= 8
Gap1	Yes	Variable-size	Structure alignment	An artifact of the File Offset to PixelArray in the Bitmap File Header
Pixel Array	No	Variable-size	To define the actual values of the pixels	The pixel format is defined by the DIB Header or Extra bit masks. Each row in the Pixel Array is padded to a multiple of 4 bytes in size
Gap2	Yes	Variable-size	Structure alignment	An artifact of the ICC Profile Data offset field in the DIB Header
ICC Color Profile	Yes	Variable-size		Can also contain a path to an external file containing the Color Profile. When loaded in memory as "non-packed DIB", it is located between the Color Table and Gap1.

DIBs in memory

A Bitmap Image File loaded into memory becomes a DIB data structure - an important component of the Windows GDI API. The in-memory DIB data structure is almost the same as the BMP file format, but it does not contain the 14-byte Bitmap File Header and begins with the DIB Header. For DIBs loaded in memory, the Color Table can also consist of 16 bit entries, that constitute indexes to the currently realized palette (an additional level of indirection), instead of explicit RGB color definitions. In all cases, the Pixel Array must begin at a memory address that is a multiple of 4 bytes. In non-packed DIBs loaded in memory, the optional Color Profile data should be located immediately after the Color Table and before the Gap1 and Pixel Array (unlike in Diag.1).

When the size of Gap1 and Gap2 is zero, the in-memory DIB data structure is customarily referred to as "packed DIB" and can be referred to by a single pointer pointing to the beginning of the DIB Header. In all cases, the Pixel Array must begin at a memory address that is a multiple of 4 bytes. In some cases it may be necessary to adjust the number of entries in the Color Table in order to force the memory address of the Pixel Array to a multiple of 4 bytes. For "packed DIBs" loaded in memory, the optional Color Profile data should immediately follow the Pixel Array, as depicted in Diag.1 (with Gap1=0 and Gap2=0). "Packed DIBs" are required by Windows clipboard API functions as well as by some Windows patterned brush and resource functions.

Bitmap file header

This block of bytes is at the start of the file and is used to identify the file. A typical application reads this block first to ensure that the file is actually a BMP file and that it is not damaged. The first two bytes of the BMP file format are the character 'B' then the character 'M' in 1-byte ASCII encoding. All of the integer values are stored in little-endian format (i.e. least-significant byte first).

! Offset#	! Size	! Purpose
0000h	2 bytes		*BM – Windows 3.1x, 95, NT, ... etc.; and it is not mandatory unless file size is greater or equal to SIGNATURE	*BA – OS/2 struct Bitmap Array	*CI – OS/2 struct Color Icon	*&CP; – OS/2 const Color Pointer	*IC – OS/2 struct Icon	**PT – OS/2 Pointer
0002h	4 bytes	the size of the BMP file in bytes
0006h	2 bytes	reserved; actual value depends on the application that creates the image
0008h	2 bytes	reserved; actual value depends on the application that creates the image
000Ah	4 bytes

''Equivalent header in C language'' #include

/* Note: the magic number has been removed from the bmpfile_header structure since it causes alignment problems struct bmpfile_magic should be written/read first followed by the struct bmpfile_header [this avoids compiler-specific alignment pragmas etc.]

/

struct bmpfile_magic { unsigned char magic[2]; };

struct bmpfile_header { uint32_t filesz; uint16_t creator1; uint16_t creator2; uint32_t bmp_offset; };

DIB Header (Bitmap Information Header)

This block of bytes tells the application detailed information about the image, which will be used to display the image on the screen. The block also matches the header used internally by Windows and OS/2 and has several different variants. All of them contain a dword (32 bit) field, specifying their size, so that an application can easily determine which header is used in the image. The reason that there are different headers is that Microsoft extended the DIB format several times. The new extended headers can be used with some GDI functions instead of the older ones, providing more functionality. Since the GDI supports a function for loading bitmap files, typical Windows applications use that functionality. One consequence of this is that for such applications, the BMP formats that they support match the formats supported by the Windows version being run. See the table below for more information.

! Size	! Header Name	! OS support	! Features Added (incremental)	! in Bitmap Files Written by
12	BITMAPCOREHEADEROS21XBITMAPHEADER	OS/2 and also all Windows versions since Windows 3.0
64	BITMAPCOREHEADER2OS22XBITMAPHEADER	OS/2	Adds Halftoning. Adds RLE and Huffman 1D compression.
40	BITMAPINFOHEADER	all Windows versions since Windows 3.0	Removes RLE-24 and Huffman 1D compression. Adds 16bpp and 32bpp pixel formats. Adds optional RGB bit masks.	Adobe Photoshop
52	BITMAPV2INFOHEADER	Undocumented.	Removes optional RGB bit masks. Adds mandatory RGB bit masks.
56	BITMAPV3INFOHEADER	Undocumented.	Adds mandatory Alpha channel bit mask.	Adobe Photoshop
108	BITMAPV4HEADER	all Windows versions since Windows [[Windows 95
124	BITMAPV5HEADER	Windows [[Windows 98

Versions after BITMAPCOREHEADER only add fields to the end of the header of the previous version. For example: BITMAPV2INFOHEADER adds fields to BITMAPINFOHEADER and BITMAPV3INFOHEADER adds fields to BITMAPV2INFOHEADER (or, BITMAPINFOHEADER is a stripped version of BITMAPV2INFOHEADER)

For compatibility reasons, most applications use the older DIB headers for saving files. With OS/2 being obsolete, for now the common format is the BITMAPINFOHEADER header. See next table for its description. All values are stored as unsigned integers, unless explicitly noted.

! Offset #	! Size	! Purpose
0Eh	4	the size of this header (40 bytes)
12h	4	the bitmap width in pixels (signed integer).
16h	4	the bitmap height in pixels (signed integer).
1Ah	2	the number of color planes being used. Must be set to 1.
1Ch	2	the number of bits per pixel, which is the color depth of the image. Typical values are 1, 4, 8, 16, 24 and 32.
1Eh	4	the compression method being used. See the next table for a list of possible values.
22h	4	the image size. This is the size of the raw bitmap data (see below), and should not be confused with the file size.
26h	4	the horizontal resolution of the image. (pixel per meter, signed integer)
2Ah	4	the vertical resolution of the image. (pixel per meter, signed integer)
2Eh	4	the number of colors in the color palette, or 0 to default to 2''n''.
32h	4	the number of important colors used, or 0 when every color is important; generally ignored.

Note: The image size field can be 0 for BI_RGB bitmaps.

''Equivalent header in C language''

typedef struct { uint32_t header_sz; int32_t width; int32_t height; uint16_t nplanes; uint16_t bitspp; uint32_t compress_type; uint32_t bmp_bytesz; int32_t hres; int32_t vres; uint32_t ncolors; uint32_t nimpcolors; } BITMAPINFOHEADER;

The compression method field (bytes #30-33) can have the following values:

! Value	! Identified by	! Compression method	! Comments
0	BI_RGB	none	Most common
1	BI_RLE8		Can be used only with 8-bit/pixel bitmaps
2	BI_RLE4	RLE 4-bit/pixel	Can be used only with 4-bit/pixel bitmaps
3	BI_BITFIELDS	Bit field or Huffman 1D compression for BITMAPCOREHEADER2	Pixel format defined by bit masks or Huffman 1D compressed bitmap for BITMAPCOREHEADER2
4	BI_JPEG	JPEG or RLE-24 compression for BITMAPCOREHEADER2	The bitmap contains a JPEG image or RLE-24 compressed bitmap for BITMAPCOREHEADER2
5	BI_PNG	PNG
6	BI_ALPHABITFIELDS	Bit field	This value is valid in Windows CE .NET 4.0 and later.

Note: BI_JPEG and BI_PNG are for printer drivers and are not supported when rendering to the screen.

''Equivalent header in C language (compression methods)''

typedef enum { BI_RGB = 0, BI_RLE8, BI_RLE4, BI_BITFIELDS, //Also Huffman 1D compression for BITMAPCOREHEADER2 BI_JPEG, //Also RLE-24 compression for BITMAPCOREHEADER2 BI_PNG, } bmp_compression_method_t;

The OS/2 BITMAPCOREHEADER header is also popular:

! Offset	! Size	! Purpose
Eh	4	the size of this header (12 bytes)
12h	2	the bitmap width in pixels.
14h	2	the bitmap height in pixels.
16h	2	the number of color planes; 1 is the only legal value
18h	2	the number of bits per pixel. Typical values are 1, 4, 8 and 24.

Note: OS/2 BITMAPCOREHEADER bitmaps cannot be compressed and cannot be 16 or 32 bits/pixel. All values in the OS/2 BITMAPCOREHEADER header are unsigned integers.

A 16-bit and 32-bit version of DIB with an integrated alpha channel has been introduced with the undocumented BITMAPV3INFOHEADER and with the documented BITMAPV4HEADER (since Windows 95) and is used within Windows XP logon and theme system as well as Microsoft Office (since v2000); it is supported by some image editing software, such as Adobe Photoshop since version 7 and Adobe Flash since version MX 2004 (then known as Macromedia Flash). It is also supported by GIMP, Google Chrome, MS-PowerPoint and MS-Word.

Color Table

The Color Table (a.k.a. Palette) occurs in the BMP image file directly after the BMP file header, the DIB header (and after optional three Red, Green and Blue bitmasks if the BITMAPINFOHEADER Header with BI_BITFIELDS option is used). Therefore, its offset is the size of the BITMAPFILEHEADER plus the size of the DIB header (plus optional 12 bytes for the three bit masks). ''Note: On Windows CE the BITMAPINFOHEADER Header can be used with the BI_ALPHABITFIELDS option in the biCompression member. In such case, four optional bitmasks follow the BITMAPINFOHEADER Header instead of the three bitmask mentioned above, thus the Color Table's offset is the size of the BITMAPFILEHEADER plus the size of the BITMAPINFOHEADER Header plus the 16 bytes of the four bitmasks (Red, Green, Blue and Alpha).''

The number of entries in the palette is either 2^''n'' or a smaller number specified in the header (in the OS/2 BITMAPCOREHEADER header format, only the full-size palette is supported). In most cases, each entry in the color table occupies 4 bytes, in the order Blue, Green, Red, 0x00 (see below for exceptions).

The Color Table is a block of bytes (a table) listing the colors used by the image. Each pixel in an indexed color image is described by a number of bits (1, 4, or 8) which is an index of a single color described by this table. The purpose of the color palette in indexed color bitmaps is to inform the application about the actual color that each of these index values corresponds to. The purpose of the Color Table in non-indexed (non-palettized) bitmaps is to list the colors used by the bitmap for the purposes of optimization on devices with limited color display capability and to facilitate future conversion to different pixel formats and paletization.

The colors in the Color Table are usually specified in the 4-byte per entry 8.8.8.0.8 format (in RGBAX notation). The Color Table used with the OS/2 BITMAPCOREHEADER uses the 3-byte per entry 8.8.8.0.0 format. For DIBs loaded in memory, the Color Table can optionally consist of 2-byte entries - these entries constitute indexes to the currently realized palette instead of explicit RGB color definitions.

Microsoft does not disallow the presence of a valid Alpha channel bit mask in BITMAPV4HEADER and BITMAPV5HEADER for 1bpp, 4bpp and 8bpp indexed color images, which indicates that the Color Table entries can also specify an Alpha component using the 8.8.8.[0-8].[0-8] format via the RGBQUAD.rgbReserved member. However, some versions of Microsoft's documentation disallow this feature by stating that the RGBQUAD.rgbReserved member "must be zero".

As mentioned above, the Color Table is normally not used when the pixels are in the 16-bit per pixel (16bpp) format (and higher); there are normally no Color Table entries in those bitmap image files. However, the Microsoft documentation (on the MSDN web site as of Nov. 16, 2010) specifies that for 16bpp (and higher), the Color Table can be present to store a list of colors intended for optimization on devices with limited color display capability, while it also specifies, that in such cases, no indexed palette entries are present in this Color Table. This may seem like a contradiction if no distinction is made between the mandatory palette entries and the optional color list.

Pixel storage

The bits representing the bitmap pixels are packed in rows. The size of each row is rounded up to a multiple of 4 bytes (a 32-bit DWORD) by padding. For images with Height > 1, multiple padded rows are stored consecutively, forming a Pixel Array.

The total number of bytes necessary to store one row of pixels can be calculated as: :$\backslash mbox\{RowSize\}\; =\; \backslash left\backslash lceil\backslash frac\; \{\; \backslash mbox\{BitsPerPixel\}\; \backslash cdot\; \backslash mbox\{ImageWidth\}\; \}\{32\}\backslash right\backslash rceil\; \backslash cdot\; 4,$ :''ImageWidth'' is expressed in pixels.

The total amount of bytes necessary to store an array of pixels in an n Bits per Pixel (bpp) image, with 2ⁿ colors, can be calculated by accounting for the effect of rounding up the size of each row to a multiple of a 4 bytes, as follows:

:$\backslash mbox\{PixelArraySize\}\; =\; \backslash mbox\{RowSize\}\; \backslash cdot\; \backslash left\; |\; \backslash mbox\{ImageHeight\}\; \backslash right\; \backslash vert$ :''ImageHeight'' is expressed in pixels. The absolute value is necessary because ImageHeight can be negative

The total Bitmap Image File size can be approximated as: :$\backslash mbox\{FileSize\}\; \backslash approx\; 54\; +\; 4\; \backslash cdot\; 2^\backslash mbox\{bpp\}\; +\; \backslash mbox\{PixelArraySize\}$, :for ''BPP'' ≤ 8 ( because for pixels larger than 8 bits, the palette is not mandatory )

Only images with 8 or fewer Bits per Pixel must account for the palette. 16bpp images (or higher), may omit the palette part from the size calculation, as follows:

:$\backslash mbox\{FileSize\}\; \backslash approx\; 54\; +\; \backslash mbox\{PixelArraySize\}$, :for ''Bits per Pixel'' > 8.

In the formulas above, the number 54 is the combined size of the 14 byte Bitmap File Header and the 40 byte popular Windows DIB Header - the BITMAPINFOHEADER  (some other DIB Header versions will be larger or smaller than that as described by the table above) and the expression $4\; \backslash cdot\; 2^n$  is the size of the color palette in bytes.

This total file size formula is only an approximation, since the size of the color palette will be $3\; \backslash cdot\; 2^n$ bytes for the OS/2 DIB Header version BITMAPCOREHEADER, and some files may define only the number of colors needed by the image, potentially fewer than $2^n$. An additional size uncertainty is introduced by the optional presence of the 12 or 16 bytes needed for the extra bit masks stored immediately after the BITMAPINFOHEADER DIB Header and the variable-size GAP depiced in Diag.1

Pixel Array (bitmap data)

The Pixel Array is a block of 32-bit DWORDs, that describes the image pixel by pixel. Normally pixels are stored "upside-down" with respect to normal image raster scan order, starting in the lower left corner, going from left to right, and then row by row from the bottom to the top of the image. Uncompressed Windows bitmaps also can be stored from the top to bottom, when the Image Height value is negative.

In the original OS/2 DIB, the only four legal values of color depth were 1, 4, 8, and 24 bits per pixel (bpp). Contemporary DIB Headers allow pixel formats with 1, 2, 4, 8, 16, 24 and 32 bits per pixel (bpp).

Padding bytes (not necessarily 0) must be appended to the end of the rows in order to bring up the length of the rows to a multiple of four bytes. When the Pixel Array is loaded into memory, each row must begin at a memory address that is a multiple of 4. This address/offset restriction is mandatory only for Pixel Array's loaded in memory. For file storage purposes, only the size of each row must be a multiple of 4 bytes while the file offset can be arbitrary. A 24-bit bitmap with Width=1, would have 3 bytes of data per row (blue, green, red) and 1 byte of padding, while Width=2 would have 2 bytes of padding, Width=3 would have 3 bytes of padding, and Width=4 would not have any padding at all.

Bitmap Image Files are typically much larger than image file formats compressed with other algorithms, for the same image. For example, the 1058×1058 Wikipedia logo, which occupies about 271 KB in the lossless PNG format, takes about 3358 KB as a 24bpp BMP Image File. Uncompressed formats are generally unsuitable for transferring images on the Internet or other slow or capacity-limited media.

Compression

Indexed color images may be compressed with 4-bit or 8-bit RLE or Huffman 1D algorithm. OS/2 BITMAPCOREHEADER2 24bpp images may be compressed with the 24-bit RLE algorithm. The 16bpp and 32bpp images are ''always'' stored uncompressed. Note that images in all color depths can be stored without compression if so desired.

Pixel Format

In bitmap image file on a disk or bitmap image in memory, the pixels can be defined by varying number of bits.

:* The 1-bit per pixel (1bpp) format supports 2 distinct colors, (for example: black and white, ...or yellow and pink). The pixel values are stored in each bit, with the first (left-most) pixel in the most-significant bit of the first byte. Each bit is an index into a table of 2 colors. This Color Table is in 32bpp 8.8.8.0.8 RGBAX format. An unset bit will refer to the first color table entry, and a set bit will refer to the last (second) color table entry.

:* The 2-bit per pixel (2bpp) format supports 4 distinct colors and stores 4 pixels per 1 byte, the left-most pixel being in the two most significant bits (Windows CE only). Each pixel value is a 2-bit index into a table of up to 4 colors. This Color Table is in 32bpp 8.8.8.0.8 RGBAX format.

:* The 4-bit per pixel (4bpp) format supports 16 distinct colors and stores 2 pixels per 1 byte, the left-most pixel being in the more significant nibble. Each pixel value is a 4-bit index into a table of up to 16 colors. This Color Table is in 32bpp 8.8.8.0.8 RGBAX format.

:* The 8-bit per pixel (8bpp) format supports 256 distinct colors and stores 1 pixel per 1 byte. Each byte is an index into a table of up to 256 colors. This Color Table is in 32bpp 8.8.8.0.8 RGBAX format.

:* The 16-bit per pixel (16bpp) format supports 65536 distinct colors and stores 1 pixel per 2 byte WORD. Each WORD can define the Alpha, Red, Green and Blue samples of the pixel.

:* The 24-bit pixel (24bpp) format supports 16,777,216 distinct colors and stores 1 pixel value per 3 bytes. Each pixel value defines the Red, Green and Blue samples of the pixel (8.8.8.0.0 in RGBAX notation). Specifically in the order (Blue, Green and Red, 8-bits per each sample ).

:* The 32-bit per pixel (32bpp) format supports 4,294,967,296 distinct colors and stores 1 pixel per 4 byte DWORD. Each DWORD can define the Alpha, Red, Green and Blue samples of the pixel.

In order to resolve the ambiguity of which bits define which samples, the DIB Headers provide certain defaults as well as specific BITFIELDS which are bit masks that define the membership of particular group of bits in a pixel to a particular channel. The following diagram defines this mechanism:

Diag.2 - The BITFIELDS mechanism for a 32 bit pixel depicted in RGBAX Sample Length notation

The sample fields defined by the BITFIELDS bit masks have to be contiguous and non-overlapping. The Red, Green and Blue bit masks are valid only when the Compression member of the DIB Header is set to BI_BITFIELDS. The Alpha bit mask is valid whenever it is present in the DIB Header or when the Compression member of the DIB Header is set to BI_ALPHABITFIELDS (Windows CE only).

Diag.3 - The pixel format with an Alpha channel in a 16 bit pixel (in RGBAX Sample Length notation) actually generated by Adobe Photohop

::::The table below lists all of the possible pixel formats of a DIB (in RGBAX notation).

The BITFIELD mechanism described above allows for the definition of tens of thousands different pixel formats, however only several of them are used in practice, such as:

8.8.8.0.0

8.8.8.0.8

8.8.8.8.0

5.5.5.0.1

5.5.5.1.0

5.6.5.0.0

4.4.4.0.4

4.4.4.4.0

All palettized formats (marked in yellow in the table above)

[The list above is in the RGBAX notation]

Example 1

Following is an example of a 2×2 pixel, 24-bit bitmap (Windows DIB Header BITMAPINFOHEADER) with pixel format 8.8.8.0.0 (in RGBAX notation).

Offset	Size	Hex Value	Value	Description

Example 2

Following is an example of a 4×2 pixel, 32-bit bitmap with opacity values in the alpha channel (Windows DIB Header BITMAPV4HEADER) with pixel format 8.8.8.8.0 (in RGBAX notation).

Offset	Size	Hex Value	Value	Description

Note that the bitmap data starts with the lower left hand corner of the image.

Usage of BMP format

The simplicity of the BMP file format, and its widespread familiarity in Windows and elsewhere, as well as the fact that this format is relatively well documented and free of patents, makes it a very common format that image processing programs from many operating systems can read and write.

Many older graphical user interfaces used bitmaps in their built-in graphics subsystems; for example, the Microsoft Windows and OS/2 platforms' GDI subsystem, where the specific format used is the ''Windows and OS/2 bitmap file format'', usually named with the file extension of .BMP or .DIB.

While most BMP files have a relatively large file size due to lack of any compression (or generally low-ratio RLE on palletized images), many BMP files can be considerably compressed with lossless data compression algorithms such as ZIP (in extreme cases of non-photographic data, up to 0.1% of original size) because they contain redundant data. Some formats, such as RAR, even include routines specifically targeted at efficient compression of such data.

Related formats

The X Window System uses a similar XBM format for black-and-white images, and XPM (''pixelmap'') for color images. There are also a variety of "raw" formats, which saves raw data with no other information. The Portable Pixmap (PPM) and Truevision TGA formats also exist, but are less often used – or only for special purposes; for example, TGA can contain transparency information.

Numerous other bitmap file formats are in existence, though most are not widely used.

See also

Comparison of graphics file formats

References

External links

For a table view of the bitmap file format at sourceforge

Bitmap File Structure

An introduction to DIBs (Device Independent Bitmaps)

BMP test images

Simple bitmap loader C++ class

The BMP File Format, Part 1 By David Charlap, Dr. Dobb's journal of software tools, Volume 20, March 1995

Category:Graphics file formats Category:Microsoft Windows multimedia technology

ar:بي إم بي (صيغة ملفات) az:BMP bg:BMP ca:BMP cs:BMP da:BMP de:Windows Bitmap es:Windows bitmap fr:Windows bitmap gl:BMP ko:BMP 파일 포맷 id:Bitmap it:Windows bitmap pam:BMP ka:BMP kk:BMP lt:BMP (formatas) hu:BMP nl:BMP (bestandsindeling) ja:Windows bitmap no:Windows Bitmap pl:BMP (format) pt:Device Independent Bitmap ro:BMP (format fișier) ru:BMP sk:BMP (formát) fi:BMP (tiedostomuoto) sv:BMP tr:Bmp uk:BMP vi:BMP yo:BMP file format zh:BMP

A file format is a particular way that information is encoded for storage in a computer file.

Since a disk drive, or indeed any computer storage, can store only bits, the computer must have some way of converting information to 0s and 1s and vice-versa. There are different kinds of formats for different kinds of information. Within any format type, e.g., word processor documents, there will typically be several different formats. Sometimes these formats compete with each other.

File formats are divided into proprietary and open formats.

Generality

Some file formats are designed for very particular sorts of data: PNG files, for example, store bitmapped images using lossless data compression. Other file formats, however, are designed for storage of several different types of data: the Ogg format can act as a container for many different types of multimedia, including any combination of audio and/or video, with or without text (such as subtitles), and metadata. A text file can contain any stream of characters, encoded for example as ASCII or Unicode, including possible control characters. Some file formats, such as HTML, Scalable Vector Graphics and the source code of computer software, are also text files with defined syntaxes that allow them to be used for specific purposes.

Specifications

Many file formats, including some of the most well-known file formats, have a published specification document (often with a reference implementation) that describes exactly how the data is to be encoded, and which can be used to determine whether or not a particular program treats a particular file format correctly. There are, however, two reasons why this is not always the case. First, some file format developers view their specification documents as trade secrets, and therefore do not release them to the public. Second, some file format developers never spend time writing a separate specification document; rather, the format is defined only implicitly, through the program(s) that manipulate data in the format.

Using file formats without a publicly available specification can be costly. Learning how the format works will require either reverse engineering it from a reference implementation or acquiring the specification document for a fee from the format developers. This second approach is possible only when there ''is'' a specification document, and typically requires the signing of a non-disclosure agreement. Both strategies require significant time, money, or both. Therefore, as a general rule, file formats with publicly available specifications are supported by a large number of programs, while non-public formats are supported by only a few programs.

Patent law, rather than copyright, is more often used to protect a file format. Although patents for file formats are not directly permitted under US law, some formats require the encoding of data with patented algorithms. For example, using compression with the GIF file format requires the use of a patented algorithm, and although initially the patent owner did not enforce it, they later began collecting fees for use of the algorithm. This has resulted in a significant decrease in the use of GIFs, and is partly responsible for the development of the alternative PNG format. However, the patent expired in the US in mid-2003, and worldwide in mid-2004. Algorithms are usually held not to be patentable under current European law, which also includes a provision that members "shall ensure that, wherever the use of a patented technique is needed for a significant purpose such as ensuring conversion of the conventions used in two different computer systems or networks so as to allow communication and exchange of data content between them, such use is not considered to be a patent infringement", which would apparently allow implementation of a patented file system where necessary to allow two different computers to interoperate.

Identifying the type of a file

A method is required to determine the format of a particular file within the filesystem—an example of metadata. Different operating systems have traditionally taken different approaches to this problem, with each approach having its own advantages and disadvantages.

Of course, most modern operating systems, and individual applications, need to use all of these approaches to process various files, at least to be able to read 'foreign' file formats, if not work with them completely.

Filename extension

One popular method in use by several operating systems, including Windows, Mac OS X, CP/M, DOS, VMS, and VM/CMS, is to determine the format of a file based on the section of its name following the final period. This portion of the filename is known as the filename extension. For example, HTML documents are identified by names that end with .htm (or .html), and GIF images by .gif. In the original FAT filesystem, filenames were limited to an eight-character identifier and a three-character extension, which is known as 8.3 filename. Many formats thus still use three-character extensions, even though modern operating systems and application programs no longer have this limitation. Since there is no standard list of extensions, more than one format can use the same extension, which can confuse the operating system and consequently users.

One artifact of this approach is that the system can easily be tricked into treating a file as a different format simply by renaming it—an HTML file can, for instance, be easily treated as plain text by renaming it from filename.html to filename.txt. Although this strategy was useful to expert users who could easily understand and manipulate this information, it was frequently confusing to less technical users, who might accidentally make a file unusable (or 'lose' it) by renaming it incorrectly.

This led more recent operating system shells, such as Windows 95 and Mac OS X, to hide the extension when displaying lists of recognized files. This separates the user from the complete filename, preventing the accidental changing of a file type, while allowing expert users to still retain the original functionality through enabling the displaying of file extensions.

A downside of hiding the extension is that it then becomes possible to have what appear to be two or more identical filenames in the same folder. This is especially true when image files are needed in more than one format for different applications. For example, a company logo may be needed both in .tif format (for publishing) and .gif format (for web sites). With the extensions visible, these would appear as the unique filenames "CompanyLogo.tif" and "CompanyLogo.gif". With the extensions hidden, these would both appear to have the identical filename "CompanyLogo", making it more difficult to determine which to select for a particular application.

A further downside is that hiding such information can become a security risk. This is because on a filename extensions reliant system all usable files will have such an extension (for example all JPEG images will have ".jpg" or ".jpeg" at the end of their name), so seeing file extensions would be a common occurrence and users may depend on them when looking for a file's format. By having file extensions hidden a malicious user can create an executable program with an innocent name such as "Holiday photo.jpg.exe". In this case the ".exe" will be hidden and a user will see this file as "Holiday photo.jpg", which appears to be a JPEG image, unable to harm the machine save for bugs in the application used to view it. However, the operating system will still see the ".exe" extension and thus will run the program, which is then able to cause harm and presents a security issue. To further trick users, it is possible to store an icon inside the program, as done on Microsoft Windows, in which case the operating system's icon assignment can be overridden with an icon commonly used to represent JPEG images, making such a program look like and appear to be called an image, until it is opened that is. This issue requires users with extensions hidden to be vigilant, and never open files which seem to have a known extension displayed despite the hidden option being enabled (since it must therefore have 2 extensions, the real one being unknown until hiding is disabled). This presents a practical problem for Windows systems where extension hiding is turned on by default.

Internal metadata

A second way to identify a file format is to store information regarding the format inside the file itself. Usually, such information is written in one (or more) binary string(s), tagged or raw texts placed in fixed, specific locations within the file. Since the easiest place to locate them is at the beginning of it, such area is usually called a ''file header'' when it is greater than a few bytes, or a ''magic number'' if it is just a few bytes long.

File header

First of all, the meta-data contained in a file header are not necessarily stored only at the beginning of it, but might be present in other areas too, often including the end of the file; that depends on the file format or the type of data it contains. Character-based (text) files have character-based human-readable headers, whereas binary formats usually feature binary headers, although that is not a rule: a human-readable file header may require more bytes, but is easily discernable with simple text or hexadecimal editors. File headers may not only contain the information required by algorithms to identify the file format alone, but also real metadata about the file and its contents. For example most image file formats store information about image size, resolution, colour space/format and optionally other authoring information like who, when and where it was made, what camera model and shooting parameters was it taken with (if any, cfr. Exif), and so on. Such metadata may be used by a program reading or interpreting the file both during the loading process and after that, but can also be used by the operating system to quickly capture information about the file itself without loading it all into memory.

The downsides of file header as a file-format identification method are at least two. First, at least a few (initial) blocks of the file need to be read in order to gain such information; those could be fragmented in different locations of the same storage medium, thus requiring more seek and I/O time, which is particularly bad for the identification of large quantities of files altogether (like a GUI browsing inside a folder with thousands or more files and discerning file icons or thumbnails for all of them to visualize). Second, if the header is binary hard-coded (i.e. the header itself is subject to a non-trivial interpretation in order to be recognized), especially for metadata content protection's sake, there is some risk that file format is misinterpreted at first sight, or even badly written at the source, often resulting in corrupt metadata (which, in extremely pathological cases, might even render the file unreadable anymore).

A more logically sophisticated example of file header is that used in wrapper (or container) file formats.

Magic number

One way to incorporate such metadata, often associated with Unix and its derivatives, is just to store a "magic number" inside the file itself. Originally, this term was used for a specific set of 2-byte identifiers at the beginning of a file, but since any undecoded binary sequence can be regarded as a number, any feature of a file format which uniquely distinguishes it can be used for identification. GIF images, for instance, always begin with the ASCII representation of either GIF87a or GIF89a, depending upon the standard to which they adhere. Many file types, most especially plain-text files, are harder to spot by this method. HTML files, for example, might begin with the string <html> (which is not case sensitive), or an appropriate document type definition that starts with <!DOCTYPE, or, for XHTML, the XML identifier, which begins with <?xml. The files can also begin with HTML comments, random text, or several empty lines, but still be usable HTML.

The magic number approach offers better guarantees that the format will be identified correctly, and can often determine more precise information about the file. Since reasonably reliable "magic number" tests can be fairly complex, and each file must effectively be tested against every possibility in the magic database, this approach is relatively inefficient, especially for displaying large lists of files (in contrast, filename and metadata-based methods need check only one piece of data, and match it against a sorted index). Also, data must be read from the file itself, increasing latency as opposed to metadata stored in the directory. Where filetypes don't lend themselves to recognition in this way, the system must fall back to metadata. It is, however, the best way for a program to check if a file it has been told to process is of the correct format: while the file's name or metadata may be altered independently of its content, failing a well-designed magic number test is a pretty sure sign that the file is either corrupt or of the wrong type. On the other hand a valid magic number does not guarantee that the file is not corrupt or of a wrong type.

So-called shebang lines in script files are a special case of magic numbers. Here, the magic number is human-readable text that identifies a specific command interpreter and options to be passed to the command interpreter.

Another operating system using magic numbers is AmigaOS, where magic numbers were called "Magic Cookies" and were adopted as a standard system to recognize executables in Hunk executable file format and also to let single programs, tools and utilities deal automatically with their saved data files, or any other kind of file types when saving and loading data. This system was then enhanced with the Amiga standard Datatype recognition system. Another method was the FourCC method, originating in OSType on Macintosh, later adapted by Interchange File Format (IFF) and derivatives.

External metadata

A final way of storing the format of a file is to explicitly store information about the format in the file system, rather than within the file itself.

This approach keeps the metadata separate from both the main data and the name, but is also less portable than either file extensions or "magic numbers", since the format has to be converted from filesystem to filesystem. While this is also true to an extent with filename extensions — for instance, for compatibility with MS-DOS's three character limit — most forms of storage have a roughly equivalent definition of a file's data and name, but may have varying or no representation of further metadata.

Note that zip files or archive files solve the problem of handling metadata. A utility program collects multiple files together along with metadata about each file and the folders/directories they came from all within one new file (e.g. a zip file with extension .zip). The new file is also compressed and possibly encrypted, but now is transmissible as a single file across operating systems by FTP systems or attached to email. At the destination, it must be unzipped by a compatible utility to be useful, but the problems of transmission are solved this way.

Mac OS type-codes

The Mac OS' Hierarchical File System stores codes for ''creator'' and ''type'' as part of the directory entry for each file. These codes are referred to as OSTypes, and for instance a HyperCard "stack" file has a ''creator'' of WILD (from Hypercard's previous name, "WildCard") and a ''type'' of STAK. The type code specifies the format of the file, while the creator code specifies the default program to open it with when double-clicked by the user. For example, the user could have several text files all with the type code of TEXT, but which each open in a different program, due to having differing creator codes. RISC OS uses a similar system, consisting of a 12-bit number which can be looked up in a table of descriptions — e.g. the hexadecimal number FF5 is "aliased" to PoScript, representing a PostScript file.

Mac OS X Uniform Type Identifiers (UTIs)

A Uniform Type Identifier (UTI) is a method used in Mac OS X for uniquely identifying "typed" classes of entity, such as file formats. It was developed by Apple as a replacement for OSType (type & creator codes).

The UTI is a Core Foundation string, which uses a reverse-DNS string. Some common and standard types use a domain called public (e.g. public.png for a Portable Network Graphics image), while other domains can be used for third-party types (e.g. com.adobe.pdf for Portable Document Format). UTIs can be defined within a hierarchical structure, known as a conformance hierarchy. Thus, public.png conforms to a supertype of public.image, which itself conforms to a supertype of public.data. A UTI can exist in multiple hierarchies, which provides great flexibility.

In addition to file formats, UTIs can also be used for other entities which can exist in OS X, including:

Pasteboard data

Folders (directories)

Translatable types (as handled by the Translation Manager)

Bundles

Frameworks

Streaming data

Aliases and symlinks

OS/2 Extended Attributes

The HPFS, FAT12 and FAT16 (but not FAT32) filesystems allow the storage of "extended attributes" with files. These comprise an arbitrary set of triplets with a name, a coded type for the value and a value, where the names are unique and values can be up to 64 KB long. There are standardized meanings for certain types and names (under OS/2). One such is that the ".TYPE" extended attribute is used to determine the file type. Its value comprises a list of one or more file types associated with the file, each of which is a string, such as "Plain Text" or "HTML document". Thus a file may have several types.

The NTFS filesystem also allows to store OS/2 extended attributes, as one of file ''forks'', but this feature is merely present to support the OS/2 subsystem (not present in XP), so the Win32 subsystem treats this information as an opaque block of data and does not use it. Instead, it relies on other file forks to store meta-information in Win32-specific formats. OS/2 extended attributes can still be read and written by Win32 programs, but the data must be entirely parsed by applications.

POSIX extended attributes

On Unix and Unix-like systems, the ext2, ext3, ReiserFS version 3, XFS, JFS, FFS, and HFS+ filesystems allow the storage of extended attributes with files. These include an arbitrary list of "name=value" strings, where the names are unique and a value can be accessed through its related name.

PRONOM Unique Identifiers (PUIDs)

The PRONOM Persistent Unique Identifier (PUID) is an extensible scheme of persistent, unique and unambiguous identifiers for file formats, which has been developed by The National Archives of the UK as part of its PRONOM technical registry service. PUIDs can be expressed as Uniform Resource Identifiers using the info:pronom/ namespace. Although not yet widely used outside of UK government and some digital preservation programmes, the PUID scheme does provide greater granularity than most alternative schemes.

MIME types

MIME types are widely used in many Internet-related applications, and increasingly elsewhere, although their usage for on-disc type information is rare. These consist of a standardised system of identifiers (managed by IANA) consisting of a ''type'' and a ''sub-type'', separated by a slash — for instance, text/html or image/gif. These were originally intended as a way of identifying what type of file was attached to an e-mail, independent of the source and target operating systems. MIME types identify files on BeOS, AmigaOS 4.0 and MorphOS, as well as store unique application signatures for application launching. In AmigaOS and MorphOS the Mime type system works in parallel with Amiga specific Datatype system.

There are problems with the MIME types though; several organisations and people have created their own MIME types without registering them properly with IANA, which makes the use of this standard awkward in some cases.

File format identifiers (FFIDs)

File format identifiers is another, not widely used way to identify file formats according to their origin and their file category. It was created for the Description Explorer suite of software. It is composed of several digits of the form NNNNNNNNN-XX-YYYYYYY. The first part indicates the organisation origin/maintainer (this number represents a value in a company/standards organisation database), the 2 following digits categorize the type of file in hexadecimal. The final part is composed of the usual file extension of the file or the international standard number of the file, padded left with zeros. For example, the PNG file specification has the FFID of 000000001-31-0015948 where 31 indicates an image file, 0015948 is the standard number and 000000001 indicates the ISO Organisation.

File content based format identification

Another but least popular way to identify the file format is to look at the file contents for distinguishable patterns among file types. As we know, the file contents are sequence of bytes and a byte has 256 unique patterns (0~255). Thus, counting the occurrence of byte patterns that is often referred as byte frequency distribution gives distinguishable patterns to identify file types. There are many content based file type identification schemes that use byte frequency distribution to build the representative models for file type and use any statistical and data mining techniques to identify file types

File structure

There are several types of ways to structure data in a file. The most usual ones are described below.

Unstructured formats (raw memory dumps)

Earlier file formats used raw data formats that consisted of directly dumping the memory images of one or more structures into the file.

This has several drawbacks. Unless the memory images also have reserved spaces for future extensions, extending and improving this type of structured file is very difficult. It also creates files that might be specific to one platform or programming language (for example a structure containing a Pascal string is not recognized as such in C). On the other hand, developing tools for reading and writing these types of files is very simple.

The limitations of the unstructured formats led to the development of other types of file formats that could be easily extended and be backward compatible at the same time.

Chunk-based formats

Electronic Arts and Commodore-Amiga pioneered this file format in 1985, with their IFF (Interchange File Format) file format. In this kind of file structure, each piece of data is embedded in a container that contains a signature identifying the data, as well the length of the data (for binary encoded files). This type of container is called a ''"chunk"''. The signature is usually called a chunk id, chunk identifier, or tag identifier.

With this type of file structure, tools that do not know certain chunk identifiers simply skip those that they do not understand.

This concept has been taken again and again by RIFF (Microsoft-IBM equivalent of IFF), PNG, JPEG storage, DER (Distinguished Encoding Rules) encoded streams and files (which were originally described in CCITT X.409:1984 and therefore predate IFF), and Structured Data Exchange Format (SDXF). Even XML can be considered a kind of chunk-based format, since each data element is surrounded by tags which are akin to chunk identifiers.

Directory-based formats

This is another extensible format, that closely resembles a file system (OLE Documents are actual filesystems), where the file is composed of 'directory entries' that contain the location of the data within the file itself as well as its signatures (and in certain cases its type). Good examples of these types of file structures are disk images, OLE documents and TIFF images.

See also

Audio file format

Chemical file format

Container format (digital)

Document file format

DROID file format identification utility

File (command), a file type identification utility

File Formats, Transformation, and Migration (related wikiversity article)

FormatFactory, a free omni file format converter.

Future proofing

Graphics file format summary

List of archive formats

List of file formats

List of free file formats

List of motion and gesture file formats

Magic number (programming)

List of file signatures, or "magic numbers"

Object file

Open format

Windows file types

References

External links

Open Directory Data Format links - File types resources on DMOZ

File Format Identification on the Forensics Wiki

File Extension Seeker - Metasearch engine for file extensions

Category:Computer file formats

ar:صيغة ملف bs:Formati datoteka bg:Файлов формат ca:Format de fitxer cs:Formát souboru de:Dateiformat el:Μορφότυπο es:Formato de archivo informático eu:Fitxategi formatu fa:قالب پرونده fr:Format de données ko:파일 형식 is:Skráasnið it:Formato di file ka:მონაცემთა ფორმატი lb:Dateiformat hu:Fájlformátum nl:Bestandsformaat ja:ファイルフォーマット mhr:Файлформат pl:Format pliku pt:Formato de arquivo ru:Формат файла simple:File format sk:Formát súboru fi:Tiedostomuoto sv:Filformat uk:Формат файлу zh:檔案格式