Expeed logo

The Nikon Expeed image processing engines (often styled EXPEED) are signal processors for Nikon's digital cameras. They perform a high number of tasks: Bayer filtering, demosaicing, image sensor corrections/dark-frame subtraction, image noise reduction, image sharpening, image scaling, gamma correction, image enhancement/Active D-Lighting, colorspace conversion, chroma subsampling, framerate conversion, lens distortion/chromatic aberration correction, image compression/JPEG encoding, video compression, display/video interface driving, digital image editing, face detection, audio processing/compression/encoding and computer data storage/data transmission.

The used multi-processor system on a chip solution integrates an image processor in multi-core processor architecture, with even each single processor-core able to compute many instructions/operations in parallel. Storage- and display-interfaces and other modules are added and a digital signal processor (DSP) increases the number of simultaneous computations. An on-chip 32-bit microcontroller initiates and controls the operation and data transfers of all processors, modules and interfaces and can be seen as the main control unit of the camera. The Expeed is an application-specific integrated circuit (ASIC) built by Fujitsu specifically for Nikon designs according to Nikon specifications.

Technology[link]

The Nikon Expeed are based on the Fujitsu Milbeaut imaging processors with 16-bit per pixel^[1] multi-core FR-V processor architecture, using a highly parallel, pipelined architecture which allows efficient hardware use, increasing throughput and reducing power consumption. Each core is using an 8-way 256-bit very long instruction word (VLIW, MIMD) and is organized in a 4-unit superscalar pipelined architecture giving a peak performance of up to 28 instructions per clock cycle and core. Due to the used 4-way single instruction, multiple data (SIMD) vector processor units, data is processed with up to 112 data-operations per cycle and core.^[2]

An on-chip 32-bit Fujitsu RISC (FR) microcontroller core is used to initiate and control all processors, modules and interfaces.^{[3][4][5][6][7]} The Expeed versions designated EI-14x and the Expeed 2 and 3 additionally include a HD video codec engine (FR-V based) and an 16-bit DSP with separate on-chip 4-block harvard RAM which is usable for example for additional image- and audio-processing. The Expeed 3 offers a highly increased speed in its image processor (with even two pipelines on the EI-160), its H.264 video encoder and is controlled with a dual-core ARM architecture microcontroller.^[8][9]

Image sensor interface[link]

CMOS image sensor connection is done by a mixed analog/digital interface which controls the sensor digitally, but receives analog signals with parallel on-chip 14-bit analog-to-digital (A/D) converters. The Expeed variants EI-142 and EI-158 connect all full-frame (FX) digital SLR sensors and additionally the Nikon D300/D300s with 12 simultaneous, parallel analog signal readout channels. Mainly due to a larger settling time it allows improved conversion accuracy compared to the 4-channels in the previous Nikon D2X / D2Xs, Nikon D200 or the Canon EOS 5D Mark II.^[10][11][12] 6-channel readout is featured by the EI-149 and EI-154 used on the D90, D5000, D7000 and D5100.

The Expeed 2 allows an increased image sensor readout clock frequency with improved A/D converter accuracy, especially when using 14-bit. Expeed A/D converters like EI-149 or all EI-142 need considerably reduced clock rates (1.8 fps on Nikon D3X) for higher accuracy, limiting for example the D3s dynamic range at low ISOs.^[13][14][15] The Expeed 3 variant EI-160, first used in the Nikon 1 series, connects a data stream with 24 digital channels (bus), using A/D converters integrated on the image sensor chip.^[16]

Variants[link]

Fujitsu specifies each Milbeaut generation with different numbers of processors. Nikon gives no details, but uses different designated processors in its professional and consumer lines. Although the Milbeaut (Expeed) is used in different Nikon designs and by other manufacturers, the software/firmware specifies many of its functions and details and the number of processors or included modules may vary in this ASIC.

Nikon Expeed EI-142

Nikon D700 Motherboard with Expeed EI-142

Expeed[link]

First used in the Nikon D3 and Nikon D300 in 2007. Later additionally in the Nikon D3X, Nikon D700 and Nikon D300s, marked EI-142, and Nikon D90 and Nikon D5000, marked EI-149. It is based on an Fujitsu Milbeaut imaging-processor with 720p Motion JPEG video encoder, DSP and FR-80 (EI-14x versions) core.

The variant EI-137 in the Nikon D60 and Nikon D3000 - additionally found in the Nikon D40, Nikon D40x and Nikon D80^[6] - is based on the older Milbeaut M-3 with FR-70 core with only 12-bit, 2-channel image sensor readout and features are reduced.^[17]

Expeed 2[link]

1080p H.264/MPEG-4 HD video encoder, improved face detection, image noise reduction and image distortion correction^[18] are the main improved features compared to EI-14x Expeeds.^[3] Used in the Nikon D7000,^[19] Nikon D3100 and Nikon D5100 and Nikon marked EI-154.^[20] Although image sensor readout clock frequency has increased by a factor of 1.75, A/D converter accuracy is improved, especially when using 14-bit. Image processor performance is increased, performing a higher continuous shooting frame rate even when high ISO noise reduction or Active D-Lighting is activated. The Expeed EI-15x are controlled by an integrated FR-81 core.

The Nikon D3s processor^[21] - also named Expeed 2 - uses nearly the same EI-142 processor as the previous D3/D3X^[22] featuring for example only 720p Motion JPEG video with the advantage of easy JPEG image extraction, and the disadvantage of larger file sizes. It offers the same image sensor interface with identical speed and A/D converter accuracy, limiting the D3s dynamic range at 200 and especially 100 ISOs lower as the D7000/D5100.^[23][13][14]

Expeed C2[link]

Variant used in some Coolpix compact cameras.

Expeed 3[link]

High-speed dual image-processing engines with 600 megapixels per second speed, enhanced H.264 HD video engine and controlled by a dual-core ARM microcontroller.^[24][16][9] Nikon marked EI-160. First used in the Nikon 1 series September 2011 which Nikon claims has world's fastest speed of 60 frames per second (10 fps with full autofocus). In the Nikon 1 series it uses 1GByte fast DDR2 RAM packaged in 2x 4Gbit chips. It is the first Expeed using digital image sensor readout. The new ARM microcontroller requires new firmware compared to the different Fujitsu FR microcontroller, which were used in all former Milbeaut and Expeed processors.

The name is also used for the EI-158 variant in the Nikon D4^[25] and Nikon D800.^[26] Compared to the previous EI-142, it offers the same improvements as the Expeed 2 EI-154, including increased A/D converter accuracy and image sensor analog signal readout clock rate, reducing rolling shutter.^{[27][13][14][15]}

The processor of the Nikon D3200 is also named Expeed 3.

Pre-Expeed[link]

Unofficial Expeed DSLRs[link]

The Expeed processor variant EI-137 is found in the Nikon D40, Nikon D40x and Nikon D80 - as it is officially in the later released Nikon D60 and Nikon D3000.

Early DSLRs[link]

DSLRs announced before August 2006 are not containing processors named Expeed (for example Nikon D70/D70s processor: EI-118), although that does not mean that these processors use a different architecture.^[28] Or the Nikon D200 processor (EI-126) uses the same, but greatly expanded firmware as the D80 (inofficial "Expeed").

Additional (co-)processors[link]

As input/output (I/O) processor to connect additional sensors and displays Nikon uses external 32-bit microcontrollers: The Fujitsu FR in the Expeed EI-14x series, and changed to MIPS architecture in the Expeed EI-15x series. The Expeed EI-160 in the Nikon 1 series with its dual ARM core needs no additional I/O processors.^[29] Former DSLRs used Hitachi microcontrollers.

The Nikon 1 series includes an Epson graphic processor.

Alternative firmware[link]

As with CHDK/Magic Lantern modified Canon digital cameras based on DIGIC processors, attempts are being made to write custom firmware^[5] with some recent progress including an FR emulator for some DSLRs.^{[30][31][32][33]} It was shown that Nikon uses the SOFTUNE Integrated Development Environment together with an µITRON realtime kernel.^[28] Currently there is some modified,^[34] but no stable alternative firmware available, as the project is still in a very early state.

DIAG raw hack[link]

This was not a firmware hack, but a method (Photopc – digital camera control^[35]) to call an already implemented diagnostic mode on some old Nikon Coolpix cameras with Fujitsu Sparclite processor. In this mode they could write raw images.

References[link]

External links[link]

• Nikon: Expeed
• Nikon: What is the difference between Expeed and Expeed2?