Gottfried’s NEF file processing Page
... the Nikon D1, in all its raw glory ...
Dedication

This page is dedicated to the Nikon D1 digital camera users who have never opened the raw, .NEF files emanating from their machine and simply accept what the other file formats (TIFF, JPEG) offer. The process described here is offered as an insight along the path the data take, from the 'buckets' of the CCD chip to the 'Photoshop' (or other) graphical file on the personal computer. This is not meant as a substitute for working on NEF files with programs such as:

Nikon View / Nikon Capture, Nikon USA, Bibble/MacBibble, or QImage Pro.


A word about numbers: Hexadecimal numbers are given in the format: 0x00, 0x0000, or 0x00000000.
All others in the usual notation are in base 10 or decimal numbers.

General NEF file format

The Nikon D1 NEF file is 3,995,832 bytes long, and like all Gaul, it is divided into three parts:

  1. front
  2. thumbnail image
  3. 12-bit CCD image


front

For want of a more descriptive name I am calling the first 5120 bytes (0x00000000 - 0x000013FF, preceding the thumbnail data) the 'front'.

This area contains a variety of scalar info: numbers, names, dates, etc. It appears to behave in a 'TIFF' manner. The 'Hex Dump' of this are yields sections similar to the following, composed of character bytes or 'ASCII' strings.

The first two file identifier bytes are: [0x4D 0x4D] 'MM'. 
0x00000000: 0x4D 0x4D 0x00 0x2A 0x00 0x00 0x00 0x08 0x00 0x19 0x00 0xFE 0x00 0x04 0x00 0x00

For example, the hexadecimal zero [0x00] terminated ASCII character type region: 0x00000312 - 0x00000324

0x00000310: 0x39 0x00 0x32 0x30 0x30 0x30 0x3A 0x30 0x39 0x3A 0x30 0x32 0x20 0x31 0x36 0x3A     9.2000:09:02 16:
0x00000320: 0x35 0x35 0x3A 0x31 0x39 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x06 0x00 0x00     55:19...........
contains the date and time of the image capture. 
0x32 0x30 0x30 0x30 0x3A 0x30 0x39 0x3A 0x30 0x32 0x20 0x31 0x36 0x3A 0x35 0x35 0x3A 0x31 0x39 = "2000:09:02 16:55:19"


The region: 0x00000440 - 0x0000046F

0x00000440: 0x43 0x4F 0x4C 0x4F 0x52 0x00 0x52 0x41 0x57 0x32 0x2E 0x37 0x4D 0x00 0x41 0x55     COLOR.RAW2.7M.AU
0x00000450: 0x54 0x4F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 0x20 0x00 0x20 0x20 0x20 0x20 0x20     TO        .     
0x00000460: 0x20 0x00 0x41 0x46 0x2D 0x53 0x20 0x20 0x00 0x20 0x20 0x20 0x20 0x20 0x20 0x20      .AF-S  .
contains zero terminated ASCII character information on image mode: COLOR, image quality: RAW2.7M, exposure mode: AUTO, and lens focus mode: AF-S.

thumbnail image

The first 'vector' organized info is the array of byte size pixels (interleaved).

#define THUMB_Start  0x00001400
#define THUMB_End    0x0000D6FF

0x00001400: 0x14 0x7B 0x36 0x14 0x7B 0x36 0x16 0x7B 0x37 0x17 0x7C 0x38 0x18 0x7D 0x37 0x17
0x00001410: 0x7C 0x36 0x18 0x7D 0x37 0x19 0x7E 0x38 0x1A 0x7F 0x39 0x19 0x7E 0x38 0x19 0x7E
        :
0x0000D6E0: 0x65 0xA8 0x2D 0x68 0xAA 0x2D 0x68 0xAA 0x2D 0x67 0xA7 0x2F 0x69 0xA9 0x2F 0x6B
0x0000D6F0: 0xA9 0x30 0x6C 0xAA 0x34 0x69 0xAB 0x33 0x68 0xAA 0x32 0x67 0xA9 0x32 0x67 0xA9


Below and to the right are thumbnail images of the two test files used in developing this program.
The color rectangles (below) were obtained from an ad hoc arrangement of some colored file folders.

NEF test thumbnail

The image on the right is composed of two not very colorful book covers.

 NEF test thumbnail
12-bit CCD image

The image (QED) is contained in the remaining 3995832 byte long region and is composed of units of 12-bit pixel data:

 column:  0 1 2 3 4 5     The color pixel data is organized as a 2-D array, in the Bayer pattern shown left.

This starts at: 0x0000E8FA and ends at file location: 0x003DE1B1.
That span translates into 3995832 (8-bit) bytes:
((4055473 - 59642) +1) = 3995832 * (8/12) = 2663888 12-bit charge coupled cells,
reflecting the underlying 1324 x 2012 two dimensional detector array.
In the usual marketing translation, 2,663,888 is rounded up to 2.7M pixels.
 line 0   B   G        
 line 1   G   R        
 line 2   B   G        
 line 3   G   R         
#define RAW_start  0x0000E8FA
#define RAW_end    0x003DE1B1

0x0000E8F0: 0x00 0x01 0x00 0x00 0x01 0x2C 0x00 0x00 0x00 0x01 0x13 0xD1 0xDD 0x16 0xE2 0x1D

0x0000E900: 0x16 0xF2 0x27 0x18 0x72 0x12 0x15 0xB2 0x1B 0x15 0xE2 0x05 0x16 0x72 0x04 0x16
0x0000E910: 0xE2 0x27 0x17 0x12 0x14 0x16 0xF2 0x34 0x18 0x91 0xFF 0x18 0x22 0x1F 0x16 0xB2
          :
0x003DE190: 0xDC 0x28 0xF0 0xD9 0x23 0x50 0xCC 0x20 0x20 0xED 0x28 0xE1 0x23 0x23 0xE0 0xCE
0x003DE1A0: 0x21 0xD0 0xDC 0x1E 0x80 0x6D 0x20 0xD0 0xBD 0x1E 0x60 0xE9 0x22 0xC0 0xD7 0x2A
0x003DE1B0: 0xD0 0xFF


The initial nine (8-bit) bytes in this example map the first six 12-bit charge coupled cell contents as follows:

0x0000E8FA: 0x13
0x0000E8FB: 0xD1
0x0000E8FC: 0xDD      ->  0x013D   0x01DD

0x0000E8FC: 0x16
0x0000E8FC: 0xE2
0x0000E8FC: 0x1D                       ->  0x016E   0x021D

0x0000E8FC: 0x16
0x0000E8FC: 0xF2
0x0000E8FC: 0x27                                        ->  0x016F   0x0227


Raw NEF file translation program

The "C" language program consists of three major routines:

  1. main
  2. get_bayer
  3. photoshop_raw


Early Results

RGB Pattern NEF file results

Initial Nikon D1 test photograph files :
NEF RAW test pattern

Continuous Tone NEF file results
NEF test book covrs

Natural light photos, both a bit darker than the original images.


To return to the home page just press Return ... still considerable work to be done here.
Modified: 12 September 2000 by: Gottfried Hogh