Rawnalyze

Raw Image Analyzer User's Guide


Rawnalyze is a computer program devised for the support of detailed analysis of raw images created by digital cameras. Click here to read the general description, limitations, downloading, etc.

Table of Content


Note: the term color often occurs in this guide relating to pixels of the raw image. The exact meaning is raw pixel with a specific color filter. It is important to understand, that a raw pixel alone does not represent a specific color. The color of the captured scenery can be evaluated only in conjunction with the surrounding pixels of other colors.

If you have suggestions, critiques regarding the program or this manual, experienced some phenomenon looking like a program error or want to express your opinion, please email to

If you want to be notified about new versions of this program, drop a line to


The dialog

 

Dialog elements

After starting Rawnalyze, following dialog window appears:

The capture has been created in the initial state


The first step after having started the program is usually locating the raw file and opening it. The file specification can be typed in directly; clicking on Process starts opening and reading the file. However, more often the Browse subdialog will be used to select the raw file to process. After a file has been selected, it will be opened and read immediately, there is no need to click on the Process button; in fact, there is no need to click on Process any more while analyzing the same raw image. The only exception is, when switching between analyzing modified and unmodified raw data, see Changing the original raw data later. Clicking on Process initiates opening and reading the raw file again, causing a considerable delay in response time.

The shortcut for starting the file browse dialog is Ctrl-O.

Displaying the manual through a Web browser via the Help function may become necessary, but if someone is reading this page, then that goal has been accompished already.

As there is not much else to say about this screen shot, let's skip ahead through opening an actual raw file.


We go through the components of this window, so that we can refer to the controls in the following chapters.

  1. The most important selection after the file is the display mode:

    The initial selection is composite. Any selection remains effective after opening another image.

  2. The black arrow represents the image orientation, relating to the normal orientation of the camera, which is landscape for most cameras (wider side horizontal), but some cameras' normal orientation is portrait.
    denotes the normal orientation,
    denotes the left turned orientation, which is how most cameras are held for portrait orientation,
    denotes the right turned orientation, which is how a few people are shooting in portrait mode,
    denotes the upside down orientation.

    The initial orientation is the one recorded in the raw file. If the orientation is not recorded, then the initial orientation is upright. When clicking on an arrow, the displayed orientation is changed to the one corresponding to that arrow.

    The image can be rotated by 90° to the left with the keyboard action Ctrl-L and 90° to the right with the keyboard action Ctrl-R.

  3. The black point control group and the white point control group are explained further down, at Pixel selection.
  4. The unlabelled slider is for lightness adjustment. The adjustment can be made at 1/3 EV, 1/2 EV or 2/3 EV points. (Note, that this adjustment is often referred to as exposure adjustment, however that is an incorrect usage of the term, for the exposure can not be adjusted after the shot has been made.)

    The initial position of this slider is usually at null, but in some cases it shows an adjustment, which is indicated in the raw file. Example: Canon 40D images shot with the Highlight Tone Priority (HTP) option start with an adjustment of +1.

  5. Depending on the selected display mode, different checkboxes appear as selection of further options. These are described with the related display mode.
  6. Selection of the mapping function decides, which mathematical function will be applied, when transforming the linear raw data in a more displayable form in composite color, channel or greyscale mode. Some of the functions are designed to vastly increase the contrast in the highlights, while creating strange side effects.

    Note, that only the color intensity of the displayed image is affected by the selected function. The colors themselves are of the camera's color space and don't get transformed in sRGB, etc.

  7. The shooting data consists of the camera maker and model, camera body serial number, date and time of the shooting, image dimensions, ISO, aperture, shutter time and lens focal length; some of these may be missing.

    Notes:

  8. The statistics under Black Point shows the number and proportion of the selected pixels in the lower range. This statistics is affected by the black point value and by the Outside range selection.
  9. The statistics under White Point shows the number and proportion of the selected pixels in the upper range. This statistics is affected by the white point value and by the Outside range selection.

    Notes on the pixel statistics:

  10. The current zooming indicator; for example indicates 3x magnification, while shows, that the reduced size image is displayed.
  11. The position and size of the current area selection, see Area selection below.
  12. Pixel statistics of the current area selection, see Area selection below.

 

Resizing the dialog window

Beside being minimized, maximized and restored, the dialog frame can be resized by dragging the right or the bottom edge, or the bottom right corner. The new size can not be narrower, nor shorter than the respective initial dimension.

The size of the image window changes together with the dialog frame size; this way the image can be viewed with less scrolling in a larger image window. Here are two examples for different dialog frame sizes:

 

Magnifying, reducing the image size

In the normal view each pixel of the raw image is displayed as a pixel on the monitor. The image can be magnified by typing in + (plus) character; repeated typing can increase the magnification to up to ten times.

Typing in the - (minus) character reduces the magnification. In composite color and greyscale display mode the display size can be effectively reduced, so that each color filter array block is represented by a single pixel.

The normal view (no magnification, no reduction) can be reinstated immediately by entering Alt, Ctrl and S.

The current magnification/reduction state is displayed above the top left corner of the image frame.

Examples:

This is the image from the selection example below in reduced size

and here the same image in six times magnification

 

Scrolling, positioning the image

Raw images from newer cameras are quite large, some of them are very large. Looking through the image, even in reduced size view can be tedious. A multitude of scolling/positioning options are available to make this task faster.

 

Working with the sliders

A slider can be adjusted by dragging it: left-clicking on the slider pointer, holding down the left button and moving the mouse. Clicking on the trackbar (the line going in the length of the slider) at the left or right to the slider pointer moves the slider by one "page".

While the focus is on a slider, it can be moved just like a scroll bar, using the arrow keys, PageUp, PageDown, Home and End. The magnitude of change the arrow keys or PageUp and PageDown cause depends on the slider.

The slider becomes focused after having clicked on any part of it, and it remains focused until clicking on another control (a button, an input field, etc.) If the slider is focused, it is framed by a thin dotted rectangle, like this one:

Each sliders is accompanied by a Reset button, which sets the slider to its initial position.

There are other, very special ways to control the black point and white point values, and thereby the respective sliders; those are described in the Pixel selection.

 

Changing the original raw data

The raw data remains unchanged in memory in the entire duration of the analysis. However, the raw pixel values may be changed on a particular way immediately after having read the data. That change can not be reversed, the raw file has to be opened and read again in order to work with the original rax pixel values.

Following options are available:

The special key combination has to be kept pressed down when clicking on the Process button or when starting processing of a raw file in the Browse dialog. It's enough to keep these keys pressed one or two seconds long.

Note, that

 

Pixel selection

An important aspect of the raw pixel analysis is determining, which pixels should be involved in it and how. The selection is based on the color and value of raw pixels.

The effect of pixel color and value selection is described in other chapters, here we deal only with the way the selection occurs.

The tools of pixel selection are:

 

Area selection

When displaying the image in normal size, a rectangular area of the display can be selected by right-clicking at a point of the display, holding down the right mouse button and dragging the selection in the direction and to the size needed. The selected area is marked by an orange rectangle. Another selection can be made at any time; the previous selection will be released. The selection can be removed by the keyboard action Ctrl-D.

The unit of selection is the color filter array pattern, i.e. a block of 2x2 pixels, not individual pixels. Thus a single pixel can not be selected.

The selection remains active between different display modes. When changing to histogram mode or to reduced size display, the selection remains defined but not displayed, nor is the related statistics displayed. When changing back to normal size display or to image or exposure display mode from histogram display, the selection will be shown again with the related statistics.

It is important to understand, that a part of the display area is selected, not a part of the image. When the image position is changed, the selection remains on the very same place, i.e. another part of the image gets selected, and the statistics is recalculated.

The selection remains unchanged even when opening another image; this is useful when comparing two images of the same scenery.

The use of a selection is:

Following screen capture helps explaining the display associated with a selection:

The orange colored rectangle frames the selected area;
the red rectangles mark the associated information.
The red markings do not appear on the dialog window, they are drawn only for the sake of explanation.


Image display

 

Display modes

There are three ways the image data can be displayed resembling a "proper" image:

 

The effects of settings

It is most important to understand the role of the settings in the process of converting the raw pixel value in the RGB value. The effects of the settings/selections are the same in the three image display modes, and the same effects are reflected by the histogram of mapped data.


Exposure display

 

The exposure display mode is another way to analyze the raw image: selected pixels are displayed with red, green or blue, according to the pixel's position in the color filter array block; however, the displayed intensity is fixed, that does not represent the raw pixels' intensity (i.e. the raw pixel value). In other words: the selection does depend on the raw pixel values, but the pixels appearance does not.

Such a display resembles a "proper" image less than the image display modes, but there is no problem with recognizing a pixel of low value.

The way of pixel selection is the same as that in image display mode, except

An entire color channel (even one of the green channels separated from the other) can be excluded from the display by setting the corresponding white balance coefficient to zero; see the detailed description of the white balance feature in the Image Display section.

The exposure display offers an additional choice: if the BL corrected checkbox is checked, then the black level corrected pixel values are compared against the selection range limits (this is independent on the black point setting); otherwise the original raw pixel values are used in the selection.

The exposure analysis is done best in maximized window: here the small images are presented.

Example 1:

Some cameras offer ISO settings at 1/3 stops as well, but are those "real ISOs"? Let's take an image, shot with ISO 125 and look for a pixel value range, which contains many pixels (the histogram helps finding such ranges). Type in a pixel value in the black point group (don't forget to Enter it) and enter the same value in the white point group as well; after pressing Enter, the control remains in that field. The level 1901 has been selected in both groups. Only red pixels will be displayed at the moment; the color needs to be selected only in the black point group, as that governs the values up to 2047.


All the red pixels we can see now have the value 1901; the statistics shows, that there are 666 such pixels in the entire image, and 1470 greens (and no blues, this is suspicios)

As the control is still in the level input field (no matter if in the black point or white point group), pressing the right cursor increases both limits by one.
Now we see the red pixels with value 1902; there are 762 such in the image

Then we do the same with 1903 and 1904

This shows the red pixels with the value 1904; there are 712 such red pixels in the image

And now the surprize with 1905:

There are no red pixels with the value 1905 in the entire image

If we keep on stepping through the values, we find that regularly every fourth/fifths value (alternating) is a "gap" in either the red or blue pixels. This indicates, that the ISO 125 values have been created by multiplying the ISO 100 values. There is no visible gap in the green pixel values, because the gaps are alternating between the two green positions of the color filter array block.

Example 2: the saturation level of a certain camera.

The histogram shows a strange behaviour of the green pixels when clipping: the clipping does not occur at once, it occupies a range of pixel values

Let's find out, how the clipping occurs with this camera. Outside range is checked, only the green pixels in the higher range are shown.

There are no pixels with 4026 and above

but there are some with 4025, so that is that ultimate clipping point here

by lowering the upper limit, the outline of a block appears

one level lower the block starts to fill out

only half of the green pixels appear in the upper part, and a quarter of them in the lower part

until the entire block is filled, but only with one of the two green pixels in each color filter array block

then from 4009 the other green pixels are appearing

and at 4001 the block is almost completely filled

The analysis of the entire image reveals, that lower limit of clipping is 3995. The difference between values in the range from 3995 to 4025 is meaningless, i.e. these pixel values do not represent true proportions of the lightness of the captured scenery. The lower limit has to be seen as saturation level.

The importance of knowing the saturation level lies in the clipping indication in image display mode.


Histogram display

There are several ways the distibution of pixel values can be presented. Common between these histograms is, that they are always based on non-demosaiced pixel values, representing the three "channels" of the sensor, and all pixels are involved in the calculation (as opposed to sampling), therefor the histograms are always accurate.

The basic raw histogram

The basic raw histogram will be shown if neither the Mapped nor the Fine checkbox are checked.

In basic mode the histogram represents the entire range of unmodified raw pixel values. The histogram consists of 512 columns; as the pixel value range is alway larger than 512, one column of the histogram represents a number of pixel values. For example if the numerical range of the pixel values is 0-3900, then each each column represents alternatingly seven or eight consecutive pixel values. If the value range is 4096 (12 bit depth), then each subrange is 8 levels wide; the first column represents the levels 0 to 7, the second one the values 8 to 15 and the 512th column represents the pixel values 4088 to 4095.

Note, that the numerical pixel value range is not always the same as the numerical range of the bit depth. Some cameras create raw data, which fully occupies the range of the bit depth, while the numerical pixel value range with some other cameras (for example Canon DSLRs) can be substantially smaller than the range given by the bit depth. In some cases the range depends on the ISO and/or on the channel. The actual range used in the histogram is declared in the camera's description, which is hard coded in Rawnalyze; it is equal to or slightly greater that the initial white point, and it is always the same for all three channels, even if the saturation levels of the channels are different.

The height of the histogram is 128 pixels; the height of a column represents the number of pixels with the value falling into the subrange of that column in proportion of all pixels of that color. If the sensor contains 8 million pixels, 2 million of those are capturing the red range. If there are 100,000 pixels in the value range 1000 to 1007, then the 126th pixel column of the histogram represents 5% of all red pixels, assumed the range of pixel values is 0-4095.

However, the height of the pixel column is not in direct proportion to the percentage of pixels it is representing, because that way small pixel amounts would not be visible - or the histogram would have to be several thousand pixels tall. Instead, the hight represents the logarythm of the percentage.

A two pixels tall column represents 0.01%; a column as tall as the first white marker from the bottom upwards represents 0.05%, the hight of the next marker is at 0.1%, etc. It is not easy to determine exactly how many pixels are in a column, but it is not important either.

The meaning of a single-pixel column is, that there is at least one pixel in that range, but less than 0.01%. Note, that this can be triggered even by a "hot" or "stuck" pixel.

The short white lines under the histogram mark the exposure value oriented distribution in the levels. If one imagines the right end as EV 0, then the first marker from the right side represents the 1/3 EV lower point, the second marker is at -2/3 EV, the third, somewhat larger marker is at -1 EV, etc. Obviously the range of the highest stop occupies half on the entire histogram.

The yellow dotted pixel column close to the left edge marks the black level (the minimum of the black correction values), the one close to the righ end marks the saturation level. The black level, if visible, is at the same location in all three histrograms; the location of the saturation marker may be different for the colors, depending on the sensor.

The left turquoise dotted pixel column (this does not have to be at the left side) marks the black point, the value is shown in the input field of the black point control group; the right turquoise pixel column represents the white point, the value is displayed in the input field of the white point control group. These two can be at the same location.

If an indicator pixel column is not visible, then it can be imagined at the left respectively right edge of the histogram.

The fine raw histogram

The basic raw histogram conveys an overview of the exposure. This is suitable in many cases; however, it is somewhat coarse, because each column represents a range of pixel values. The fine raw histogram shows the detailed distibution of pixel levels, as each column of the histogram represents a single pixel level. The fine raw histogram will be shown if the Fine checkbox is checked.

The histogram consists of 512 columns, each maximum 128 pixels tall. These 512 columns show a "crop" of the complete fine histogram, starting with the pixel level indicated in the black point field. As the black point value is changed, the histogram's range moves in the total pixel value range, i.e. it represents the crop starting with the new black point value.

The height of the a column represents the number of pixels with the respective value in proportion of all pixels of that color, relative to the total pixel value range, on the same logarythmic scale as in the basic raw histogram.

The white markers at the bottom of the histograms indicate the locations of 1/10th of the 512 wide range. In other words, the markers are in 51 pixels distances. The position of a marker added to the current black point yields the pixel value represented by the column over the marker. For example if the current black point value is 128, then the second marker indicates the position correspoding to the pixel value 230.

Gaps in the histogram indicate, that there is no pixel with the respective value. The gaps in the above example are due to the lossy compression of the image file.

The mapped histogram

If the Mapped checkbox is checked, then the histogram represents the pixel values after following adjustments:

This histogram too is based on non-demosaiced data, i.e. the red, green and blue histograms still represent the pixels with the corresponding color filters. Thus this this histogram reflects the image displayed in composite, channel or greyscale mode, except for the clipping indication.

The appearance of the mapped histogram differs from the basic raw histogram in following aspects:

A particular combination of setting worth of mentioning is, when the black point and white point are at their initial values, the lightness adjustment is null, the mapping method is "Linear" and the white balance is set. The mapped histogram with white balance applied appears like a white balanced raw histogram.

Examples

This is the initial state of the raw histogram of an image created by a Canon 40D. The saturation level is 12740 at ISO 160, and the histogram range ends at 12800.
The first dotted column contains both the black level and black point indicators, the second one contains both the white point and the saturation level indicators (the saturation levels happen to be the same for all three colors in this case).

The histogram shows, how well (or not) exposed the shot was. In this case only slightly more than one third of a stop could have been added to the exposure without causing clipping, even though the shot is strongly underexposed (the dynamic range exceeded the camera's capturing capacity)

This is the typical sign of clipping caused by specular spots or direct light; in this case the sun was the cuplrit, but it caused less than 0.1% of all pixels clipping

The saturation levels of the three colors are very different with this camera.
Over 10% of the pixels clipped

This is the effect of the lossy raw compression by some cameras, for example Nikon: groups of levels are simpy left out. The range of these group is increasing at higher pixel values. Therefor there are no gaps, only spikes at the dark end, but the ranges of "gaps" are becoming larger towards the bright end, so far, that there is no pixel in the entire subrange associated with some columns of the histogram, causing the gaps in the histogram

The effect of the lossy raw compression up-close, pixel level by pixel level

The mapping of the six hundred some different raw values to the 256 RGB values eliminates the gaps, but some strange spikes are still appearing

The application of white balance blends the levels more; these spikes are not perceivable on the resulting image when displaying or printing

The Stouffer wedge is a colorless transparent consisting of strips with transparency 1/3 stop apart. Despite looking boring as a photographic object, it is useful in the explanation of some subjects.

This is a crop of the image before white balancing. As the strips are colorless, they ought to appear grey; as such they are suitable for white balance picking

This is the raw histogram; the spikes/bumps represent the strips. Had they been captured as grey, i.e. red, green and blue with equal intensity, the spikes/bumps would line up nicely

The histogram of the mapped data without white balance too shows a shift between the intensities of the colors of strips

White balancing multiplied the red pixel values by 1.09 and the green ones by 0.6094, causing the three spikes from the strips lining up

As the result, the strips now appear grey

Linear mapping of the white balanced image shows the effect of white balancing on the raw data

Setting the black point to 500 and the white point to 2000 increases the contrast, while pushing the lower values in black and the higher values in white

the histogram shows how the contast is increased: a range of the raw values is mapped on the entire RGB range

With black point 1000 and white point 2500, only 1500 levels are mapped on the entire RGB range, which increases the contrast even more

the histogram shows, that less strips are mapped to the same RGB range, which in turn increases the spacing between them even more


Did you know, that JPEG images can be used to hide data? Here is how:
Steganography by CryptoBola


Modified: 2008-04-30