LINEAL RAW PROCESSING                   © LUIS A GUEVARA luis@sigmacumlaude.com

Return to Sigma Cum Laude .com

This is one of the most elusive Digital Concepts , because ,when seen globally , all Digital Processing Systems are Lineal. Otherwise the images we make will not make any sense to us.

We expect a Scene to look the same way ,whether we look at them through a window , through a Roof mounted Video camera , at the Computer Screen or in a Print hanged in the wall . This would not be possible if their overall processing systems did not respect all the tonal relationships of the Scene , such as Color Balance , Illumination Dynamic Range , Contrast , Sharpness ,etc. When it does , we say that the system is Lineal and that there are no distortions introduced by the process itself.

In practice our systems are lineal to a degree that we call "Realistic", and we recognize that some combinations of Hardware and Software brands produce better "Image Quality" or that there is an improved "Realism" in their resulting images.

Finally when dealing with this kind of elusive concepts , we can reach another plateau in the Image Reproduction arena, where the image quality achieved is so highly realistic ,that we say it has achieved "Three Dimensionality" , that is ,it can fool the eye into believing that is real.

The European Renaissance painters referred to this concept ,as "Trompe-l'œil " (trick the eye") , and worked hard to achieve it.

So what do we mean by Lineal Processing as opposed to Lineal Systems?

Simple. Although, as a whole ,all the elements of an Imaging System behave Linearly , their individual Components are mostly Non Lineal , but their non linearity has been compensated at different points, to achieve the final desired overall System Linearity. The tool used to do this is called Gamma Encoding.

Gamma is nothing but the slope of a function graph that describe the relationship between the input and the output of a system or a system element.

GammaThe image shown is a Monitor Gamma Curve that is definitely non linear and non uniform . Its value , represented by the slope of the curve at any given point ,is not one , but continuously variable , so it is therefore defined by convention , as being the slope of a line drawn from the origin, Tangent to the Lowest point of the Curve. This point happens to fall very near the center of the scale , so it is generally assumed that Contrast is the Gamma of the Mid Tones.

This last observation is crucial to understanding the benefits of Linear processing. Since Contrast is nothing but the tonal separation between adjacent tones , a Gamma encoded image ,since it is the result of a Curve , expands the separation of the Lower Tones, of lower gamma ,and Compresses the Upper Tones that ,not only have the highest Gamma, but their gamma value keeps increasing infinitely and rapidly until it reaches pure white and Black. No tones in between.

IN OTHER WORDS , GAMMA ENCODING EXPANDS THE LOWER TONES ,AND SEVERILY DAMAGE THE REPRODUCTION OF THE UPPER VALUES

Gamma 1.0 , on the other hand ,means that the output divided by the input equals ONE. No transformation takes place , and we call it LINEAL. The output equals the input. In a Graph it will be represented by a Straight line of uniform Slope of 45 degrees, that is , a constant Transformation value of 1. ADJACENT TONES RETAIN THEIR ORIGINAL SEPARATION AT ALL POINTS ALONG THE SCALE

We certainly want linearity in all our overall systems , be they music amplifiers or Digital Cameras .However in Digital Imaging this is harder to achieve because our Computer Monitors , Cameras LCD Displays , and all Printing Devices are Highly Non Lineal . Monitors for example have a Gamma of 2.5 and more, but our Camera Sensors are very Lineal with a Gamma very close to 1.0 , so how do we conciliate this disparity ?

Simple , again , but largely unknown to most people, in spite that this has been with us since the creation of Video Cameras :

An artificial Gamma Correction is introduced to the Captured RAW Data that is exactly the inverse of the Display Gamma so that the overall effect is that they cancel each other out.

Since today imaging systems operate within Standardized Color Spaces , each one designed with its own Target Display Gamma , of which the most ubiquitous is the sRGB Color Space that was designed to represent the Gamma of the Average , Uncalibrated , PC CRT Monitor , of Gamma 2.2 , then the inverse of this 1/2.2=0.45 , is applied, as a transformation function, to the RAW data ,DURING ITS CONVERSION from RAW DATA to Displayable Image File .

This is done in the RAW developer, as a first step , right before before any other necessary processes of the conversion from data to image ,have taken place.The following graphic illustrates how the two ,opposite curves combine , mathematically, to produce a Mean Value of 1.0 , the straight Gray Line of Gamma 1.0 , that you see in between the two red ones.

http://en.wikipedia.org/wiki/Gamma_correction

That is ,our RAW Data that was Linearly captured ,thanks to the painstaking efforts of the Sensor Designers , is artificially DISTORTED when and where it hurts the most , right when Color Channels and Color Balance are being created from it.

NOTE:The RAW file is just a Matrix of Data with no Color Channels , Color Spaces or even Color , for that matter.It is not an image file. For more on RAW , please see my article "The Raw Story" at: http://www.sigmacumlaude.com/The_Raw_Story.aspx

SO , WHAT IS THIS LINEAR PROCESSING AGAIN?

Simple. I am sure that you will agree that is simple. Linear processing is avoiding the introduction of this Inverse Gamma at the point where it hurts the most ,to reintroduce later, where it hurts the least ,as one of the last steps of Image Optimization, during Post Processing , rather than on camera ,at the Image Creation. A big substantial Difference.

Of course we must reintroduce it , to achieve again the necessary overall System Gamma of 1.0. If we don't , we will end up with an image that possess the same Gamma as the display does , that is G2.2 , which for most people is VERY DARK AND CONTRASTY and TOTALLY UNUSABLE. ( Please take a mental note of this , because when you process Lineal Images and you forget to reintroduce the compensating Gamma of 0.45 , your image will be like that , dark and concentrated at the lower tones.)

For example , this is a RAW image , converted in dcRAW-X into a LINEAL Photoshop file, that was opened without assigning it a Custom Lineal Profile. As you can see because is missing the Gamma encoding , the image looks dark and contrasted , because nothing is counteracting the high Monitor Gamma.


Please note that its color settings are indicating sRGB , my Photoshop preference as a Workspace for those images that don't come with a Gamma Tag. sRGB has a Gamma of 2.2 , so we end up with 2.2 ( sRGB gamma) + 2.2 ( Monitor gamma) = Gamma 4.4 , instead of the desired Gamma 1.0 . No wonder it looks so dark and Contrasted with almost no detail, or color , in the highlights.

 

If I change my workspace to Adobe Prophoto , that has a Gamma of only 1.8 , as seen here , the Colors get better , but the Tones , although slightly brighter ,are still wrong and harsh .Detail is poor in the HIGHLIGHTS and barely acceptable in the Mid Tones . But clearly we are moving in the right direction.

 

.

 

Now this is the same image , after changing the workspace to my Custom Profile of Gamma 1.0 , THAT I HAVE NAMED AS "LINEAL PROPHOTO"

As you can see , the Custom Profile is nothing but a Prophoto RGB profile ,where the Gamma was changed to 1.0 , and saved with a name of my choice , as a Workspace preset.:

This is a very impressive Visual difference , that comes from the fact that all instructions that the camera writes to the Metadata of the RAW file have been stripped from it , by my LINEAL CONVERTER OF CHOICE , "DCRAW-X"

This 16 Bit Photoshop PSD file was created, directly into Adobe PROPHOTO RGB Color Space , without even opening the image in a Viewer , so that I could quickly get to open it in Photoshop in the most Pristine condition possible.

Some people do not believe this , so here you have a screen shot of the RAW file Metadata , as presented by Photoshop CS4;

This is the Metadata of the Lineal PSD image .It is very short containing mostly creation information Please notice how both Photoshop and Camera RAW frames are showing the File number IMG30989.psd :

For Comparison here is the extensive Metadata obtained from the same image , after normal processing through Adobe Camera RAW
YOU CAN CLEARLY SEE ALL THE EXTRA INSTRUCTIONS , HIGHLIGHTED IN RED

<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 4.2.2-c063 53.352624, 2008/07/30-18:05:41 ">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
xmlns:tiff="http://ns.adobe.com/tiff/1.0/">
<tiff:Make>SIGMA</tiff:Make>
<ti

ff:Model>SIGMA SD9</tiff:Model>
<tiff:XResolution>240/1</tiff:XResolution>
<tiff:YResolution>240/1</tiff:YResolution>
<tiff:ResolutionUnit>2</tiff:ResolutionUnit>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:exif="http://ns.adobe.com/exif/1.0/">
<exif:ExifVersion>0221</exif:ExifVersion>
<exif:ExposureTime>1/125</exif:ExposureTime>
<exif:ShutterSpeedValue>6965784/1000000</exif:ShutterSpeedValue>
<exif:FNumber>56/10</exif:FNumber>
<exif:ApertureValue>4970854/1000000</exif:ApertureValue>
<exif:ExposureProgram>3</exif:ExposureProgram>
<exif:DateTimeOriginal>2008-10-02T08:57:58-04:00</exif:DateTimeOriginal>
<exif:ExposureBiasValue>0/1</exif:ExposureBiasValue>
<exif:MaxApertureValue>0/1</exif:MaxApertureValue>
<exif:MeteringMode>5</exif:MeteringMode>
<exif:PixelXDimension>4096</exif:PixelXDimension>
<exif:PixelYDimension>2731</exif:PixelYDimension>
<exif:ISOSpeedRatings>
<rdf:Seq>
<rdf:li>200</rdf:li>
</rdf:Seq>
</exif:ISOSpeedRatings>
<exif:Flash rdf:parseType="Resource">
<exif:Fired>False</exif:Fired>
</exif:Flash>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:xmp="http://ns.adobe.com/xap/1.0/">
<xmp:ModifyDate>2008-10-02T12:56:29-04:00</xmp:ModifyDate>
<xmp:CreateDate>2008-10-02T08:57:58-04:00</xmp:CreateDate>
<xmp:CreatorTool>Adobe Photoshop Lightroom</xmp:CreatorTool>
<xmp:Label>Green</xmp:Label>
<xmp:Rating>5</xmp:Rating>
<xmp:MetadataDate>2008-10-02T12:56:29-04:00</xmp:MetadataDate>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:aux="http://ns.adobe.com/exif/1.0/aux/">
<aux:SerialNumber>01007372</aux:SerialNumber>
<aux:LensID>0</aux:LensID>
<aux:Firmware>1.4.0.1302 Release</aux:Firmware>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:xmpRights="http://ns.adobe.com/xap/1.0/rights/">
<xmpRights:Marked>True</xmpRights:Marked>
<xmpRights:UsageTerms>
<rdf:Alt>
<rdf:li xml:lang="x-default">ALL REPRODUCTION PROHIBITED </rdf:li>
</rdf:Alt>
</xmpRights:UsageTerms>
<xmpRights:WebStatement>http://www.sigmacumlaude.com</xmpRights:WebStatement>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:creator>
<rdf:Seq>
<rdf:li>LUIS A GUEVARA</rdf:li>
</rdf:Seq>
</dc:creator>
<dc:rights>
<rdf:Alt>
<rdf:li xml:lang="x-default">© LUIS A GUEVARA luis@sigmacumlaude.com</rdf:li>
</rdf:Alt>
</dc:rights>
<dc:subject>
<rdf:Bag>
<rdf:li>Tests of 19 and apo 2x</rdf:li>
<rdf:li>photar 25</rdf:li>
</rdf:Bag>
</dc:subject>
<dc:description>
<rdf:Alt>
<rdf:li xml:lang="x-default">19 f2.8 ELMARIT</rdf:li>
</rdf:Alt>
</dc:description>
<dc:format>image/tiff</dc:format>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:Iptc4xmpCore="http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/">
<Iptc4xmpCore:CreatorContactInfo rdf:parseType="Resource">
<Iptc4xmpCore:CiAdrExtadr>6337 NW 174 TERRACE</Iptc4xmpCore:CiAdrExtadr>
<Iptc4xmpCore:CiAdrCity>MIAMI</Iptc4xmpCore:CiAdrCity>
<Iptc4xmpCore:CiAdrRegion>FLORIDA </Iptc4xmpCore:CiAdrRegion>
<Iptc4xmpCore:CiAdrPcode>33015-4462</Iptc4xmpCore:CiAdrPcode>
<Iptc4xmpCore:CiAdrCtry>UNITED STATES</Iptc4xmpCore:CiAdrCtry>
<Iptc4xmpCore:CiTelWork>305 8269254</Iptc4xmpCore:CiTelWork>
<Iptc4xmpCore:CiEmailWork>luis@sigmacumlaude.com</Iptc4xmpCore:CiEmailWork>
<Iptc4xmpCore:CiUrlWork>http://www.sigmacumlaude.com</Iptc4xmpCore:CiUrlWork>
</Iptc4xmpCore:CreatorContactInfo>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/">
<photoshop:AuthorsPosition>FINE ART PHOTOGRAPHER</photoshop:AuthorsPosition>
<photoshop:SidecarForExtension>X3F</photoshop:SidecarForExtension>
<photoshop:ColorMode>3</photoshop:ColorMode>
<photoshop:ICCProfile>ProPhoto RGB</photoshop:ICCProfile>
</rdf:Description>
<rdf:Description rdf:about=""
xmlns:crs="http://ns.adobe.com/camera-raw-settings/1.0/">
<crs:RawFileName>IMG30989.X3F</crs:RawFileName>
<crs:Version>5.0</crs:Version>
<crs:WhiteBalance>Custom</crs:WhiteBalance>
<crs:Temperature>4700</crs:Temperature>
<crs:Tint>-1</crs:Tint>
<crs:Exposure>-0.95</crs:Exposure>
<crs:Shadows>6</crs:Shadows>
<crs:Brightness>+43</crs:Brightness>
<crs:Contrast>+25</crs:Contrast>
<crs:Saturation>+9</crs:Saturation>
<crs:Sharpness>25</crs:Sharpness>
<crs:LuminanceSmoothing>0</crs:LuminanceSmoothing>
<crs:ColorNoiseReduction>0</crs:ColorNoiseReduction>
<crs:ChromaticAberrationR>0</crs:ChromaticAberrationR>
<crs:ChromaticAberrationB>0</crs:ChromaticAberrationB>
<crs:VignetteAmount>0</crs:VignetteAmount>
<crs:ShadowTint>0</crs:ShadowTint>
<crs:RedHue>0</crs:RedHue>
<crs:RedSaturation>0</crs:RedSaturation>
<crs:GreenHue>0</crs:GreenHue>
<crs:GreenSaturation>0</crs:GreenSaturation>
<crs:BlueHue>0</crs:BlueHue>
<crs:BlueSaturation>0</crs:BlueSaturation>
<crs:FillLight>0</crs:FillLight>
<crs:Vibrance>+35</crs:Vibrance>
<crs:HighlightRecovery>89</crs:HighlightRecovery>
<crs:Clarity>+57</crs:Clarity>
<crs:Defringe>0</crs:Defringe>
<crs:HueAdjustmentRed>0</crs:HueAdjustmentRed>
<crs:HueAdjustmentOrange>0</crs:HueAdjustmentOrange>
<crs:HueAdjustmentYellow>0</crs:HueAdjustmentYellow>
<crs:HueAdjustmentGreen>0</crs:HueAdjustmentGreen>
<crs:HueAdjustmentAqua>0</crs:HueAdjustmentAqua>
<crs:HueAdjustmentBlue>0</crs:HueAdjustmentBlue>
<crs:HueAdjustmentPurple>0</crs:HueAdjustmentPurple>
<crs:HueAdjustmentMagenta>0</crs:HueAdjustmentMagenta>
<crs:SaturationAdjustmentRed>0</crs:SaturationAdjustmentRed>
<crs:SaturationAdjustmentOrange>0</crs:SaturationAdjustmentOrange>
<crs:SaturationAdjustmentYellow>0</crs:SaturationAdjustmentYellow>
<crs:SaturationAdjustmentGreen>0</crs:SaturationAdjustmentGreen>
<crs:SaturationAdjustmentAqua>0</crs:SaturationAdjustmentAqua>
<crs:SaturationAdjustmentBlue>0</crs:SaturationAdjustmentBlue>
<crs:SaturationAdjustmentPurple>0</crs:SaturationAdjustmentPurple>
<crs:SaturationAdjustmentMagenta>0</crs:SaturationAdjustmentMagenta>
<crs:LuminanceAdjustmentRed>0</crs:LuminanceAdjustmentRed>
<crs:LuminanceAdjustmentOrange>0</crs:LuminanceAdjustmentOrange>
<crs:LuminanceAdjustmentYellow>0</crs:LuminanceAdjustmentYellow>
<crs:LuminanceAdjustmentGreen>0</crs:LuminanceAdjustmentGreen>
<crs:LuminanceAdjustmentAqua>0</crs:LuminanceAdjustmentAqua>
<crs:LuminanceAdjustmentBlue>0</crs:LuminanceAdjustmentBlue>
<crs:LuminanceAdjustmentPurple>0</crs:LuminanceAdjustmentPurple>
<crs:LuminanceAdjustmentMagenta>0</crs:LuminanceAdjustmentMagenta>
<crs:SplitToningShadowHue>0</crs:SplitToningShadowHue>
<crs:SplitToningShadowSaturation>0</crs:SplitToningShadowSaturation>
<crs:SplitToningHighlightHue>0</crs:SplitToningHighlightHue>
<crs:SplitToningHighlightSaturation>0</crs:SplitToningHighlightSaturation>
<crs:SplitToningBalance>0</crs:SplitToningBalance>
<crs:ParametricShadows>0</crs:ParametricShadows>
<crs:ParametricDarks>0</crs:ParametricDarks>
<crs:ParametricLights>0</crs:ParametricLights>
<crs:ParametricHighlights>0</crs:ParametricHighlights>
<crs:ParametricShadowSplit>25</crs:ParametricShadowSplit>
<crs:ParametricMidtoneSplit>50</crs:ParametricMidtoneSplit>
<crs:ParametricHighlightSplit>75</crs:ParametricHighlightSplit>
<crs:SharpenRadius>+1.0</crs:SharpenRadius>
<crs:SharpenDetail>25</crs:SharpenDetail>
<crs:SharpenEdgeMasking>0</crs:SharpenEdgeMasking>
<crs:PostCropVignetteAmount>0</crs:PostCropVignetteAmount>
<crs:ConvertToGrayscale>False</crs:ConvertToGrayscale>
<crs:ToneCurveName>Medium Contrast</crs:ToneCurveName>
<crs:ToneCurve>
<rdf:Seq>
<rdf:li>0, 0</rdf:li>
<rdf:li>32, 22</rdf:li>
<rdf:li>64, 56</rdf:li>
<rdf:li>128, 128</rdf:li>
<rdf:li>192, 196</rdf:li>
<rdf:li>255, 255</rdf:li>
</rdf:Seq>
</crs:ToneCurve>
<crs:CameraProfile>Embedded</crs:CameraProfile>
<crs:CameraProfileDigest>1241CA94FDB2D3E6D897CD3E2908F5A3</crs:CameraProfileDigest>
<crs:HasSettings>True</crs:HasSettings>

<crs:HasCrop>False</crs:HasCrop>
<crs:AlreadyApplied>True</crs:AlreadyApplied>

</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>

This second example Metadata was so long that I could not make a Screen Shot of it and had to Copy and Paste , instead.

The point that I am trying to make is that , when Utmost Image Quality matters , and your image content lies in the middle and upper tones , linear processing is the way to go. Lineal Image Converters , like dcRAW-X and others , not only remove Gamma but also , Compression , Sharpening ,De-noising , Anti-aliasing and Interpolation. What you want is that your color channels are made from data that has been touched the least . Ideally ,not touched at all , because all data manipulation throws away data.

All this , Compression , Sharpening ,De-noising , Anti-aliasing and Interpolation, can be best done in a more powerful Desktop Computer , rather than on the camera minimalistic generic processor , tailored to the image type, under the control of the Artist and not by invisible factory presets that only work for Generic ,Average Scenes.

 

Here is the final image after some tonal adjustments and sharpening:

PLEASE CLICK ON IT TO SEE IT FULL SIZE.

So what have we gained by Linear processing and Why?

Well ,both ways of processing a RAW file ultimately achieve a overall System Gamma of 1 with respect to the Scene , but ,and here is where it can become hard to understand , the distribution of tones of the resulting images are different. Gamma encoded images have Normal tone separation of the Mid Tones , with expanded Shadow regions and Compressed highlights. Those Spikes in the Histogram show where Tones have lost Diferentiation , becoming One and the Same.

 

Linear Processing on the other hand , generates images with expanded Mid and Upper Tone Pallettes and very compressed Shadow regions that posterize easily. The Histogram shows how few tones exists at the lower end , and the abundance of tones to describe the Highlights , such as the image of the Flower above.

 

What have we gained ? Clearly we have gained a new , very usefull Tool, to handle Imagery that has abundant upper tones detail , that normally gets lost with normal processing . It is not an easy tool . It requires some understanding of Color Management and Photoshop. So it is mainly a tool that belongs in the Artists Craftmanship tool box.

Why is the outcome so different? because linear files retain the numeric characteristics of Binary numbers that , like f-stops , grow by a Factor of 2 , so on a 16 bit capture , there will be a total of 2>16 = 4096 Tonal values available to describe the entire Scene Dynamic Range, but half of those 4096 levels will be devoted to the BRIGHTEST STOP ( Remember F Stops are separated by a factor of 2X ) , half of the remainder, 2048 ,to the next stop and so on , so in a typical Scene of Six Stops of Dynamic Range , there will be only 64 Tones left to describe the Extreme Shadows, which can become insufficient, and require yet another tool; Dual Processing in Photoshop,to combine the upper part of Linear Images to the Lower end of Gamma Encoded ones ,and that will be the subject of another article, but here you have a sample of such a technique:

 

These differences becomes more of a factor when the image file has to be reduced to an 8Bit file , for Display or Printing , since there are not yet Printers or Display Monitors capable of handling 16 Bits. There are efforts being made to create those Printers and displays , but they are lagging behind. Not only they are not here yet , but in the mean time digital computing, including digital Imaging ,is quickly moving to 32 Bit Floating Point binary numbers ,that will eliminate the Quantization problems of our present 16 Bit INTEGER numbers ,which are unable to describe those subtle tonalities that lie between integers . Today, tone values such as 1.6 1.7 1.8 1.9 ,that exist in the Shadow end ,for example, are all converted to the same integer value of 2, so 3 Tones will be lost in the translation . The same happens in the other direction, so 1.5 1.4 1.3 1.2 become Integer 1.0 and another 3 tones are lost , for a total of six , when our first stop only have 64 Tones , six tones does make a difference of a whooping 10 % .More so the same will happen to all those small numbers between 2 and 3 , 3 and 4 and so on . This problem remains significant until you reach numbers larger than hundreds , when the loss becomes smaller than 1 %

Floating point Binary numbers can handle fractional , decimal numbers , so our tonal scales will one day increase dramatically, and perhaps then the difference between Lineal and Gamma encoded processing will not be noticeable in that superabundance of tones.

 

If you want to , you can download the 7 MB IMG30989.X3F Zip file, to try all this by yourself . Just click here , but remember that all this document is copyrighted and of course so are the images .

You can see more of my images at my personal galleries:
http://photo.net/photos/Luis-A-Guevara
http://www.pbase.com/luis_a_guevara/galleries
http://gallery.leica-users.org/v/LUIS+A+GUEVARA/
http://www.summiluxart.com/
http://www.sigmacumlaude.com/
http://luis-a-guevara.deviantart.com/

 

PLEASE CLICK HERE TO RETURN TO SIGMACUMLAUDE.COM