Image Capture Beyond 24-Bit RGB
Donald S. Brown, Eastman Kodak Company
The human visual system can distinguish roughly 10 million distinct colors. A 24-bit color image (8 bits per color) can encode over 16 million color values. Why would it ever be necessary to capture an image using 30-, 36-, or 48-bit color? And, why use more than three-color channels? The answer is found in the details of how images are captured, displayed, and viewed.
Any practical color-reproduction system (photography, television, lithography) relies on the trichromatic nature of the human visual system (HVS). The eye has three kinds of color sensors, called cones. As a result, only three primary colors (red, green, and blue) are needed to excite these cones in different combinations to create those 10 or so million colors. However, having only three receptors means the HVS cannot sense the individual contributions each wavelength of a particular color stimuli make, only the aggregate effect. As a consequence, two colors made of physically different materials, and thus having different spectral composition, can look identical, depending on the illumination and viewing conditions. (1).
Three color channels (RGB) are sufficient to encode all hues, but are the 256 levels (8 bits) in each channel of a 24-bit image adequate? Theoretical calculations and practical systems suggest the right 256 levels can create very high-quality reproductions, depending on the quality of those 8 bits and the type of output. (2) Greater bit depth is unquestionably needed at capture to achieve 256 useful levels in each color channel, for reasons described below.
Digital Image Capture
For simplicity, Figure 1 illustrates a monochrome grayscale; however, the same concept applies to each channel in an RGB image. Most digital images are captured using a CCD (charge-coupled device). CCDs are actually analog devices, in that the voltage output for each picture element on a CCD varies continuously in proportion to the intensity of the light striking it (Figure 1a). (3) The digitization happens in the analog-to-digital (A/D) converter. A/D converters take the CCD-output voltage range and divide it into a discrete number of levels (e.g., a 5-bit A/D converter has 32 levels or code values). The conversion from voltage to code value is also proportional, or linear (Figure 1b). Therefore, the difference from one code value to the next represents an equal increment in the light intensity as seen by the sensor.
a. CCD detector
b. 5-bit A/D Converter
c. Human Visual Response
Figure 1. Relationship of Luminance to CCD Voltage, Digital Code Value, and Perceived Brightness
Here is the catch. The human visual system does not see equal changes in light intensity as equal changes in perceived brightness. The response is nonlinear, something more like a power function as shown in Figure 1c. Figure 2 illustrates an image as seen by the sensor (top) and by the HVS (bottom).
Figure 2. An Image as Seen by the Sensor and the HVS
In visual terms, the A/D converter has assigned too many code values to light colors and possibly too few to dark ones as evidenced by the loss of detail in shadows (Figure 2). Scanners and digital cameras solve this problem by over-assigning bit values throughout an image to improve dark sections, which results in too many values in the light areas. Keeping all the visually redundant code values around would require more memory, larger file sizes, and slower image processing. Typically, capture systems will reduce the bit depth (from 12 bits to 8 bits, for example) by choosing fewer levels, but levels that represent more visually equal increments. Figure 3 demonstrates the idea by reducing the 5-bit (32-level) linear data to 3-bit (8-level) data with roughly equal visual increments. This step picks the best 8 values from the 32 levels available. If too few bits are used to begin with, the resulting image may show quantization artifacts (posterization), which is manifested by the appearance of visible tonal steps (Figure 4).
Figure 3. Reducing the 5-bit (32-level) Linear Data to 3-bit (8-level) Data
Figure 4. Quantization Artifacts
High-end scanners and digital cameras use 12-, 14-, or 16-bit A/D converters, and 12 bits per record (36-bit color) is probably the minimum for high-quality work. (4) Within the scanner software, the linear data can be modified by a variety of automatic and manual adjustments, which will include a reshaping of the data to a more visually acceptable scale. The data is then delivered as 8 bits per channel, ready for display, printing, etc. It is more common now for scanners to provide access to the higher bit-depth data directly. (5) And, improved 16-bit-per-color capabilities in Adobe Photoshop software version 5.0 make this data easier to manipulate. Also, the International Color Consortium (ICC) color management system is compatible with 12- and 16-bit-per-color images. It is always an advantage to store higher bit-depth data to ensure the greatest flexibility in subsequent processing for various purposes; however, more storage resources are required.
Bit depth is only part of the picture. The number of bits per color used in a particular scanner or digital camera says nothing about dynamic range or noise. The recorded range from lightest white to darkest black of any two scanners can vary greatly, even though both might use the same number of bits per channel. And, simply putting a 16-bit A/D converter in a scanner does not prevent noise from reducing the useful number of levels to 12 bits or fewer. To decrease noise, increase dynamic range, and improve image quality, there is no substitute for high-quality components and for techniques such as cooling the sensor and time delay and integration (TDI). Published scanner specifications should be read like the marketing documents they are. Even clear-cut specifications such as bit depth have been subject to questionable interpretation. (6) Scanner and digital camera specifications are an indication of quality, but the full story is known only when real images are captured and evaluated.
Obviously, capturing good data in each color channel is very important, but since the HSV is trichromatic, what is the motivation for multispectral imaging, which is based on capturing more than three channels? The answer is more accurate color.
It should be noted that multispectral imaging also includes systems that obtain information from parts of the spectrum beyond the visual region. For instance, remote-sensing systems have infrared channels to supplement or enhance data captured in the visual spectrum. Only systems in pursuit of more faithful color capture are discussed here.
Given identical viewing conditions, a colorimetric scanner, such as IBM Pro/3000, would essentially see the world like the HVS. Commercial scanners generally do not have such responses because of practical design considerations. Instead, a color-correction step involving look-up tables, matrices, or 3-D look-up tables is applied to the scanner data to improve color matching performance. Unfortunately, color correction, or calibration, provides only a partial solution. The correction is only valid for the particular light source and media type for which it was derived. In many cases, this is perfectly acceptable. However, even small changes, such as dye-set differences from one generation of color film or paper to another, technically require different color corrections.
For high-fidelity color, three-channel capture is limited, especially in the case of original artwork created with multiple pigments. It is not surprising, then, that multispectral capture has been applied in the direct capture of fine artwork. Of particular note is the VASARI project (Visual Arts System for Archiving and Retrieval of Images) (7), which has produced systems in association with The National Gallery, London and The Uffizi Gallery, Florence. Multispectral systems in these applications are designed to more accurately estimate CIE tristimulus values (which characterize the HVS trichromatic response) or, better yet, estimate an entire spectral reconstruction for each pixel. Full spectral reconstruction has the advantage of allowing differences in illumination between the capture and display to be corrected. Calibration of such systems will be much less dependent on using the same or similar materials in both the object and the calibration target. Systems have been constructed or proposed that use from five to sixteen separate channels. (8) The number of channels and their spectral characteristics, as well as methods to regress spectral or colorimetric data, are topics of current research.
Multispectral systems are very computationally intensive because of the greater amount of information to process, and the algorithmic conversions to spectral or colorimetric data. It should be noted that the same concerns about bit depth in three-color imaging apply, and that there seems to be a consensus that 12-bit-per-channel capture is sufficient. (9) Though spectrally reconstructed data is more complete, file size is very large. Colorimetric data, on the other hand, can be stored in three 8-bit channels. To retain greater color precision, a 32-bit (10 bit L*, 11 bit a*, 11 bit b*) image has been used.
Greater bit depth (12-bits or more per channel) is necessary to achieve an appropriate 8-bit-per-channel image for display. However, different 24-bit images are required for different output options, so it is desirable to retain the higher bit-depth image for multiple purposes. Of course, retaining high-bit depth images puts a significant burden on storage resources. Additionally, there are still few file formats and compression options for images with greater than 8 bits per channel.
Three-color capture in most all cases requires a color-correction step to measure colorimetric values. The validity of such a color correction is limited to the specific set of colorants for which it was derived. Multispectral imaging is a way to overcome these limitations. Colorimetric fidelity, however, does not guarantee perfect reproductions because gamut limitations of the output media may, in some cases, cause greater harm than in-gamut color errors.
By their nature, multispectral imaging systems process a greater amount of data than an RGB system, and currently require custom tools and less-common file formats. Images produced by multispectral systems are generally not directly viewable, rather, they serve as master images from which derivatives are produced for display or printing. The same is often true for higher bit-depth images. Multispectral systems for precise color capture are still in the realm of custom, research-oriented devices. As digital imaging becomes more powerful and less expensive, multispectral systems may become more common.
This article based on RLG DigiNews, Vol 3, No. 3, 1999
1. The effect of viewing environment, including flare and viewer adaptation can be very significant. An excellent explanation of this topic is given in: Edward J. Giorgianni and Thomas E. Madden, Digital Color Management (Reading, MA: Addison-Wesley, 1998), 14.
2. R.W.G. Hunt, "Bits, Bytes, and Square Meals in Digital Imaging," Proceedings of IS&T/SID Fifth Color Imaging Conference: Color Science, Systems, and Applications. (1997): 1-5; Shin Ohno, "Color in Digital Photography, Color Quality of Digital Photography Prints," Proceedings of IS&T/SID Fifth Color Imaging Conference: Color Science, Systems, and Applications. (1997): 100-104; Fred Mintzer, "Developing Digital Libraries of Cultural Content for Internet Access," IEEE Communications Magazine v37 (Jan 1999): 72-78.
3. Gerald C. Holst, CCD Arrays, Cameras and Displays, 2nd ed. (Bellingham, WA: SPIE Optical Engineering Press, 1998), 78.
4. John Cupitt, David Saunders, and Kirk Martinez, "Digital Imaging in European Museums," Proceedings of SPIE vol. 3025, Very High Resolution and Quality Imaging II (1997): 144-151; Franziska Frey and Sabine Süsstrunk, "Image Quality Requirements for the Digitization of Photographic Collections," Proceedings of SPIE vol. 2663, Very High Resolution and Quality Imaging (1996): 2-7.
5. Andrew Rodney, "Correcting at the Scan Stage," PEI Magazine (Jan 1999): 18-26; "Product Watch: Colorful Scanning," Publish (Aug 1998).
6. "News Beat: Court Rules Umax Scanners a Bit Shy," Publish (March 1999).
7. A. Abrardo, et al., "Art-Works Colour Calibration by Using the VASARI Scanner," Proceedings of IS&T/SID Fourth Color Imaging Conference: Color Science, Systems, and Applications (1996): 94-97; K. Martinez, "High Quality Digital Imaging of Art in Europe," Proceedings of SPIE vol. 2663, Very High Resolution and Quality Imaging (1996): 69-75; Kirk Martinez, John Cupitt, and David Saunders, "High Resolution Colorimetric Imaging of Paintings," Proceedings of SPIE vol. 1901, Cameras, Scanners, and Image Acquisition Systems (1993): 25-36.
8. Y. Miyake et al., "Development of Multiband Color Imaging Systems for Recordings of Art Paintings," Proceedings of SPIE vol. 3648, IS&T/SPIE Conference on Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts IV, (1999): 218-225; Bernhard Hill, "Multispectral Color Technology: A Way toward High Definition Color Image Scanning and Encoding," Proceedings of SPIE vol. 3409, EUROPTO Conference on Electronic Imaging: Processing, Printing, and Publishing in Color, (1998): 2-13; Shoji Tominaga, "Spectral Imaging by a Multi-Channel Camera," Proceedings of SPIE vol. 3648, IS&T/SPIE Conference on Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts IV, (1999): 38-47; Roy S. Berns et al., "Multi-Spectral-Based Color Reproduction Research at the Munsell Color Science Laboratory," Proceedings of SPIE vol. 3409, EUROPTO Conference on Electronic Imaging: Processing, Printing, and Publishing in Color (1998): 14-25.
9. Hill, "Multispectral Color Technology," 11; Friedhelm König and Werner Praefcke, "The Practice of Multispectral Image Acquisition," Proceedings of SPIE vol. 3409, EUROPTO Conference on Electronic Imaging: Processing, Printing, and Publishing in Color, (1998): 34-41; Henri Maître et al., "Spectrometric Image Analysis of Fine Art Paintings," Proceedings of IS&T/SID Fourth Color Imaging Conference: Color Science, Systems, and Applications. (1996): 50-53; David Saunders and Anthony Hamber, "From Pigments to Pixels: Measurement and Display of the Colour Gamut of Paintings," Proceedings of SPIE vol. 1250, Perceiving, Measuring, and Using Color (1990): 90-10