-
Notifications
You must be signed in to change notification settings - Fork 26
doc introduction
The process of rendering raw data captured by a camera into a displayable image is both technical and artistic. Technical transforms can get us from one colorspace to another, apply display device encoding, or map input value to output value in a precise way. But at some point we need to craft image appearance to satisfy artistic goals. We want our images to look good and most of all to serve the story we are trying to tell. The processes involved are fascinating and diverse.
I would be the first to admit that I do not have all the answers. The tools here represent a process of learning and experimentation. I am curious and want to learn how things work. I am not a color scientist. Maybe more of a color pragmatist. I love reading scientific studies and papers, but always with the goal of making better looking images.
I find it problematic that display transform processes are often "black box" proprietary secrets. The only "open" system for display rendering I am aware of is ACES. To encourage technical and artistic innovation, as well as general understanding of the topic, I believe there should be more open solutions available for these increasingly common problems.
My frame of reference for all of this is that of a feature film VFX color pipeline, because this is where my experience lies. However I believe the information here could be valuable for everyone from DI Colorists to video game engineers and all the hobbyists in between.
A display transform can go by many names, or no name at all.
Display Rendering Transform (DRT)
View Transform
Output Transform
"S-Shaped View Transform"
Image Formation
Display Prep
LUT
First, an image is captured by a digital camera sensor. The camera stores this image in a digital file. Preferably, the image will be scene-referred. This means there is a known relationship between the original scene light intensity, and the intensity values recorded in the pixels of the image.
We need a display transform to prepare this scene-referred image data for presentation on a display device. This transform is lossy. Once you transform for a display, you can't get back the original data. Image data that has been transformed in this way is called display-referred.
Many image processing operations need to be performed in a linear domain where we have a known relationship back to the original light intensity captured by the camera. Therefore, it makes sense to push the display transform as late as possible in the image processing pipeline.
But there always needs to be a display transform in order for us image makers to see what we are doing. The display transform is the window that we look through in order to see the scene that we are creating.
A display transform prepares image data for presentation on a display device. It is a combination of technical transforms, and creative transforms to achieve a desired image appearance or rendering. It can be very simple, or very very complex.
The technical components are pretty straightforward. Usually there is
- A mapping to compress input intensity into the more limited range of the display device.
- A mapping to compress input colors into the more limited color range of the display device.
- Colorspace conversions to convert between the input gamut and the output gamut for the display device.
- Some type of encoding to account for the characteristics of the display device
Creative components might be much more nuanced and complex. There might be engineered hue shifts, neutral color appearance adjustments, specific nonlinear adjustments to chroma over certain ranges of color.
Ideally we would design the system to separate technical and artistic as much as we can, and allow as much control over the individual processes as possible.
Hopefully this serves as a good high-level introduction to the topic. Read on for more detailed information...
Next...