Outline Shader

March 2023

A small experiment I did during my studies. All objects in the scene use a “normal” sub-shader to control the actual colors rendered on the screen, and an “outline” substitute sub-shader to determine what is written into the discontinuity map. This map is then processed in a compute kernel using a naive pixel comparison approach. When the difference between neighboring pixels exceeds a defined threshold, an outline is generated. The resulting outline map is then combined with the colors from the “normal” sub-shader and rendered to the screen.

Alt text Discontinuity map

+

Alt text Actual color pass

=

Alt text Final render

The grass blades are instanced procedurally, with the use of a surface shader. All other objects (two planes and a sphere) use a minimal custom vertex-fragment shader. Below is a snippet of the shader setup for the grass blades, featuring options for both the “normal” and the “outlines” shaders. Other objects - such as the grass plane, the sphere, and the background plane - use a similar setup that allows controlling both the actual rendered colors and the colors written into the discontinuity map.

Alt text Shader setup for the grass blades.

Changing the discontinuity map color of the sphere in the following way - Alt text Discontinuity map with the sphere blending in with the background.

results in the final render: Alt text Final render with the outline-less sphere.

Code snippet of the outline kernel.

void Outline(uint3 id : SV_DispatchThreadID)
{
    float3 thisPixelColor = float3(outlineMap[id.xy].x, outlineMap[id.xy].y, outlineMap[id.xy].z);
    float4 thisPixel = outlineMap[id.xy];
    float4 thisPixelSource = source[id.xy];
    float outline = 0;
    
    float thisPixelColorLength = length(thisPixelColor);
    
    if (thisPixelColorLength == 0)
    {
        outputOutline[id.xy] = thisPixel;
        return;
    }
    
    uint2 lowerPixelId = uint2(id.x, (id.y + 1));
        
    if (withinScreen(lowerPixelId))
    {
        float3 lowerPixelColor = float3(outlineMap[lowerPixelId].x, outlineMap[lowerPixelId].y, outlineMap[lowerPixelId].z);
        float4 lowerPixel = outlineMap[lowerPixelId];
        if (abs(length(lowerPixelColor) - thisPixelColorLength) > outlineThreshold)
        {
            thisPixel = float4(0, 0, 0, 1);
            outline = 1;
        }
    }
    
    uint2 rightPixelId = uint2((id.x + 1), (id.y));
    if (withinScreen(rightPixelId))
    {
        float3 rightPixelColor = float3(outlineMap[rightPixelId].x, outlineMap[rightPixelId].y, outlineMap[rightPixelId].z);
        float4 rightPixel = outlineMap[rightPixelId];
        if (abs(length(rightPixelColor) - thisPixelColorLength) > outlineThreshold)
        {
            thisPixel = float4(0, 0, 0, 1);
            outline = 1;
        }
    }
    
    outputOutline[id.xy] = lerp(thisPixelSource, thisPixel, outline);
}