Normal-based Edge Detection with Sobel Operator -Screenspace
Spatial shader that uses the normal texture to find edges. This is a less-complex look at edge detection, perfect for learning screenspace shaders.
Instructions:
- Create a new MeshInstance3D in your 3D scene
- Add a QuadMesh mesh to the instance
- Optional? Move the mesh out of the center of the screen
- In `Geometry` for the instance, increase the `Extra Cull Margin` to the max
- As of writing, 16384 m
- In the Quadmesh, increase the size from 1×1 m to 2×2 m
- In the Quadmesh, check the `Flip Faces` box so that it’s `True`
- In the Quadmesh, in `Material`, choose `New Shader Material`
- In the Material, in `Shader`, choose `New Shader`
- Here you can choose a name for your shader and we can start customizing!
Notes:
This shader does an okay job at edge detection, but it has some issues.
- If two objects at different heights have similar normal vectors, an edge won’t be detected.
- example – a square box on a flat floor. The top of the box and the floor have the same normal, so a perspective view wouldn’t show an edge.
- in an orthogonal or straight on view, normals have trouble flagging edges due to this
- there are lots of ways to solve this issue, the one I’ve heard about most is to use the depth vectors
- basically to make sure the two pixels are on different planes
- you can get thinner lines by reducing your offset
vec2 offset = 0.5 / VIEWPORT_SIZE;- this effectively upscales your normal texture when looking at neighboring pixels
- I have been told this has a large impact on performance, so be wary
Check out my full write-up here: https://github.com/nuzcraft/unga-dungeon/blob/main/shader_notes/normal_based_edge_detection_w_sobel.md
Models by Kenny – https://kenney.nl/
Shader code
shader_type spatial;
render_mode unshaded;
uniform sampler2D SCREEN_TEXTURE: hint_screen_texture, filter_linear_mipmap;
uniform sampler2D NORMAL_TEXTURE : hint_normal_roughness_texture, filter_linear_mipmap;
uniform float edge_threshold = 0.2;
uniform vec3 line_color: source_color = vec3(0.043, 0.282, 0.467);
uniform vec3 background_color: source_color = vec3(0.408, 0.969, 0.694);
const mat3 sobel_y = mat3(
vec3(1.0, 0.0, -1.0),
vec3(2.0, 0.0, -2.0),
vec3(1.0, 0.0, -1.0)
);
const mat3 sobel_x = mat3(
vec3(1.0, 2.0, 1.0),
vec3(0.0, 0.0, 0.0),
vec3(-1.0, -2.0, -1.0)
);
void vertex(){
POSITION = vec4(VERTEX, 1.0);
}
void fragment() {
vec2 uv = SCREEN_UV;
vec4 screen_color = texture(SCREEN_TEXTURE, uv);
vec3 normal = texture(NORMAL_TEXTURE, uv).rgb;
normal = normal * 2.0 - 1.0;
vec2 offset = 1.0 / VIEWPORT_SIZE;
vec3 n = texture(NORMAL_TEXTURE, uv + vec2(0.0, -offset.y)).rgb;
vec3 s = texture(NORMAL_TEXTURE, uv + vec2(0.0, offset.y)).rgb;
vec3 e = texture(NORMAL_TEXTURE, uv + vec2(offset.x, 0.0)).rgb;
vec3 w = texture(NORMAL_TEXTURE, uv + vec2(-offset.x, 0.0)).rgb;
vec3 nw = texture(NORMAL_TEXTURE, uv + vec2(-offset.x, -offset.y)).rgb;
vec3 ne = texture(NORMAL_TEXTURE, uv + vec2(offset.x, -offset.y)).rgb;
vec3 sw = texture(NORMAL_TEXTURE, uv + vec2(-offset.x, offset.y)).rgb;
vec3 se = texture(NORMAL_TEXTURE, uv + vec2(offset.x, offset.y)).rgb;
mat3 surrounding_pixels = mat3(
vec3(length(nw-normal), length(n-normal), length(ne-normal)),
vec3(length(w-normal), length(normal-normal), length(e-normal)),
vec3(length(sw-normal), length(s-normal), length(se-normal))
);
float edge_x = dot(sobel_x[0], surrounding_pixels[0]) + dot(sobel_x[1], surrounding_pixels[1]) + dot(sobel_x[2], surrounding_pixels[2]);
float edge_y = dot(sobel_y[0], surrounding_pixels[0]) + dot(sobel_y[1], surrounding_pixels[1]) + dot(sobel_y[2], surrounding_pixels[2]);
float edge = sqrt(pow(edge_x, 2.0)+pow(edge_y, 2.0));
if (edge > edge_threshold) {
ALBEDO = line_color;
} else {
ALBEDO = background_color;
}
}
