GLSL shaders | Grapple

What you write

Write one entry function that takes a position in the frame and returns a colour, plus any helper functions you need:

vec4 shade(vec2 uv) {
  vec4 color = gp_sample_input_frames(uv);
  return vec4(color.rgb * gp_amount, color.a);
}

uv runs from 0 to 1 across the frame, with (0, 0) at the top left.
The colour you return becomes that pixel: red, green, blue and alpha.
Grapple writes the rest of the program: the #version line, the bindings, the uniforms and main. Don't declare any of those, or global in and out variables.
Name the entry function anything that isn't reserved; shade is the usual choice.

Shaders are GLSL 4.50 for Vulkan, compiled inside Grapple. A shader with a frame input is a filter. One with no frame input is a generator: it makes the picture from nothing.

Built-in values

Built-in shader values
Name	Type	Meaning
`gp_time`	`float`	The time in composition seconds.
`gp_frame_index`	`uint`	The frame number.
`gp_frame_rate`	`float`	The composition's frames per second.
`gp_resolution`	`vec2`	The size in pixels of the frame being rendered. Previews and contact sheets can be smaller than the composition.
`gp_composition_resolution`	`vec2`	The composition's own size in pixels, the same in preview and export. `gp_composition_width` and `gp_composition_height` are its parts.
`gp_pixel`	`vec2`	One rendered pixel in `uv` units, for sampling a neighbour.
`gp_` and a parameter's name	by type	Each parameter, such as `gp_amount`.
`gp_sample_` and an input's name	function	Samples a frame input at `uv`, such as `gp_sample_input_frames(uv)`.

For a size that should look the same in a small preview and in the export, measure it in composition pixels and divide by gp_composition_resolution: twelve pixels across is 12.0 / gp_composition_resolution.x in uv.

Parameters

Each parameter becomes a control on the shader in the Inspector, which you can key, drive or publish in Controls. In the shader it's gp_ followed by its name, with a type that follows its value:

A number: float. Its control is a slider or an angle.
True or false: bool, from a toggle.
A point or a colour: vec2 for two values, vec3 for three.
A rectangle: vec4: x, y, width and height.

Frame inputs

A filter placed on a clip, a layer or the composition gets one input, named input_frames, so it samples with gp_sample_input_frames(uv). A shader can declare up to three frame inputs, each with its own gp_sample_ function, such as a foreground and a backdrop to combine. Sampling returns the input's colour and alpha at that point.

On a clip, the frame is the clip's own canvas, before its transform, mask, opacity and blend. On a layer or the composition, it's the composed picture in the composition's frame.

Colour and alpha

Colours are in the project's linear working colour space, the space where light adds up: twice the value is twice the light. Mixing, blurring and brightening behave the way light does.
Alpha is straight: the colour you return isn't multiplied by its alpha. A half-transparent red is vec4(1.0, 0.0, 0.0, 0.5).
Values can go above 1.0. Inside the composition, a bright highlight keeps its light for the effects after it; the export clips at white.
A colour you type into a shader is linear, so the sRGB value from a design file looks too light if you use it as it is. Convert it first:

vec3 srgb_to_linear(vec3 c) {
  return mix(c / 12.92, pow((c + 0.055) / 1.055, vec3(2.4)), step(0.04045, c));
}

vec4 shade(vec2 uv) {
  vec4 c = gp_sample_input_frames(uv);
  vec3 orange = srgb_to_linear(vec3(255.0, 90.0, 31.0) / 255.0);   // #FF5A1F from a design file
  return vec4(mix(c.rgb, orange, gp_amount), c.a);
}

Time

gp_time is the composition time in seconds, so a shader that reads it moves on its own and renders the same frame the same way every time. Use it for motion that never stops, such as a heat haze. For motion that starts and stops, key a parameter instead: a shader with an amount you key from 0 to 1 over a few frames reads better on the Timeline, and Ape and you can both change it there.

The camera

Every shader can read the camera the renderer uses at that moment. Its values are functions named gp_camera_ and the field:

Where it is: gp_camera_eye(), and the directions gp_camera_right(), gp_camera_up() and gp_camera_forward(), in world pixels.
The lens: gp_camera_focal_length() and gp_camera_sensor_width() in millimetres, gp_camera_fov_x() and gp_camera_fov_y() in radians, gp_camera_focus_distance() in world pixels and gp_camera_aperture_f_stop().
The projection: gp_camera_projection() is GP_CAMERA_PERSPECTIVE or GP_CAMERA_ORTHOGRAPHIC. gp_camera_near(), gp_camera_far() and the matrices gp_camera_world_to_view(), gp_camera_view_to_world(), gp_camera_world_to_clip() and gp_camera_clip_to_world().

Two helpers do the projection for you, including a solved camera's lens distortion and the footage's framing:

GPProjectResult r = gp_project(point);   // r.visible, r.uv, r.depth for a world point
GPViewRay ray = gp_view_ray(uv);         // ray.origin, ray.direction through a pixel

World space is composition pixels: the origin at the top left, x to the right, y down and z toward the viewer, the same as the Inspector. A point a shader projects appears where Grapple draws that point.

Lights

The composition's lights are a small table. gp_lights_count() says how many there are, and each column is a function that takes a row and fills in a value, returning false if the row isn't there:

bool gp_lights_kind(uint row, out uint value);       // GP_LIGHT_AMBIENT, GP_LIGHT_DIRECTIONAL or GP_LIGHT_POINT
bool gp_lights_color(uint row, out vec3 value);      // linear colour
bool gp_lights_intensity(uint row, out float value);
bool gp_lights_direction(uint row, out vec3 value);
bool gp_lights_position(uint row, out vec3 value);   // world pixels
bool gp_lights_range(uint row, out float value);     // world pixels
bool gp_lights_casts_shadow(uint row, out bool value);

Paths and tables

A shader can take more of the project as resource inputs: a shape's outline, or a table a Python program makes. They're connections in the project, and they follow their source: drag a point of the path and the shader reads the new route.

A path: Every shape clip offers its outline, after its animation, trims, transforms and any copies a generator makes. Connected to an input named route, it arrives as tables of contours and points, with two helpers: gp_route_sample(contour, distance, sample) and gp_route_sample_progress(contour, progress, sample). Each fills a GPPathSample with a position and a tangent, and returns false past the end.
A table: A Python program can output a gp.Table with named, typed columns. Connected to an input named dots, the shader gets gp_dots_count() and a function per column, such as gp_dots_position(row, value).

Every function a shader can call is listed when it's added, in the result Ape gets back, and in any compile error. Dots along a path below is a complete example.

Add a shader to a project

Ape adds shaders to a project. In the Ape tab, paste your function, or describe the look you want, and say where it goes: on a clip, on a layer, or on the whole composition. Name the controls you want, such as "a Strength from 0 to 1". The shader is added as an effect or a generator, and its parameters appear as controls in the Inspector.

A script or a connected agent adds one with the program.author step, in the shader language grapple-shader-0:

layer = gp.comp("Main")["Plate"]
gp.step("program.author", displayName="Channel split", language="grapple-shader-0",
        entrypoint="shade", placement={"kind": "track", "trackNodeId": layer.id},
        params=[{"name": "split", "label": "Split",
                 "description": "How far red and blue move apart, in composition pixels.",
                 "value": 6, "editor": {"family": "scalar", "min": 0, "max": 40}}],
        source="""
vec4 shade(vec2 uv) {
  vec2 offset = vec2(gp_split / gp_composition_resolution.x, 0.0);
  vec4 c = gp_sample_input_frames(uv);
  return vec4(gp_sample_input_frames(uv + offset).r, c.g,
              gp_sample_input_frames(uv - offset).b, c.a);
}
""")

Placed on frames with no ports declared, a shader is a filter with the input input_frames. Where a program sits lists the places a shader can go.

When it doesn't compile

Grapple compiles the shader when it's added or changed. An error names the line in your own source and shows it, then lists the functions generated for that shader:

line 3: 'gp_strenght' : undeclared identifier
  >   return vec4(c.rgb * gp_strenght, c.a);
Generated functions for this program:
...

If nothing points at a line, the message is "Define vec4 shade(vec2 uv).": the entry function is missing or has another name.

Performance

The function runs once for every pixel of every frame. A loop of 24 steps on a 4K frame is about 200 million steps, so keep loops short and leave them early when you can.
Each sample of a frame input is a texture read. A blur of 25 samples costs 25 times a plain filter.
Path helpers search the path's points each time you call them. Sample a few dozen points, not hundreds.
Preview a short range at the size you'll deliver. Previews can render smaller, and a shader whose sizes are in composition pixels looks the same in both.

Worked examples

Each example says which parameters it uses. Give them those names when you or Ape add the shader.

Channel split

vec4 shade(vec2 uv) {
  vec2 offset = vec2(gp_split / gp_composition_resolution.x, 0.0);
  vec4 c = gp_sample_input_frames(uv);
  float r = gp_sample_input_frames(uv + offset).r;
  float b = gp_sample_input_frames(uv - offset).b;
  return vec4(r, c.g, b, c.a);
}

One parameter, split, in composition pixels. Key it from 0 to 12 and back over a few frames for a hit on a cut.

Vignette

vec4 shade(vec2 uv) {
  vec4 c = gp_sample_input_frames(uv);
  vec2 aspect = vec2(gp_composition_width / gp_composition_height, 1.0);
  float d = length((uv - 0.5) * aspect);
  float edge = smoothstep(gp_radius - gp_softness, gp_radius, d);
  return vec4(c.rgb * (1.0 - gp_amount * edge), c.a);
}

Parameters radius, softness and amount. The aspect correction keeps it round on a wide frame.

Heat haze

vec4 shade(vec2 uv) {
  float wave = sin(uv.y * gp_frequency + gp_time * gp_speed);
  float shift = wave * gp_amplitude / gp_composition_resolution.x;
  return gp_sample_input_frames(vec2(uv.x + shift, uv.y));
}

Parameters frequency, speed and amplitude (in composition pixels). Because it reads gp_time, the haze moves on its own. Key amplitude to fade it in and out.

A soft glow

vec4 shade(vec2 uv) {
  vec4 c = gp_sample_input_frames(uv);
  vec2 spacing = gp_radius / gp_composition_resolution;
  vec3 sum = vec3(0.0);
  for (int y = -2; y <= 2; ++y) {
    for (int x = -2; x <= 2; ++x) {
      vec3 s = gp_sample_input_frames(uv + vec2(x, y) * spacing).rgb;
      sum += max(s - vec3(gp_threshold), vec3(0.0));   // only light above the threshold
    }
  }
  return vec4(c.rgb + sum / 25.0 * gp_strength, c.a);
}

Parameters radius (composition pixels between samples), threshold and strength. It works in linear light, so bright highlights glow more than mid-tones, and they can go past white inside the composition.

Rings that grow

vec4 shade(vec2 uv) {
  vec2 p = (uv - 0.5) * gp_composition_resolution;     // composition pixels from the centre
  float r = length(p);
  float wave = 0.5 + 0.5 * sin(r / gp_spacing * 6.2831853 - gp_time * gp_speed);
  vec3 color = mix(gp_dark, gp_light, smoothstep(0.4, 0.6, wave));
  return vec4(color, 1.0);
}

A generator: it has no frame input and makes the picture itself. Parameters spacing (pixels between rings), speed, and two colours, dark and light, given as linear colours.

Lit by the scene's ambient light

vec4 shade(vec2 uv) {
  vec4 c = gp_sample_input_frames(uv);
  vec3 ambient = vec3(0.0);
  for (uint i = 0u; i < gp_lights_count(); ++i) {
    uint kind; vec3 color; float intensity;
    if (gp_lights_kind(i, kind) && kind == GP_LIGHT_AMBIENT &&
        gp_lights_color(i, color) && gp_lights_intensity(i, intensity))
      ambient += color * intensity;
  }
  vec3 lit = c.rgb * ambient;
  return vec4(mix(c.rgb, lit, gp_amount), c.a);
}

One parameter, amount. Change the composition's ambient light and a flat graphic takes on its colour, so it sits with the 3D layers around it.

Dots along a path

vec4 shade(vec2 uv) {
  vec4 base = gp_sample_input_frames(uv);
  vec2 here = uv * gp_composition_resolution;
  float dot_alpha = 0.0;
  for (int i = 0; i < 24; ++i) {
    GPPathSample s;
    if (!gp_route_sample_progress(0u, float(i) / 23.0, s)) continue;
    GPProjectResult r = gp_project(s.position);
    if (!r.visible) continue;
    float d = distance(here, r.uv * gp_composition_resolution);
    dot_alpha = max(dot_alpha, 1.0 - smoothstep(gp_size - 1.5, gp_size, d));
  }
  return vec4(mix(base.rgb, vec3(1.0), dot_alpha), max(base.a, dot_alpha));
}

Twenty-four white dots spaced evenly along a shape's path, with a parameter size for their radius in pixels. The shader declares a resource input named route of type path_set, connected to the shape clip's outlines output. Because it projects through the camera with gp_project, the dots stay on the path in a 3D view too.

program = gp.step("program.author", displayName="Path dots", language="grapple-shader-0",
                  entrypoint="shade", placement={"kind": "track", "trackNodeId": layer.id},
                  inputPorts=[{"name": "input_frames", "type": "frame_stream"},
                              {"name": "route", "type": "resource", "resource": {"typeId": "path_set"}}],
                  outputPorts=[{"name": "frames", "type": "frame_stream"}],
                  params=[{"name": "size", "label": "Dot size", "description": "Each dot's radius in pixels.",
                           "value": 8, "editor": {"family": "scalar", "min": 1, "max": 40}}],
                  source=DOTS)
gp.step("effect.connect_ports", sourceNodeId=route_shape.id, sourcePort="outlines",
        targetNodeId=program.id, targetPort="route")