Model providers in Python

What a provider is

A provider is a folder with two files. provider.json describes the service: its name, where it lives, the key it needs, and the operations and models it offers. provider.py does the work. Grapple's own connections to OpenRouter, OpenAI, Google and others are built the same way, and they're in the resources\providers folder where Grapple is installed, so you can read one before you write your own.

The operations a provider can implement are chat (text.chat.v1), image generation (image.generate.v1), speech (audio.speech.generate.v1), and video generation and extension (video.generate.v1, video.extend.v1).

Add and approve a script

Open Settings (Ctrl+,) and go to Models.
Under Your scripts, choose Add provider script… and pick the script's provider.json. Open folder shows where your scripts are kept, and Reload reads the folder again after you edit a file.
Open the provider. It reads "Your script · Waiting for your approval". Read the script, then press Approve.
Paste the key if the service needs one, and turn on the models you want.

Approval is for the exact files you approved: change either one, and the provider waits for your approval again. Revoke approval switches a script off, and Delete script removes its folder. If a script can't be read, the page says which one and why. A provider's id must be its own: one that matches a provider Grapple includes is refused.

The two files

my_service\
  provider.json    what the service is and offers
  provider.py      run(context) and/or converse(context)

The script runs in Grapple's own Python, with its helper package grapple_provider. A provider that needs another package lists it under dependencies in its manifest, with the version, and the provider's page in Settings shows the Python packages it uses.

provider.json

The top of a manifest, from the provider Grapple includes for any OpenAI-style server:

{
  "schemaVersion": 2,
  "customEndpoint": true,
  "id": "openai_compatible",
  "label": "OpenAI-compatible server",
  "description": "A server of your own, such as Ollama or LM Studio, or any service with an OpenAI-style API.",
  "runtime": "python",
  "script": "provider.py",
  "credential": {
    "required": false,
    "label": "API key, if the server needs one"
  },
  "setupHint": "Needs the server's address, such as http://localhost:11434/v1, entered on its page in Settings > Models.",
  "operations": ["text.chat.v1"],
  "models": [ ... ]
}

id, label, description: Its identity and how it reads in Settings.
runtime, script: A script you add is always python, and script names its file.
endpoint, customEndpoint: The address a new connection starts with, and whether a person can change it.
credential: Whether a key is required, what it's called where it's entered, and a helpUrl shown beside the field for where to get one. Keys live in the vault and reach the script for its own service only.
operations: The operations the provider implements. Each model may declare only these.
models: The models to offer, each with its operations, the inputs it takes (text, images, choices such as a thinking level) and the outputs it gives. A model marked "template": true is a starting point for a model name the person types.

provider.py

A script defines run(context) for one-shot operations, such as making an image, and converse(context) for chat turns. A provider that speaks the OpenAI chat format can hand chat to a helper in one line:

from grapple_provider import chat_completions


def converse(context):
    return chat_completions.converse(context, reasoning="reasoning_effort", prompt_cache=False)

An image provider reads the request's inputs, calls the service, and returns each image as a file output:

import base64

from grapple_provider import ProviderError, http


def run(context):
    if context.operation != "image.generate.v1":
        raise ProviderError(f"This provider does not implement {context.operation}.",
                            code="app.provider_operation_unsupported")
    prompt = context.text("prompt").strip()
    if not prompt:
        raise ProviderError("Describe the image to generate.", code="app.provider_input_missing")
    body = {"model": context.model, "prompt": prompt, "n": int(context.value("count", 1) or 1)}
    context.parameters(body)                      # inputs the manifest maps to service fields
    port = context.media_output("image")["portId"]
    answer = http.post_json(context, http.join(context.require_endpoint(), "/images/generations"),
                            body, headers=http.bearer(context.require_key()))
    return [context.file_output(port, data=base64.b64decode(item["b64_json"]),
                                mime_type="image/png", ordinal=i)
            for i, item in enumerate(answer.get("data") or [])]

What the script receives

context.operation, context.model: The operation asked for, and the service's own name for the model.
context.require_key(), context.require_endpoint(): The connection's key and address, or a clear failure that tells the person to set them in Settings, Models.
context.text(port), context.value(port, default), context.values(port): The request's inputs. A choice gives its value for the service, and a resolution gives {"width", "height"}.
context.media(port): The files supplied for an input, such as reference images, each with its path and mime_type.
context.parameters(body): Copies the inputs the manifest maps to service fields into a request body.
context.media_output(kind), context.output_dir: The declared output for "image", "audio" or "video", and the folder to write files in.
context.file_output(port, data= or path=, mime_type=): A file result, from bytes or a file under output_dir. Grapple checks the bytes against the type, and the type against what the model declares.
context.output(port, value): A text, number, choice or structured result.
context.progress(stage, completed, total, message): Reports how far a long job has got.
context.checkpoint(sequence, payload): Records evidence such as a remote job id before waiting on it.

Errors and cancelling

Raise ProviderError(message, code=...) with a sentence a person can act on, such as "Describe the image to generate." It's the message the failed operation reports. context.cancelled() says whether the person has stopped the operation, and context.on_cancel(handler) runs your handler when they do, so you can cancel the job on the service too.

Helpers

grapple_provider.http: post_json, get_json, request_bytes, stream_events and download, with join for addresses and bearer(key) for the usual header. They turn a service's error into a ProviderError with its detail.
chat_completions, anthropic_messages, gemini: Complete chat turns for the three common chat formats, with tools and streaming.
grapple_provider.media: File helpers, such as sniff_mime and pcm16_to_wav for raw speech audio.

Ask Ape to write one

Tell Ape the service and the model, such as "Write a provider for our studio's image server; it takes a prompt and returns PNGs at /generate". Ape can write the two files for you. Add them under Your scripts, read them, approve the script, and add the key when its page asks.