Integration 302: Build a Python Module

Objective

Create a module whose function is implemented in Python and compiled into a standalone binary with PyInstaller. The binary runs without a Python installation on the target machine.

This tutorial builds on Integration 301. You will reuse the same Docker-based workflow, credentials, and Agent setup — so those steps are referenced rather than repeated.

By the end you will know how to:

• Scaffold a Python module project with the Istari Digital CLI
• Implement a function using the scaffold's framework (Pydantic models, function registry)
• Build the module into a self-contained executable
• Test, package, publish, and run it on the platform

Namespace: As in Integration 301, replace <namespace> with your own unique value (e.g. alice, teamx).

Prerequisites

• You have completed Integration 301.
• Docker and Docker Compose installed.
• The Istari Digital CLI and Agent installer in a downloads/ directory (same packages as Integration 301). Replace <VERSION> with the CLI version and X.Y.Z with the Agent version. This tutorial was checked with CLI 0.21.20 and Agent 11.11.2.

Step 1: Start the Python Development Container

The Docker image for this tutorial includes Python, Poetry, the Istari Digital CLI, and the Agent — all pre-installed. Create these files in a new project directory (e.g. python-module-tutorial/).

1.1 Place downloads

mkdir -p downloads
# cp ~/Downloads/stari_ubuntu_24_04-amd64-v<VERSION>.tar.gz downloads/
# cp ~/Downloads/istari-agent_X.Y.Z_amd64.deb downloads/

Put one CLI archive and the non-FIPS Agent .deb in downloads/. The Dockerfile copies whichever file matches.

1.2 Create the Dockerfile

FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 \
    python3-pip \
    python3-venv \
    zip \
    unzip \
    ca-certificates \
    jq \
    nano \
    binutils \
    libpython3.12 \
    libmpv1

RUN pip install --break-system-packages "pydantic>=2" poetry

COPY downloads/stari_ubuntu_24_04-amd64-v*.tar.gz /tmp/stari.tar.gz
RUN cd /tmp && tar -xzf stari.tar.gz \
    && cd stari_ubuntu_24_04-amd64 \
    && ./stari_ubuntu_24_04-amd64 path add --yes \
    && rm -rf /tmp/stari*
ENV PATH="/root/.istari_digital/bin:${PATH}"

COPY downloads/istari-agent_*_amd64.deb /tmp/agent.deb
RUN dpkg -i /tmp/agent.deb && rm /tmp/agent.deb

WORKDIR /workspace

CMD ["sleep", "infinity"]

Compared with Integration 301 this image adds Python development packages and pre-installs the CLI and Agent, so they are ready when the container starts.

1.3 Create docker-compose.yml

services:
  istari-tutorial:
    build: .
    image: istari-python-tutorial:24.04
    platform: linux/amd64
    container_name: istari-python-tutorial
    volumes:
      - .:/workspace
    working_dir: /workspace
    stdin_open: true
    tty: true

1.4 Create the .env file

Copy .env and istari-credentials.json from Integration 301 into this project folder, or create them the same way. See Integration 301 — Step 2.2.

API_URL=https://api.your-instance.istari.app
WEB_APP_URL=https://your-instance.istari.app
CLI_CREDENTIALS_FILE=/workspace/istari-credentials.json

These names are only for the commands in this tutorial. The CLI does not read .env. The Agent key is created in Step 8, the same way as Install the agent on Linux.

1.5 Start the container and initialise

docker compose up -d --build
docker compose exec istari-tutorial bash

Inside the container, initialise the CLI with the key file:

source /workspace/.env
stari client init "$API_URL" --identity-service --credentials-file "$CLI_CREDENTIALS_FILE" --yes
stari client ping

stari client init writes a cli: section into /root/.config/istari_digital/istari_digital_config.yaml. Leave that section in place when you configure the Agent in Step 8.

The Agent reads a default: section. Add it when it is missing:

CONFIG=/root/.config/istari_digital/istari_digital_config.yaml
grep -q '^default:' "$CONFIG" || printf 'default: {}\n' >> "$CONFIG"

Step 2: Scaffold the Module

The CLI generates a complete Python module project:

cd /workspace
stari module scaffold my-python-module \
  --type python-module \
  --author "Your Name" \
  --email "you@example.com" \
  --description "Tutorial: Python compiled module" \
  --version "1.0.0"

This creates the following structure:

my-python-module/
├── module_manifest.json
├── pyproject.toml
├── module/
│   ├── __init__.py
│   ├── __main__.py            ← entry point (arg parsing, function dispatch)
│   ├── logging_config.py
│   ├── module_config.py
│   └── functions/
│       ├── __init__.py         ← auto-discovers function files at import time
│       ├── registry.py         ← register() / get_function()
│       ├── base/
│       │   └── function_io.py  ← Input, Output, OutputType types
│       ├── data_extraction.py
│       ├── model_to_artifact_no_auth.py   ← simplest template (we will use this one)
│       ├── model_to_artifacts_no_auth.py
│       ├── model_to_artifacts_basic_auth.py
│       ├── model_to_artifacts_oidc_auth.py
│       └── model_params_to_artifacts_no_auth.py
├── tests/
├── scripts/
├── hooks/
└── README.md

The scaffold ships with several template functions covering common patterns. Each one is a working example you can run immediately. Open the project's README.md to see the full table:

Use case	Template file
1 input → 1 output, no auth	model_to_artifact_no_auth.py
1 input → multiple outputs	model_to_artifacts_no_auth.py
1 input + parameters → multiple outputs	model_params_to_artifacts_no_auth.py
Basic auth (username/password)	model_to_artifacts_basic_auth.py
OIDC/JWT auth	model_to_artifacts_oidc_auth.py

Install dependencies

cd my-python-module
poetry install

Verify the scaffold compiles and the existing templates' tests pass:

poetry run pytest

Step 3: Create Your Function from a Template

The recommended workflow is to copy the closest template and modify it. Our function takes one input file and produces one output file with no authentication, so model_to_artifact_no_auth.py is the right starting point.

3.1 Copy the template

cp module/functions/model_to_artifact_no_auth.py module/functions/file_copy.py

3.2 Understand the template structure

Open module/functions/file_copy.py. The template has two clearly separated sections:

# ============================================================================
# YOUR BUSINESS LOGIC - REPLACE THIS SECTION
# ============================================================================
def process_model(input_path: Path, output_path: Path) -> None:
    """Replace this with your actual processing logic."""
    content = input_path.read_text(encoding="utf-8")
    output_path.write_text(f"Processed: {content}", encoding="utf-8")
# ============================================================================
# END OF YOUR BUSINESS LOGIC
# ============================================================================

# --- Framework code below (usually no changes needed) ---

The business logic section is the only part you must change. The framework code below it handles input parsing, error handling, and output metadata — you typically leave it as-is, though you will rename a few things.

3.3 Edit the file

Make the following changes to module/functions/file_copy.py:

1. Replace the business logic — add a timestamp header when copying:

def process_model(input_path: Path, output_path: Path) -> None:
    """Copies the input file and prepends a timestamp header."""
    from datetime import datetime

    content = input_path.read_text(encoding="utf-8")
    header = f"# Processed at {datetime.now().isoformat()}\n\n"
    output_path.write_text(header + content, encoding="utf-8")

2. Rename the Pydantic input class from ModelToArtifactNoAuthInput to FileCopyInput:

class FileCopyInput(BaseModel):
    """Input schema for the function."""
    ...

3. Rename the framework function and update its internals — change the function name, use FileCopyInput, and derive the output name from the input filename:

def file_copy(input_json: str, temp_dir: str) -> List[Output]:
    ...
    function_input = FileCopyInput.model_validate_json(input_json)
    ...
    output_path = Path(temp_dir) / f"copy_{input_path.name}"
    ...

4. Update the register() call at the bottom:

register("FileCopy", file_copy)

3.4 Final file

After all edits, module/functions/file_copy.py should look like this:

"""
File copy function — copies input file to output with a timestamp header.
Based on the ModelToArtifactNoAuth template.
"""

import logging
from datetime import datetime
from pathlib import Path
from typing import List

from pydantic import BaseModel, ConfigDict, Extra, Field, ValidationError

from module.functions.base.function_io import Input, Output, OutputType
from module.functions.registry import register

logger = logging.getLogger(__name__)


# ============================================================================
# YOUR BUSINESS LOGIC
# ============================================================================
def process_model(input_path: Path, output_path: Path) -> None:
    """Copies the input file and prepends a timestamp header."""
    content = input_path.read_text(encoding="utf-8")
    header = f"# Processed at {datetime.now().isoformat()}\n\n"
    output_path.write_text(header + content, encoding="utf-8")


# ============================================================================
# END OF YOUR BUSINESS LOGIC
# ============================================================================


class FileCopyInput(BaseModel):
    """Input schema for the function."""

    input_model: Input[str] = Field(
        ...,
        description="The input model file to process.",
    )

    model_config = ConfigDict(extra=Extra.allow)


def file_copy(input_json: str, temp_dir: str) -> List[Output]:
    """
    Processes a single model file and produces a single artifact.
    """
    logger.info("Starting FileCopy execution.")

    try:
        function_input = FileCopyInput.model_validate_json(input_json)
    except ValidationError as e:
        raise ValueError(f"Invalid input JSON for FileCopy: {e}") from e

    outputs: List[Output] = []

    input_path = Path(function_input.input_model.value)
    output_path = Path(temp_dir) / f"copy_{input_path.name}"

    try:
        process_model(input_path, output_path)
    except OSError:
        logger.exception(f'Failed to process model at "{input_path}".')
        return outputs

    outputs.append(
        Output(
            name="output_artifact",
            type=OutputType.FILE,
            path=str(output_path),
        ),
    )

    return outputs


register("FileCopy", file_copy)

What changed compared to the template

What	Template (model_to_artifact_no_auth.py)	Our function (file_copy.py)
Business logic	Prepends "Processed: "	Prepends a timestamp header
Input class	ModelToArtifactNoAuthInput	FileCopyInput
Function name	model_to_artifact_no_auth	file_copy
Output name	"output_artifact"	"output_artifact" (unchanged)
Registered name	"ModelToArtifactNoAuth"	"FileCopy"

The framework code (argument parsing, error handling, output writing) is unchanged — that is the point of using a template.

3.5 Update __init__.py to import only your function

By default, module/functions/__init__.py auto-discovers and imports every .py file in the directory. That means all the scaffold templates would be registered alongside your function. Replace the auto-discovery with an explicit import of your function only:

Edit module/functions/__init__.py:

"""Istari functions offered by the module."""

import module.functions.file_copy

This keeps the project clean — only FileCopy is registered and available to the Agent.

Step 4: Update the Manifest

Replace the contents of module_manifest.json (replace <namespace>):

{
  "module_key": "@<namespace>:python-tutorial",
  "module_version": "1.0.0",
  "module_checksum": "placeholder",
  "module_type": "function",
  "module_display_name": "Python Tutorial Module",
  "tool_key": "<namespace>-tool",
  "tool_display_name": "<Namespace> Tool",
  "tool_versions": ["1.0.0"],
  "operating_systems": ["Ubuntu 24.04"],
  "agent_version": ">=9.0.0",
  "internal": false,
  "authors": [
    {
      "name": "Your Name",
      "email": "you@example.com"
    }
  ],
  "functions": {
    "@<namespace>:copy": [
      {
        "entrypoint": "python_module",
        "run_command": "{entrypoint} FileCopy --input-file {input_file} --output-file {output_file} --temp-dir {temp_dir}",
        "operating_systems": ["Ubuntu 24.04"],
        "tool_versions": ["1.0.0"],
        "function_schema": {
          "inputs": {
            "input_model": {
              "type": "user_model",
              "validation_types": ["@extension:txt"],
              "optional": false,
              "display_name": "Input File"
            }
          },
          "outputs": [
            {
              "name": "output_artifact",
              "type": "file",
              "required": true,
              "upload_as": "artifact",
              "display_name": "Copied File"
            }
          ]
        }
      }
    ]
  }
}

Key differences from Integration 301:

• entrypoint points to python_module — the compiled binary that PyInstaller produces (placed at the zip root alongside the manifest).
• run_command passes FileCopy as the first argument — this is the registered function name (the one in register()), not the Python file name.
• The inputs key input_model must match the field name in the Pydantic model (FileCopyInput.input_model).
• The outputs name output_artifact must match the name in the Output object returned by the function.

Validate the manifest:

stari module lint module_manifest.json

tip

JSON schemas for validation and code assistants

The CLI ships with JSON schemas that it uses for validation. You can export them for use in your editor or with LLM code assistants:

stari schemas get module_manifest
stari schemas get function_schema
stari schemas get function_input
stari schemas get function_output

Each command saves the schema to a .json file in the current directory. Providing these schemas as context to an AI coding assistant helps it generate valid manifests and function I/O structures.

Step 5: Test Locally

5.1 Create test files

mkdir -p test_run
echo "Hello from the Python module!" > test_run/input.txt

Create test_run/input.json:

{
  "input_model": {
    "type": "user_model",
    "value": "test_run/input.txt"
  }
}

5.2 Run with Poetry (pre-build)

Before compiling, test directly with Python:

poetry run python -m module FileCopy \
  --input-file test_run/input.json \
  --output-file test_run/output.json \
  --temp-dir test_run

5.3 Verify output

cat test_run/output.json
cat test_run/copy_input.txt

Expected output:

# Processed at 2026-02-25T14:00:00.000000

Hello from the Python module!

Step 6: Build the Binary

PyInstaller bundles Python, your code, and all dependencies into a single executable. The scaffold includes a pre-configured build task:

poetry run poe build_binary

This runs pyinstaller --onefile --additional-hooks-dir=./hooks --name=python_module module/__main__.py and produces dist/python_module.

Test the compiled binary

rm test_run/output.json test_run/copy_input.txt

./dist/python_module FileCopy \
  --input-file test_run/input.json \
  --output-file test_run/output.json \
  --temp-dir test_run

cat test_run/copy_input.txt

The binary should produce the same result as the Poetry run.

Step 7: Package and Publish

7.1 Package

The Agent only needs the compiled binary and the manifest. Zip them together without the dist/ folder level:

cd /workspace/my-python-module
zip /workspace/my-python-module.zip module_manifest.json
cd dist && zip /workspace/my-python-module.zip python_module && cd ..

7.2 Publish

stari client publish module_manifest.json

7.3 Grant access

Same procedure as Integration 301 — Step 5.3: open Admin Settings → User → Manage Tool Access and enable your tool.

Step 8: Deploy and Run

The Agent package is already installed in the image. Give it a key the same way as Install the agent on Linux, then deploy your module. The container runs as root, so these commands omit sudo.

1. In the web app, open Agents and select Generate Key. Select Download credentials while the key is on screen. The browser saves agent-credentials-<key id>.json. It is shown once.
2. Copy that file into this project folder.
3. Inside the container, move it into root's home and close it to everyone else:

mkdir -p ~/.istari_digital
mv /workspace/agent-credentials-<key id>.json ~/.istari_digital/
chmod 600 ~/.istari_digital/agent-credentials-<key id>.json

4. Open /root/.config/istari_digital/istari_digital_config.yaml. It already has default: and cli: from Step 1.5. Add an agent: section and leave cli: in place. The container has no desktop session, so include headless mode:

agent:
  istari_digital_agent_digital_api_url: "https://api.example.istari.app"
  istari_digital_agent_identity_service_secret_file: "/root/.istari_digital/agent-credentials-<key id>.json"
  istari_digital_agent_identity_service_enabled: true
  istari_digital_agent_headless_mode: true

Use the API URL from your .env. The configuration stores the path to the credentials file.

Deploy the module, then start the Agent as root:

mkdir -p /opt/local/istari_agent/istari_modules/my-python-module
unzip -o /workspace/my-python-module.zip \
  -d /opt/local/istari_agent/istari_modules/my-python-module
ls /opt/local/istari_agent/istari_modules/my-python-module/module_manifest.json \
   /opt/local/istari_agent/istari_modules/my-python-module/python_module

/opt/local/istari_agent/istari_agent_X.Y.Z

Got display name in the log is the name under All Agents. No compatible jobs and Checking for available jobs repeat about every 15 seconds. That is the Agent waiting for work. Leave this terminal running. Open another shell with docker compose exec istari-tutorial bash before you continue.

8.1 Submit a job

In the Istari Digital web app:

• Upload a .txt file.
• Select the file and choose Create Job.
• In the Select a tool/function dropdown, select the tool and function you created.
• Execute the job.

This starts a Job that you can monitor in the Jobs section.

8.2 Check Agent logs

In the terminal where the Agent is running, you should see:

Found available job 6b1d8333-d0c8-4d9b-992e-975724a5e8c6
CLAIMING JOB STATE: job 6b1d8333...
EXECUTING JOB STATE: job 6b1d8333...
Running process 6b1d8333... in cwd /opt/local/istari_agent/istari_modules/my-python-module ...
Process 6b1d8333... completed succesfully
Return code for job 6b1d8333...: 0
Job 6b1d8333... completed successfully
EXECUTION SUCCESS STATE: job 6b1d8333...

8.3 Check results

In the Istari Digital web app:

1. Go to Resources in the sidebar.
2. Find your Resource and view the artifact generated by the job.

---

Summary

What you did	How
Scaffolded a Python module project	stari module scaffold --type python-module
Implemented a function	Pydantic input model + processing logic + register()
Tested before compiling	poetry run python -m module FileCopy ...
Built a standalone binary	poetry run poe build_binary (PyInstaller)
Packaged, published, and ran	Same workflow as Integration 301

Next steps

• Add more functions: Copy another template (e.g. model_to_artifacts_no_auth.py for multiple outputs, or model_to_artifacts_basic_auth.py for authenticated access), edit the business logic, update the manifest. The auto-discovery in __init__.py picks up new files automatically.
• Use parameters: See model_params_to_artifacts_no_auth.py for a template that accepts user-supplied parameters alongside the input file.
• Read the scaffold reference: The generated project includes REFERENCE.md and DEVELOPMENT.md with detailed documentation on testing, configuration, error handling, and deployment.
• Explore the manifest spec: See the Module Manifest API Reference for all available fields.

---

Troubleshooting

ModuleNotFoundError: No module named 'module'

You are running Python from outside the project directory, or Poetry's virtual environment is not active.

Fix: Run from the my-python-module/ directory using poetry run.

PyInstaller build fails with missing Python.h

PyInstaller needs Python development headers to compile.

Fix: Install them inside the container:

apt-get install -y python3.12-dev

The tutorial Dockerfile already includes libpython3.12, but if you use a different base image you may need this.

Module tool X does not match manifest tool Y

You already published a module with a given module_key and tool name, then changed the tool field in the manifest and tried to publish again. The platform binds the tool name to the module_key on first publish — subsequent versions must use the same tool value.

Fix: Either revert the tool field to the original value, or use a new module_key for the renamed tool.

Module version 1.0.0 already exists

Module versions are immutable once published.

Fix: Bump module_version in module_manifest.json (e.g. 1.0.0 → 1.0.1) and publish again.

Other issues

See Integration 301 — Troubleshooting for Agent-related errors (DisplayNameError, KeyError: 'default').