Integration 102: Build a Python Module

Objective

Create a module whose function is implemented in Python and compiled into a standalone binary with PyInstaller. The binary runs without a Python installation on the target machine.

This tutorial builds on Integration 101. You will reuse the same Docker-based workflow, credentials, and Agent setup — so those steps are referenced rather than repeated.

By the end you will know how to:

• Scaffold a Python module project with the Istari Digital CLI
• Implement a function using the scaffold's framework (Pydantic models, function registry)
• Build the module into a self-contained executable
• Test, package, publish, and run it on the platform

Namespace: As in Integration 101, replace <namespace> with your own unique value (e.g. alice, teamx).

Prerequisites

• You have completed Integration 101 and have a working .env file with REGISTRY_URL, API_KEY, and AGENT_PAT.
• Docker and Docker Compose installed.
• The Istari Digital CLI and Agent installer in a downloads/ directory (same as Integration 101).

Step 1: Start the Python Development Container

The Docker image for this tutorial includes Python, Poetry, the Istari Digital CLI, and the Agent — all pre-installed. Create these files in a new project directory (e.g. python-module-tutorial/).

1.1 Place downloads

mkdir -p downloads
# Copy the same files as Integration 101:
# cp /path/to/stari_ubuntu_24_04-amd64-v0.21.1.tar.gz downloads/
# cp /path/to/istari-agent_10.1.2_amd64.deb downloads/

1.2 Create the Dockerfile

FROM ubuntu:24.04

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 \
    python3-pip \
    python3-venv \
    zip \
    unzip \
    ca-certificates \
    jq \
    nano \
    binutils \
    libpython3.12

RUN pip install --break-system-packages "pydantic>=2" poetry

COPY downloads/stari_ubuntu_24_04-amd64-v0.21.1.tar.gz /tmp/stari.tar.gz
RUN cd /tmp && tar -xzf stari.tar.gz \
    && cd stari_ubuntu_24_04-amd64 \
    && ./stari_ubuntu_24_04-amd64 path add --yes \
    && rm -rf /tmp/stari*
ENV PATH="/root/.istari_digital/bin:${PATH}"

COPY downloads/istari-agent_10.1.2_amd64.deb /tmp/agent.deb
RUN dpkg -i /tmp/agent.deb && rm /tmp/agent.deb

WORKDIR /workspace

CMD ["sleep", "infinity"]

Compared with Integration 101 this image adds Python development packages and pre-installs the CLI and Agent, so they are ready when the container starts.

1.3 Create docker-compose.yml

services:
  istari-tutorial:
    build: .
    image: istari-python-tutorial:24.04
    platform: linux/amd64
    container_name: istari-python-tutorial
    volumes:
      - .:/workspace
    working_dir: /workspace
    stdin_open: true
    tty: true

1.4 Create the .env file

If you don't already have one from Integration 101, create a .env file with your credentials (see Integration 101 — Step 2.2):

REGISTRY_URL=<registry-url>
API_KEY=<your-api-key>
AGENT_PAT=<your-agent-pat>

1.5 Start the container and initialise

docker compose up -d --build
docker compose exec istari-tutorial bash

Inside the container, initialise the CLI with your credentials:

source /workspace/.env
stari client init "$REGISTRY_URL" "$API_KEY" --yes
stari client ping

Step 2: Scaffold the Module

The CLI generates a complete Python module project:

cd /workspace
stari module scaffold my-python-module \
  --type python-module \
  --author "Your Name" \
  --email "you@example.com" \
  --description "Tutorial: Python compiled module" \
  --version "1.0.0"

This creates the following structure:

my-python-module/
├── module_manifest.json
├── pyproject.toml
├── module/
│   ├── __init__.py
│   ├── __main__.py            ← entry point (arg parsing, function dispatch)
│   ├── logging_config.py
│   ├── module_config.py
│   └── functions/
│       ├── __init__.py         ← auto-discovers function files at import time
│       ├── registry.py         ← register() / get_function()
│       ├── base/
│       │   └── function_io.py  ← Input, Output, OutputType types
│       ├── data_extraction.py
│       ├── model_to_artifact_no_auth.py   ← simplest template (we will use this one)
│       ├── model_to_artifacts_no_auth.py
│       ├── model_to_artifacts_basic_auth.py
│       ├── model_to_artifacts_oidc_auth.py
│       └── model_params_to_artifacts_no_auth.py
├── tests/
├── scripts/
├── hooks/
└── README.md

The scaffold ships with several template functions covering common patterns. Each one is a working example you can run immediately. Open the project's README.md to see the full table:

Use case	Template file
1 input → 1 output, no auth	model_to_artifact_no_auth.py
1 input → multiple outputs	model_to_artifacts_no_auth.py
1 input + parameters → multiple outputs	model_params_to_artifacts_no_auth.py
Basic auth (username/password)	model_to_artifacts_basic_auth.py
OIDC/JWT auth	model_to_artifacts_oidc_auth.py

Install dependencies

cd my-python-module
poetry install

Verify the scaffold compiles and the existing templates' tests pass:

poetry run pytest

Step 3: Create Your Function from a Template

The recommended workflow is to copy the closest template and modify it. Our function takes one input file and produces one output file with no authentication, so model_to_artifact_no_auth.py is the right starting point.

3.1 Copy the template

cp module/functions/model_to_artifact_no_auth.py module/functions/file_copy.py

3.2 Understand the template structure

Open module/functions/file_copy.py. The template has two clearly separated sections:

# ============================================================================
# YOUR BUSINESS LOGIC - REPLACE THIS SECTION
# ============================================================================
def process_model(input_path: Path, output_path: Path) -> None:
    """Replace this with your actual processing logic."""
    content = input_path.read_text(encoding="utf-8")
    output_path.write_text(f"Processed: {content}", encoding="utf-8")
# ============================================================================
# END OF YOUR BUSINESS LOGIC
# ============================================================================

# --- Framework code below (usually no changes needed) ---

The business logic section is the only part you must change. The framework code below it handles input parsing, error handling, and output metadata — you typically leave it as-is, though you will rename a few things.

3.3 Edit the file

Make the following changes to module/functions/file_copy.py:

1. Replace the business logic — add a timestamp header when copying:

def process_model(input_path: Path, output_path: Path) -> None:
    """Copies the input file and prepends a timestamp header."""
    from datetime import datetime

    content = input_path.read_text(encoding="utf-8")
    header = f"# Processed at {datetime.now().isoformat()}\n\n"
    output_path.write_text(header + content, encoding="utf-8")

2. Rename the Pydantic input class from ModelToArtifactNoAuthInput to FileCopyInput:

class FileCopyInput(BaseModel):
    """Input schema for the function."""
    ...

3. Rename the framework function and update its internals — change the function name, use FileCopyInput, and derive the output name from the input filename:

def file_copy(input_json: str, temp_dir: str) -> List[Output]:
    ...
    function_input = FileCopyInput.model_validate_json(input_json)
    ...
    output_path = Path(temp_dir) / f"copy_{input_path.name}"
    ...

4. Update the register() call at the bottom:

register("FileCopy", file_copy)

3.4 Final file

After all edits, module/functions/file_copy.py should look like this:

"""
File copy function — copies input file to output with a timestamp header.
Based on the ModelToArtifactNoAuth template.
"""

import logging
from datetime import datetime
from pathlib import Path
from typing import List

from pydantic import BaseModel, ConfigDict, Extra, Field, ValidationError

from module.functions.base.function_io import Input, Output, OutputType
from module.functions.registry import register

logger = logging.getLogger(__name__)


# ============================================================================
# YOUR BUSINESS LOGIC
# ============================================================================
def process_model(input_path: Path, output_path: Path) -> None:
    """Copies the input file and prepends a timestamp header."""
    content = input_path.read_text(encoding="utf-8")
    header = f"# Processed at {datetime.now().isoformat()}\n\n"
    output_path.write_text(header + content, encoding="utf-8")


# ============================================================================
# END OF YOUR BUSINESS LOGIC
# ============================================================================


class FileCopyInput(BaseModel):
    """Input schema for the function."""

    input_model: Input[str] = Field(
        ...,
        description="The input model file to process.",
    )

    model_config = ConfigDict(extra=Extra.allow)


def file_copy(input_json: str, temp_dir: str) -> List[Output]:
    """
    Processes a single model file and produces a single artifact.
    """
    logger.info("Starting FileCopy execution.")

    try:
        function_input = FileCopyInput.model_validate_json(input_json)
    except ValidationError as e:
        raise ValueError(f"Invalid input JSON for FileCopy: {e}") from e

    outputs: List[Output] = []

    input_path = Path(function_input.input_model.value)
    output_path = Path(temp_dir) / f"copy_{input_path.name}"

    try:
        process_model(input_path, output_path)
    except OSError:
        logger.exception(f'Failed to process model at "{input_path}".')
        return outputs

    outputs.append(
        Output(
            name="output_artifact",
            type=OutputType.FILE,
            path=str(output_path),
        ),
    )

    return outputs


register("FileCopy", file_copy)

What changed compared to the template

What	Template (model_to_artifact_no_auth.py)	Our function (file_copy.py)
Business logic	Prepends "Processed: "	Prepends a timestamp header
Input class	ModelToArtifactNoAuthInput	FileCopyInput
Function name	model_to_artifact_no_auth	file_copy
Output name	"output_artifact"	"output_artifact" (unchanged)
Registered name	"ModelToArtifactNoAuth"	"FileCopy"

The framework code (argument parsing, error handling, output writing) is unchanged — that is the point of using a template.

3.5 Update __init__.py to import only your function

By default, module/functions/__init__.py auto-discovers and imports every .py file in the directory. That means all the scaffold templates would be registered alongside your function. Replace the auto-discovery with an explicit import of your function only:

Edit module/functions/__init__.py:

"""Istari functions offered by the module."""

import module.functions.file_copy

This keeps the project clean — only FileCopy is registered and available to the Agent.

Step 4: Update the Manifest

Replace the contents of module_manifest.json (replace <namespace>):

{
  "module_key": "@<namespace>:python-tutorial",
  "module_version": "1.0.0",
  "module_checksum": "placeholder",
  "module_type": "function",
  "module_display_name": "Python Tutorial Module",
  "tool_key": "<namespace>-tool",
  "tool_versions": ["1.0.0"],
  "operating_systems": ["Ubuntu 24.04"],
  "agent_version": ">=9.0.0",
  "internal": false,
  "authors": [
    {
      "name": "Your Name",
      "email": "you@example.com"
    }
  ],
  "functions": {
    "@<namespace>:copy": [
      {
        "entrypoint": "python_module",
        "run_command": "{entrypoint} FileCopy --input-file {input_file} --output-file {output_file} --temp-dir {temp_dir}",
        "operating_systems": ["Ubuntu 24.04"],
        "tool_versions": ["1.0.0"],
        "function_schema": {
          "inputs": {
            "input_model": {
              "type": "user_model",
              "validation_types": ["@extension:txt"],
              "optional": false,
              "display_name": "Input File"
            }
          },
          "outputs": [
            {
              "name": "output_artifact",
              "type": "file",
              "required": true,
              "upload_as": "artifact",
              "display_name": "Copied File"
            }
          ]
        }
      }
    ]
  }
}

Key differences from Integration 101:

• entrypoint points to python_module — the compiled binary that PyInstaller produces (placed at the zip root alongside the manifest).
• run_command passes FileCopy as the first argument — this is the registered function name (the one in register()), not the Python file name.
• The inputs key input_model must match the field name in the Pydantic model (FileCopyInput.input_model).
• The outputs name output_artifact must match the name in the Output object returned by the function.

Validate the manifest:

stari module lint module_manifest.json

Step 5: Test Locally

5.1 Create test files

mkdir -p test_run
echo "Hello from the Python module!" > test_run/input.txt

Create test_run/input.json:

{
  "input_model": {
    "type": "user_model",
    "value": "test_run/input.txt"
  }
}

5.2 Run with Poetry (pre-build)

Before compiling, test directly with Python:

poetry run python -m module FileCopy \
  --input-file test_run/input.json \
  --output-file test_run/output.json \
  --temp-dir test_run

5.3 Verify output

cat test_run/output.json
cat test_run/copy_input.txt

Expected output:

# Processed at 2026-02-25T14:00:00.000000

Hello from the Python module!

Step 6: Build the Binary

PyInstaller bundles Python, your code, and all dependencies into a single executable. The scaffold includes a pre-configured build task:

poetry run poe build_binary

This runs pyinstaller --onefile --additional-hooks-dir=./hooks --name=python_module module/__main__.py and produces dist/python_module.

Test the compiled binary

rm test_run/output.json test_run/copy_input.txt

./dist/python_module FileCopy \
  --input-file test_run/input.json \
  --output-file test_run/output.json \
  --temp-dir test_run

cat test_run/copy_input.txt

The binary should produce the same result as the Poetry run.

Step 7: Package and Publish

7.1 Package

The Agent only needs the compiled binary and the manifest. Zip them together without the dist/ folder level:

cd /workspace/my-python-module
zip /workspace/my-python-module.zip module_manifest.json
cd dist && zip /workspace/my-python-module.zip python_module && cd ..

7.2 Publish

stari client publish module_manifest.json

7.3 Grant access

Same procedure as Integration 101 — Step 5.3: open Admin Settings → User → Manage Tool Access and enable your tool.

Step 8: Deploy and Run

The Agent is already installed in the Docker image. Initialise it and deploy your module:

# Initialise the Agent
source /workspace/.env
stari agent init "$REGISTRY_URL" "$AGENT_PAT"
echo "default: {}" >> /root/.config/istari_digital/istari_digital_config.yaml
echo "    istari_digital_agent_headless_mode: true" >> /root/.config/istari_digital/istari_digital_config.yaml

# Deploy the module
mkdir -p /opt/local/istari_agent/istari_modules/my-python-module
unzip -o /workspace/my-python-module.zip \
  -d /opt/local/istari_agent/istari_modules/my-python-module

# Start the Agent
/opt/local/istari_agent/istari_agent_10.1.2

8.1 Submit a job

In the Platform UI:

• Upload a .txt file.
• Select the file and choose Create Job.
• In the Select a tool/function dropdown, select the tool and function you created.
• Execute the job.

This starts a Job that you can monitor in the Jobs section.

8.2 Check Agent logs

In the terminal where the Agent is running, you should see:

Found available job 6b1d8333-d0c8-4d9b-992e-975724a5e8c6
CLAIMING JOB STATE: job 6b1d8333...
EXECUTING JOB STATE: job 6b1d8333...
Running process 6b1d8333... in cwd /opt/local/istari_agent/istari_modules/my-python-module ...
Process 6b1d8333... completed succesfully
Return code for job 6b1d8333...: 0
Job 6b1d8333... completed successfully
EXECUTION SUCCESS STATE: job 6b1d8333...

8.3 Check results

In the Platform UI:

1. Go to the Files tab.
2. Find your file and view the artifact generated by the job.

---

Summary

What you did	How
Scaffolded a Python module project	stari module scaffold --type python-module
Implemented a function	Pydantic input model + processing logic + register()
Tested before compiling	poetry run python -m module FileCopy ...
Built a standalone binary	poetry run poe build_binary (PyInstaller)
Packaged, published, and ran	Same workflow as Integration 101

Next steps

• Add more functions: Copy another template (e.g. model_to_artifacts_no_auth.py for multiple outputs, or model_to_artifacts_basic_auth.py for authenticated access), edit the business logic, update the manifest. The auto-discovery in __init__.py picks up new files automatically.
• Use parameters: See model_params_to_artifacts_no_auth.py for a template that accepts user-supplied parameters alongside the input file.
• Read the scaffold reference: The generated project includes REFERENCE.md and DEVELOPMENT.md with detailed documentation on testing, configuration, error handling, and deployment.
• Explore the manifest spec: See the Module Manifest API Reference for all available fields.

---

Troubleshooting

ModuleNotFoundError: No module named 'module'

You are running Python from outside the project directory, or Poetry's virtual environment is not active.

Fix: Run from the my-python-module/ directory using poetry run.

PyInstaller build fails with missing Python.h

PyInstaller needs Python development headers to compile.

Fix: Install them inside the container:

apt-get install -y python3.12-dev

The tutorial Dockerfile already includes libpython3.12, but if you use a different base image you may need this.

Module tool X does not match manifest tool Y

You already published a module with a given module_key and tool name, then changed the tool field in the manifest and tried to publish again. The platform binds the tool name to the module_key on first publish — subsequent versions must use the same tool value.

Fix: Either revert the tool field to the original value, or use a new module_key for the renamed tool.

Module version 1.0.0 already exists

Module versions are immutable once published.

Fix: Bump module_version in module_manifest.json (e.g. 1.0.0 → 1.0.1) and publish again.

Other issues

See Integration 101 — Troubleshooting for Agent-related errors (DisplayNameError, KeyError: 'default').