Skip to main content

Function Test File Generation

The Istari Digital CLI provides a powerful command to automatically generate function input and output test files from module manifests. This feature helps developers create test data for their functions quickly and consistently.

Overview​

The Istari Digital CLI provides two main commands for working with function test files:

  1. stari function list - Discover available functions and their indices
  2. stari function generate test_files - Generate test files for specific functions

The generate command reads a module manifest file, extracts function schemas, and generates corresponding JSON test files through interactive prompts. It supports both simple validation types (@string, @number, @boolean) and complex nested structures (@array:[@object:[@string, @boolean]]).

Discovering Available Functions​

List Command Syntax​

stari function list [OPTIONS]

List Command Options​

OptionShortDescriptionDefaultRequired
--manifest-mPath to module manifest file. Defaults to module_manifest.json in current directory.module_manifest.json❌

List Command Example​

# List functions from default manifest
stari function list

# List functions from custom manifest
stari function list --manifest path/to/custom_manifest.json

List Command Output​

The list command provides detailed information about each function:

Function: @istari:extract
   Available indices: 0-2 (3 implementation(s))
   0: cameo_module_2021x-refresh2/cameo_module_2021x-refresh2.exe
      OS: Windows 10, Windows Server 2019, Windows 11, Windows Server 2022
   1: cameo_module_2022x-refresh2/cameo_module_2022x-refresh2.exe
      OS: Windows 10, Windows Server 2019, Windows 11, Windows Server 2022
   2: cameo_module_2024x-refresh2/cameo_module_2024x-refresh2.exe
      OS: Windows 10, Windows Server 2019, Windows 11, Windows Server 2022

Generate Test Files Command​

Command Syntax​

stari function generate test_files [OPTIONS]

Options​

OptionShortDescriptionDefaultRequired
--function-fFunction name (e.g., @istari:extract). If not specified, generates files for ALL functions. Use stari function list to see available functions.None❌
--index-iFunction implementation index. Use stari function list to see available indices.0❌
--manifest-mPath to module manifest file. Defaults to module_manifest.json in current directory.module_manifest.json❌
--output-dir-oOutput directory for test filestest_files❌
--placeholderGenerate files with placeholder values instead of prompting for input. Perfect for batch generation.False❌

Auto-Completion Support​

The CLI provides auto-completion for function names when available. The completion system will:

  • Suggest available function names when typing the --function flag
  • Show available indices when typing the --index flag
  • Use the manifest file specified with --manifest for accurate suggestions

Default Manifest Behavior​

Both the list and generate test_files commands default to looking for module_manifest.json in your current working directory. This provides a convenient workflow where you can simply run:

# Works if module_manifest.json exists in current directory
stari function list
stari function generate test_files -f @istari:extract

Error Handling​

When the default manifest file is not found, the CLI provides guidance:

If alternative manifest files are found:

Error: Default manifest file 'module_manifest.json' not found in current directory
Current directory: /path/to/your/project

Found these potential manifest files in the current directory:
  • example_module_manifest.json
  • custom_manifest.json

Try one of these commands:
  stari function list --manifest example_module_manifest.json
  stari function list --manifest custom_manifest.json

If no manifest files are found:

Error: Default manifest file 'module_manifest.json' not found in current directory
Current directory: /path/to/your/project

To fix this issue:
  1. Navigate to a directory containing 'module_manifest.json', or
  2. Use --manifest/-m to specify the path to your manifest file

Example:
  stari function list --manifest /path/to/your/manifest.json

Enhanced Error Messages​

The CLI provides helpful error messages when invalid options are provided:

Invalid Function Name​

$ stari function generate test_files -f @istari:invalid -m example_module_manifest.json

Error: Function '@istari:invalid' not found in manifest. Available functions: [...]

To see all available functions and indices, run:
   stari function list --manifest example_module_manifest.json

Available functions in this manifest:
   @istari:extract (indices: 0-2)
   @istari:twc_extract (indices: 0-2)
   @istari:delete_diagram (indices: 0-2)
   ...

Invalid Index​

$ stari function generate test_files -f @istari:extract -i 10 -m example_module_manifest.json

Error: Function index 10 out of range. Function '@istari:extract' has 3 implementations (0-2)

To see all available functions and indices, run:
   stari function list --manifest example_module_manifest.json

Basic Usage​

Generate Test Files for a Function​

stari function generate test_files --function "@istari:extract" --manifest example_module_manifest.json

This command will:

  1. Load the manifest file
  2. Find the @istari:extract function (first implementation by default)
  3. Prompt you to enter values for each input parameter
  4. Generate two JSON files:
    • test_files/input.json - Function input data
    • test_files/output.json - Expected output data

Specify Function Implementation Index​

If a function has multiple implementations, use the --index option:

stari function generate test_files -f "@istari:extract" -i 1 -m example_module_manifest.json

Custom Output Directory​

stari function generate test_files -f "@istari:extract" -m manifest.json -o my_test_files

Placeholder Mode​

The --placeholder flag allows you to generate test files with automatically generated placeholder values instead of interactive prompts. This is useful for:

  • Batch generation of test files for multiple functions
  • CI/CD pipelines where interactive input isn't possible
  • Quick scaffolding of test structure
  • Initial development when you just need file templates

Generate Files with Placeholder Values​

# Generate files for a specific function with placeholder values
stari function generate test_files -f "@istari:extract" --placeholder

# Custom manifest and output directory
stari function generate test_files -f "@istari:extract" -m example_module_manifest.json -o skeleton_tests --placeholder

Output:

Generating test files for function: @istari:extract (index: 0)
Output directory: test_files
Using placeholder values

Generated test files:
   Input:  test_files/input.json
   Output: test_files/output.json

Generate Files for All Functions​

When you omit the --function flag, the CLI will generate test files for ALL functions in the manifest:

# Generate test files for all functions with placeholder values
stari function generate test_files --placeholder

# Custom manifest and output directory
stari function generate test_files -m example_module_manifest.json -o all_tests --placeholder

Output:

Generating test files for ALL functions in manifest
Base output directory: all_tests
Using placeholder values

Generated test files for 3 function implementation(s):
   @istari:extract (index 0):
     Input:  all_tests/@istari_extract/index_0/input.json
     Output: all_tests/@istari_extract/index_0/output.json
   @istari:transform (index 0):
     Input:  all_tests/@istari_transform/index_0/input.json
     Output: all_tests/@istari_transform/index_0/output.json
   @istari:validate (index 0):
     Input:  all_tests/@istari_validate/index_0/input.json
     Output: all_tests/@istari_validate/index_0/output.json

Directory Structure for All Functions​

When generating files for all functions, the CLI creates an organized directory structure:

all_tests/
├── @istari_extract/
│   └── index_0/
│       ├── input.json
│       └── output.json
├── @istari_transform/
│   ├── index_0/
│   │   ├── input.json
│   │   └── output.json
│   └── index_1/
│       ├── input.json
│       └── output.json
└── @istari_validate/
    └── index_0/
        ├── input.json
        └── output.json

Placeholder Value Types​

The CLI generates intelligent placeholder values based on input validation types:

Input TypeValidation TypePlaceholder ValueExample
Parameter@string"example_string""example_string"
Parameter@number4242
Parameter@booleantruetrue
Parameter@array:@stringArray of strings["item1", "item2", "item3"]
Parameter@array:@numberArray of numbers[1, 2, 3]
Parameter@object:[string, number]Object with string keys, number values{"setting1": 123, "setting2": 456}
Parameter@array:[@object:[@string, @string, @boolean, @object]]Complex update tags array[{"element_id": "_example_element_id_123", "element_name": "Example Element Name", "replace_existing": true, "tags": {"key1": "value1", "status": "active"}}]
Parameter@object:[@string, @array:[@object:[@string, @number]]]Nested object with array{"name_1": "example_string", "items_2": [{"name_1": "example_string", "count_2": 42}]}
Parameter@array:[@array:[@number]]Matrix/2D array[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
Parameter@object:[@string, @object:[@array:[@string], @boolean]]Deep nested object{"name_1": "example_string", "config_2": {"items_1": ["item1", "item2"], "field_2": true}}
Parameter@extension:pdfFile with extension"example_file.pdf"
Parameter@mime_type:application/jsonFile hint"<file_with_mime_type_application_json>"
User Model@extension:zipModel file"model_file.zip"
User Model@mime_type:application/zipArchive file"model_archive.zip"
User ModelDefaultGeneric model"model_file.dat"
Auth Info@basic:Basic authentication{"username": "example_user", "password": "example_password"}
Auth Info@bearer:Bearer token{"token": "example_bearer_token"}
Auth Info@api_key:API key{"api_key": "example_api_key"}
User LinkAnyURL placeholder"<replace-with-url>"

Optional Parameters in Placeholder Mode​

Optional parameters are automatically skipped in placeholder mode, keeping the generated files focused on required inputs only.

Interactive Prompts​

The command will prompt you for input values based on the function schema. The prompts vary by input type:

Parameter Types​

Validation Support Overview​

The CLI provides comprehensive validation support for both simple and complex validation types:

Validation LevelDescriptionExamples
✅ Fully SupportedCLI validates input format, converts types, and handles complex nesting@string, @number, @boolean, @array:, @object:, complex nested types
ℹ️ Display HintsCLI shows requirements but doesn't enforce them@mime_type:, @extension:, @basic: (auth)
❌ Not ValidatedNo validation performedFile existence, credential validity

Enhanced Complex Type Support:

  • Nested Objects: Full support for @object:[@string, @array:[@object:[@string, @number]]]
  • Arrays of Arrays: Support for @array:[@array:[@number]] and similar matrix structures
  • Deep Nesting: Unlimited nesting depth with intelligent parsing
  • Contextual Field Names: Smart field naming (e.g., element_id, tags for update patterns)
  • JSON Input Mode: Interactive JSON entry for complex structures with dynamic examples

Important Notes:

  • File paths: The CLI accepts any file path input but doesn't verify file existence or accessibility
  • MIME types: Expected MIME types are displayed but not validated against actual files
  • Extensions: File extensions are shown as hints but not enforced
  • Authentication: Credential format requirements are displayed but not validated

String Parameters​

--- Input: Model Name (model_name) ---
Type: parameter
Validation types: @string
Enter string value: my_test_model

Number Parameters​

--- Input: Timeout (timeout_seconds) ---
Type: parameter
Validation types: @number
Enter number value: 30.5

Boolean Parameters​

--- Input: Enable Debug (debug_mode) ---
Type: parameter
Validation types: @boolean
Enter boolean value [y/N]: y

Array Parameters​

--- Input: Tags (tag_list) ---
Type: parameter
Validation types: @array:string
Enter array values for type string (comma-separated)
Values: tag1, tag2, tag3

Object Parameters​

--- Input: Configuration (config) ---
Type: parameter
Validation types: @object:[string, number]
Enter JSON object: {"setting": 123, "count": 42}

Complex Nested Arrays​

For complex array structures like @array:[@object:[@string, @string, @boolean, @object]], the CLI provides JSON input mode with dynamic examples:

--- Input: Updates (updates) ---
Type: parameter
Validation types: @array:[@object:[@string, @string, @boolean, @object]]
This parameter expects a complex array structure.
Type: @object:[@string, @string, @boolean, @object]

Please enter the array as JSON. For example:
[
  {
    "element_id": "_example_element_id_123",
    "element_name": "Example Element Name",
    "replace_existing": true,
    "tags": {
      "key1": "value1",
      "key2": "value2",
      "status": "active"
    }
  },
  {
    "element_id": "_example_element_id_123_1",
    "element_name": "Example Element Name Variant 1",
    "replace_existing": true,
    "tags": {
      "key1_v1": "value1_modified",
      "key2_v1": "value2_modified",
      "status": "active"
    }
  }
]

JSON array: [{"element_id": "my_element_123", "element_name": "My Element", "replace_existing": false, "tags": {"status": "updated"}}]

Nested Object Structures​

For deeply nested objects like @object:[@string, @array:[@object:[@string, @number]]]:

--- Input: Complex Config (complex_config) ---
Type: parameter
Validation types: @object:[@string, @array:[@object:[@string, @number]]]
This parameter expects a complex object structure.

Please enter the object as JSON. For example:
{
  "name_1": "example_string",
  "items_2": [
    {
      "name_1": "example_string",
      "count_2": 42
    },
    {
      "name_1": "example_string",
      "count_2": 42
    }
  ]
}

JSON object: {"name": "My Config", "items": [{"field": "value1", "count": 10}]}

Matrix Data (Arrays of Arrays)​

For matrix-like structures @array:[@array:[@number]]:

--- Input: Matrix Data (matrix) ---
Type: parameter
Validation types: @array:[@array:[@number]]
This parameter expects a complex array structure.

Please enter the array as JSON. For example:
[
  [1, 2, 3],
  [1, 2, 3],
  [1, 2, 3]
]

JSON array: [[1, 2], [3, 4], [5, 6]]

Model Files​

--- Input: CAD Model (cameo_model) ---
Type: user_model
Validation types: @mime_type:application/zip, @extension:mdzip
Expected file types based on validation:
  MIME type: application/zip
  Extension: mdzip
Enter file path for model: /path/to/model.mdzip

Validation Level: ℹ️ Display Hints Only

  • The CLI shows expected MIME types and file extensions as guidance
  • No validation is performed on:
    • File existence or accessibility
    • Actual MIME type of the provided file
    • File extension matching
    • File size or content validity

Authentication Information​

--- Input: API Credentials (auth_token) ---
Type: auth_info
Validation types: @basic:teamwork_cloud
Authentication types:
  Type: basic
Enter authentication token/credential: my_auth_token_12345

Validation Level: ℹ️ Display Hints Only

  • The CLI shows the expected authentication type (basic, token, etc.)
  • No validation is performed.
--- Input: Metadata Link (twc_link) ---
Type: user_link
Validation types: @extension:istari_teamwork_cloud_metadata_mdzip
Expected link extensions:
  Extension: istari_teamwork_cloud_metadata_mdzip
Enter link/file path: /path/to/metadata.istari_teamwork_cloud_metadata_mdzip

Validation Level: ℹ️ Display Hints Only

  • The CLI shows expected file extensions for linked resources
  • No validation is performed.

Optional Parameters​

For optional parameters, you'll be asked if you want to skip them:

--- Input: Optional Config (optional_param) ---
Type: parameter
Validation types: @string
This input is optional
Skip this optional parameter? [y/N]: n
Enter string value: optional_value

Output Files​

The command generates two JSON files:

Input File Format​

{
  "cameo_model": {
    "type": "user_model",
    "value": "/path/to/model.mdzip"
  },
  "timeout": {
    "type": "parameter",
    "value": 30.5
  },
  "debug_mode": {
    "type": "parameter",
    "value": true
  }
}

Output File Format​

[
  {
    "name": "blocks",
    "type": "file",
    "path": "output_blocks.json"
  },
  {
    "name": "diagrams",
    "type": "directory",
    "path": "output_diagrams_dir"
  }
]

Follow these steps for the best experience:

1. Discover Available Functions​

Start by listing all available functions in your manifest:

stari function list --manifest your_manifest.json

This shows you:

  • All function names you can use with --function
  • Available indices for each function with --index
  • Implementation details for each index

2. Choose Function and Index​

From the list output, select:

  • The function name (e.g., @istari:extract)
  • The implementation index (e.g., 0, 1, 2)

3. Generate Test Files​

Run the generate command with your chosen options:

stari function generate test_files \
  --function "@istari:extract" \
  --index 0 \
  --manifest your_manifest.json \
  --output-dir test_files

4. Follow Interactive Prompts​

The CLI will guide you through providing values for each input parameter.

5. Review Generated Files​

Check the generated test files:

ls test_files/
cat test_files/input.json
cat test_files/output.json

Real-World Examples​

Example 1: Cameo Data Extraction​

stari function generate test_files \
  --function "@istari:extract" \
  --index 0 \
  --manifest example_module_manifest.json \
  --output-dir example_test_data

Sample Interaction:

Generating test files for function: @istari:extract (index: 0)
Output directory: cameo_test_data

=== Generating Function Input File ===

--- Input: cameo_model (cameo_model) ---
Type: user_model
Validation types: @mime_type:application/zip, @extension:mdzip
Expected file types based on validation:
  MIME type: application/zip
  Extension: mdzip
Enter file path for model: /Users/dev/models/sample_model.mdzip

=== Generating Function Output File ===
Output files represent expected function outputs
Include optional output 'blocks'? [Y/n]: y
Enter path for output 'blocks': blocks_output.json
Include optional output 'requirements'? [Y/n]: y
Enter path for output 'requirements': requirements_output.json
Include optional output 'diagrams'? [Y/n]: y
Enter path for output 'diagrams': diagrams_output_dir

✅ Generated test files:
   Input:  cameo_test_data/input.json
   Output: cameo_test_data/output.json

Example 2: Teamwork Cloud Integration​

stari function generate test_files \
  -f "@istari:twc_extract" \
  -m example_module_manifest.json \
  -o twc_tests

Generated Files:

  • twc_tests/input.json
  • twc_tests/output.json

Example 3: Multiple Function Implementations​

# Generate for 2021x version
stari function generate test_files -f "@istari:extract" -i 0 -m manifest.json -o test_2021x

# Generate for 2022x version
stari function generate test_files -f "@istari:extract" -i 1 -m manifest.json -o test_2022x

# Generate for 2024x version
stari function generate test_files -f "@istari:extract" -i 2 -m manifest.json -o test_2024x

Example 4: Complex Validation Types (Update Tags Function)​

stari function generate test_files \
  --function "@istari:update_tags" \
  --manifest example_module_manifest.json \
  --output-dir update_tags_test

Sample Interaction with Complex Array Structure:

Generating test files for function: @istari:update_tags (index: 0)
Output directory: update_tags_test

=== Generating Function Input File ===

--- Input: cameo_model (cameo_model) ---
Type: user_model
Validation types: @extension:mdzip
Enter file path for model: /Users/dev/models/cameo_model.mdzip

--- Input: updates (updates) ---
Type: parameter
Validation types: @array:[@object:[@string, @string, @boolean, @object]]
This parameter expects a complex array structure.
Type: @object:[@string, @string, @boolean, @object]

Please enter the array as JSON. For example:
[
  {
    "element_id": "_example_element_id_123",
    "element_name": "Example Element Name",
    "replace_existing": true,
    "tags": {
      "key1": "value1",
      "key2": "value2",
      "status": "active"
    }
  },
  {
    "element_id": "_example_element_id_123_1",
    "element_name": "Example Element Name Variant 1",
    "replace_existing": true,
    "tags": {
      "key1_v1": "value1_modified",
      "key2_v1": "value2_modified",
      "status": "active"
    }
  }
]

JSON array: [
  {
    "element_id": "_2021x_2_38b206bc_1744829738919_924302_3837",
    "tags": {
      "Logs": null,
      "Requirement Verification Pass or Fail": "Failed_UPDATED_FROM_CODE",
      "Risk": "High",
      "Text": "Text Update From CODE for ELEMENT WITH ID _2021x_2_38b206bc_1744829738919_924302_3837"
    }
  },
  {
    "element_name": "Requirement Brake Deployment ",
    "tags": {
      "Hyperlink": "www.istaridigital.com",
      "Requirement Verification Pass or Fail": "PASSED",
      "Text": "Text Update From CODE for ELEMENT WITH NAME Requirement Brake Deployment ",
      "Version History": "v1.3.4"
    }
  }
]

✅ Generated test files:
   Input:  update_tags_test/input.json
   Output: update_tags_test/output.json

Generated Files:

The input file will contain the complex nested structure exactly as entered:

{
  "cameo_model": {
    "type": "user_model",
    "value": "/Users/dev/models/cameo_model.mdzip"
  },
  "updates": {
    "type": "parameter",
    "value": [
      {
        "element_id": "_2021x_2_38b206bc_1744829738919_924302_3837",
        "tags": {
          "Logs": null,
          "Requirement Verification Pass or Fail": "Failed_UPDATED_FROM_CODE",
          "Risk": "High",
          "Text": "Text Update From CODE for ELEMENT WITH ID _2021x_2_38b206bc_1744829738919_924302_3837"
        }
      },
      {
        "element_name": "Requirement Brake Deployment ",
        "tags": {
          "Hyperlink": "www.istaridigital.com",
          "Requirement Verification Pass or Fail": "PASSED",
          "Text": "Text Update From CODE for ELEMENT WITH NAME Requirement Brake Deployment ",
          "Version History": "v1.3.4"
        }
      }
    ]
  }
}

Troubleshooting​

Common Issues​

Manifest File Not Found​

Error: Manifest file 'my_manifest.json' not found

Solution: Check the file path and ensure the manifest file exists.

Function Not Found​

Error: Function '@istari:nonexistent' not found in manifest. Available functions: [@istari:extract, @istari:update_tags]

Solution: Use one of the available functions listed in the error message.

Invalid Function Index​

Error: Function index 5 out of range. Function '@istari:extract' has 3 implementations (0-2)

Solution: Use a valid index (0-based) within the available range.

Input Validation Limitations​

Issue: Generated test files contain invalid or unrealistic data.

Common Problems:

  • File paths that don't exist on the target system
  • Invalid MIME types or file extensions
  • Incorrect authentication credential formats
  • Malformed URLs or links

Solutions:

  1. Manually verify file paths after generation - ensure files exist and are accessible
  2. Check file extensions match the displayed validation hints (.mdzip, .xml, etc.)
  3. Validate authentication formats - use proper format for basic auth (username:password) or tokens
  4. Test URLs and links - verify accessibility and correct extensions

Note: Complex nested validation types like @array:[@object:[@string, @boolean]] are now fully supported with intelligent parsing and contextual placeholder generation. For file paths, MIME types, and authentication, the CLI shows validation requirements but doesn't enforce them. Always review generated files before use.

Validation Tips​

  1. Validate your manifest before generating test files:

    stari module lint my_manifest.json

  2. Check function schemas are properly formatted:

    stari schemas get function_schema

  3. Use proper file extensions for models and links as specified in validation types.