Accelerator Docs

Appearance

🔧 Inbuilt Routines ​

Inbuilt routines are predefined, reusable routines available across all project spaces on the platform. They are designed to support common, general-purpose tasks and help users avoid having to re-invent the wheel.

These routines can be used as-is or combined into jobflows — either through the GUI or in code via wkube.py.

🧩 Key Features ​

  • Available in all project spaces by default
  • Can be launched from the GUI or used programmatically
  • Fully compatible with custom routines in pipelines or jobflows
  • Support GUI configuration and dynamic parameter injection via JSON Schema

📸 Inbuilt Routines in GUI ​

Inbuilt Routines in GUI
Predefined inbuilt routines visible in all project spaces

🧱 Reusing Inbuilt Routines in Code ​

Below are examples for using each inbuilt routine with a corresponding WKubeTask definition and configuration options.

1. Regional Timeseries Validator ​

python
from accli import WKubeTask

regional_validator = WKubeTask(
    name="Regional Timeseries Validator",
    repo_url="https://github.com/iiasa/accelerator-common-routines.git",
    repo_branch="master",
    docker_filename="csv_regional_timeseries_validator/Dockerfile",
    command="python main.py",
    input_mappings="selected_files:/code/inputs/",
    required_cores=1,
    required_ram=1024*1024*1024,
    required_storage_local=1024*1024*1024,
    required_storage_workflow=1024,
    timeout=3600
)

Configuration Options:

  • dataset_template_id (number, required): Template ID for validation
  • VERIFY_ONLY (string, optional): Set to "true" to skip result registration

2. Regional Timeseries Merger ​

python
regional_merger = WKubeTask(
    name="Regional Timeseries Merger",
    repo_url="https://github.com/iiasa/accelerator-common-routines.git",
    repo_branch="master",
    docker_filename="csv_regional_timeseries_merger/Dockerfile",
    command="python main.py",
    input_mappings="selected_files:/code/inputs/",
    required_cores=1,
    required_ram=1024*1024*1024,
    required_storage_local=1024*1024*1024,
    required_storage_workflow=1024,
    timeout=3600
)

Configuration Options:

  • merged_filename (string, required): Output filename (without extension)
  • MERGE_ONLY (string, optional): Set to "true" to skip validation/push

3. GeoTiFF to Cloud Optimized GeoTIFF Converter ​

python
tif_to_cog = WKubeTask(
    name="GeoTiFF to Cloud Optimized GeoTiFF Converter",
    repo_url="https://github.com/iiasa/accelerator-common-routines.git",
    repo_branch="master",
    docker_filename="tif_to_cog_converter/Dockerfile",
    command="python main.py",
    input_mappings="selected_files:/code/inputs/",
    required_cores=1,
    required_ram=1024*1024*1024,
    required_storage_local=1024*1024*1024,
    required_storage_workflow=1024,
    timeout=3600
)

Configuration Options:

  • dataset_template_id (number, required): Template ID for CRS and metadata
  • INPUT_FILE_CRS (string, optional): EPSG code (e.g., EPSG:4326)
  • INPUT_FILE_NODATA (string, optional): Nodata value override

4. DVC Powered Git Push ​

python
dvc_git_push = WKubeTask(
    name="DVC powered git push",
    repo_url="https://github.com/iiasa/accelerator-common-routines.git",
    repo_branch="master",
    docker_filename="git_dvc_push/Dockerfile",
    command="python main.py",
    input_mappings="selected_files:/code/workdir/newfiles",
    required_cores=1,
    required_ram=1024*1024*1024,
    required_storage_local=1024*1024*1024,
    required_storage_workflow=1024,
    timeout=3600
)

Configuration Options:

  • GIT_REPO_URL_HTTP (string, required): Git repo HTTPS URL
  • BRANCH_NAME (string, required): Target branch
  • DVC_S3_ENDPOINT_URL (string, required): S3 endpoint
  • DVC_S3_BUCKET (string, required): S3 bucket name
  • DVC_S3_PREFIX (string, required): Prefix within bucket
  • REPO_DATA_FOLDER (string, required): Folder path in repo
  • COMMIT_MESSAGE (string, required): Commit message

Secrets Required:

  • GIT_PAT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

5. DVC Powered Git Pull ​

python
dvc_git_pull = WKubeTask(
    name="DVC powered git pull",
    repo_url="https://github.com/iiasa/accelerator-common-routines.git",
    repo_branch="master",
    docker_filename="git_dvc_pull/Dockerfile",
    command="python main.py",
    required_cores=1,
    required_ram=1024*1024*1024,
    required_storage_local=1024*1024*1024,
    required_storage_workflow=1024,
    timeout=3600
)

Configuration Options:

  • GIT_REPO_URL_HTTP (string, required)
  • BRANCH_NAME (string, required)
  • DVC_S3_ENDPOINT_URL (string, required)
  • DVC_S3_BUCKET (string, required)
  • DVC_S3_PREFIX (string, required)

Secrets Required:

  • GIT_PAT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

✅ Summary ​

  • Inbuilt routines simplify common workflows like validation, merging, conversion, and DVC operations
  • They are available in the GUI and can be reused in wkube.py
  • All configuration options are clearly defined and support form-based overrides

These routines save time, promote consistency, and are fully ready for reuse across your projects.