Skip to content

🧩 Routine from Remote Source and Custom Dependency ​

This guide extends the basic "Getting Started" instructions by showing how to run routines from a remote Git repository and how to define custom execution environments using either a containerfile (the Accelerator currently supports specifically the Dockerfile format) or a prebuilt container image.

These methods offer greater flexibility for advanced use cases where code is managed remotely or dependencies exceed what’s provided in the default base stacks.


πŸš€ Option 1: Run Routine from Git Repository ​

Instead of referencing a local folder with job_folder, you can point to a Git repository using:

  • repo_url: The HTTPS URL of the Git repository
  • repo_branch: The name of the branch containing the code

Updated wkube.py Example ​

python
from accli import WKubeTask

task = WKubeTask(
    name="My Git-based Routine",
    repo_url="https://github.com/your-org/your-repo.git",
    repo_branch="main",
    base_stack="PYTHON3_7",
    command="python script.py",
    required_cores=1,
    required_ram=512*1024*1024,
    required_storage_local=1024*1024*2,
    required_storage_workflow=1024*1024,
    timeout=3600,
    conf={
        "input_mappings": "acc://mybucket/input.csv:/code/input.csv",
        "output_mappings": "/code/output/:acc://mybucket/output/"
    }
)

βœ… Note: repo_url and repo_branch are both required.


🧱 Adding Custom Dependencies with a Containerfile ​

If the base stack is not enough for your task, you can specify a custom containerfile to build an environment tailored to your needs.

Use the docker_filename parameter:

python
docker_filename="Dockerfile"
  • The value is the path to your containerfile relative to the root of the Git repo or project folder
  • This can be used with either:
    • job_folder (for local projects)
    • repo_url + repo_branch (for Git-based projects)

Example with a containerfile ​

python
task = WKubeTask(
    name="Routine with Custom Containerfile",
    repo_url="https://github.com/your-org/your-repo.git",
    repo_branch="main",
    docker_filename="Dockerfile",
    command="python script.py",
    ...
)

πŸ“¦ This will trigger the platform to build a new container image from the containerfile before running the job.


πŸ›°οΈ Option 2: Run with Prebuilt Container Image ​

You may also provide a fully built container image containing both the code and all dependencies. In this case:

  • Omit: job_folder, repo_url, repo_branch, and docker_filename
  • Use only: docker_image

Example ​

python
task = WKubeTask(
    name="Routine from Container Image",
    docker_image="docker.io/yourname/yourimage:latest",
    command="python script.py",
    ...
)

πŸ“Œ This method is ideal for fully packaged, production-ready workflows.


🧠 Summary ​

ModeRequired KeysDescription
Local Projectjob_folderDefault mode for local dev
Git Reporepo_url, repo_branchPull code from remote Git source
Custom Containerfiledocker_filename (w/ job_folder or Git)Build custom image for job
Prebuilt Container Imagedocker_imageRun using your own container image

βœ… Best Practices ​

  • Make sure paths in docker_filename are relative to project root
  • Use prebuilt images for highly stable or performance-optimized jobflows.

This approach gives you full control over how your routine is sourced and how it runs, supporting everything from rapid prototyping to enterprise-grade workflows.