π§© Routine from Remote Source and Custom Dependency β
This guide extends the basic "Getting Started" instructions by showing how to run routines from a remote Git repository and how to define custom execution environments using either a containerfile (the Accelerator currently supports specifically the Dockerfile format) or a prebuilt container image.
These methods offer greater flexibility for advanced use cases where code is managed remotely or dependencies exceed whatβs provided in the default base stacks.
π Option 1: Run Routine from Git Repository β
Instead of referencing a local folder with job_folder, you can point to a Git repository using:
repo_url: The HTTPS URL of the Git repositoryrepo_branch: The name of the branch containing the code
Updated wkube.py Example β
from accli import WKubeTask
task = WKubeTask(
name="My Git-based Routine",
repo_url="https://github.com/your-org/your-repo.git",
repo_branch="main",
base_stack="PYTHON3_7",
command="python script.py",
required_cores=1,
required_ram=512*1024*1024,
required_storage_local=1024*1024*2,
required_storage_workflow=1024*1024,
timeout=3600,
conf={
"input_mappings": "acc://mybucket/input.csv:/code/input.csv",
"output_mappings": "/code/output/:acc://mybucket/output/"
}
)β Note:
repo_urlandrepo_branchare both required.
π§± Adding Custom Dependencies with a Containerfile β
If the base stack is not enough for your task, you can specify a custom containerfile to build an environment tailored to your needs.
Use the docker_filename parameter:
docker_filename="Dockerfile"- The value is the path to your containerfile relative to the root of the Git repo or project folder
- This can be used with either:
job_folder(for local projects)repo_url+repo_branch(for Git-based projects)
Example with a containerfile β
task = WKubeTask(
name="Routine with Custom Containerfile",
repo_url="https://github.com/your-org/your-repo.git",
repo_branch="main",
docker_filename="Dockerfile",
command="python script.py",
...
)π¦ This will trigger the platform to build a new container image from the containerfile before running the job.
π°οΈ Option 2: Run with Prebuilt Container Image β
You may also provide a fully built container image containing both the code and all dependencies. In this case:
- Omit:
job_folder,repo_url,repo_branch, anddocker_filename - Use only:
docker_image
Example β
task = WKubeTask(
name="Routine from Container Image",
docker_image="docker.io/yourname/yourimage:latest",
command="python script.py",
...
)π This method is ideal for fully packaged, production-ready workflows.
π§ Summary β
| Mode | Required Keys | Description |
|---|---|---|
| Local Project | job_folder | Default mode for local dev |
| Git Repo | repo_url, repo_branch | Pull code from remote Git source |
| Custom Containerfile | docker_filename (w/ job_folder or Git) | Build custom image for job |
| Prebuilt Container Image | docker_image | Run using your own container image |
β Best Practices β
- Make sure paths in
docker_filenameare relative to project root - Use prebuilt images for highly stable or performance-optimized jobflows.
This approach gives you full control over how your routine is sourced and how it runs, supporting everything from rapid prototyping to enterprise-grade workflows.