Build Data Science pipeline step configurations with job dependencies and infrastructure settings.
Last verified: May 2026
Build Data Science pipeline step configurations with job dependencies, infrastructure, and environment settings.
Required Fields
compartmentIdprojectIddisplayNamepipelineStepsOutput will appear here...Build Data Science pipeline step configurations with job dependencies and infrastructure settings. This tool helps OCI engineers generate valid configurations quickly without consulting documentation, reducing errors and accelerating infrastructure deployment. All processing runs in your browser with no data sent to external servers.
Your data science team is repeatedly running the same training workflow (data prep → train → validate → register in catalog) by manually launching individual jobs. The builder generates a Pipeline definition: 4 steps with proper dependencies, infrastructure right-sized per step (small VM for prep, GPU for training, CPU for validation, small VM for registration). Now the team triggers entire workflow runs with one command. Each run is tracked with full lineage — when a production model misbehaves, the team has the exact training run, data version, and parameters to debug.
Pipelines orchestrate ML workflows: data prep → training → validation → deployment. Each step runs in its own job with its own infrastructure (CPU for data prep, GPU for training, CPU for inference). Without pipelines, you'd manually coordinate jobs and pass artifacts — error-prone and not reproducible.
Use parameter overrides to run the same pipeline with different hyperparameters. The base pipeline definition stays stable; runtime parameters control learning rates, batch sizes, model architectures. This is how teams do hyperparameter tuning at scale.
Pipeline runs are versioned and tracked — every execution has full lineage. When a model misbehaves in production, you can trace back through the exact pipeline run, parameters, and data versions that produced it. Without this, ML model debugging is nearly impossible.
The builder constructs OCI Data Science pipeline configurations: pipeline resource (compartment, project association, configuration overrides), pipeline steps (each with step type: ML_JOB or CUSTOM_SCRIPT, dependency array referencing prerequisite step names, custom container image references, infrastructure config: shape + count, environment variables, parameter overrides). Output is generated as oci data-science pipeline commands and Terraform oci_datascience_pipeline resources.
Was this tool helpful?
Disclaimer: This tool runs entirely in your browser. No data is sent to our servers. Always verify outputs before using them in production. AWS, Azure, and GCP are trademarks of their respective owners.