Posted On: Jun 10, 2021
Amazon SageMaker Pipelines, the first purpose-built continuous integration and continuous delivery (CI/CD) service for machine learning (ML), now supports a new callback step that allows customers to integrate any task or job outside Amazon SageMaker as a step in the model building pipeline. When a callback step is invoked, the current execution of a SageMaker model building pipeline will pause and wait for an external task or job to return a task token that was generated by SageMaker at the start of call back step execution. You can use the call back step to include processing jobs external to SageMaker such a Spark job running on an Amazon EMR cluster or an extract-transform-load (ETL) task in AWS Glue as part of the SageMaker model building pipeline.
The new callback capability in SageMaker Pipelines enables customers to leverage other AWS services as part of the ML workflow. This feature is available in all regions where SageMaker Pipelines is available. To learn more, please visit the documentation page.