AWS SageMaker Updates: End-to-End Machine Learning Features

Approximately three years following its initial release, Amazon Web Services’ SageMaker platform has received a substantial enhancement through the introduction of new functionalities. These improvements aim to streamline and expand the capabilities of developers in automating and scaling each phase of the process for constructing advanced automation and machine learning applications, as stated by the company.

With machine learning becoming increasingly prevalent, various departments within organizations are discovering applications for automation. AWS is focused on simplifying the creation of these customized applications for its user base.

“A significant advantage of a widely used service like SageMaker is the wealth of feedback we receive from customers, which directly influences our future development efforts,” explained Swami Sivasubramanian, AWS vice president of machine learning. “Today, we are introducing a suite of tools for Amazon SageMaker designed to significantly simplify the process for developers to create complete machine learning pipelines. These pipelines will facilitate the preparation, construction, training, explanation, inspection, monitoring, debugging, and deployment of tailored machine learning models, all with increased visibility, interpretability, and automation at scale.”

According to AWS, companies such as 3M, ADP, AstraZeneca, Avis, Bayer, Capital One, Cerner, Domino’s Pizza, Fidelity Investments, Lenovo, Lyft, T-Mobile, and Thomson Reuters are already integrating SageMaker tools into their operations.

Among the new offerings is Amazon SageMaker Data Wrangler, which provides a method for standardizing data originating from diverse sources, ensuring consistent usability. Data Wrangler also simplifies the process of consolidating various data sources into relevant features for data analysis. This tool incorporates over 300 pre-built data transformation functions, enabling customers to normalize, modify, and combine features without the need for coding.

Amazon also introduced the Feature Store, which enables customers to establish repositories for simplified storage, updating, retrieval, and sharing of machine learning features used for both training and inference.

Another newly released tool from Amazon Web Services is Pipelines, a workflow management and automation system. Pipelines offers orchestration and automation capabilities comparable to those found in conventional programming. This system allows developers to define each step within a complete machine learning workflow, as detailed in a company statement. Developers can utilize these tools to reproduce an entire workflow from SageMaker Studio with identical settings, guaranteeing consistent model results, or to rerun the workflow with updated data for model refinement.

Addressing the persistent challenge of data bias in artificial intelligence and machine learning models, Amazon launched SageMaker Clarify. Announced recently, this tool is designed to detect bias throughout the machine learning workflow, empowering developers to prioritize transparency in model development. While acknowledging the existence of open-source tools for similar testing, Amazon notes that these tools are typically manual and demand significant effort from developers.

Further products aimed at simplifying machine learning application development include SageMaker Debugger, which accelerates model training by monitoring system resource usage and alerting developers to potential performance limitations; Distributed Training, which accelerates the training of large, complex deep learning models by automatically distributing data across multiple GPUs; and SageMaker Edge Manager, a model management tool for edge devices, enabling developers to optimize, secure, monitor, and manage models deployed on edge device fleets.

Finally, Amazon unveiled SageMaker JumpStart, offering developers a searchable interface to discover algorithms and example notebooks to facilitate their machine learning endeavors. The company states that this will provide developers new to machine learning with the option to select and deploy several pre-built machine learning solutions within SageMaker environments.