Azure Machine Learning SDK Documentation

Introduction to Azure Machine Learning SDK

The Azure Machine Learning SDK for Python is a powerful tool for data scientists and ML engineers to develop, train, and manage machine learning solutions on Azure. It provides a high-level abstraction over Azure services, allowing you to focus on your ML workflows.

With the SDK, you can:

Connect to your Azure Machine Learning workspace.
Manage compute resources for training and deployment.
Prepare and version your data.
Train models using various frameworks (TensorFlow, PyTorch, scikit-learn, etc.).
Deploy models as real-time endpoints or batch inference jobs.
Track experiments, log metrics, and version models.
Automate your ML pipelines for MLOps.

Getting Started

To begin using the Azure Machine Learning SDK, you need an Azure subscription and an Azure Machine Learning workspace. Follow these steps:

Install the SDK:
pip install azure-ai-ml azure-identity
Authenticate: Use Azure Identity to authenticate. The most common method for local development is the DefaultAzureCredential.
from azure.identity import DefaultAzureCredential from azure.ai.ml import MLClient # Authenticate using your Azure identity credential = DefaultAzureCredential() # Specify your workspace details subscription_id = "YOUR_SUBSCRIPTION_ID" resource_group = "YOUR_RESOURCE_GROUP" workspace_name = "YOUR_WORKSPACE_NAME" # Initialize MLClient ml_client = MLClient(credential, subscription_id, resource_group, workspace_name) print("Successfully connected to Azure ML Workspace.")
Explore your workspace:
print(f"Workspace Name: {ml_client.workspace_name}") print(f"Location: {ml_client.workspace_location}")

Core Concepts

Workspaces

An Azure Machine Learning workspace is the top-level resource that provides a centralized place to work with all the artifacts you create when you use Azure Machine Learning. It's an organized repository for your ML projects.

Compute Targets

Compute targets are specific environments where you run your training jobs or deploy your models. The SDK supports various compute types:

Compute Instance: A managed cloud-based workstation.
Compute Cluster: Scalable clusters of VMs for training.
Inference Cluster: Kubernetes clusters for model deployment.
Attached Compute: Existing Azure compute resources like Azure Databricks or HDInsight.

Datastores

Datastores are references to storage locations in your Azure subscription. They allow you to securely connect to data stored in Azure Blob Storage, Azure Data Lake Storage, or Azure SQL Database without exposing credentials.

Datasets

Datasets are abstractions representing data in your datastores. They enable efficient data access, versioning, and integration with training jobs.

Experiments

Experiments are logical groupings of related runs. They help you organize and compare different training attempts or model evaluations.

Runs

A run represents a single execution of your training script or pipeline. Each run records metrics, parameters, and outputs, providing a detailed history of your experiments.

Models

A model is a trained artifact that you register in your workspace. Once registered, you can version and deploy models.

Endpoints

Endpoints provide a way to expose your deployed models for real-time inference or batch scoring, making them accessible to applications.

SDK Features

Data Preparation

The SDK simplifies data loading, transformation, and feature engineering. You can easily create and manage Datasets.

Model Training

Integrate with popular ML frameworks like TensorFlow, PyTorch, and scikit-learn. Submit training jobs to managed compute targets and track their progress.

from azure.ai.ml.entities import CommandJob

job = CommandJob(
    code="./src",
    command="python train.py --learning-rate ${{inputs.learning_rate}}",
    inputs={
        "training_data": Input(type="uri_folder", path="azureml://datastores/workspaceblobstore/paths/mnist/training"),
        "learning_rate": 0.01,
    },
    environment="azureml://registries/azureml/environments/sklearn-1.0/versions/1",
    compute="cpu-cluster",
    display_name="mnist-sklearn-training",
)

returned_job = ml_client.jobs.create_or_update(job)
print(f"Submitted job: {returned_job.name}")
                    

Model Deployment

Deploy your registered models to managed endpoints for real-time scoring or batch inference.

from azure.ai.ml.entities import OnlineEndpoint, ManagedOnlineDeployment

# Create an online endpoint
endpoint = OnlineEndpoint(name="my-online-endpoint", description="Online endpoint for my model")
ml_client.online_endpoints.begin_create_or_update(endpoint).result()

# Create a managed online deployment
deployment = ManagedOnlineDeployment(
    name="blue",
    endpoint_name="my-online-endpoint",
    model=ml_client.models.get(name="my-registered-model", version="1"),
    instance_type="Standard_DS3_v2",
    instance_count=1,
)
ml_client.online_deployments.begin_create_or_update(deployment).result()
                    

MLOps & Automation

Build reproducible and automated ML pipelines using the SDK. Manage model lifecycle, CI/CD integration, and model governance.

API Reference

The SDK provides a comprehensive set of classes and functions for interacting with Azure Machine Learning. Key modules include:

`MLClient`

The primary client for interacting with your Azure ML workspace.

MLClient(credential, subscription_id, resource_group, workspace_name)

`azure.ai.ml.entities`

Contains classes for defining ML assets like Jobs, Models, Endpoints, Compute, Datastores, and Environments.

CommandJob, PipelineJob, Model, OnlineEndpoint, ManagedOnlineDeployment, Environment

`azure.ai.ml.operations`

Provides methods for managing and interacting with ML assets.

ml_client.jobs.create_or_update(), ml_client.models.register(), ml_client.online_endpoints.begin_create_or_update()

For detailed API documentation, please refer to the official Azure ML Python SDK documentation.

Examples

Explore the following resources for practical examples of using the Azure Machine Learning SDK:

Navigation