Process Data in Azure Analysis Services
After creating or modifying your Azure Analysis Services (AAS) tabular or multidimensional model, you need to populate it with data. This process is called "processing." Processing updates the data in the model by querying the underlying data sources and loading the results into the model's in-memory cache. You can process entire models, individual tables, or partitions.
Why Process Data?
- Initial Data Load: To populate your model with data for the first time.
- Data Refresh: To update your model with the latest data from your sources.
- Schema Changes: After making changes to the data source or model metadata that require a data refresh.
Processing Methods
You can process data using several methods:
- SQL Server Management Studio (SSMS): A graphical tool that provides a user-friendly interface for managing AAS.
- Tabular Editor: A popular third-party tool for managing tabular models, offering advanced features for processing.
- Analysis Services Management Objects (AMO) and TOM: Programmatic interfaces for automating processing tasks.
- Azure portal: For basic processing operations and monitoring.
Processing Types
There are different types of processing that determine how data is updated:
- Full Process: Clears all existing data in the selected object (model, table, or partition) and then reloads it. This is the most comprehensive but can be time-consuming.
- Data Process: Reloads data into the selected object without clearing existing data. This is faster but may not be suitable if there are significant structural changes.
- Add Process: Appends new data to existing data in the selected object. This is useful for incremental loads.
- Calculation Group Process: Processes calculation groups.
- Clear Process: Clears all data from the selected object without reloading it.
Processing an Entire Model
This section describes how to perform a full process on your entire model using SSMS.
Processing Individual Tables or Partitions
Processing individual tables or partitions is often more efficient, especially for large models.
Automating Processing with AMO/TOM
For scheduled or automated processing, you can use AMO (Analysis Services Management Objects) or TOM (Tabular Object Model). Here's a simplified C# example using TOM:
using Microsoft.AnalysisServices.Tabular;
using System;
// ...
// Connect to the Azure Analysis Services server
string serverName = "your_aas_server.asazure.windows.net";
string databaseName = "your_database_name";
string connectionString = $"Provider=MSOLAP;Data Source={serverName};Initial Catalog={databaseName};Impersonation Level=Impersonate;";
using (var server = new Server())
{
server.ConnectionString = connectionString;
server.Connect();
var database = server.Databases.GetByName(databaseName);
// Process the entire database
Console.WriteLine($"Processing database: {database.Name}");
var processingResult = database.Process(ProcessType.Full); // Or ProcessType.Data, ProcessType.Add
if (processingResult.Status == Microsoft.AnalysisServices.OperationStatus.Completed)
{
Console.WriteLine("Database processed successfully!");
}
else
{
Console.WriteLine($"Database processing failed. Status: {processingResult.Status}, Error: {processingResult.ErrorDescription}");
}
// Or process a specific table
// var table = database.Tables.GetByName("YourTableName");
// Console.WriteLine($"Processing table: {table.Name}");
// var tableProcessingResult = table.Process(ProcessType.Full);
// ... handle result ...
}
Monitoring Processing Jobs
You can monitor the status of your processing jobs in the Azure portal by navigating to your Analysis Services resource, then selecting Models, and clicking on the relevant database. The Activity tab will show recent processing operations and their outcomes.
Troubleshooting Processing Errors
If a processing job fails, check the following:
- Data Source Credentials: Ensure that the credentials used to connect to the data source are valid and have sufficient permissions.
- Data Source Availability: Verify that the data source is accessible from Azure Analysis Services.
- Query Errors: Examine the error messages for any issues with the queries used to extract data from the source.
- Resource Constraints: For very large datasets, consider scaling up your AAS instance or optimizing your model design.
Detailed error information can often be found in SSMS logs or by examining the results of programmatic processing operations.