Data Explorer Pool in Azure Synapse Analytics

This article provides a detailed reference for the Data Explorer pool (also known as Kusto pools) within Azure Synapse Analytics. Data Explorer pools are optimized for log and telemetry analytics, offering powerful Kusto Query Language (KQL) capabilities for real-time data exploration.

Key Features

  • Kusto Query Language (KQL): Leverage the expressive and efficient KQL for querying large datasets.
  • Real-time Ingestion: Stream data into the pool for near real-time analysis.
  • Scalability: Dynamically scale compute and storage resources to meet demand.
  • Data Formats: Supports various data formats including CSV, JSON, Parquet, Avro, and more.
  • Integration: Seamless integration with other Azure services like Azure Data Lake Storage, Azure Blob Storage, and Power BI.
  • Machine Learning: Built-in support for machine learning functions within KQL.

Data Explorer Pool SKU and Configuration

Data Explorer pools are provisioned with specific SKUs that determine performance and cost. The primary configuration options include:

Compute SKU

Determines the processing power and memory of the nodes in the pool. Examples include:

  • ComputeSKU: A string representing the SKU name (e.g., "Compute-Optimized-4", "General-Purpose-2").

Node Count

The number of compute nodes allocated to the pool.

  • NodeCount: An integer representing the number of nodes.

Cache Size

The amount of memory allocated for caching data, improving query performance.

  • CacheSizeMB: An integer representing the cache size in megabytes.

Management Operations

You can manage Data Explorer pools using Azure portal, Azure CLI, PowerShell, or REST API. Common operations include:

  • Creating a Data Explorer Pool: Provisioning a new pool with specified configurations.
  • Scaling a Data Explorer Pool: Adjusting the number of nodes or compute SKU.
  • Deleting a Data Explorer Pool: Removing an existing pool.
  • Monitoring: Tracking performance metrics and logs.

KQL Examples

Here are some common KQL query examples:

// Count records in a table
MyTable
| count

// Filter records based on a condition
MyTable
| where Timestamp > ago(1h) and EventLevel == "Error"
| project Timestamp, Message

// Aggregate data
MyTable
| summarize count() by EventLevel, bin(Timestamp, 1m)
| order by Timestamp asc, EventLevel asc

Related Topics