Azure Cosmos DB Documentation

Azure Cosmos DB: Partition Key Best Practices

Choosing the right partition key is crucial for the performance, scalability, and cost-effectiveness of your Azure Cosmos DB solution. A well-designed partition key distributes your data and request load evenly across physical partitions.

Understanding the Goal

The primary goals of a good partition key are:

• Even Data Distribution: Ensure data is spread as evenly as possible across all available partitions.
• Even Request Distribution: Distribute read and write operations uniformly to avoid hot partitions.
• Scalability: Allow the database to scale horizontally by adding more partitions as your data grows.
• Cost Efficiency: Optimize Request Unit (RU) consumption by preventing bottlenecks.

Key Characteristics of a Good Partition Key

A good partition key typically has:

• High Cardinality: A large number of distinct values. This helps in distributing data across more partitions.
• Inclusiveness: The partition key should be present in most (ideally all) documents within a logical partition.
• Query Predictability: Queries should be able to target a specific set of partitions efficiently.

Common Partition Key Anti-Patterns to Avoid

• Low Cardinality: A partition key with very few unique values (e.g., a boolean flag, a status). This leads to hot partitions.
• Sequential or Ordered Values: Using timestamps or auto-incrementing IDs as partition keys can lead to hot partitions, as recent data is always written to the same partition.
• Uncommon or Irregularly Accessed Values: If a small subset of partition key values are queried far more often than others, it can create hot spots.
• Non-Deterministic Values: Avoid partition keys whose values might change frequently, as this can lead to data migration and complexity.

Strategies for Choosing a Partition Key

1. Leverage Existing Properties with High Cardinality

Look for properties within your documents that naturally have a wide variety of values. Common examples include:

• User IDs
• Tenant IDs (for multi-tenant applications)
• Geographic Locations (e.g., country, city)
• Product IDs

2. Composite Partition Keys

If a single property doesn't offer enough cardinality, consider combining two properties into a composite partition key. This is achieved by concatenating their values (often with a delimiter).

// Example data
{
    "id": "doc1",
    "userId": "user123",
    "eventDate": "2023-10-27T10:00:00Z",
    "eventType": "login",
    "details": "..."
}

// Composite partition key value: "user123#2023-10-27"
// This distributes data by user and then by date for that user.
                

3. Use a "Synthetic" Partition Key

If no suitable property exists, you can create a synthetic property that provides good distribution. This might involve hashing a value or using a combination of properties.

4. Partitioning for Specific Query Patterns

Design your partition key based on your most frequent and critical query patterns. If you often query by user ID and then by date, a composite key like userId#date can be effective.

Best Practices Summary

• Analyze your data and query patterns thoroughly.
• Prioritize high cardinality.
• Avoid sequential or low-cardinality keys.
• Consider composite keys for better distribution.
• For multi-tenant applications, use the tenant ID as the partition key.
• Monitor your partition utilization regularly for hot spots.
• If necessary, plan for partition key migration if your initial choice proves suboptimal.

Migrating Partition Keys

Changing a partition key after data has been inserted is a complex operation. It typically involves creating a new container with the desired partition key, copying data from the old container to the new one, and then deleting the old container. Azure Cosmos DB provides tools and guidance for this process.

For detailed information and advanced scenarios, please refer to the official Azure Cosmos DB documentation on partitioning.