MSDN Documentation

Troubleshooting Azure Cosmos DB Storage

This document provides guidance on diagnosing and resolving common issues encountered when working with Azure Cosmos DB, focusing on storage-related aspects.

Common Storage-Related Issues

1. High Storage Consumption

Unexpectedly high storage usage can impact costs and performance. Here are common causes and solutions:

• Large Documents: Review your document size. Consider if all data needs to be stored within a single document. You might benefit from de-normalization or splitting large documents.
• Indexing: While indexing improves query performance, it also consumes storage. Analyze your indexing policy. Remove indexes that are not frequently used or necessary.
• Time-to-Live (TTL): Ensure that TTL is configured correctly if you intend to automatically expire data. Incorrect TTL settings might lead to data persisting longer than expected.
• Deleted Data (Logical Deletion): If you implement logical deletion (e.g., marking documents as deleted), ensure a mechanism is in place to purge these documents physically, or consider using TTL.

Troubleshooting Steps:

1. Use Azure Monitor to track the DocumentSize metric.
2. Inspect your indexing policy via the Azure portal or SDK.
3. Verify TTL configuration in your container settings.
4. Query your data to identify potentially "deleted" but still stored documents.

2. Slow Ingestion or Write Operations

When data cannot be written to Cosmos DB at the expected rate, it can manifest as slow ingestion.

• Throughput Provisioning: Ensure you have provisioned sufficient Request Units (RUs) for your write operations. Consider autoscale throughput.
• Request Rate Too Large (429 Errors): This is a direct indicator of exceeding provisioned throughput. Implement exponential backoff and retry logic in your application.
• Large Document Size: Larger documents consume more RUs for writes. Optimize document size as mentioned above.
• Indexing Latency: Complex indexing policies or indexing on frequently updated fields can increase write latency.

Troubleshooting Steps:

1. Monitor ProvisionedThroughput and ConsumedDocumentStorage metrics in Azure Monitor.
2. Track the frequency and percentage of 429 (Request Rate Too Large) responses in your application logs or Azure Monitor.
3. Analyze the RU consumption per request using the x-ms-request-charge header.

3. Data Consistency Issues

While Cosmos DB offers tunable consistency levels, understanding their implications is crucial.

• Strong Consistency: Guarantees that reads always return the most recent committed write. This comes with higher latency.
• Bounded Staleness: Reads may be up to a specified number of versions or time behind the leader.
• Session, Consistent Prefix, Eventual: Offer lower latency but may return stale data.

If you're experiencing unexpected data staleness, verify your chosen consistency level and ensure your application logic correctly handles potential inconsistencies based on that level.

4. Indexing Performance Degradation

As your dataset grows and your access patterns evolve, indexing performance can sometimes degrade.

• Inefficient Indexes: Regularly review your indexing policy. Remove unused indexes. Consider index exclusions for specific paths if they are not queried.
• Index Size: Indexes consume storage and can increase the cost of write operations.
• Wildcard Indexes: Use wildcard indexes (/*) judiciously, as they can impact performance and storage.

Troubleshooting Steps:

1. Use the cosmosdb-index-tool (if available for your SDK version) or manual analysis to identify index usage and potential inefficiencies.
2. Review the "Index Management" section of your Cosmos DB account in the Azure portal.

Tools and Resources

• Azure Monitor: Essential for tracking metrics like storage, RU consumption, latency, and error rates.
• Azure Portal: Provides a visual interface for managing your Cosmos DB account, containers, indexing policies, and viewing metrics.
• Cosmos DB Emulator: A local development tool that mimics Cosmos DB. Useful for testing and debugging without incurring cloud costs.
• SDKs and APIs: Use the diagnostic logging capabilities of your chosen SDK to capture detailed information about requests and responses.