.NET SDK – Getting Started with Azure Data Lake Storage Gen2

This guide walks you through creating a simple .NET console application that connects to Azure Data Lake Storage Gen2, creates a file system, and uploads/downloads a file.

Prerequisites

• Azure subscription (free trial works)
• Azure Storage account with hierarchical namespace enabled
• .NET 6.0 SDK or later (download)
• Visual Studio 2022 or VS Code
• Azure CLI (optional, for creating resources)

Install the SDK

Run the following command in your console to add the Azure.Storage.Files.DataLake package:

dotnet add package Azure.Storage.Files.DataLake --version 12.*

Authenticate

You can authenticate using a connection string, a shared key, or Azure AD. Below is an example using a connection string stored in appsettings.json:

{
  "Storage": {
    "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=<account_name>;AccountKey=<account_key>;EndpointSuffix=core.windows.net"
  }
}

Read the configuration in code:

using Azure.Storage.Files.DataLake;
using Microsoft.Extensions.Configuration;

var config = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json", optional: false)
    .Build();

string connectionString = config["Storage:ConnectionString"];
var serviceClient = new DataLakeServiceClient(connectionString);

Create a File System

Once you have a DataLakeServiceClient, create a file system (container) if it doesn't exist:

string fileSystemName = "sample-fs";
var fileSystemClient = serviceClient.GetFileSystemClient(fileSystemName);
await fileSystemClient.CreateIfNotExistsAsync();

Upload & Download Files

The snippet below uploads a local sample.txt file, then reads it back.

string localPath = "sample.txt";
string remotePath = "data/sample.txt";

var fileClient = fileSystemClient.GetFileClient(remotePath);

// Upload
await using var fileStream = File.OpenRead(localPath);
await fileClient.UploadAsync(fileStream, overwrite:true);

// Download
string downloadPath = "downloaded.txt";
await using var downloadStream = File.OpenWrite(downloadPath);
await fileClient.ReadAsync(downloadStream);
Console.WriteLine($"File downloaded to {downloadPath}");

Cleanup

Delete the file system when you're done (optional):

await fileSystemClient.DeleteIfExistsAsync();