Building an Image Analysis Application with Azure AI Services

This tutorial will guide you through the process of creating a powerful image analysis application using Azure AI Services. We'll leverage the Azure Computer Vision service to extract meaningful information from images, such as object detection, facial recognition, and content moderation.

Prerequisites

  • An Azure subscription. If you don't have one, sign up for a free account.
  • Visual Studio Code or your preferred IDE.
  • Node.js and npm installed.
  • Basic understanding of JavaScript and web development.

Step 1: Set up Azure Resources

First, you need to create an Azure AI Services resource that includes the Computer Vision capability. This will provide you with an endpoint and an API key to authenticate your requests.

  1. Navigate to the Azure portal.
  2. Click on "Create a resource".
  3. Search for "Azure AI Services" and select it.
  4. Click "Create".
  5. Fill in the required details:
    • Subscription: Choose your Azure subscription.
    • Resource group: Create a new one or select an existing one.
    • Region: Select a region close to you.
    • Name: A unique name for your resource (e.g., my-image-analyzer).
    • Type of AI resource: Select "Multi-service account".
    • Pricing tier: Choose a suitable tier (e.g., the free tier for testing).
  6. Review and create the resource.

Once the resource is deployed, navigate to it in the Azure portal. Under "Keys and Endpoint", you'll find your Endpoint and Key1 (or Key2). Keep these handy; you'll need them later.

Step 2: Initialize your Project

Let's set up a simple Node.js project for our application.


mkdir azure-image-analyzer
cd azure-image-analyzer
npm init -y
npm install @azure/cognitiveservices-computervision express dotenv

Create a .env file in your project's root directory and add your Azure credentials:


COMPUTER_VISION_ENDPOINT=YOUR_AZURE_COMPUTER_VISION_ENDPOINT
COMPUTER_VISION_KEY=YOUR_AZURE_COMPUTER_VISION_KEY

Replace YOUR_AZURE_COMPUTER_VISION_ENDPOINT and YOUR_AZURE_COMPUTER_VISION_KEY with the values you obtained from the Azure portal.

Step 3: Create the Backend API

Create a file named server.js and add the following code to handle image analysis requests:


require('dotenv').config();
const express = require('express');
const ComputerVisionClient = require('@azure/cognitiveservices-computervision');
const msRest = require('@azure/ms-rest-js');

const app = express();
app.use(express.json());
app.use(express.static('public')); // For serving static frontend files

const endpoint = process.env.COMPUTER_VISION_ENDPOINT;
const key = process.env.COMPUTER_VISION_KEY;

const computerVisionClient = new ComputerVisionClient(
    new msRest.ApiKeyCredentials(key, { 'Ocp-Apim-Subscription-Key': key }),
    endpoint
);

app.post('/analyze-image', async (req, res) => {
    const { imageUrl } = req.body;

    if (!imageUrl) {
        return res.status(400).json({ error: 'imageUrl is required' });
    }

    try {
        const analysisResult = await computerVisionClient.analyzeImage(imageUrl, {
            visualFeatures: ['Categories', 'Tags', 'Description', 'Objects', 'Faces', 'ImageType', 'Adult'],
        });
        res.json(analysisResult);
    } catch (error) {
        console.error('Error analyzing image:', error);
        res.status(500).json({ error: 'Failed to analyze image' });
    }
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
    console.log(`Server running on port ${PORT}`);
});

Step 4: Build the Frontend

Create a public folder in your project root and inside it, create an index.html file and a style.css file.

public/index.html


<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Azure Image Analyzer</title>
    <link rel="stylesheet" href="style.css">
</head>
<body>
    <div class="container">
        <h1>Azure Image Analysis Tool</h1>
        <div class="input-section">
            <label for="imageUrl">Enter Image URL:</label>
            <input type="text" id="imageUrl" placeholder="e.g., https://example.com/image.jpg">
            <button id="analyzeBtn">Analyze Image</button>
        </div>
        <div id="results" class="results-section">
            <img id="previewImage" src="" alt="Image Preview">
            <h2>Analysis Results</h2>
            <div id="analysisOutput"></div>
        </div>
    </div>
    <script src="script.js"></script>
</body>
</html>

public/style.css


:root {
    --primary-color: #0078D4;
    --secondary-color: #60A5FA;
    --background-color: #F9FAFB;
    --surface-color: #FFFFFF;
    --text-primary: #111827;
    --text-secondary: #6B7280;
    --border-color: #E5E7EB;
    --shadow-color: rgba(0, 0, 0, 0.1);
}

body {
    font-family: 'Inter', sans-serif;
    line-height: 1.6;
    color: var(--text-primary);
    background-color: var(--background-color);
    margin: 0;
    padding: 0;
}

.container {
    max-width: 1000px;
    margin: 40px auto;
    padding: 30px;
    background-color: var(--surface-color);
    border-radius: 12px;
    box-shadow: 0 4px 20px var(--shadow-color);
}

h1 {
    text-align: center;
    color: var(--primary-color);
    margin-bottom: 30px;
    font-size: 2.2em;
}

.input-section {
    display: flex;
    flex-direction: column;
    align-items: center;
    gap: 15px;
    margin-bottom: 40px;
    padding: 25px;
    background-color: var(--background-color);
    border-radius: 8px;
}

.input-section label {
    font-weight: 600;
    font-size: 1.1em;
}

.input-section input[type="text"] {
    width: 100%;
    max-width: 400px;
    padding: 12px 15px;
    border: 1px solid var(--border-color);
    border-radius: 6px;
    font-size: 1em;
    outline: none;
    transition: border-color 0.3s ease;
}

.input-section input[type="text"]:focus {
    border-color: var(--primary-color);
}

button {
    background-color: var(--primary-color);
    color: white;
    border: none;
    padding: 12px 25px;
    border-radius: 6px;
    font-size: 1.1em;
    cursor: pointer;
    transition: background-color 0.3s ease, transform 0.2s ease;
    font-weight: 600;
}

button:hover {
    background-color: var(--secondary-color);
    transform: translateY(-2px);
}

button:active {
    transform: translateY(0);
}

.results-section {
    text-align: center;
    margin-top: 40px;
    padding-top: 30px;
    border-top: 1px solid var(--border-color);
}

#previewImage {
    max-width: 100%;
    max-height: 400px;
    margin-bottom: 30px;
    border-radius: 8px;
    box-shadow: 0 4px 15px var(--shadow-color);
    display: none; /* Hidden by default */
}

.results-section h2 {
    color: var(--primary-color);
    margin-bottom: 20px;
    font-size: 1.8em;
}

#analysisOutput {
    text-align: left;
    font-size: 1.05em;
    color: var(--text-secondary);
    background-color: var(--background-color);
    padding: 30px;
    border-radius: 8px;
}

#analysisOutput h3 {
    color: var(--primary-color);
    margin-top: 0;
    margin-bottom: 15px;
    font-size: 1.3em;
}

#analysisOutput p {
    margin-bottom: 15px;
}

#analysisOutput ul {
    padding-left: 25px;
    margin-bottom: 15px;
}

#analysisOutput li {
    margin-bottom: 8px;
}

#analysisOutput strong {
    color: var(--text-primary);
}

/* Specific styling for different analysis types */
.categories, .tags, .objects, .faces, .adult-content {
    margin-bottom: 25px;
    padding: 20px;
    background-color: var(--surface-color);
    border-radius: 8px;
    box-shadow: 0 2px 8px rgba(0,0,0,0.05);
}

.adult-content {
    border-left: 5px solid orange;
}

.categories ul, .tags ul, .objects ul {
    list-style: disc;
    padding-left: 30px;
}

.objects ul li {
    display: flex;
    justify-content: space-between;
    align-items: center;
    margin-bottom: 10px;
}

.object-confidence {
    font-size: 0.9em;
    color: var(--text-secondary);
    background-color: #e9ecef;
    padding: 3px 7px;
    border-radius: 4px;
}

.faces ul {
    list-style: none;
    padding: 0;
    display: grid;
    grid-template-columns: repeat(auto-fill, minmax(150px, 1fr));
    gap: 20px;
}

.face-item {
    border: 1px solid var(--border-color);
    border-radius: 6px;
    padding: 15px;
    background-color: var(--background-color);
    text-align: center;
}

.face-item img {
    width: 80px;
    height: 80px;
    border-radius: 50%;
    object-fit: cover;
    margin-bottom: 10px;
    border: 2px solid var(--primary-color);
}

.face-item p {
    margin: 5px 0;
    font-size: 0.95em;
}

/* Responsive adjustments */
@media (max-width: 768px) {
    .container {
        margin: 20px auto;
        padding: 20px;
    }
    h1 {
        font-size: 1.8em;
    }
    .input-section {
        flex-direction: column;
        align-items: stretch;
    }
    .input-section input[type="text"] {
        max-width: none;
    }
    .results-section h2 {
        font-size: 1.6em;
    }
    .faces ul {
        grid-template-columns: repeat(auto-fill, minmax(120px, 1fr));
    }
}

public/script.js


document.getElementById('analyzeBtn').addEventListener('click', async () => {
    const imageUrlInput = document.getElementById('imageUrl');
    const previewImage = document.getElementById('previewImage');
    const analysisOutput = document.getElementById('analysisOutput');

    const imageUrl = imageUrlInput.value.trim();
    if (!imageUrl) {
        alert('Please enter an image URL.');
        return;
    }

    // Display image preview
    previewImage.src = imageUrl;
    previewImage.style.display = 'block';

    analysisOutput.innerHTML = '<p>Analyzing...</p>';

    try {
        const response = await fetch('/analyze-image', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
            },
            body: JSON.stringify({ imageUrl }),
        });

        if (!response.ok) {
            const errorData = await response.json();
            throw new Error(errorData.error || `HTTP error! status: ${response.status}`);
        }

        const data = await response.json();
        displayAnalysisResults(data);

    } catch (error) {
        console.error('Error:', error);
        analysisOutput.innerHTML = `<p style="color: red;">Error: ${error.message}</p>`;
    }
});

function displayAnalysisResults(data) {
    const analysisOutput = document.getElementById('analysisOutput');
    analysisOutput.innerHTML = ''; // Clear previous results

    // Description
    if (data.description && data.description.captions.length > 0) {
        const caption = data.description.captions[0];
        analysisOutput.innerHTML += `
            <div class="description">
                <h3>Description</h3>
                <p><strong>Caption:</strong> ${caption.text} (Confidence: ${Math.round(caption.confidence * 100)}%)</p>
            </div>
        `;
    }

    // Tags
    if (data.tags && data.tags.length > 0) {
        analysisOutput.innerHTML += `
            <div class="tags">
                <h3>Tags</h3>
                <ul>${data.tags.map(tag => `<li>${tag.name} (Confidence: ${Math.round(tag.confidence * 100)}%)</li>`).join('')}</ul>
            </div>
        `;
    }

    // Categories
    if (data.categories && data.categories.length > 0) {
        analysisOutput.innerHTML += `
            <div class="categories">
                <h3>Categories</h3>
                <ul>${data.categories.map(cat => `<li>${cat.name} (Score: ${Math.round(cat.score * 100)}%)</li>`).join('')}</ul>
            </div>
        `;
    }

    // Objects
    if (data.objects && data.objects.length > 0) {
        analysisOutput.innerHTML += `
            <div class="objects">
                <h3>Detected Objects</h3>
                <ul>${data.objects.map(obj => `
                    <li>
                        ${obj.object}
                        <span class="object-confidence">Confidence: ${Math.round(obj.confidence * 100)}%</span>
                    </li>
                `).join('')}</ul>
            </div>
        `;
    }

    // Faces
    if (data.faces && data.faces.length > 0) {
        analysisOutput.innerHTML += `
            <div class="faces">
                <h3>Detected Faces</h3>
                <ul>${data.faces.map((face, index) => `
                    <li class="face-item">
                        <img src="${document.getElementById('previewImage').src}" alt="Face ${index + 1}">
                        <p><strong>Age:</strong> ${face.age || 'N/A'}</p>
                        <p><strong>Gender:</strong> ${face.gender || 'N/A'}</p>
                    </li>
                `).join('')}</ul>
            </div>
        `;
    }

    // Adult Content
    if (data.adult && data.adult.isAdultContent) {
        analysisOutput.innerHTML += `
            <div class="adult-content">
                <h3>Adult Content Detection</h3>
                <p><strong>Is Adult Content:</strong> Yes</p>
                <p><strong>Adult Score:</strong> ${Math.round(data.adult.adultScore * 100)}%</p>
                <p><strong>Racy Score:</strong> ${Math.round(data.adult.racyScore * 100)}%</p>
                <p><strong>Gore Score:</strong> ${Math.round(data.adult.goreScore * 100)}%</p>
            </div>
        `;
    } else if (data.adult) {
         analysisOutput.innerHTML += `
            <div class="adult-content">
                <h3>Adult Content Detection</h3>
                <p><strong>Is Adult Content:</strong> No</p>
            </div>
        `;
    }
}

// Add font link if not already present (e.g., if this HTML is loaded in isolation)
if (!document.querySelector('link[href^="https://fonts.googleapis.com"]')) {
    const fontLink = document.createElement('link');
    fontLink.rel = 'preconnect';
    fontLink.href = 'https://fonts.googleapis.com';
    document.head.appendChild(fontLink);

    const fontLink2 = document.createElement('link');
    fontLink2.rel = 'preconnect';
    fontLink2.href = 'https://fonts.gstatic.com';
    fontLink2.crossOrigin = '';
    document.head.appendChild(fontLink2);

    const fontLink3 = document.createElement('link');
    fontLink3.href = 'https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;700&display=swap';
    fontLink3.rel = 'stylesheet';
    document.head.appendChild(fontLink3);
}

Step 5: Run the Application

Now, you can run your application:

node server.js

Open your web browser and navigate to http://localhost:3000. You should see the image analysis tool. Enter an image URL and click "Analyze Image" to see the results!

Further Enhancements

  • Implement error handling for invalid image URLs.
  • Add more visual features for analysis (e.g., OCR for text extraction).
  • Integrate with Azure Blob Storage for uploading images.
  • Deploy the application to Azure App Service for a production-ready solution.

This tutorial provides a foundational understanding of building an image analysis application with Azure AI Services. Explore the Azure Computer Vision documentation for more advanced features and capabilities.