Microsoft Docs Windows APIs

Core Concepts

Understanding these fundamental concepts is crucial for working effectively with Media Foundation:

• Media Session: The central orchestrator of media processing. It manages the pipeline and events.
• Topology: A directed graph that describes the flow of data through the Media Foundation pipeline.
• Nodes: The building blocks of a topology, representing sources, sinks, or transforms.
• Media Sink: A component that receives processed media data (e.g., writes to a file, displays on screen).
• Media Source: A component that provides raw media data (e.g., reads from a file, captures from a camera).
• Media Transform (MFT): A component that processes media data (e.g., decodes video, applies an audio effect).
• Media Type: Defines the format of media data being passed between nodes (e.g., video codec, resolution, frame rate, audio format).

Key Topics

Media Foundation Pipeline

The pipeline is the heart of Media Foundation. It's a sequence of connected components (nodes) that process media data. A typical pipeline might involve:

1. A Media Source to read or capture data.
2. One or more Media Transforms (MFTs) for decoding, encoding, or other processing.
3. A Media Sink to output the processed data.

The structure of the pipeline is defined by a Topology, which is built and submitted to the Media Session.

Media Sources

Media Sources are responsible for providing media data. Examples include:

• File Source (MFReadWrite.dll): Reads media from local files.
• Capture Source: Captures audio and video from devices like webcams and microphones.
• Network Source: Receives media streams over a network.

Each Media Source exposes one or more output streams, each with a specific Media Type.

Media Sinks

Media Sinks consume processed media data. Common examples include:

• File Sink (MFReadWrite.dll): Writes media to a local file.
• Audio Renderer: Plays audio through the system's sound card.
• Video Renderer: Displays video on a window.
• Network Sink: Sends media streams over a network.

Media Transforms (MFTs)

MFTs perform a specific media processing task. They can be chained together to form complex processing pipelines. Key types of MFTs include:

• Decoders: Convert compressed media data into raw, uncompressed formats (e.g., H.264 decoder, AAC decoder).
• Encoders: Convert raw media data into compressed formats for storage or transmission.
• Video Effects: Apply visual enhancements or filters to video.
• Audio Effects: Process audio signals (e.g., normalization, equalization).

MFTs typically have one or more input streams and one or more output streams, each with defined Media Types.

Media Types

A Media Type is a set of attributes that describe the format of media data. It defines details like:

• Major Type (e.g., MFMediaType_Video, MFMediaType_Audio)
• Subtype (e.g., MFVideoFormat_H264, MFAudioFormat_PCM)
• Video Resolution (Width, Height)
• Frame Rate
• Audio Sample Rate, Channels, Bits Per Sample
• Bit Rate

Components in the pipeline negotiate compatible Media Types for their connections.

API Reference

The Media Foundation SDK exposes a rich set of COM interfaces and functions. Key interfaces include:

• IMFMediaSession: Manages the media pipeline and playback state.
• IMFTopology: Represents the data flow graph.
• IMFTopologyNode: Represents a source, sink, or transform within a topology.
• IMFMediaSource: Interface for media sources.
• IMFMediaSink: Interface for media sinks.
• IMFTransform: Interface for media transforms (MFTs).
• IMFMediaType: Interface for describing media data formats.
• IMFPresentationClock: Manages timing for playback.

Refer to the specific API documentation for detailed usage, parameters, and return values.

Explore the following resources for detailed API descriptions:

• IMFMediaSession Interface
• IMFTransform Interface
• IMFMediaType Interface

Tutorials

Get started with practical examples:

• Basic Media Playback Tutorial
• Capturing Video and Audio
• Encoding and Saving Media to File
• Implementing a Custom Media Transform