Model Evaluation Metrics in .NET AI
Introduction
When building machine‑learning models with ML.NET, selecting the right evaluation metrics is essential to understand how well your model performs on real‑world data. This article covers the most common metrics for classification, regression, and ranking tasks, and shows how to compute them using the ML.NET API.
Classification Metrics
- Accuracy – Overall correctness.
- Precision – Positive predictive value.
- Recall (Sensitivity) – True positive rate.
- F1‑Score – Harmonic mean of precision and recall.
- ROC‑AUC – Area under the ROC curve.
- LogLoss – Cross‑entropy loss.
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Trainers;
// Load data
var mlContext = new MLContext();
IDataView data = mlContext.Data.LoadFromTextFile<ModelInput>("train.csv", separatorChar: ',', hasHeader: true);
// Split
var split = mlContext.Data.TrainTestSplit(data, testFraction: 0.2);
var train = split.TrainSet;
var test = split.TestSet;
// Build pipeline
var pipeline = mlContext.Transforms.Conversion.MapValueToKey("Label")
.Append(mlContext.Transforms.Text.FeaturizeText("Features", "Text"))
.Append(mlContext.MulticlassClassification.Trainers.SdcaMaximumEntropy())
.Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabel"));
// Train
var model = pipeline.Fit(train);
// Evaluate
var predictions = model.Transform(test);
var metrics = mlContext.MulticlassClassification.Evaluate(predictions);
// Output results
Console.WriteLine($"MacroAccuracy: {metrics.MacroAccuracy:F2}");
Console.WriteLine($"MicroAccuracy: {metrics.MicroAccuracy:F2}");
Console.WriteLine($"LogLoss: {metrics.LogLoss:F2}");
Console.WriteLine($"LogLossReduction: {metrics.LogLossReduction:F2}");
Regression Metrics
- Mean Absolute Error (MAE)
- Mean Squared Error (MSE)
- Root Mean Squared Error (RMSE)
- R‑Squared (R²)
var regressionPipeline = mlContext.Transforms.Concatenate("Features", "Feature1", "Feature2")
.Append(mlContext.Regression.Trainers.Sdca(labelColumnName: "Label", maximumNumberOfIterations: 100));
var regressionModel = regressionPipeline.Fit(train);
var regressionPred = regressionModel.Transform(test);
var regressionMetrics = mlContext.Regression.Evaluate(regressionPred);
Console.WriteLine($"R2: {regressionMetrics.RSquared:F2}");
Console.WriteLine($"MAE: {regressionMetrics.MeanAbsoluteError:F2}");
Console.WriteLine($"MSE: {regressionMetrics.MeanSquaredError:F2}");
Console.WriteLine($"RMSE: {regressionMetrics.RootMeanSquaredError:F2}");
Comments