MSDN - Python Data Science & ML

Decision Trees - Practical Examples

Explore real-world applications and code examples of decision trees in Python, demonstrating their versatility in classification and regression tasks.

Example 1: Predicting Customer Churn (Classification)

A common use case for decision trees is predicting whether a customer will churn (leave a service). This example uses a simplified dataset to illustrate the process.

Dataset Overview:

Python Code (using scikit-learn):

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import pandas as pd

# Load and preprocess data (assuming 'data.csv' exists)
df = pd.read_csv('customer_churn_data.csv')
df.dropna(inplace=True) # Handle missing values

# Feature engineering and encoding (simplified)
df['Churn'] = df['Churn'].map({'Yes': 1, 'No': 0})
df = pd.get_dummies(df, columns=['ContractType', 'PaymentMethod'], drop_first=True)

features = ['Tenure', 'MonthlyCharges', 'TotalCharges', 'ContractType_One year', 'ContractType_Two year', 'PaymentMethod_Credit card']
X = df[features]
y = df['Churn']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Initialize and train the decision tree classifier
model = DecisionTreeClassifier(max_depth=5, random_state=42)
model.fit(X_train, y_train)

# Predict and evaluate
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.4f}")

# You can further visualize the tree using graphviz (requires installation)
# from sklearn.tree import plot_tree
# import matplotlib.pyplot as plt
# plt.figure(figsize=(20,10))
# plot_tree(model, feature_names=features, class_names=['No Churn', 'Churn'], filled=True)
# plt.show()

Example 2: Predicting House Prices (Regression)

Decision trees can also be used for regression tasks, such as predicting continuous values like house prices.

Dataset Overview:

Python Code (using scikit-learn):

from sklearn.tree import DecisionTreeRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error
import pandas as pd
import numpy as np

# Load and preprocess data (assuming 'house_prices.csv' exists)
df = pd.read_csv('house_prices_data.csv')
df.dropna(inplace=True)

# Simple encoding for categorical features (e.g., NeighborhoodQuality)
df = pd.get_dummies(df, columns=['NeighborhoodQuality'], drop_first=True)

features = ['SquareFeet', 'NumberOfBedrooms', 'YearBuilt', 'NeighborhoodQuality_Good', 'NeighborhoodQuality_Excellent']
X = df[features]
y = df['Price']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Initialize and train the decision tree regressor
model = DecisionTreeRegressor(max_depth=4, random_state=42)
model.fit(X_train, y_train)

# Predict and evaluate
y_pred = model.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
print(f"Root Mean Squared Error (RMSE): ${rmse:.2f}")

Example 3: Visualizing a Simple Decision Tree

Understanding the structure of a decision tree is key. This example shows how to visualize a small tree for better interpretability. This often requires external libraries like `graphviz`.

Python Code (conceptual):

from sklearn.tree import DecisionTreeClassifier
from sklearn.datasets import load_iris
import matplotlib.pyplot as plt
# from sklearn.tree import plot_tree # Uncomment if you have matplotlib and scikit-learn installed

# Load a small, built-in dataset
iris = load_iris()
X = iris.data
y = iris.target

# Train a simple decision tree
model = DecisionTreeClassifier(max_depth=3, random_state=0)
model.fit(X, y)

# Visualize the tree (requires matplotlib)
# plt.figure(figsize=(12, 8))
# plot_tree(model, 
#           feature_names=iris.feature_names, 
#           class_names=iris.target_names, 
#           filled=True, 
#           rounded=True, 
#           fontsize=10)
# plt.title("Decision Tree Visualization (Iris Dataset)")
# plt.show()

# To actually render this, you would need to:
# 1. Install matplotlib: pip install matplotlib
# 2. Uncomment the plotting code above.
# 3. Potentially install graphviz and its Python bindings if plot_tree requires it on your system.
print("Visualization code is commented out but shows how to generate tree plots.")