CatBoost Python Tutorial: Handle Categorical Features Like a Pro

CatBoost handles categorical features natively, which means you can stop obsessing over preprocessing and actually focus on building models that work. Let me show you how to use this beast of a library like you actually know what you’re doing.

Why CatBoost Makes Other Libraries Look Bad

Here’s the thing about traditional gradient boosting libraries: they hate categorical data. XGBoost and LightGBM force you to encode everything into numbers before they’ll even look at your dataset. Ever tried one-hot encoding a feature with 100+ categories? Your feature space explodes faster than you can say “curse of dimensionality.”

CatBoost takes a different approach. It uses something called ordered target statistics to handle categorical features automatically. What does that mean for you? You literally just tell CatBoost which columns are categorical, and it figures out the rest. No manual encoding. No preprocessing nightmares. Just results.

Plus, CatBoost has some seriously impressive features:

• Built-in categorical handling that actually works
• Robust overfitting detection so you don’t embarrass yourself in production
• GPU support for when you’re feeling fancy
• Symmetric trees that make predictions lightning-fast

IMO, it’s one of the most underrated libraries in the Python ecosystem. Let me prove it to you.

Getting Started: Installation and Basic Setup

First things first — let’s get CatBoost installed. Fire up your terminal and run:

bash

pip install catboost

That’s it. No complicated dependencies, no configuration files, no sacrificing a USB drive to the tech gods. Just one command and you’re ready.

Now let’s load up the essentials:

python

import pandas as pd
import numpy as np
from catboost import CatBoostClassifier, CatBoostRegressor, Pool
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, roc_auc_score

Notice that Pool import? That's CatBoost's secret weapon for handling data efficiently. We'll get to that in a minute.

Loading and Preparing Your Data

Let me show you a real example. I’ll use a dataset with both numerical and categorical features because that’s where CatBoost really shines.

python

# Load your data
df = pd.read_csv('your_dataset.csv')

# Separate features and target
X = df.drop('target_column', axis=1)
y = df['target_column']

# Split the data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

Pretty standard stuff so far, right? Here’s where it gets interesting.

Identifying Categorical Features: The Right Way

You need to tell CatBoost which columns are categorical. You’ve got two options here, and trust me, the method you choose matters.

Option 1: List the indices

python

categorical_features_indices = [0, 2, 4, 7]  # Column positions

Option 2: List the names (my preferred method)

python

categorical_features_names = ['color', 'brand', 'category', 'region']

Why do I prefer names? Because when you inevitably shuffle your columns around or add new features, indices break. Names don’t. FYI, you’ll thank me later when you’re not debugging at 2 AM wondering why your model suddenly went haywire.

Creating a CatBoost Pool: Your New Best Friend

Here’s what separates beginners from pros: using Pool objects. A Pool bundles your data, labels, and categorical feature info into one efficient package.

python

train_pool = Pool(
    data=X_train,
    label=y_train,
    cat_features=categorical_features_names
)

test_pool = Pool(
    data=X_test,
    label=y_test,
    cat_features=categorical_features_names
)

Why bother with Pools? They make training faster and let CatBoost optimize memory usage. Plus, you specify your categorical features once and never worry about them again.

Training Your First CatBoost Model

Alright, let’s build a classifier. I’m using a classification example here, but the process for regression is nearly identical — just swap CatBoostClassifier for CatBoostRegressor.

python

model = CatBoostClassifier(
    iterations=1000,
    learning_rate=0.03,
    depth=6,
    loss_function='Logloss',
    verbose=100,
    random_seed=42
)

model.fit(
    train_pool,
    eval_set=test_pool,
    early_stopping_rounds=50
)

Let me break down what’s happening here:

• iterations: How many boosting rounds to run (1000 is a solid starting point)
• learning_rate: Controls how much each tree contributes (lower = more stable but slower)
• depth: How deep each tree goes (6–10 works well for most datasets)
• verbose: Shows progress every 100 iterations so you don’t wonder if your code froze
• early_stopping_rounds: Stops training if performance doesn’t improve for 50 rounds

The model automatically handles those categorical features you specified in the Pool. No encoding. No preprocessing. Just pure, unadulterated machine learning magic.

Making Predictions and Evaluating Performance

Once training finishes, you can predict on new data:

python

# Get predictions
predictions = model.predict(test_pool)
prediction_probs = model.predict_proba(test_pool)

# Calculate accuracy
accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy:.4f}")

# For binary classification, get AUC score
auc = roc_auc_score(y_test, prediction_probs[:, 1])
print(f"AUC-ROC: {auc:.4f}")

Ever wondered why CatBoost often outperforms other libraries right out of the box? It’s because those ordered target statistics prevent target leakage during training. Your categorical features get encoded using information from previous examples only, which creates more robust predictions.

Hyperparameter Tuning: The Parameters That Actually Matter

Look, you can spend weeks tuning every single parameter, or you can focus on the ones that move the needle. Here are the big three:

1. Learning Rate and Iterations

Lower learning rate + more iterations = better performance but slower training. Start with learning_rate=0.03 and iterations=1000, then adjust.

python

model = CatBoostClassifier(
    learning_rate=0.01,
    iterations=2000,
    depth=6
)

2. Tree Depth

Deeper trees capture complex patterns but risk overfitting. Most datasets work well with depths between 4–10.

3. L2 Regularization

Controls overfitting by penalizing large weights:

python

model = CatBoostClassifier(
    l2_leaf_reg=3,  # Higher = more regularization
    iterations=1000
)

Want my honest advice? Start simple. Train a basic model, check if it’s overfitting (training score way higher than validation score), then tune from there.

Handling Missing Values: CatBoost’s Hidden Superpower

Here’s something that’ll blow your mind: CatBoost handles missing values automatically. You don’t need to impute them. You don’t need to create “is_missing” flags. You literally do nothing. :/

python

# Data with missing values? No problem.
X_train_with_nans = X_train  # Keep those NaNs right where they are

train_pool = Pool(
    data=X_train_with_nans,
    label=y_train,
    cat_features=categorical_features_names
)

model.fit(train_pool)

CatBoost treats missing values as a separate category for categorical features and learns optimal splits for numerical features with missingness. It’s one less thing to worry about, and honestly, it’s glorious.

Feature Importance: Understanding What Drives Your Model

After training, you’ll want to know which features actually matter:

python

feature_importances = model.get_feature_importance(train_pool)
feature_names = X_train.columns

importance_df = pd.DataFrame({
    'feature': feature_names,
    'importance': feature_importances
}).sort_values('importance', ascending=False)

print(importance_df.head(10))

This shows you which features have the biggest impact on predictions. You can use this to simplify your model, identify data quality issues, or just feel smug about your feature engineering skills.

Saving and Loading Models

You built an awesome model. Now what? Save it so you don’t have to retrain every time:

python

# Save the model
model.save_model('catboost_model.cbm')

# Load it later
loaded_model = CatBoostClassifier()
loaded_model.load_model('catboost_model.cbm')

# Make predictions with loaded model
predictions = loaded_model.predict(X_test)

The .cbm format is CatBoost's native format and loads instantly. You can also save as JSON if you need human-readable output, but honestly, stick with .cbm for production.

Wrapping Up: You’re Now a CatBoost Pro

Let’s recap what you’ve learned. CatBoost handles categorical features natively, which eliminates hours of preprocessing work. You use Pool objects to bundle your data efficiently. You specify categorical features once and let the library handle the rest. You can tune a handful of key parameters to improve performance without getting lost in hyperparameter hell.

The best part? CatBoost often delivers state-of-the-art results with minimal effort. No complicated pipelines. No endless feature engineering. Just clean, effective machine learning that actually works.

So next time you’re staring at a dataset full of categorical features, don’t reach for that one-hot encoder. Fire up CatBoost instead. Your future self will thank you — probably while sipping coffee and watching your models train in a fraction of the time it used to take.

Now go build something awesome.