Machine learning sounds like it should require a PhD and a stack of linear algebra textbooks before you write a single line of code. In practice, scikit-learn, Python’s standard machine learning library, lets you train, evaluate, and use a real model in under twenty lines of code, with the underlying math handled for you. This Python for Machine Learning tutorial walks through the entire workflow, from raw data to a working, evaluated model, with runnable code at every step.

You don’t need advanced math to get started here. What you need is the Python fundamentals covered in our Python tutorial and the array and data-handling basics from our NumPy tutorial, since scikit-learn builds directly on both.

Setting Up Your Environment

As of mid-2026, scikit-learn’s current stable release is version 1.9.0, requiring Python 3.11 or newer. Install it alongside the libraries it’s commonly paired with:

pip install scikit-learn pandas numpy matplotlib

Verify the installation:

import sklearn
print(sklearn.__version__)

scikit-learn doesn’t handle deep learning, that’s the territory of PyTorch or TensorFlow, but for the large majority of real-world machine learning problems involving structured, tabular data, it remains the standard starting point and often the final choice too.

The Machine Learning Workflow

Nearly every machine learning project follows the same basic sequence, regardless of the specific algorithm involved: load and explore your data, clean and preprocess it, split it into training and testing sets, train a model on the training data, evaluate its performance on the held-out test data, and then use it to make predictions on new, unseen data. This tutorial builds through each of these stages in order, using the same running example throughout.

Types of Machine Learning

Supervised learning trains on labeled data, where the correct answer is already known for each example, and covers two main problem types: classification predicts a category (is this email spam or not?), while regression predicts a continuous number (what will this house sell for?).

Unsupervised learning works with unlabeled data, finding structure on its own, most commonly through clustering, grouping similar data points together without being told in advance what the groups should be.

Reinforcement learning is a different paradigm entirely, where an agent learns through trial and error by receiving rewards or penalties for actions taken in an environment, and it’s less commonly needed for typical business or data analysis problems compared to the first two.

This tutorial focuses on supervised learning, since it covers the large majority of practical machine learning work you’ll encounter as a beginner.

Loading and Exploring Your Data

scikit-learn ships with several small, well-known datasets built in, which makes them useful for learning without needing to source and clean external data first.

from sklearn.datasets import load_iris
import pandas as pd

iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names)
df['species'] = iris.target

print(df.head())
print(df.describe())
print(df['species'].value_counts())

The Iris dataset contains measurements of flower petals and sepals across three species, and it’s a classic starting point specifically because it’s small, clean, and has a clear classification target, letting you focus on learning the workflow rather than fighting messy data on your first attempt.

Always start a real project by actually looking at your data before touching a model. df.describe() shows the range, mean, and spread of each numeric column, catching obvious issues like an impossible value or a column that’s mostly missing before it silently breaks a model down the line.

Working With Real CSV Data

Built-in datasets are useful for learning, but real projects start with pandas.read_csv() loading data from a file, database export, or API response instead.

import pandas as pd

df = pd.read_csv('customer_data.csv')

print(df.shape)          # (rows, columns)
print(df.dtypes)         # data type of each column
print(df.isnull().sum()) # missing values per column
print(df.duplicated().sum())  # count of exact duplicate rows

Real, external data almost always needs more inspection than a clean, built-in dataset: checking for duplicate rows that could bias a model toward overrepresented examples, verifying that a column’s data type matches what you expect (a numeric column accidentally loaded as text, for instance, silently breaks calculations later), and looking for inconsistent categorical values, “NY,” “New York,” and “ny” all representing the same thing but treated as three separate categories unless cleaned first. Budgeting real time for this inspection step, rather than rushing straight to model training, is consistently what separates a model that performs well in practice from one that looked fine in a notebook but breaks down on genuinely new data.

Preprocessing Data for Machine Learning

Real data is rarely ready to feed directly into a model. A few preprocessing steps come up in nearly every project.

Handling missing values:

# Check for missing values
print(df.isnull().sum())

# Fill missing numeric values with the column median
df['some_column'] = df['some_column'].fillna(df['some_column'].median())

# Or drop rows with missing values, if there aren't many
df = df.dropna()

Encoding categorical variables, since most models require numeric input:

from sklearn.preprocessing import OneHotEncoder
import pandas as pd

# One-hot encoding converts a category into multiple binary columns
sample_df = pd.DataFrame({'color': ['red', 'blue', 'green', 'blue']})
encoded = pd.get_dummies(sample_df, columns=['color'])
print(encoded)

Feature scaling, important for algorithms sensitive to the scale of input features, like logistic regression or anything distance-based:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(df[iris.feature_names])

StandardScaler transforms each feature to have a mean of 0 and a standard deviation of 1, which prevents a feature measured in the thousands from dominating a model’s calculations purely due to scale, rather than actual predictive importance, compared to a feature measured in single digits.

Splitting Data Into Training and Test Sets

Evaluating a model on the same data it trained on gives a falsely optimistic picture of how well it will actually perform on new, unseen data. Splitting data into separate training and test sets is non-negotiable for honest evaluation.

from sklearn.model_selection import train_test_split

X = df[iris.feature_names]
y = df['species']

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(f"Training set: {X_train.shape[0]} samples")
print(f"Test set: {X_test.shape[0]} samples")

test_size=0.2 reserves 20% of the data for testing, a common default. random_state=42 makes the split reproducible, so running the same code again produces an identical split rather than a different random one each time, which matters for debugging and for comparing results across experiments.

Training Your First Classification Model

With clean, split data ready, training a model in scikit-learn follows a consistent pattern regardless of which algorithm you choose.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

predictions = model.predict(X_test)
print(predictions[:10])

Every scikit-learn model follows this same fit() then predict() pattern, which is a big part of why the library is so approachable: once you understand this pattern with one algorithm, switching to a completely different one, a decision tree, a random forest, a support vector machine, requires changing essentially nothing except the import and the model constructor.

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

tree_model = DecisionTreeClassifier(random_state=42)
tree_model.fit(X_train, y_train)

forest_model = RandomForestClassifier(n_estimators=100, random_state=42)
forest_model.fit(X_train, y_train)

A random forest trains many decision trees on different random subsets of the data and features, then combines their predictions, which typically produces more accurate and more stable results than a single decision tree alone, at the cost of being somewhat less directly interpretable.

Evaluating Model Performance

Accuracy alone can be misleading, particularly with imbalanced classes, so it’s worth using a fuller set of metrics from the start.

from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

accuracy = accuracy_score(y_test, predictions)
print(f"Accuracy: {accuracy:.2%}")

print(classification_report(y_test, predictions))
print(confusion_matrix(y_test, predictions))

classification_report gives precision, recall, and F1 score broken down by class, not just an overall average, which matters because a model can perform well on one class while performing poorly on another, a pattern a single overall accuracy number hides completely. The confusion matrix shows exactly which classes get confused with which, which is often more diagnostically useful than any single summary metric when you’re trying to actually improve a model.

Regression: Predicting a Continuous Value

Classification predicts a category; regression predicts a number. The workflow is nearly identical, just with different models and evaluation metrics.

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
import numpy as np

# Example: predicting a continuous value from features
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

reg_model = LinearRegression()
reg_model.fit(X_train, y_train)
reg_predictions = reg_model.predict(X_test)

mse = mean_squared_error(y_test, reg_predictions)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, reg_predictions)

print(f"RMSE: {rmse:.2f}")
print(f"R² score: {r2:.2f}")

Root mean squared error (RMSE) measures the typical size of prediction errors, in the same units as the target variable, which makes it more directly interpretable than mean squared error alone. R² score indicates how much of the variance in the target the model explains, ranging from 0 (no better than always predicting the average) to 1 (perfect prediction), with negative values indicating a model performing worse than that simple average baseline.

Clustering: An Unsupervised Learning Example

Unlike classification and regression, clustering doesn’t require labeled data at all. It groups similar data points together based purely on their features, useful for tasks like customer segmentation where you don’t already know what the groups should be.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(df[iris.feature_names])

kmeans = KMeans(n_clusters=3, random_state=42, n_init=10)
clusters = kmeans.fit_predict(X_scaled)

df['cluster'] = clusters
print(df.groupby('cluster')[iris.feature_names].mean())

KMeans groups data into a specified number of clusters (n_clusters=3 here) by iteratively assigning points to the nearest cluster center and updating those centers based on the points assigned to them. Choosing the right number of clusters isn’t always obvious in advance; a common approach, called the elbow method, involves running KMeans across a range of cluster counts and plotting how much the within-cluster variance decreases with each additional cluster, looking for the point where adding more clusters stops providing a meaningful improvement.

Cross-Validation: A More Reliable Evaluation

A single train-test split can produce a misleadingly good or bad score purely by chance, depending on which specific rows happened to land in the test set. Cross-validation addresses this by testing across several different splits and averaging the results.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(model, X, y, cv=5)
print(f"Cross-validation scores: {scores}")
print(f"Mean accuracy: {scores.mean():.2%} (+/- {scores.std():.2%})")

cv=5 performs 5-fold cross-validation, splitting the data into five parts, training on four and validating on the fifth, rotating through all five combinations. The resulting mean and standard deviation give a far more honest picture of expected real-world performance than a single lucky (or unlucky) train-test split.

Overfitting, Underfitting, and Feature Importance

Overfitting happens when a model learns the training data too closely, including its noise and quirks, and performs noticeably worse on new data than it did during training. Underfitting happens when a model is too simple to capture the real pattern in the data, performing poorly on both training and test data alike.

train_accuracy = model.score(X_train, y_train)
test_accuracy = model.score(X_test, y_test)

print(f"Training accuracy: {train_accuracy:.2%}")
print(f"Test accuracy: {test_accuracy:.2%}")

A large gap, high training accuracy but noticeably lower test accuracy, is the classic signature of overfitting. Tree-based models are particularly prone to this if left unrestricted, which is why parameters like max_depth exist to deliberately limit how complex a tree is allowed to grow.

Tree-based models also expose which features actually drove their predictions:

importances = forest_model.feature_importances_
for feature, importance in zip(iris.feature_names, importances):
    print(f"{feature}: {importance:.3f}")

This is genuinely useful beyond satisfying curiosity: it can reveal that a feature you assumed mattered contributes almost nothing, or that a feature you overlooked is actually doing most of the work, both of which inform decisions about what data is actually worth collecting for future versions of a model.

Hyperparameter Tuning

Every model has hyperparameters, settings chosen before training rather than learned from the data, like how deep a decision tree is allowed to grow or how many trees a random forest builds. Choosing good hyperparameter values can meaningfully improve performance, and GridSearchCV automates the process of testing multiple combinations systematically.

from sklearn.model_selection import GridSearchCV

param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 10, 20],
    'min_samples_split': [2, 5, 10]
}

grid_search = GridSearchCV(
    RandomForestClassifier(random_state=42),
    param_grid,
    cv=5,
    scoring='accuracy'
)

grid_search.fit(X_train, y_train)

print(f"Best parameters: {grid_search.best_params_}")
print(f"Best cross-validation score: {grid_search.best_score_:.2%}")

best_model = grid_search.best_estimator_

GridSearchCV trains and cross-validates a model for every combination in param_grid, which can get computationally expensive quickly as the number of parameters and values grows. RandomizedSearchCV offers a faster alternative for larger search spaces, sampling a fixed number of random combinations rather than testing every single one exhaustively.

Handling Imbalanced Datasets

Many real-world classification problems, fraud detection, disease diagnosis, churn prediction, have significantly more examples of one class than another, which can cause a model to effectively ignore the minority class since predicting the majority class alone already achieves high accuracy.

# Tell the model to weight the minority class more heavily
weighted_model = LogisticRegression(class_weight='balanced', max_iter=200)
weighted_model.fit(X_train, y_train)

Setting class_weight='balanced' automatically adjusts a model’s internal loss calculation to penalize mistakes on the minority class more heavily, without requiring you to modify the dataset itself. Other common approaches include oversampling the minority class or undersampling the majority class before training, with libraries like imbalanced-learn providing more sophisticated techniques like SMOTE, which generates synthetic minority-class examples rather than simply duplicating existing ones. Regardless of the technique used, always evaluate an imbalanced classification problem with precision, recall, and F1 score rather than accuracy alone, since accuracy alone hides exactly the failure mode class imbalance tends to cause.

Building a Complete Pipeline

Combining preprocessing and modeling into a single Pipeline object keeps your workflow organized and, importantly, prevents a common and serious mistake called data leakage.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('classifier', LogisticRegression(max_iter=200))
])

pipeline.fit(X_train, y_train)
pipeline_predictions = pipeline.predict(X_test)
print(f"Pipeline accuracy: {accuracy_score(y_test, pipeline_predictions):.2%}")

A Pipeline ensures preprocessing steps like scaling are fit only on training data and then correctly applied to test data using those same fitted parameters, rather than accidentally fitting the scaler on the full dataset before splitting, which would let information from the test set leak into training and produce an artificially inflated performance estimate.

Once you have a trained model you’re happy with, save it for later use without retraining:

import joblib

joblib.dump(pipeline, 'model.joblib')

loaded_model = joblib.load('model.joblib')
new_predictions = loaded_model.predict(X_test)

Common Beginner Mistakes

Data leakage from scaling before splitting. Fitting a scaler on the entire dataset before splitting into train and test sets lets test set information influence the training process, producing an evaluation score that looks better than what the model will actually achieve on genuinely new data. Always fit preprocessing steps only on training data, ideally through a Pipeline as shown above.

Evaluating only on training data. A model can achieve near-perfect accuracy on data it was trained on while performing far worse on anything new, which is exactly why a held-out test set, or cross-validation, is essential rather than optional.

Ignoring class imbalance. If one class makes up 95% of your data, a model that always predicts that class achieves 95% accuracy while learning nothing useful. Checking class balance early, and using precision, recall, and F1 score rather than accuracy alone, catches this before it becomes a misleading final result.

Not setting a random seed. Without random_state set consistently across your splits and models, results become difficult to reproduce or fairly compare between experiments, since each run introduces new randomness.

Jumping straight to complex models. A simple model like logistic regression or a single decision tree, trained and evaluated properly, often performs surprisingly well and gives you a meaningful baseline before reaching for a more complex model whose extra complexity may not even be justified by the data.

Practice Project

A solid way to apply this Python for Machine Learning tutorial’s concepts end to end: load the built-in load_breast_cancer dataset from sklearn.datasets, split it properly, train both a logistic regression and a random forest classifier, evaluate both with a full classification report and confusion matrix, and use cross-validation to determine which model actually generalizes better rather than trusting a single train-test split. That single exercise touches every stage of the workflow covered in this tutorial: preprocessing, splitting, training multiple model types, evaluation, and cross-validation.

Frequently Asked Questions

Do I need to understand the math behind machine learning algorithms to use scikit-learn effectively?

Not to get started productively, no. scikit-learn handles the underlying math, and you can build working, reasonably good models by understanding the workflow, what each preprocessing step does and why, and how to properly evaluate results. Deeper mathematical understanding becomes more valuable as you move toward tuning models, diagnosing subtle problems, or working with more advanced techniques.

What’s the difference between scikit-learn and TensorFlow or PyTorch?

scikit-learn focuses on classical machine learning algorithms, well suited to structured, tabular data. TensorFlow and PyTorch are built for deep learning, neural networks with many layers, which excel at unstructured data like images, audio, and text at scale. Many real-world projects use scikit-learn for a large share of their work and reach for deep learning frameworks specifically when the problem and data genuinely call for it.

How much data do I need before machine learning actually works well?

It varies enormously by problem complexity, but classical algorithms in scikit-learn can produce genuinely useful results with datasets ranging from a few hundred to a few thousand rows for reasonably simple problems, which is considerably less than the volumes typically required for deep learning approaches to outperform simpler methods.

Why did my model get 100% accuracy? Is that a good sign?

Almost certainly not, in most real situations. Perfect accuracy on a test set usually signals either data leakage (test information influencing training somehow), an unrealistically easy or duplicated dataset, or evaluating on the training data by mistake rather than a genuine held-out test set. Treat a suspiciously perfect score as a signal to double-check your pipeline before celebrating it.

Which algorithm should I try first for a new classification problem?

Logistic regression or a simple decision tree, both fast to train and easy to interpret, make a strong starting baseline. If performance isn’t sufficient, a random forest or gradient boosting model (like XGBoost) is a reasonable next step, since both tend to perform well on tabular data with relatively little tuning required to get solid initial results.

Is scikit-learn still relevant given how much attention deep learning gets?

Yes, very much so. For the large majority of real-world business problems involving structured, tabular data, customer churn, fraud detection, sales forecasting, classical algorithms in scikit-learn remain both effective and considerably easier to train, interpret, and deploy than a deep learning approach, which is overkill for most of these problems.

What’s the difference between GridSearchCV and manually trying different hyperparameters?

GridSearchCV systematically tests every combination in a defined search space using cross-validation for each one, giving a more reliable estimate of which combination actually generalizes best rather than performing well by chance on a single split. Manual tuning is faster for quick experimentation but is more prone to accidentally overfitting your choices to a single train-test split rather than genuinely better hyperparameters.

Do I need a GPU to use scikit-learn?

No. Nearly every algorithm in scikit-learn runs on the CPU, and for the dataset sizes classical machine learning typically handles, a GPU provides little to no benefit. GPUs matter far more for deep learning frameworks like PyTorch and TensorFlow, where the computations involved genuinely benefit from the massive parallelization a GPU provides.

Where to Go From Here

This Python for Machine Learning tutorial covered the complete workflow: loading and exploring data, preprocessing, splitting, training both classification and regression models, evaluating them properly, cross-validation, understanding overfitting, and building a production-ready pipeline you can actually save and reuse. From here, the fastest way to build real skill is working through a dataset from Kaggle or a similar source end to end yourself, since real, messier data surfaces preprocessing decisions this tutorial’s clean example dataset doesn’t force you to make.

If you’re still solidifying the Python fundamentals this tutorial assumes, our Python tutorial and NumPy tutorial with examples cover the foundation everything here builds on. Once you’re comfortable with the workflow in this guide, our Data Science Interview Questions guide covers how these same concepts, precision and recall, cross-validation, overfitting, get tested in real interviews. For complete reference on every algorithm and parameter mentioned here, the official scikit-learn documentation is worth bookmarking as your primary ongoing resource.