01 — Python Environment Setup¶

The single most common source of wasted hours in ML engineering is environment management: wrong Python version, conflicting CUDA libraries, a conda environment that worked yesterday and doesn’t today, or PyTorch installed in the wrong environment. This document gives you the setup that actually works in 2025-2026, the reasoning behind each choice, and exact commands to reproduce it.

The Decision: Which Environment Manager?¶

As of 2025, you have four realistic options. Here is an honest comparison:

Tool	Speed	CUDA/Binary deps	Best for	Verdict
uv (Astral)	⚡⚡⚡ (10-100x faster than pip)	❌ No CUDA support yet	Pure Python ML projects	New standard for pure Python
conda / miniconda	🐢 Slow resolver	✅ Best CUDA support	PyTorch with CUDA, complex scientific stack	Still necessary for CUDA work
venv + pip	⚡ OK	❌ No CUDA management	Simple projects, quick prototypes	Fine but manual
pyenv	N/A (Python version manager)	N/A	Managing multiple Python versions	Useful alongside uv or venv

The 2025 community consensus:

uv is the new default for pure Python project management. It replaces pip+venv+pip-tools in one tool. 10-100x faster resolver. Written in Rust.
conda / miniconda is still the right choice when you need CUDA + PyTorch on Linux, or when your stack includes non-Python binary dependencies (HDF5, GDAL, etc.).
On macOS with Apple Silicon (M1/M2/M3): No CUDA. Use uv or standard venv. PyTorch MPS works without CUDA.

⚠️ Licensing note on conda: The defaults channel from Anaconda Inc. is NOT free for commercial use at companies with >200 employees. Use conda-forge channel only, or use miniconda (which avoids this by default). For a learner/individual, this is not an issue.

Python Version: 3.11¶

Use Python 3.11 as of 2025.

PyTorch 2.4.x supports: Python 3.8, 3.9, 3.10, 3.11, 3.12
Python 3.12 support is available but some ML ecosystem libraries lag
Python 3.11 offers significant performance improvements over 3.10 (10-60% speedups on interpreter benchmarks)
Do not use Python 3.8 or 3.9 for new projects — many libraries are dropping support

CUDA compatibility matrix (for reference, Linux GPU cloud):

CUDA	PyTorch	Python
12.1	2.4.x	3.8-3.12
12.4	2.4.x	3.8-3.12
11.8	2.4.x	3.8-3.12

On macOS (Apple Silicon): CUDA not applicable — use MPS backend.

Setup Option A: uv (Recommended for macOS, Pure Python ML)¶

Install uv¶

# Install uv (macOS / Linux)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Restart terminal or run:
source ~/.bashrc  # or ~/.zshrc on macOS

# Verify
uv --version
# Expected: uv 0.4.x or later

Create a new ML project with uv¶

# Create project directory
mkdir ml_project && cd ml_project

# Initialize with Python 3.11
uv init --python 3.11

# This creates:
# .python-version  (pins Python version)
# pyproject.toml   (project metadata)
# .venv/           (virtual environment, auto-created)

# Add core ML dependencies
uv add torch torchvision torchaudio
uv add numpy pandas matplotlib scikit-learn
uv add jupyter jupyterlab ipykernel
uv add transformers datasets accelerate

# Add development dependencies
uv add --dev black ruff isort pre-commit

# Verify the environment
uv run python -c "import torch; print(torch.__version__)"

# Run any command in the environment
uv run jupyter lab
uv run python train.py

Key uv commands¶

uv add <package>          # Install a package (like pip install)
uv add --dev <package>    # Install a dev dependency
uv remove <package>       # Uninstall
uv sync                   # Sync environment to pyproject.toml (like pip install -r)
uv run <command>          # Run command in the virtual environment
uv pip list               # List installed packages
uv lock                   # Generate uv.lock (reproducible installs)

Setup Option B: Miniconda (For CUDA + Complex Binary Deps)¶

Use this if: you’re on a Linux cloud machine with NVIDIA GPU, or you need non-Python binary dependencies.

Install Miniconda¶

# macOS (Apple Silicon)
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
bash Miniconda3-latest-MacOSX-arm64.sh

# macOS (Intel)
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash Miniconda3-latest-MacOSX-x86_64.sh

# Linux
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

# After install, initialize conda:
conda init zsh  # or bash
# Restart terminal

Create a clean ML environment with conda¶

# Create environment with Python 3.11
conda create -n ml_env python=3.11 -y

# Activate
conda activate ml_env

# Install PyTorch — CRITICAL: use the official install command from pytorch.org
# For macOS (Apple Silicon, no CUDA):
pip install torch torchvision torchaudio

# For Linux with CUDA 12.1:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# For Linux with CUDA 12.4:
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# Install remaining stack
pip install numpy pandas matplotlib scikit-learn
pip install jupyter jupyterlab ipykernel
pip install transformers datasets accelerate peft trl
pip install wandb mlflow

# Register the kernel in Jupyter
python -m ipykernel install --user --name ml_env --display-name "Python (ml_env)"

# Export environment for reproducibility
conda env export > environment.yml

Key conda commands¶

conda activate ml_env           # Activate environment
conda deactivate                # Deactivate
conda env list                  # List all environments
conda env export > env.yml      # Export
conda env create -f env.yml     # Recreate from export
conda env remove -n ml_env      # Delete environment

The Core ML Stack: What to Install¶

Every ML project needs this baseline. Install in this order — PyTorch first, then the rest.

# Core computation
torch torchvision torchaudio    # PyTorch ecosystem
numpy                            # Array operations (PyTorch dependency but install explicitly)
scipy                            # Scientific computing

# Data manipulation
pandas                           # Tabular data
datasets                         # Hugging Face datasets (replaces manual data loading)
Pillow                           # Image handling

# Visualization
matplotlib                       # Base plotting
seaborn                          # Statistical plots
plotly                           # Interactive plots (optional, heavy)

# ML ecosystem
scikit-learn                     # Classical ML, preprocessing, metrics
transformers                     # HuggingFace transformers (BERT, GPT, etc.)
accelerate                       # Multi-GPU / mixed precision training
peft                             # Parameter-efficient fine-tuning (LoRA, etc.)
trl                              # Reinforcement learning from human feedback

# Experiment tracking
wandb                            # Weights & Biases (free tier)

# Development
jupyter                          # Notebook server
jupyterlab                       # Better notebook UI
ipykernel                        # Jupyter kernel
notebook                         # Classic notebook interface

requirements.txt vs pyproject.toml¶

Use pyproject.toml for new projects (2025 standard). It’s the official Python packaging standard (PEP 517/518/621).

Minimal pyproject.toml for an ML project:

[project]
name = "my-ml-project"
version = "0.1.0"
description = "ML experiment"
requires-python = ">=3.11"
dependencies = [
    "torch>=2.4.0",
    "numpy>=1.26.0",
    "pandas>=2.1.0",
    "scikit-learn>=1.4.0",
    "transformers>=4.40.0",
    "wandb>=0.16.0",
]

[project.optional-dependencies]
dev = [
    "black",
    "ruff",
    "pre-commit",
    "pytest",
]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

If you need requirements.txt (e.g., for a cloud service that doesn’t accept pyproject.toml):

# From a working environment
pip freeze > requirements.txt

# Or with uv, generate a pinned requirements file:
uv pip compile pyproject.toml -o requirements.txt

Virtual Environment Best Practices¶

One environment per project. Never share environments across projects. The day you do, a dependency update in Project A breaks Project B.
Never install into base. conda activate base && pip install is how you create dependency hell. Always activate a named environment first.
Pin your versions. torch>=2.4.0 is fine for a requirements file. torch==2.4.1 is what you want in a uv.lock for exact reproducibility.
Commit lock files. uv.lock, poetry.lock, or requirements.txt with pinned versions should be committed to git. pyproject.toml with version ranges should also be committed. The .venv/ directory should be in .gitignore.
Document your CUDA version. If your code depends on CUDA 12.1, say so in README.md. “Requires PyTorch 2.4.0 with CUDA 12.1. Install with: pip install torch --index-url https://download.pytorch.org/whl/cu121”

What most people get wrong¶

Using the system Python. python3 on macOS is Apple’s system Python. It’s not for development. If you run which python3 and see /usr/bin/python3, you are using the system Python. Stop. Install uv or conda and use a project-local Python.

Conda environment bloat. Creating one giant “ml_env” with everything installed over 6 months. By month 6 it has 200 packages, unresolvable conflicts, and nobody knows what’s needed. Create a new environment per project, or at minimum per major project phase.

Installing PyTorch from the wrong channel. The official PyTorch install command (from pytorch.org/get-started) changes with CUDA versions. Do not install pip install torch and assume you’ll get CUDA support. You won’t. Always generate the install command from pytorch.org with your specific OS/CUDA/Python combination.

Return to README.md · Next: 02_gpu_and_compute_strategy.md