05 — Cloud GPU Access for CUDA C++ (Phase 5, No NVIDIA on Mac)

Hard constraint. Your M-series Mac has zero NVIDIA GPU. Phase 5 (06_phase_5_cpp_for_ml_data/) includes CUDA C++ work — kernels, Thrust, cuBLAS bindings, TensorRT integration, llama.cpp GPU path. None of this runs on Metal. You need cloud access, and this file names the cheapest sane path.

§1. The Metal / MPS honest verdict (July 2026)

Apple’s Metal Performance Shaders (MPS) and MLX give you GPU-accelerated tensor ops on M-series silicon. For your local ML notebook work (PyTorch MPS backend, MLX for local LLM inference) this is enough and sometimes excellent — MLX on M3 Max is a legitimate 4090-adjacent inference platform for quantized 7B–13B models.

But: Metal is not CUDA. If your Phase 5 goal is to learn how NVIDIA GPUs work at the C++ level (which it is), you must write CUDA kernels and profile them with Nsight. That requires an NVIDIA GPU. There is no substitute.

Rule for Phase 5: run everything you can on M-series (PyTorch MPS, MLX, CPU inference paths, small llama.cpp). Rent cloud NVIDIA time only when you’re specifically writing CUDA C++ or benchmarking against CUDA-only libraries (TensorRT, cuDNN, cuBLAS custom kernels).

§2. The 2026 cloud GPU market — prices verified July 2026

Prices are USD/hour on-demand unless marked “spot”. Add ~2–3% for Indian bank forex + GST implications. Availability changes weekly — verify at the provider’s pricing page before committing.

Provider

A100 40GB

A100 80GB

H100 SXM/PCIe

Billing

Signup friction

Best for

Thunder Compute

$1.09/hr

~$1.29/hr

$2.19/hr

Per-second

Email only

Cheapest sane pick

Vast.ai

$0.40–$1.20/hr spot

$0.60–$1.60 spot

$1.50–$3.00 spot

Per-second

Email + card

Cheapest overall, spot pricing

Lambda Labs

$1.29/hr

$1.79/hr

$2.79–$3.99/hr

Per-minute

Email + card

Reliable but often sold out

RunPod

$1.39/hr

$1.89/hr

$2.89/hr

Per-second

Email + card

Good UI, good docs

Modal

~$2.50/hr

—

~$3.95/hr

Per-second serverless

GitHub OAuth

Serverless per-function invocation

Paperspace

$2.30/hr

$3.09/hr

~$5.95/hr

Per-minute

Email + card

Overpriced now; used to lead

GMI Cloud

—

~$1.79/hr

$2.00/hr

Per-hour

Enterprise-ish

Cheap H100 if you can get in

Google Colab Pro

T4/A100 flexible

limited

none

Monthly $10

Google login

Notebook only, no ssh

Kaggle Notebooks

P100 free 30 hrs/week

—

none

Free

Google login

Free tier, notebook only

AWS SageMaker

~$4/hr

~$5.50/hr

~$8/hr

Per-second

AWS account

Overpriced for hobbyists

AWS EC2 p4d/p5

~$4.10/hr

—

~$12/hr on-demand

Per-second

AWS account

Skip unless enterprise

Azure ND-series

Similar to AWS

Similar

Similar

Per-second

Azure account

Skip

GCP A2 instances

~$3.70/hr

~$5/hr

~$11/hr

Per-second

GCP account

Skip unless Google $300 credit

Key insight verified from Reddit / HN in 2026: the same H100 costs $1.79/hr on Lambda when available and $12/hr on AWS on-demand — a 6.7× spread for identical silicon. Never rent GPUs from a hyperscaler for personal learning; always rent from a specialist.

§4. Concrete “$50/month” plan for Phase 5

Month split (typical):

  • $10 Colab Pro (~₹850) — always-on for quick notebook cells

  • $40 Thunder Compute A100 (~₹3,400) — ~36 hours/month of real CUDA C++ work

Total: $50/mo ≈ ₹4,250. Fits inside your paid-resources budget from 09_resources/05_paid_platforms.md (which allocates ~₹8,000/mo across learning tools during active phases).

Session pattern:

  1. Weekend morning: SSH into Thunder Compute A100 instance.

  2. git clone your CUDA repo, work for 3–4 hours.

  3. Snapshot to persistent storage.

  4. Shut down instance. Billing stops immediately.

  5. Total cost per weekend: 4 hrs × $1.09 = $4.36 (~₹370).

You do NOT keep instances running. Every provider bills for wall-clock, not utilization. A forgotten instance overnight is a ₹1,000 mistake.

§5. Billing setup from India — the tested path

International hourly-metered services are unfriendly to standard Indian debit cards. What actually works in 2026:

  1. Virtual USD-denominated card via Niyo, Jupiter, or Fi Money. Load ₹5,000 as USD, use it for cloud GPU subscriptions. RBI International-transaction rules apply but hobbyist-scale is fine. Confirm at the bank before use.

  2. HDFC or ICICI Forex Card for larger loads if you need multi-month commit.

  3. Zoho corporate card — only if the work is Zoho-relevant. Confirm with finance.

  4. Wise (formerly TransferWise) multi-currency USD account — requires KYC, but reliable and now widely available to Indian residents. Use only if you want a real USD balance.

Do NOT use your primary Indian debit card. The International-transaction daily limit on standard debit cards (often ₹10,000/day) will trip after 2 days of A100 use.

Also enable card-level fraud alerts via SMS. Cloud GPU providers occasionally place holds ($1 verification charge, etc.) that look like fraud to Indian banks.

§6. What you install on the cloud GPU (one-time setup, ~15 min)

Assume you rented a Thunder Compute A100 running Ubuntu 22.04 or 24.04 with NVIDIA driver 550+ and CUDA 12.4+ preinstalled.

# SSH in
ssh <user>@<instance-ip>

# Verify GPU + CUDA
nvidia-smi
# expected: NVIDIA A100 40GB, driver 550+, CUDA 12.4+

nvcc --version
# expected: Cuda compilation tools, release 12.4 or newer

# Install the C++ development bundle (Ubuntu apt)
sudo apt-get update && sudo apt-get install -y \
    build-essential cmake ninja-build \
    clang-18 clang-tidy-18 clang-format-18 lld-18 \
    git gh \
    python3-pip pipx \
    cuda-nsight-compute-12-4 cuda-nsight-systems-12-4

# Or: bring your Docker image from §4 of 04_docker_and_devcontainers.md
docker pull ghcr.io/<yourname>/cxx-dev:2026-cuda

Then VS Code Remote-SSH connects to it directly:

  1. Cmd+Shift+P → “Remote-SSH: Connect to Host…”

  2. Enter <user>@<instance-ip>

  3. VS Code re-installs your extensions on the remote, opens the folder.

  4. Terminal, debugger, everything now runs on the A100 box.

Total re-setup time on a fresh cloud instance: ~15 min. That’s why the Dockerfile in 04_docker_and_devcontainers.md matters — you can pull your image and skip 90% of setup.

§7. Nsight Compute and Nsight Systems — the CUDA C++ profiling tools

You are not fully doing CUDA C++ if you have not opened Nsight Compute at least once. On the cloud instance:

# Nsight Compute — kernel-level profiling
sudo apt-get install -y cuda-nsight-compute-12-4
ncu --version

# Nsight Systems — system-level timeline
sudo apt-get install -y cuda-nsight-systems-12-4
nsys --version

Use ncu to profile individual kernels:

ncu --set full ./my_kernel_app

Use nsys for timeline profiling of the whole app:

nsys profile -o report ./my_app
nsys stats report.nsys-rep

Copy .nsys-rep and .ncu-rep files back to your Mac via scp and open them in Nsight Compute / Nsight Systems GUI which runs on macOS. Yes, NVIDIA ships macOS-native Nsight viewers. Download from developer.nvidia.com/nsight-compute and .../nsight-systems.

This is the specific workflow that makes cloud A100 rental worth the ₹5,000/month: you profile there, you analyze locally.

§8. llama.cpp / vLLM / TensorRT-specific notes

Since your positioning is “ML engineer who drops into the C++ inference layer” (09_resources/07_zoho_alignment_reads.md), the three specific stacks you touch in Phase 5 are:

  • llama.cpp (Georgi Gerganov, C++11-compatible codebase, CUDA optional). Runs on M-series Metal (no cloud needed) OR CUDA. Read the CUDA path on cloud; run the Metal path locally.

  • vLLM (Python + CUDA C++). Cannot run on Mac at all. Cloud only. 30 min A100 sessions are enough to inspect the C++ paged-attention kernels.

  • TensorRT (NVIDIA proprietary, C++). Cloud only. Its trtexec benchmark tool is the single most-cited “does this model fit under my latency budget” test in India ML-eng studies (NVIDIA Bangalore, Krutrim, Sarvam AI, Ola Electric).

Budget guidance per stack:

  • llama.cpp CUDA: ~5 hours of A100 total — $5.5

  • vLLM inspection: ~10 hours A100 — $11

  • TensorRT hands-on: ~15 hours A100 — $16

  • Total Phase 5 cloud spend: ~$50 = ₹4,300 (one-off, not monthly)

Plus $10/mo Colab Pro for ongoing notebook work.

§9. Cost-control checklist (habits to bake in month 1)

  • Set cost alerts at the provider dashboard for every $10.

  • Set auto-shutdown after N hours idle in provider settings where supported (Thunder Compute, RunPod).

  • Never enable auto-recharge above ₹2,000. Force yourself to re-authorize spend monthly.

  • Use spot/interruptible instances where the workload is restart-safe (Vast.ai, RunPod spot).

  • Persistent storage is cheap (~$0.15/GB/mo). GPU time is expensive. Save state to storage, kill the GPU, resume later.

  • Weekly billing review: 5 min every Sunday to reconcile the provider dashboard against your bank statement.

  • Never leave a browser tab open with ssh — one accidental laptop lid-open overnight is ₹1,000.

§10. Longer-term option — buy vs rent (2027+)

If you sustain Phase 5+ CUDA work beyond your 13-month window, at some threshold buying beats renting.

  • Used RTX 3090 (24 GB) in India secondary market: ₹60,000–75,000 as of July 2026.

  • RTX 4070 Ti Super (16 GB) new: ~₹75,000.

  • RTX 4090 (24 GB) new: ~₹1,80,000–2,10,000.

You would also need:

  • Desktop chassis + PSU 850W+ + Ryzen/Intel mid-range CPU + 64GB RAM + NVMe — ~₹80,000–1,20,000 depending on trims.

Break-even math: ₹1.4L build (RTX 3090 used + midrange PC) ÷ ₹4,300/mo cloud budget = ~33 months to break even.

Verdict: rent through Phase 7 (13 months). Consider buying only if you land a role that requires ongoing CUDA work at home. In every other case, cloud stays cheaper because you pay only when you use.

Note: RTX 3090/4090 do not run inside a MacBook Pro chassis. You would need a second desktop machine. That’s a non-trivial life change; do not assume-away.

§11. India-specific gotchas

  • GST on cloud services. Some providers (Modal, RunPod) charge GST to Indian users; others don’t. Reconcile with your invoices. GST is claimable if this is a business expense; otherwise not.

  • Latency to US providers. SSH latency to US-East providers is ~230 ms from Bengaluru — noticeable for interactive typing but irrelevant for batch jobs. US-West is ~180 ms. Use mosh instead of raw ssh to make the terminal feel responsive.

  • Bandwidth caps. Home fiber plans (Airtel Xstream 300 Mbps) are usually uncapped; some ISPs FUP at 3 TB/month. Downloading a 40 GB model twice a week × 4 weeks = 320 GB/month — fine on Airtel/Jio.

  • PowerCut resilience. Buy a small UPS for your Mac if you plan 8-hour cloud-GPU training runs; a home power-cut kills your SSH session mid-training. Not because it kills the GPU (that keeps running and billing) but because you lose the tunnel and can’t stop it fast.

  • RBI reporting. Under RBI rules, aggregate foreign remittances above USD 250,000/year require reporting. You will not hit this. But monthly amounts above USD 500 may trigger extra bank scrutiny — keep it under.

§12. Quick provider signup recipes

Thunder Compute (recommended):

  1. Sign up at https://thundercompute.com.

  2. Add virtual USD card ($10 minimum load).

  3. Launch → A100 40GB → SSH key upload → Ubuntu 24.04.

  4. Instance ready in ~90 seconds. ssh in.

Vast.ai (cheap spot):

  1. Sign up at https://vast.ai.

  2. Add funds ($10 minimum).

  3. Search offers → filter by GPU model, DL image, price.

  4. Rent → SSH key upload → Jupyter or bare SSH.

  5. Instance ready in ~60 seconds.

Google Colab Pro (baseline):

  1. https://colab.research.google.com/signup.

  2. Pay $10/mo via Google Payments.

  3. Runtime → Change runtime type → A100 (when available).

  4. Use for notebook-only exploration; NOT a substitute for real C++ development.

§13. What NOT to do

  • Do not use AWS Free Tier for GPU. There is no GPU tier that’s free.

  • Do not use Colab Free for CUDA C++ development. It’s notebook-only, no root, and the runtime disconnects after 90 min idle.

  • Do not buy an eGPU box for a Mac. Apple Silicon does not support eGPUs.

  • Do not use Docker on the cloud GPU to run CUDA workloads unless the Docker daemon and NVIDIA Container Toolkit are set up — verify by running docker run --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi and confirming it prints the GPU.

  • Do not keep an instance running “just in case”. Snapshot state, terminate, spin up when needed.