11.06 · Local Database & Infrastructure

When you touch this: M4 (JDBC), M7-M8 (Spring Data, Hibernate, real DB), M8-M9 (Kafka, Redis caching), M10-M11 (Testcontainers in CI).

You will not install Postgres, Redis, or Kafka natively on your Mac. Ever. You will run everything as containers via docker-compose.yml, tear it all down with one command, and rebuild in 30 seconds. This keeps your laptop clean and your production dependencies close to production.

The one exception is your JVM — that runs native, via SDKMAN. Everything else that has state or listens on a port: container.

The Default Stack

Service

Image

Port

Use in months

Postgres 16

postgres:16-alpine

5432

M4+ (default relational DB)

Redis 7

redis:7-alpine

6379

M7+ (caching, session, rate-limit)

Kafka (KRaft)

bitnami/kafka:3.7

9092

M8+ (event-driven)

Redpanda (alt)

redpandadata/redpanda:latest

9092

M8+ (Kafka API, lighter, 200 MB vs 2 GB)

pgAdmin

dpage/pgadmin4:latest

5050

Optional — DBeaver is better

MailHog

mailhog/mailhog

8025

M8 (test email sending)

The Golden docker-compose.yml

Create this at your project root or in a dedicated ~/dev/infra/ folder for a shared local stack. docker compose up -d postgres redis starts only what you need.

version: "3.9"
name: java-prep-infra

services:
  postgres:
    image: postgres:16-alpine
    container_name: pg
    environment:
      POSTGRES_USER: dev
      POSTGRES_PASSWORD: devpass
      POSTGRES_DB: appdb
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dev -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    container_name: redis
    ports: ["6379:6379"]
    command: ["redis-server", "--appendonly", "yes"]
    volumes: [redisdata:/data]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s

  kafka:
    image: bitnami/kafka:3.7
    container_name: kafka
    ports: ["9092:9092"]
    environment:
      KAFKA_CFG_NODE_ID: 0
      KAFKA_CFG_PROCESS_ROLES: controller,broker
      KAFKA_CFG_LISTENERS: PLAINTEXT://:9092,CONTROLLER://:9093
      KAFKA_CFG_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 0@kafka:9093
    volumes: [kafkadata:/bitnami/kafka]

  kafka-ui:
    image: provectuslabs/kafka-ui:latest
    container_name: kafka-ui
    ports: ["8090:8080"]
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092
    depends_on: [kafka]

volumes:
  pgdata:
  redisdata:
  kafkadata:

Commands you will run daily:

docker compose up -d              # start everything, background
docker compose up -d postgres     # only Postgres
docker compose logs -f kafka      # tail kafka logs
docker compose ps                 # health status
docker compose down               # stop containers (keeps volumes)
docker compose down -v            # nuke volumes too (fresh DB)

Postgres: The Default. Not MySQL.

For new projects in 2026, Postgres is the default. MySQL/MariaDB is only relevant if your employer mandates it (some Zoho legacy products, older TCS/Infosys projects). The reasons:

  • Better JSON support (jsonb indexes rival MongoDB for many use cases).

  • Better window functions, CTEs, RETURNING clauses.

  • Extensions: pgvector (semantic search — relevant for your ML background), pg_trgm (fuzzy text), TimescaleDB (time series), PostGIS.

  • Better default isolation (Read Committed with proper snapshot isolation).

Connection URL pattern:

jdbc:postgresql://localhost:5432/appdb?user=dev&password=devpass

GUI client: DBeaver Community (free, cross-platform, brew install --cask dbeaver-community). Skip pgAdmin — the web UI is slow and ugly. TablePlus is nicer but paid ($89, ~₹7,500).

Redis: For Cache, Session, Rate-Limit

Redis in 2026 is table stakes. Every Spring Boot service worth deploying uses it for something. Learn:

  • Cache-aside pattern (@Cacheable in Spring — M7).

  • Distributed lock (Redisson library or Spring Integration).

  • Rate limiting (token bucket via Redis + Lua script).

  • Pub/Sub (lightweight event fan-out; not a Kafka replacement).

CLI: docker exec -it redis redis-cli, then KEYS *, GET foo, MONITOR. GUI: RedisInsight (free, redis.com/redis-enterprise/redis-insight).

Note: Redis licence changed to SSPL/RSAL in 2024. Valkey (Linux Foundation fork) is a drop-in alternative if your legal team objects. docker pull valkey/valkey:8-alpine — identical API.

Kafka: The Skip Question

Kafka locally is heavy — a broker + controller eats ~1 GB RAM on idle. Two escape hatches:

  1. Redpanda — Kafka-API-compatible, single binary, C++, no JVM, ~200 MB idle. Perfect for local. In docker-compose.yml, swap the kafka service for a redpandadata/redpanda:latest container listening on 9092.

  2. Testcontainers — spin up Kafka only during test runs, not 24/7.

For M8 learning, use real Kafka (Bitnami image above) so you meet ZooKeeper-vs-KRaft, consumer groups, partitions in their native habitat. For M10 portfolio work, Redpanda is fine.

Kafka UI: provectuslabs/kafka-ui (baked into the compose above at localhost:8090). Better than the deprecated Confluent Control Center.

Testcontainers: The M9 Unlock

Testcontainers is the library that made “real DB in tests” cheap. Instead of an in-memory H2 that lies to you about Postgres semantics, you spin up a real Postgres container for the duration of your test run.

@Testcontainers
@SpringBootTest
class UserRepositoryIT {

    @Container
    @ServiceConnection  // Spring Boot 3.1+ auto-wires DataSource
    static PostgreSQLContainer<?> pg = new PostgreSQLContainer<>("postgres:16-alpine");

    @Autowired UserRepository repo;

    @Test
    void findsByEmail() {
        repo.save(new User("a@b.com"));
        assertThat(repo.findByEmail("a@b.com")).isPresent();
    }
}

Why this matters: every @Query you write against Postgres will actually run against Postgres, not a Frankenstein compat layer. Native jsonb queries, RETURNING, upserts — all real.

The @ServiceConnection annotation (Spring Boot 3.1+) removes 15 lines of @DynamicPropertySource boilerplate. Use it. There are container types for Postgres, MySQL, Redis, Kafka, Elasticsearch, MongoDB, LocalStack (fake AWS), Wiremock, and 40+ more — java.testcontainers.org/modules.

Speed tip: Testcontainers Desktop (free from Docker/AtomicJar) enables container reuse across test JVM runs — cuts your test suite from 45s to 5s by keeping Postgres warm.

What Not to Install Natively

  • Postgres via brew install postgresql — will conflict with the container’s port and eat startup RAM. If you already did this: brew services stop postgresql.

  • Redis via brew install redis — same reason.

  • Java Kafka distribution as tarball — painful. Container or nothing.

  • MongoDB Compass alongside the container — fine, GUI is welcome.

Data Seeding & Migrations

By M7 you own Flyway or Liquibase for schema migrations. Never use ddl-auto=update past a demo. Configure Flyway in Spring Boot:

spring.flyway.enabled=true
spring.flyway.locations=classpath:db/migration
spring.jpa.hibernate.ddl-auto=validate

Migration files live at src/main/resources/db/migration/V1__init.sql, V2__add_users.sql, etc. Never edit an old V* file after it has run. Add V3__fix.sql instead.

Backup / Restore Locally

docker exec pg pg_dump -U dev appdb > backup.sql
docker exec -i pg psql -U dev -d appdb < backup.sql

Use for demos where you want to freeze a data state, or when moving between machines.

What You Will Get Wrong First

  • Volumes not mounted. Container restarts wipe the DB. Confirm volumes: block exists in compose.

  • localhost inside the container. From inside the app container, Postgres is db:5432 or postgres:5432 (service name), not localhost:5432. Only from host machine is it localhost:5432.

  • Port already in use. Existing native install grabbing 5432 → lsof -iTCP:5432 -sTCP:LISTEN. Kill it.

  • Kafka advertised.listeners misconfigured. Producer connects, consumer times out. The address Kafka advertises must be reachable from the client. localhost:9092 for host-side clients, kafka:9092 for other containers.

  • No health checks. Your app starts before Postgres is ready → connection refused → Spring retries → confusion. Use depends_on: { db: { condition: service_healthy } }.

Practice Milestones

  • M4: Raw JDBC against the Postgres container. Read schema, batch insert, transactions.

  • M7: Spring Data JPA against Postgres. Add Flyway. Write one custom @Query using jsonb.

  • M8: Add Redis for a caching layer. Add Kafka; publish + consume in the same Spring Boot app with @KafkaListener.

  • M9: Rewrite your integration tests with Testcontainers. Delete every H2 dependency.

  • M10-M11: Portfolio project ships with a docker-compose.yml that spins up its entire dependency graph. A reviewer clones your repo and runs docker compose up; if it doesn’t work in 60 seconds, you failed.

Return to README.md · Next: 07_ai_agent_workflow_discipline.md