Caching & Distributed State — Redis as the Second Database

Once you have more than one service, you have a distributed state problem. Redis is the pragmatic answer for 90% of cases: fast, single-threaded (predictable), well-supported in Spring, and cheap to run. This file covers what to cache, how to invalidate, when locks are the wrong tool, and the four failure modes that will bite you in production.

Caching is not “put Redis in front of it.” Caching is a consistency contract you negotiate with your users.

Cache Patterns — Pick One and Own It

Pattern

Read path

Write path

Consistency

Use when

Cache-aside

App checks cache → miss → load from DB → populate cache

App writes DB, then invalidates cache

Eventually consistent; brief staleness after write

Default for 90% of cases

Write-through

Cache handles read misses; app never reads DB directly

App writes cache; cache writes DB synchronously

Strong within cache, slower writes

Read-heavy with strict consistency

Write-behind

Same as write-through

App writes cache; cache queues DB writes asynchronously

Weak; data loss risk on cache crash

Very high write throughput, loss-tolerant

Refresh-ahead

Cache proactively refreshes hot keys before expiry

Same as cache-aside

Same as cache-aside

Skewed access (small hot set)

⚠️ What most people get wrong: they mix patterns. Some endpoints cache-aside, some write-through, some write nothing to cache. Six months later, no one can reason about consistency. Pick one and enforce it as a review rule.

Spring Data Redis — Wiring It Up

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

spring:
  data:
    redis:
      host: redis
      port: 6379
      timeout: 200ms
      lettuce:
        pool:
          enabled: true
          max-active: 16
          max-idle: 8
          min-idle: 2

Lettuce (Netty-based, thread-safe, non-blocking) is the default and the right choice. Jedis is legacy — avoid.

Cache-Aside in Practice

@Service
@RequiredArgsConstructor
class LinkResolver {
    private static final Duration TTL = Duration.ofHours(1);
    private final StringRedisTemplate redis;
    private final LinkRepository repo;

    public Optional<String> resolveUrl(String slug) {
        String key = "link:%s".formatted(slug);
        String cached = redis.opsForValue().get(key);
        if (cached != null) {
            return Optional.of(cached);
        }
        return repo.findBySlug(slug)
                   .map(Link::getUrl)
                   .map(url -> {
                       // SET with TTL and NX to avoid stampede overwriting
                       redis.opsForValue().setIfAbsent(key, url, TTL);
                       return url;
                   });
    }

    public void invalidate(String slug) {
        redis.delete("link:%s".formatted(slug));
    }
}

Or use Spring’s @Cacheable abstraction with a RedisCacheManager — fine for simple cases, harder to control TTLs per key and stampede behavior. For anything hot, write the cache logic explicitly.

The Four Failure Modes

1. Cache Stampede (Thundering Herd)

When a hot key expires, N concurrent requests all miss, all hit the DB, all recompute the value. The DB melts.

Solutions in order of increasing sophistication:

  • Randomize TTLs — don’t set every key to expire at exactly 60 minutes. Use TTL + jitter(0, TTL/10) so expiries spread out.

  • Single-flight lock — first miss acquires a short-lived Redis lock (SET NX PX 5000); other misses wait 50ms and retry the cache read.

  • Probabilistic early expiration (XFetch) — recompute the value before it expires with probability that rises as TTL approaches. Advanced but elegant.

  • Never-expiring cache with async refresh — the cache always returns a value; a background job refreshes it. Highest availability, slightly stale data.

2. Cache Penetration

Requests for keys that don’t exist in the DB either. Each request hits the cache (miss), hits the DB (miss), returns nothing. DB is now under attack.

Solution: cache the negative result with a short TTL (30–60 seconds). Prevents attackers from hammering non-existent slugs.

if (cached == null) {
    return repo.findBySlug(slug)
               .map(link -> { redis.opsForValue().set(key, link.getUrl(), TTL); return link.getUrl(); })
               .or(() -> {
                   redis.opsForValue().set(key, "__NOT_FOUND__", Duration.ofSeconds(60));
                   return Optional.empty();
               });
}
if ("__NOT_FOUND__".equals(cached)) return Optional.empty();

3. Cache Avalanche

A large fraction of the cache expires at the same time (e.g., you loaded 10k keys during startup with the same TTL). All of them miss simultaneously.

Solution: the TTL jitter above; also stagger cache warming.

4. Consistency Skew After Write

App writes to DB, invalidates cache — but between DB commit and cache delete, a concurrent reader loads the old value into the cache. Now the cache is stale until the TTL.

Solutions:

  • Delete twice — delete cache before AND after the DB write.

  • Write-through — acceptable when write throughput is low.

  • CDC-driven invalidation — Debezium reads the WAL and publishes cache-invalidation events. Eliminates the race entirely but is heavyweight infra.

Distributed Locks — Only When You Must

Redisson gives you RLock semantics on top of Redis. The API looks nice; the semantics are subtle.

@Bean
RedissonClient redisson() {
    var cfg = new Config();
    cfg.useSingleServer().setAddress("redis://redis:6379");
    return Redisson.create(cfg);
}

// Usage — note the two timeouts
RLock lock = redisson.getLock("job:nightly-rollup");
boolean acquired = lock.tryLock(2, 30, TimeUnit.SECONDS);   // wait up to 2s, hold up to 30s
if (!acquired) throw new LockUnavailable();
try {
    // ... work ...
} finally {
    if (lock.isHeldByCurrentThread()) lock.unlock();
}

The Redlock Debate — Say This in studies

Martin Kleppmann wrote a widely-cited critique of the Redlock algorithm (multi-node Redis distributed lock). Antirez responded. The honest position:

  • For safety-critical exclusion (money, identity), do NOT rely on Redis locks alone. Use a fencing token: the lock returns a monotonically increasing token; the resource rejects operations with a stale token. Or use ZooKeeper / etcd / a database advisory lock.

  • For best-effort mutual exclusion (only-run-one-cron, deduplicate a burst of work), single-node Redisson locks are fine and simple.

Don’t reach for distributed locks by default. They’re a code smell that says “my design has a shared mutable resource across services.” Usually you can refactor to make the operation idempotent or move it into a single-partition Kafka topic where ordering handles the exclusion for you.

⚠️ What most people get wrong: they use a distributed lock to prevent duplicate work, then discover the lock TTL was shorter than the work. Two workers now hold “the lock.” Always: lock TTL > worst-case work duration, OR use fencing tokens.

Session Externalization

If you must use stateful sessions (some legacy apps do), do NOT keep them in the JVM. As soon as you scale beyond one instance, sticky sessions become a load-balancing nightmare.

<dependency>
  <groupId>org.springframework.session</groupId>
  <artifactId>spring-session-data-redis</artifactId>
</dependency>

spring:
  session:
    store-type: redis
    timeout: 30m
    redis:
      namespace: shortly:session

Done. Sessions now live in Redis; any pod can serve any request.

For new services, prefer stateless JWT. Reserve sessions for legacy or where you need server-side revocation without a token-blacklist scheme.

Idempotency Storage in Redis

Revisiting the Idempotency-Key pattern from file 01, with Redis specifics:

String key = "idem:%s".formatted(clientKey);
// SET NX + EX = atomic "insert if absent with TTL"
Boolean claimed = redis.opsForValue().setIfAbsent(key, "PENDING", Duration.ofHours(24));
if (Boolean.FALSE.equals(claimed)) {
    String state = redis.opsForValue().get(key);
    if ("PENDING".equals(state)) throw new ConflictException("in-flight");
    return json.readValue(state, LinkResponse.class);   // cached success
}
try {
    var res = service.doWork();
    redis.opsForValue().set(key, json.writeValueAsString(res), Duration.ofHours(24));
    return res;
} catch (Exception e) {
    redis.delete(key);   // let the client retry
    throw e;
}

Observability for Cache Layers

  • Micrometer auto-instruments Lettuce commands (lettuce_command_completion_seconds).

  • Track your own hit ratio: cache_hits_total / (cache_hits_total + cache_misses_total).

  • Alert when hit ratio drops below the historical p10 — usually a code deploy invalidated a key format.

  • Alert on redis_connected_clients approaching maxclients (default 10000).

  • Alert on redis_evicted_keys_total — you’ve exceeded maxmemory and Redis is dropping keys.

Redis Deployment Sanity

Deployment

When

Single node

Local dev, tiny services

Redis Sentinel

HA needed, standard failover

Redis Cluster

Sharded across nodes; needed above ~25 GB or 100k ops/sec

Managed (ElastiCache, MemoryStore, Upstash, Redis Cloud)

You’re not running a Redis cluster yourself — use the managed offering

Persistence config — pick one:

  • RDB snapshots — default, low overhead, potential data loss window

  • AOF (append-only file) — durable, small perf hit

  • AOF + RDB — durable + fast restore, production standard

  • maxmemory-policy: allkeys-lru for pure cache; volatile-lru (default) if you mix cache + non-cache data

Sins Checklist

  • Same TTL on every cache key (avalanche risk)

  • No negative caching for missing entries (penetration risk)

  • Distributed lock used for correctness without fencing tokens

  • Sessions in JVM memory

  • Cache invalidation only on the write path (races with slow readers)

  • Jedis instead of Lettuce

  • Redis with no maxmemory set (grows until OOM-kill)

  • Storing large blobs (>100KB) in Redis — you have an object store, use it

Practice

  1. Add cache-aside for /{slug} in shortly. Measure p95 latency with and without the cache under k6 load.

  2. Introduce cache stampede: force 1000 concurrent requests on a slug whose cache just expired. Watch DB CPU spike. Then implement the single-flight lock and confirm the spike disappears.

  3. Add negative caching for unknown slugs. Confirm a burst of /randomjunk requests hits Redis, not Postgres.

Return to README.md · Previous: 02_event_driven_kafka.md · Next: 04_java_for_ml_ai_applications.md