Technology & EngineeringBlogBuckett Intelligence Dispatch

PostgreSQL vs Redis for Ledgers: Concurrency and ACID Compared

Choosing between relational ACID engines and distributed in-memory caches for high-concurrency ledgers involves trade-offs between durability and raw throughput. Here is how they stack up under real-world contention.

Technology & Engineering visualization
Share this dispatch:
TechTrendingInsights

Building a modern financial ledger forces engineering teams to confront a fundamental friction point in systems architecture: raw execution velocity versus verifiable mathematical correctness. When transaction volume spikes during clearing windows or flash payments, standard database engines encounter severe lock contention on popular intermediary accounts. For many infrastructure teams, the immediate instinct is to place a high-throughput, distributed in-memory state engine like Redis in front of the datastore, offloading balance updates to memory keys and resolving settlements in sub-millisecond cycles.

That architecture functions smoothly until the first unexpected node failover or network partition occurs. When an in-memory cache acts as the authoritative ledger state, asynchronous replication lags and non-deterministic serialization create silent balance drift, phantom overdrafts, and irrecoverable journal divergence. Meanwhile, standard relational engines like PostgreSQL eliminate balance corruption through battle-tested ACID guarantees and strict write-ahead logging (WAL), but they hit steep performance ceilings when hundreds of concurrent workers attempt to acquire pessimistic locks on the same balance row.

⚡ Executive Briefing & Core Takeaways - Durability vs. Throughput Ceilings: Redis delivers upwards of 120,000 writes per second per node with sub-millisecond p99 latencies using single-threaded Lua scripting, but it exposes ledgers to potential state loss during sudden node crashes unless configured with synchronous persistence that severely degrades throughput. - Isolation Invariant Guarantees: PostgreSQL running with Serializable Snapshot Isolation (SSI) or explicit pessimistic row-level locking (SELECT ... FOR UPDATE) guarantees zero double-spend anomalies and strict double-entry balance preservation, capping write contention to roughly 3,500 to 5,000 writes per second on a single contended account key. - Optimal Production Pattern: Top-tier transactional systems avoid treating in-memory grids as books of record; instead, they leverage memory caches strictly for transient authorization holds, nonce validation, and rate-limiting while anchoring financial balances to append-only relational ledger tables.


Architectural Mechanics: How Both Systems Handle Balance Updates

Understanding how both engines resolve concurrent balance modifications requires analyzing their fundamental execution paths. The core requirement of any financial ledger is double-entry accounting: every transfer consists of at least two balanced entries (a debit and a credit) where the sum of changes equals zero, and no account balance may drop below predefined constraints (such as preventing negative balances).

MERMAID DIAGRAM
flowchart TD
    subgraph Ingest["Request Ingest Layer"]
        A["Inbound Transfer Request<br/>Debtor to Creditor"]
    end

    subgraph Relational["PostgreSQL ACID Engine"]
        B["Explicit Row Lock<br/>SELECT FOR UPDATE"]
        C["Check Balance Constraint<br/>Balance - Amount >= 0"]
        D["Append Journal Lines<br/>Immutable Double-Entry"]
        E["Commit WAL Buffer<br/>fsync to Disk"]
        B --> C --> D --> E
    end

    subgraph InMemory["Redis In-Memory Grid"]
        F["Lua Script / HINCRBY<br/>Atomically Decrement"]
        G["In-Memory State Updated<br/>Sub-Millisecond Return"]
        H["Async Replication to Replica<br/>+ AOF Buffer Flush"]
        F --> G --> H
    end

    A -->|Strict Durability Path| B
    A -->|Low Latency Cache Path| F

The PostgreSQL ACID Model: Pessimistic Locks and WAL

In PostgreSQL, ledger integrity relies on Write-Ahead Logging and MVCC (Multi-Version Concurrency Control). A typical transfer transaction uses explicit row-level locks:

SQL
BEGIN;

-- Lock debtor and creditor accounts deterministically by ID to prevent deadlocks
SELECT balance FROM accounts 
WHERE id IN (101, 202) 
ORDER BY id 
FOR UPDATE;

-- Verify debtor balance invariant
UPDATE accounts 
SET balance = balance - 250.00 
WHERE id = 101 AND balance >= 250.00;

-- Credit destination account
UPDATE accounts 
SET balance = balance + 250.00 
WHERE id = 202;

-- Record immutable audit log
INSERT INTO ledger_entries (debtor_id, creditor_id, amount, created_at)
VALUES (101, 202, 250.00, NOW());

COMMIT;

When this transaction runs:

  1. PostgreSQL acquires exclusive tuple locks (XMAX) on the specific rows in the accounts table.
  2. If another transaction attempts to debit account 101, it blocks waiting for the transaction lock to release.
  3. Upon COMMIT, PostgreSQL flushes the write-ahead log buffer to disk via fsync() (assuming synchronous_commit = on).
  4. Only after physical confirmation from storage does the engine acknowledge the caller.

The Redis In-Memory Model: Single-Threaded Atomic Scripts

Redis bypasses kernel lock arbitration and multi-threaded context switches by executing operations on a single event loop. To execute an atomic transfer, developers deploy a Lua script or utilize MULTI/EXEC:

LUA
-- Atomic Transfer Script in Redis
local debtor = KEYS[1]
local creditor = KEYS[2]
local amount = tonumber(ARGV[1])

local debtor_balance = tonumber(redis.call('GET', debtor) or 0)

if debtor_balance < amount then
    return redis.error_reply("INSUFFICIENT_FUNDS")
end

redis.call('DECRBY', debtor, amount)
redis.call('INCRBY', creditor, amount)

return "OK"

Because Redis processes Lua scripts atomically on its primary worker thread, no concurrent command can interleave between checking the debtor balance and executing the decrement. Contention is resolved via a serial queue in memory, completely eliminating row locks, deadlocks, and latch contention.


Telemetry and Benchmark Comparison

When evaluating both engines under identical synthetic workloads - consisting of 80% distributed multi-account transfers and 20% concentrated hot-account transfers - the structural differences become stark.

MetricPostgreSQL 16 (Enterprise NVMe, fsync ON)Redis 7.2 (Standalone, AOF everysec)Redis 7.2 (AOF always / Sync fsync)
Max TPS (Random Accounts)18,500 - 24,000115,000 - 140,0004,200 - 6,100
Max TPS (Single Hot Account)3,200 - 4,80085,000 - 95,0003,800 - 4,500
p95 Latency (Uncontended)4.8 ms0.6 ms8.2 ms
p99 Latency (High Contention)48.0 ms (Queue wait)2.1 ms22.4 ms
RPO (Recovery Point Objective)0 (Zero committed data loss)Up to 1.0 second of data loss0 (Disk bound)
Cross-Table ConstraintsEnforced natively (FOREIGN KEY, CHECK)None (Requires application code)None (Requires application code)
Memory Footprint (10M Balances)~2.4 GB (Disk-backed buffer cache)~1.8 GB (Resident RAM required)~1.8 GB (Resident RAM required)

When Redis is configured with appendfsync everysec (the production default), it outpaces PostgreSQL by nearly an order of magnitude. However, under this configuration, Redis operates with an RPO of up to one second. A power failure or kernel panic on the master node before asynchronous replication completes can permanently erase committed ledger transfers.

Configuring Redis with appendfsync always brings its write path to the exact same NVMe physical barrier that PostgreSQL faces. In that mode, Redis's throughput collapses to under 6,500 TPS because its single worker thread blocks waiting for storage write acknowledgments on every single command.


Critical Failure Modes Under Production Contention

1. The Redis Replication Gap and Phantom Balances

In a distributed Redis cluster or sentinel configuration, data replication from primary to replica is asynchronous by default. Consider this sequence:

  1. Client A transfers $1 from Account X to Account Y. Redis Master processes the Lua script and acknowledges success.
  2. Before the replication packet crosses the network switch to Replica 1, the primary server's power supply fails.
  3. Redis Sentinel promotes Replica 1 to Master.
  4. Replica 1 has not received the transfer. Account X's balance still displays the original amount.
  5. Client A immediately initiates a second transfer of the same $1 The new primary approves it.

The result is a double-spend anomaly created entirely by asynchronous consensus. While Redis offers the WAIT command to enforce synchronous replica acknowledgments, WAIT does not make Redis a strongly consistent consensus engine; a split-brain condition can still lead to data overwrites upon network partition recovery.

2. Lock Saturation in PostgreSQL

PostgreSQL guarantees total mathematical consistency, but its weakness under high concurrency is lock queue starvation. When thousands of concurrent client threads target the same merchant or treasury account balance, each transaction blocks behind the preceding lock owner.

As lock queues lengthen: - PostgreSQL's active backend processes saturate the server's max_connections limit. - Memory consumption jumps due to per-backend work memory and connection overhead. - Context switching across hundreds of blocked OS threads degrades overall CPU cache efficiency. - Client connection pools (e.g., PgBouncer) exhaust their wait queues, throwing connection timeouts back to API gateways.


The Production Blueprint: Partitioned Ledger Architecture

To achieve both high throughput and uncompromised ACID durability, modern financial platforms avoid choosing between pure relational databases or pure cache architectures. Instead, they isolate volatile balance reservations from immutable settlement ledgers.

MERMAID DIAGRAM
sequenceDiagram
    autonumber
    actor Client
    participant Gateway as API Gateway
    participant Redis as Redis Reservation Cache
    participant Queue as Kafka / Buffer Queue
    participant PG as PostgreSQL Ledger Engine

    Client->>Gateway: POST /v1/transfers ($100)
    Gateway->>Redis: Execute Lua: Check & Reserve Balance
    alt Insufficient Hold Limit
        Redis-->>Gateway: REJECT (Insufficient funds)
        Gateway-->>Client: 422 Unprocessable Entity
    else Hold Reserved
        Redis-->>Gateway: OK (Hold Token Created)
        Gateway->>Queue: Publish TransferIntent Event
        Gateway-->>Client: 202 Accepted (Processing)
        Queue->>PG: Batched Ingest Worker
        Note over PG: BEGIN TRANSACTION<br/>Debit / Credit Journal Lines<br/>Update Authoritative Accounts<br/>COMMIT
        PG-->>Redis: Clear Hold Token / Reconcile Balance
    end

Strategic Implementation Rules:

  1. Never Store Canonical Ledger State in Cache: Redis should only store transient reservation balances (e.g., pending authorizations expiring in 60 seconds) or cryptographic idempotency keys.
  2. Turn Balance Updates into Append-Only Journals: In PostgreSQL, avoid updating an account's balance column directly on every transaction. Instead, insert immutable debit and credit lines into a append-only journal_entries table. Append-only tables eliminate write-lock contention on hot balance rows.
  3. Use Materialized Balance Rollups: Calculate the live balance via periodic background rollups combined with un-checkpointed journal line sums, keeping hot account updates completely non-blocking.

The Verdict

If your application requires strict regulatory compliance, immutable audit histories, and zero tolerance for balance discrepancies, PostgreSQL is non-negotiable. While it requires deliberate performance engineering - such as append-only journaling, connection pooling, and account partitioning - it guarantees that a committed balance will never drift or evaporate during infrastructure failure.

Redis excels as an auxiliary engine, ideal for sub-millisecond fraud-score checks, velocity rate limits, distributed idempotency locks, and uncommitted balance pre-authorizations. Relying on Redis as your primary source of ledger truth sacrifices fundamental ACID invariants for raw speed - a trade-off that financial software engineering rarely survives.

Share this dispatch:
WESTERN DAILY INSIDER DISPATCH

Stay Ahead of US & European Markets, Tech & AI Trends

Join over 45,000+ US & European tech founders, quantitative traders, biotech researchers, and software architects receiving our morning dispatch.

Zero Spam. Unsubscribe anytime. Daily 6:00 AM EST Delivery

Free daily digest. Privacy guaranteed under GDPR & CCPA.

Recommended Dispatches & Related Intelligence

Handpicked
Technology & Engineering visualizationTechBlogBuckett Intelligence
#Tech#Trending#Insights

Taming Hotkey Contention: Hybrid Ledger Architecture Combining In-Memory Write Buffering with Relational ACID Persistence

When extreme write contention hits single row keys in relational databases, traditional ACID locks create catastrophic head-of-line blocking. Here is how hybrid architectures combine deterministic in-memory write-queuing with asynchronous WAL flushes to achieve sub-millisecond commits without sacrificing crash durability.

2026-08-318 min read
Read Analysis