Payment System
IntermediateOverview
A payment system processes financial transactions - charges, refunds, and payouts - across merchants, users, and external payment providers (card networks, banks). The defining challenge is correctness under failure: money must never be lost, duplicated, or charged without authorization. Every design decision is anchored in exactly-once semantics, auditability, and regulatory compliance.
Functional Requirements
- Accept a payment: charge a user's payment method (card, wallet) for an order.
- Process refunds: reverse a completed payment, partially or fully.
- Track payment status with state transitions visible to both the merchant and the user.
- Reconcile internal records with external payment provider statements.
- Support multiple payment methods and currencies.
Non-Functional Requirements
- Exactly-once processing: a payment must never be charged twice, even if the client retries.
- Durability: a confirmed payment must survive any single-node failure.
- Consistency: payment state must be strongly consistent - no stale reads of transaction status.
- Low latency: payment authorization should return in under 2 seconds.
- Auditability: every state change must be logged immutably for regulatory and dispute resolution.
Capacity Estimation
Assume 10M transactions/day with an average payload of 2 KB.
- QPS:
- Payments: 10M / 86,400 s ~ 116 writes/sec (peak ~5x ~ 580/sec during flash sales).
- Status reads: assume 3x the write rate -> 350 reads/sec (peak ~1,750/sec).
- Storage:
- Per day: 10M x 2 KB = 20 GB/day.
- Over 7 years (regulatory retention): 20 GB x 365 x 7 ~ 51 TB.
- Bandwidth: write ingress 116/s x 2 KB ~ 0.23 MB/s - payments are tiny but must be durable.
The QPS is modest, but the correctness requirements make this design hard, not the scale.
High-Level Architecture
The core architecture consists of an API Gateway that routes requests to a Payment Service. This service manages a strict state machine backed by a highly consistent relational database (PostgreSQL). It interacts with external Payment Service Providers (PSPs) like Stripe or Adyen. A secondary background Reconciliation Service continuously matches internal ledger entries with reports from the PSPs to ensure zero money was lost or duplicated.
Data Model
| Entity | Fields / Schema | Storage Choice |
|---|---|---|
| payment_order | payment_id (PK), merchant_id, amount, currency, status, idempotency_key, created_at | Relational DB (strong consistency, ACID) |
| ledger_entry | entry_id, payment_id, account, debit, credit, created_at | Append-only relational table (double-entry bookkeeping) |
| idempotency_record | idempotency_key (PK), payment_id, response, created_at, expires_at | Relational or key-value with TTL |
| payment_event | event_id, payment_id, status, ts, details | Append-only event log |
| reconciliation_record | rec_id, payment_id, internal_status, external_status, matched, ts | Relational |
Detailed Design
Idempotency - The Most Important Design Element
Every payment request carries a client-generated idempotency_key. Before processing, the Payment Service checks the idempotency store:
- Key exists, completed -> return the stored response immediately (no re-charge).
- Key exists, in-progress -> return a "processing" status (avoid duplicate work).
- Key absent -> insert the key, proceed with the payment.
This is what prevents double-charging when a client retries after a network timeout. The idempotency key is the foundation of exactly-once semantics.
Payment Flow (Happy Path)
- Client sends
POST /paymentswith anidempotency_key. - Payment Service validates the request and creates a
payment_orderwith statusCREATED. - Service calls the external PSP to authorize/capture the charge.
- On PSP success: update status to
COMPLETED, write double-entry ledger entries (debit buyer, credit merchant), emit aPaymentCompletedevent. - On PSP failure: update status to
FAILED, emit aPaymentFailedevent. - Store the final response against the idempotency key.
Double-Entry Ledger
Every money movement creates two ledger rows: a debit from one account and a credit to another. The sum of all debits must equal the sum of all credits. This invariant is checked continuously and makes auditing trivial. This is how banks track every cent.
Reconciliation
A periodic reconciliation service compares internal ledger entries against settlement files from the PSP (usually delivered nightly via SFTP). Mismatches (e.g., a payment the PSP confirmed but that is missing internally) trigger alerts for manual resolution. This catches bugs, network failures, and fraud.
Bottlenecks & Solutions
The primary bottleneck is the External PSP. Calling a third-party API over the public internet is slow and prone to timeouts. If the PSP goes down, your system cannot process payments. Therefore, large companies often integrate with multiple PSPs (Stripe, Braintree, Adyen) and route traffic dynamically based on health checks and routing rules (e.g., using PSP A for US cards and PSP B for European cards).