Run a mock round

A saga replaces a single ACID transaction across services with a sequence of local transactions, each publishing an event or invoking the next step. If a step fails, previously completed steps are undone by compensating transactions, which are semantic inverses such as refunding a payment or releasing a reservation.

Two coordination styles:

• Orchestration: a central saga orchestrator tells each service what to do and handles failures. Easier to understand and monitor, but adds a coordinator.
• Choreography: each service reacts to events and emits the next. Loosely coupled, but the overall flow is harder to see and can turn into a tangle.

CreateOrder -> ReserveStock -> ChargePayment -> Ship
     cancel <-  release      <- refund        <- fail

Compensations are not true rollbacks: an email already sent cannot be unsent, so design steps to be reversible or idempotent. Combine with the outbox pattern and idempotent consumers to avoid lost or duplicate events.

A microservice architecture structures an application as a set of small, independently deployable services, each built around a business capability, owning its own data store and communicating over the network.

A monolith packages all functionality into one deployable unit. It is simpler to develop, test and deploy early, and intra-process calls are fast and transactional.

Microservices bring independent scaling, team autonomy, fault isolation and the freedom to choose technology per service. The costs are real: network latency, partial failure, distributed data consistency, harder debugging, and significant operational and observability investment. You cannot refactor across service boundaries with the compiler as easily.

The right question is not which is better but whether the organization, domain understanding and operational maturity justify the overhead. Many successful systems start as a modular monolith and extract services only when a clear boundary and scaling pressure exist.

A circuit breaker stops a caller from hammering a failing dependency. It tracks failures and moves between three states:

• Closed: calls pass through while failures stay under a threshold.
• Open: after too many failures, calls fail fast immediately without touching the dependency, giving it time to recover.
• Half-open: after a cool-down, a few trial calls are allowed. Success closes the breaker; failure reopens it.

CircuitBreaker cb = CircuitBreaker.ofDefaults("payment");
Supplier<String> decorated =
    CircuitBreaker.decorateSupplier(cb, () -> callPayment());

It prevents resource exhaustion and cascading failures, because threads and connections are not tied up waiting on timeouts. Pair it with timeouts, limited retries with backoff and jitter, bulkheads to isolate pools, and a fallback such as a cached response or a graceful error. Without a timeout a circuit breaker is much less effective, since slow calls are as damaging as failed ones.

Boundaries should follow the business domain, not technical layers. Techniques:

• Domain-Driven Design bounded contexts: each context has its own model and ubiquitous language. A word like "account" may mean different things in billing and in identity, which signals separate services.
• Business capability mapping: align services to what the business does, such as pricing, fulfilment or recommendations.
• Subdomain classification: invest careful design in the core domain and keep generic or supporting subdomains simpler.
• Team topology: a service should be ownable end to end by a team, consistent with Conway's Law.
• Change frequency and scale: things that change together or scale together belong together.

Good seam: Ordering | Payments | Inventory | Notifications
Bad seam:  Controllers | Services | DAOs

Validate boundaries with real usage; wrong ones show up as chatty synchronous calls, shared tables and coordinated releases. The Strangler Fig pattern lets you peel services out of a monolith incrementally along these seams rather than doing a risky big-bang rewrite.

With many services and network hops, you cannot debug from a single log file. Observability rests on three pillars plus correlation:

• Structured logs: JSON with consistent fields and a correlation or trace id on every line.
• Metrics: counters, gauges and histograms for request rate, error rate and latency, plus resource usage. Track percentiles, not just averages.
• Distributed tracing: propagate a trace context (for example W3C traceparent) so one request can be followed across services; OpenTelemetry is the common standard.
• Health and readiness endpoints so orchestrators route traffic only to healthy instances.

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

Add centralized dashboards and meaningful alerts on symptoms such as error rate and saturation rather than every host metric. A service mesh can emit much of this automatically. The goal is to answer "what happened to this request and why" without SSHing into production hosts.

Avoid microservices when:

• The domain is still being explored. Wrong boundaries are far more expensive to move than within a monolith.
• The team is small. Each service multiplies deployment, monitoring, on-call and security effort; a handful of engineers cannot operate dozens of services well.
• You lack operational maturity: automated CI/CD, centralized logging, metrics, tracing and infrastructure as code.
• The workload is low scale and the business needs fast iteration, where a monolith ships faster.
• Strong transactional consistency is central and distribution would only add complexity.
• There is no clear independent scaling or ownership driver.

A modular monolith with clear internal boundaries gives most of the maintainability benefit at a fraction of the cost, and lets you extract services later along proven seams. The rule of thumb is to earn distribution: adopt it only when a concrete, measurable problem makes it necessary.

Two-phase commit is rarely used in microservices because it needs a coordinating transaction manager, holds locks across services, blocks on coordinator failure and hurts availability. Instead, prefer an eventual-consistency design:

• Saga: model the workflow as local transactions with compensations (orchestrated or choreographed). Each step commits independently and failures trigger undo actions.
• Outbox pattern: write the business row and an event row in one local transaction, then publish the event asynchronously. This avoids the dual-write problem where a crash after the DB commit but before the publish loses the event.
• Idempotent consumers: deduplicate by message id or use upserts so retries and duplicate deliveries are harmless.
• Reconciliation: background jobs compare states and repair drift, and reads may use a read model projected from events.

BEGIN;
INSERT INTO orders ...;
INSERT INTO outbox (event_type, payload) VALUES ('order.created', :json);
COMMIT;

Choose compensations that are reversible or harmless, and make every step idempotent, because distributed atomicity is replaced by retries and repair.

Synchronous calls (HTTP/REST, gRPC) are simple, easy to reason about and return results immediately, which suits queries and request-response workflows. The downside is temporal coupling: if the callee is slow or down, the caller waits or fails, and failures can cascade.

Asynchronous messaging decouples services through a broker. The producer publishes an event and moves on, consumers process at their own pace, and the system absorbs traffic spikes and partial outages. The cost is eventual consistency, harder debugging, duplicate delivery and the need for idempotent consumers, plus a broker to operate.

Order HTTP -> Payment service            (sync, needs answer now)
Order -> order.created event -> Email    (async, fire and forget)

A common design uses sync for the user-facing read path and async for side effects such as emails, analytics, inventory updates and notifications. Prefer async when the caller does not need an immediate answer and when decoupling or burst handling matters.

In a dynamic environment, service instances start, stop and move, so callers cannot rely on fixed hostnames. Service discovery maintains a registry of healthy instances and lets clients resolve a logical service name to a live address.

Two patterns:

• Client-side discovery: the caller queries the registry (Consul, Eureka) and load-balances itself. Fewer hops but discovery logic in each client.
• Server-side discovery: the caller hits a load balancer or virtual IP, and the platform (Kubernetes Services, cloud load balancers) routes to a healthy instance. Simpler clients, one extra hop.

# Kubernetes resolves this name to ready pods
http://order-service.default.svc.cluster.local

Health checks are essential so unhealthy instances are removed quickly. In Kubernetes, DNS plus a Service is built-in discovery; for VMs you typically need a registry. Combine discovery with retries, timeouts and circuit breakers so a stale address does not cascade into an outage.