NNODE LOOP LABruntime observatoryNNEON · Articles in 90 languages
CONNECTING
v24.18.0linux/x64
Capability → contract → transport
13

Microservices: boundaries, messages, and failures

Understand why services are separated, how commands and events cross a network, and which failure modes distributed systems introduce.

PROCESS IDcurrent server
UPTIMEsince startup
LOOP DELAY P95perf_hooks
UTILIZATIONevent loop
HTTP ROUNDTRIPbrowser → server
LIVE TRACE

Timeline

READY
0 ms
Events will appear hereRun the selected scenario
#TIMESOURCEEVENT
Waiting for an experiment…
CHAPTER 13
DEEP DIVE · FROM BASICS TO CODE

Understanding: Microservices: boundaries, messages, and failures

You can read this chapter before running the experiment. Then return to the live trace and match each concept to a real event.

FIRST, IN PLAIN LANGUAGE

A monolith resembles one large office: teams communicate quickly inside one building, but a move or outage affects everyone. Microservices are independent offices with explicit responsibilities. They communicate over a network, so every conversation can be delayed, duplicated, or lost.

TECHNICAL FOUNDATION

A microservice is an independently deployable application boundary around a business capability and its owned data. Services communicate synchronously with request-response or asynchronously with messages and events. Nest provides transport abstraction and handler decorators, but network failures, contract evolution, delivery guarantees, and cross-service observability remain application concerns.

Why it mattersMicroservices help when independent teams, different scaling profiles, or separate failure and deployment boundaries justify distributed complexity. They often hurt small systems by adding networking, brokers, tracing, contract testing, and eventual consistency without organizational benefit.
WHERE THE WORK RUNS
01PRODUCERNest controller · use case
02CONTRACTcommand · event · schema
03TRANSPORTTCP · Kafka · RabbitMQ · NATS
04CONSUMERhandler · owned data · side effect
01 · GLOSSARY

Terms used in this experiment

Understand the words first, then the execution order.

01

Service boundary

A deployment, responsibility, and data-ownership boundary around a business capability.

02

Request-response

A producer sends a request or command and expects one answer or error before a deadline.

03

Event

A fact that already happened, such as OrderCreated; the publisher does not require one consumer’s business response.

04

Message broker

Infrastructure that accepts, stores, and delivers messages, such as Kafka, RabbitMQ, NATS, or Redis Streams.

05

Delivery semantics

Whether delivery is at-most-once, at-least-once, or made practically exactly-once through constraints and idempotency.

06

Idempotency

Repeating an operation with the same key does not produce a second business effect.

07

Eventual consistency

Services may temporarily observe different state versions but converge after messages are processed.

08

Distributed trace

A connected request and message path across processes using trace and span identifiers.

02 · MECHANICS

What happens step by step

Each step maps to an observable runtime state.

  1. 01
    Find a capability

    Draw a boundary around business ownership such as orders, inventory, or payments, not technical folders.

  2. 02
    Define a contract

    Include version, operation or correlation identifiers, and a minimal payload that does not leak internal schemas.

  3. 03
    Choose interaction

    Use request-response for an immediate answer and an event for independent reactions to an established fact.

  4. 04
    Cross the transport

    Serialization turns a message into bytes; transport delivers it and the consumer deserializes it.

  5. 05
    Handle failure

    Timeout, retry, circuit-breaker, and dead-letter policies bound partial failure.

  6. 06
    Make repeats safe

    Persist an operation ID or use a UNIQUE constraint so duplicate delivery does not repeat a side effect.

  7. 07
    Connect observability

    Propagate correlation and trace context through every message boundary.

03 · CONTEXT

Where the result needs context

These details explain why similar code can sometimes produce a different trace.

01

A microservice is not merely a small REST API

Code size is secondary; independent business ownership, deployment, and protected data matter.

02

The network is part of the program

A server can complete while the client times out, so “no response” does not mean “not executed.”

03

At-least-once implies duplicates

A broker may redeliver after an acknowledgement failure; consumers must be idempotent.

04

Published events cannot be edited in place

Independent consumers already depend on the contract, so use compatible fields, versioning, and migrations.

05

Database per service describes ownership

Databases may share a physical cluster, but services should not directly mutate another owner’s tables.

06

TCP is a teaching transport here

The lab demonstrates a real Nest boundary without infrastructure; durable production delivery may need Kafka, RabbitMQ, or NATS.

01
Theory

First understand which parts of Node participate in execution.

02
Simplified code

Then remove instrumentation and focus on the central mechanism.

03
Runtime code

Finally match the model to the code that produces the live trace.

04 · Simplified code

A minimal model without instrumentation

src/demos.js · educational snippetJavaScript
@Controller()
export class InventoryMessages {
  @MessagePattern({ cmd: 'inventory.reserve' })
  reserve(@Payload() command: ReserveInventoryCommand) {
    return this.inventory.reserve(command);
  }
}

const reservation = await firstValueFrom(
  inventoryClient
    .send({ cmd: 'inventory.reserve' }, command)
    .pipe(timeout(2_000)),
);
05 · Runtime code

The complete code executed by the scenario

This is not an alternative example: these are the functions and files used by the Run button.

ACTUAL SOURCE

The source is generated from the real server function. Scenarios using a child process or Worker include every participating file.

src/microservices-lab.js
microservice236 lines
import 'reflect-metadata';
import net from 'node:net';
import {
  Controller,
  Dependencies,
  Module,
} from '@nestjs/common';
import { NestFactory } from '@nestjs/core';
import {
  ClientProxyFactory,
  EventPattern,
  MessagePattern,
  Payload,
  RpcException,
  Transport,
} from '@nestjs/microservices';
import {
  firstValueFrom,
  lastValueFrom,
  timeout,
} from 'rxjs';

const LAB_TRACE = Symbol('MICROSERVICES_LAB_TRACE');

async function reserveTcpPort() {
  const server = net.createServer();
  await new Promise((resolve, reject) => {
    server.once('error', reject);
    server.listen(0, '127.0.0.1', resolve);
  });
  const address = server.address();
  const port = typeof address === 'object' ? address.port : null;
  await new Promise((resolve) => server.close(resolve));
  if (!port) throw new Error('Не удалось выбрать локальный TCP-порт');
  return port;
}

class InventoryMessageController {
  constructor(trace) {
    this.trace = trace;
    this.reservations = new Map();
  }

  reserve(command) {
    this.trace.emit(
      'consumer',
      'request',
      `Inventory получил command inventory.reserve для order=${command.orderId}`,
    );

    const existing = this.reservations.get(command.operationId);
    if (existing) {
      this.trace.emit(
        'idempotency',
        'duplicate',
        `Повтор operationId=${command.operationId} вернул прежний reservation`,
      );
      return { ...existing, reused: true };
    }

    if (command.sku === 'sold-out') {
      throw new RpcException({
        code: 'OUT_OF_STOCK',
        message: 'Товар закончился',
      });
    }

    const reservation = {
      reservationId: `reservation-${this.reservations.size + 1}`,
      operationId: command.operationId,
      reused: false,
    };
    this.reservations.set(command.operationId, reservation);
    return reservation;
  }

  orderCreated(event) {
    this.trace.emit(
      'event-consumer',
      'event',
      `Notification получил event order.created для order=${event.orderId}`,
    );
  }
}

Dependencies(LAB_TRACE)(InventoryMessageController);
Controller()(InventoryMessageController);

const reserveDescriptor = Object.getOwnPropertyDescriptor(
  InventoryMessageController.prototype,
  'reserve',
);
MessagePattern({ cmd: 'inventory.reserve' })(
  InventoryMessageController.prototype,
  'reserve',
  reserveDescriptor,
);
Payload()(
  InventoryMessageController.prototype,
  'reserve',
  0,
);

const eventDescriptor = Object.getOwnPropertyDescriptor(
  InventoryMessageController.prototype,
  'orderCreated',
);
EventPattern('order.created')(
  InventoryMessageController.prototype,
  'orderCreated',
  eventDescriptor,
);
Payload()(
  InventoryMessageController.prototype,
  'orderCreated',
  0,
);

class MicroservicesLabModule {}
Module({
  controllers: [InventoryMessageController],
})(MicroservicesLabModule);

export async function microservicesMessaging(emit) {
  const port = await reserveTcpPort();
  const module = {
    module: MicroservicesLabModule,
    providers: [
      {
        provide: LAB_TRACE,
        useValue: { emit },
      },
    ],
  };
  const options = {
    transport: Transport.TCP,
    options: {
      host: '127.0.0.1',
      port,
    },
  };
  const service = await NestFactory.createMicroservice(module, {
    ...options,
    logger: false,
  });
  const client = ClientProxyFactory.create(options);

  try {
    emit(
      'boundary',
      'start',
      'Запускаем отдельную Nest application boundary с TCP transport',
    );
    await service.listen();
    await client.connect();

    const command = {
      operationId: 'reserve-order-42',
      orderId: 'order-42',
      sku: 'node-book',
      quantity: 1,
    };
    emit(
      'producer',
      'send',
      'Checkout вызывает request-response pattern inventory.reserve и ждёт один ответ',
    );
    const first = await firstValueFrom(
      client
        .send({ cmd: 'inventory.reserve' }, command)
        .pipe(timeout(2_000)),
    );
    emit(
      'request-response',
      'result',
      `Получен reservation=${first.reservationId}; reused=${first.reused}`,
    );

    const duplicate = await firstValueFrom(
      client
        .send({ cmd: 'inventory.reserve' }, command)
        .pipe(timeout(2_000)),
    );
    emit(
      'idempotency',
      'result',
      `Повторный command не создал вторую запись; reused=${duplicate.reused}`,
    );

    emit(
      'producer',
      'emit',
      'Checkout публикует event order.created и не ждёт business-ответ consumer-а',
    );
    await lastValueFrom(
      client
        .emit('order.created', { orderId: command.orderId })
        .pipe(timeout(2_000)),
      { defaultValue: undefined },
    );

    try {
      await firstValueFrom(
        client
          .send(
            { cmd: 'inventory.reserve' },
            {
              ...command,
              operationId: 'reserve-order-43',
              orderId: 'order-43',
              sku: 'sold-out',
            },
          )
          .pipe(timeout(2_000)),
      );
    } catch (error) {
      const code = error?.code ?? 'REMOTE_ERROR';
      emit(
        'remote-error',
        'error',
        `Remote error пересёк transport boundary: ${code}`,
      );
    }

    emit(
      'architecture',
      'result',
      'Микросервисы дают независимые границы deploy и владения данными, но добавляют сеть, partial failures, contracts, observability и delivery semantics',
    );
  } finally {
    client.close();
    await service.close();
    emit('cleanup', 'done', 'TCP client и Nest microservice остановлены');
  }
}

The emit(...) calls become live-trace rows. await and Promise keep the HTTP stream open until the scenario completes.

06 · RECIPES

Practical patterns worth keeping nearby

Compare the goal, code, and caveats instead of memorizing syntax without a model.

01

Command consumer

Let Inventory answer a command and return a producer result.

@Controller()
export class InventoryMessages {
  @MessagePattern({ cmd: 'inventory.reserve' })
  reserve(@Payload() command: ReserveInventoryCommand) {
    return this.inventory.reserve(command);
  }
}
  • The pattern addresses a message inside the transport.
  • @Payload extracts the deserialized body.
  • The return value becomes the response.
02

Request-response producer

Send a command and wait for a bounded time.

const response = await firstValueFrom(
  inventoryClient
    .send(
      { cmd: 'inventory.reserve' },
      { operationId, orderId, sku, quantity },
    )
    .pipe(timeout(2_000)),
);
  • send creates an Observable request.
  • firstValueFrom turns its first response into a Promise.
  • timeout does not necessarily cancel the consumer.
03

Event consumer

Let Notifications react to an order that already exists.

@Controller()
export class OrderEvents {
  @EventPattern('order.created.v1')
  handle(@Payload() event: OrderCreatedV1) {
    return this.notifications.sendConfirmation(event);
  }
}
  • The name describes a completed fact.
  • A version supports contract evolution.
  • The publisher does not know every consumer.
04

Event publisher

Notify independent consumers after a successful commit.

this.events.emit('order.created.v1', {
  eventId: randomUUID(),
  occurredAt: new Date().toISOString(),
  orderId,
  customerId,
});
  • A production event is often published by an outbox relay.
  • eventId supports deduplication and tracing.
05

Idempotent consumer

Avoid repeating a side effect after duplicate delivery.

await db.query(
  `INSERT INTO processed_messages (consumer, message_id)
   VALUES ($1, $2)
   ON CONFLICT DO NOTHING
   RETURNING message_id`,
  ['email-confirmation', event.eventId],
);
  • UNIQUE(consumer, message_id) protects races.
  • Zero rowCount means duplicate.
  • The marker and side effect need a consistent transaction boundary.
06

Transactional outbox

Avoid losing an event between database commit and broker publish.

await db.transaction(async (tx) => {
  const order = await orders.insert(tx, input);
  await outbox.insert(tx, {
    type: 'order.created.v1',
    aggregateId: order.id,
    payload: { orderId: order.id },
  });
});
  • The order and outbox row commit atomically.
  • A relay retries broker publishing.
  • Consumers still need duplicate safety.
07 · PRODUCTION CASES

How a learning mistake becomes an incident

A realistic service: the original code, observable failure, corrected implementation, and why the correction works.

CASE 01

Checkout creates a long synchronous chain across four services

A Nest CheckoutService creates an order and then sequentially waits for Inventory, Payments, and Notifications. Every remote call is required before HTTP 200.

INCIDENT CONTEXT

Latencies add, availability multiplies, and a timeout after a successful payment leaves an unknown outcome. Retrying the whole HTTP request may charge or reserve twice.

BEFOREPROBLEMATIC IMPLEMENTATION
@Injectable()
export class CheckoutService {
  async checkout(input: CheckoutDto) {
    const order = await this.orders.create(input);

    await firstValueFrom(
      this.inventory.send('reserve', order),
    );
    await firstValueFrom(
      this.payments.send('charge', order),
    );
    await firstValueFrom(
      this.notifications.send('email', order),
    );

    return { ...order, status: 'completed' };
  }
}

The HTTP request owns a distributed chain without one transaction manager. A successful remote side effect and a lost response are indistinguishable to the producer.

AFTERCORRECTED IMPLEMENTATION
@Injectable()
export class CheckoutService {
  async checkout(input: CheckoutDto) {
    return this.db.transaction(async (tx) => {
      const order = await this.orders.createPending(tx, input);
      const eventId = randomUUID();

      await this.outbox.add(tx, {
        eventId,
        type: 'order.placed.v1',
        payload: { orderId: order.id },
      });

      return { orderId: order.id, status: 'pending' };
    });
  }
}

@Controller()
export class InventoryEvents {
  @EventPattern('order.placed.v1')
  handle(@Payload() event: OrderPlacedV1) {
    return this.idempotency.once(
      event.eventId,
      () => this.inventory.reserve(event.orderId),
    );
  }
}

The order and outbox event commit atomically. A relay retries publishing, while the consumer accepts possible duplicates through eventId. HTTP returns pending as the workflow converges asynchronously.

FUNCTIONS AND CONSTRUCTS

What the unfamiliar calls from both code samples actually do.

ClientProxy.send(pattern, data)
Creates a Nest request-response Observable whose producer expects one response or remote error.
firstValueFrom(observable)
Turns the first emitted RxJS response into a Promise suitable for await.
db.transaction(callback)
Runs order and outbox writes on one database connection with one COMMIT or ROLLBACK.
outbox.add(tx, event)
Stores a future event in the same transaction instead of trying to atomically coordinate PostgreSQL and a broker.
@EventPattern(name)
Registers a Nest consumer handler for an event pattern on the selected transport.
idempotency.once(eventId, work)
A representative application helper that atomically remembers eventId and skips work on duplicate delivery.
WHY THE CORRECTION WORKS

This is not universal: an immediate critical answer may need bounded request-response. Email should not extend checkout, while multi-service workflows require sagas, compensation, and explicit states.

WHAT PRODUCTION SHOWED
  • Checkout p99 equals the sum of all downstream p99 values.
  • A client timeout coincides with a successful charge in payment logs.
  • One correlation ID crosses a long sequential trace chain.
08 · DO NOT CONFUSE

Common misconceptions

The myth is on the left; the accurate model is on the right.

MYTH

Microservices automatically make a system scalable.

ACTUALLY

They permit independent scaling, but poor contracts and a shared database preserve coupling.

MYTH

A service may synchronously call five more services.

ACTUALLY

Long request chains multiply latency and partial-failure probability.

MYTH

A broker guarantees there are no duplicates.

ACTUALLY

Practical delivery usually requires idempotent consumers and deduplication.

MYTH

A shared database is harmless convenience.

ACTUALLY

Direct writes into another service’s tables destroy ownership and independent deployment.

MYTH

A pet project should begin with ten services.

ACTUALLY

A modular monolith is often cheaper until a measurable reason for extraction appears.

09 · SELF-CHECK

Explain it in your own words

If you can explain the answer without quoting documentation, your mental model is starting to take shape.

  1. When is request-response better than an event, and when is it worse?
  2. Why does a timeout not prove that remote work did not execute?
  3. Why do messages need eventId, operationId, and version?
  4. Why must an at-least-once consumer be idempotent?
  5. How does a transactional outbox close the dual-write gap?
  6. Why should two services not mutate the same business table?
  7. When is a modular monolith the more reasonable choice?