CHAPTER 09
DEEP DIVE · FROM BASICS TO CODE

Understanding: Closures, GC, and heap snapshots

You can read this chapter before running the experiment. Then return to the live trace and match each concept to a real event.

FIRST, IN PLAIN LANGUAGE

A closure is a function with access to variables from its creation environment. It is useful and safe by itself. A leak appears when a long-lived reference to the function accidentally extends the lifetime of a large payload. A heap snapshot lets you walk backward from the object through retaining edges to a GC root.

TECHNICAL FOUNDATION

V8 manages the JavaScript heap and traverses from GC roots such as globals, active stacks, closures, and internal handles. Unreachable objects may be collected; reachable objects remain alive regardless of business intent. Generational collection optimizes for the common case where many young objects die quickly and survivors move to an older generation.

Why it mattersIn production, a slow leak can resemble a cache for weeks. GC work, pause time, and RSS then rise until a container is OOM-killed. A restart only hides the cause temporarily; diagnosis needs a repeatable workload, time series, and a proven retainer path.
WHERE THE WORK RUNS
01YOUR JS CODEfunctions · callbacks
02NODE APIsfs · crypto · timers
03V8 + LIBUVheap · loop · pool
04OPERATING SYSTEMI/O · threads · memory
01 · GLOSSARY

Terms used in this experiment

Understand the words first, then the execution order.

01

Closure

A function together with access to the lexical environment where it was created.

02

Shallow size

Memory occupied by the object itself, excluding all objects reached through its references.

03

Retained size

An estimate of memory that could become unreachable if this object and its retaining paths disappeared.

04

Retainer

An object or edge that keeps the inspected object reachable from a GC root.

05

Dominator

A node through which paths from GC roots to a group of objects pass, useful for finding an owner of a large retained subtree.

06

Heap snapshot

A serialized graph of objects and references for one V8 isolate at a point in time.

02 · MECHANICS

What happens step by step

Each step maps to an observable runtime state.

  1. 01
    Confirm the symptom

    Look for sustained heap or RSS growth after equivalent load cycles, not one random peak.

  2. 02
    Capture a baseline

    After warm-up and stabilization, create the first snapshot on a safe replica.

  3. 03
    Reproduce growth

    Repeat one operation a controlled number of times and let temporary work finish.

  4. 04
    Capture a second snapshot

    Compare object counts, shallow and retained sizes, and constructor or group deltas.

  5. 05
    Follow retaining paths

    Trace grown objects back to an array of closures, global Map, listener, or another root.

  6. 06
    Fix ownership and retest

    Add cleanup, TTL, LRU, a size bound, or listener removal, then repeat the same workload.

03 · CONTEXT

Where the result needs context

These details explain why similar code can sometimes produce a different trace.

01

A closure is not simply a full scope copy

Think in terms of the available lexical environment. The actual retaining edge in the current V8 snapshot is what matters.

02

A snapshot belongs to one isolate

A snapshot of the main Next.js process cannot contain the memory child or a Worker Thread heap, so this lab invokes writeHeapSnapshot inside the child.

03

Buffer appears differently from Array

Snapshots show JS wrappers and edges, while most Buffer backing memory is external. Correlate it with external, arrayBuffers, and RSS.

04

Taking a snapshot is itself risky

Serialization synchronously blocks the target Event Loop and may require about twice the heap. Use a replica, memory limits, and a restart plan.

05

RSS may not fall immediately

GC returns objects to an allocator that can retain pages for reuse. Validate the new plateau and heap trend.

01
Theory

First understand which parts of Node participate in execution.

02
Simplified code

Then remove instrumentation and focus on the central mechanism.

03
Runtime code

Finally match the model to the code that produces the live trace.

04 · Simplified code

A minimal model without instrumentation

src/demos.js · educational snippetJavaScript
const retainedClosures = [];
const globalCache = new Map();

function createHandler() {
  const payload = buildLargePayload();
  return () => payload.id; // closure retains payload
}

retainedClosures.push(createHandler());
globalCache.set(requestId, buildLargePayload());

// Fix the ownership lifetime:
retainedClosures.length = 0;
globalCache.clear();
05 · Runtime code

The complete code executed by the scenario

This is not an alternative example: these are the functions and files used by the Run button.

ACTUAL SOURCE

The source is generated from the real server function. Scenarios using a child process or Worker include every participating file.

src/memory-lab.js
supervisor346 lines
import { fork } from 'node:child_process';
import { unlink } from 'node:fs/promises';
import path from 'node:path';
import { labProfile } from './lab-profile.js';

const childPath = path.resolve(
  process.env.NODE_LOOP_SOURCE_DIR ||
    path.join(/* turbopackIgnore: true */ process.cwd(), 'src'),
  'memory-leak-child.js',
);

const MB = 1024 * 1024;
const childEnvironmentKeys = [
  'NODE_ENV',
  'TZ',
  'LANG',
  'LC_ALL',
  'TEMP',
  'TMP',
  'TMPDIR',
  'SystemRoot',
  'WINDIR',
];

function isolatedChildEnvironment() {
  return {
    NODE_LOOP_LAB_MEMORY_CHILD: '1',
    ...Object.fromEntries(
      childEnvironmentKeys
        .filter((key) => process.env[key] !== undefined)
        .map((key) => [key, process.env[key]]),
    ),
  };
}

function safeConfig(input = {}, profile = labProfile) {
  const memory = profile.memory;
  const defaultConfig = memory.defaultConfig;
  return {
    kind: memory.kinds.includes(input.kind) ? input.kind : defaultConfig.kind,
    allocationMb: memory.allocationMb.includes(Number(input.allocationMb))
      ? Number(input.allocationMb)
      : defaultConfig.allocationMb,
    intervalMs: memory.intervalMs.includes(Number(input.intervalMs))
      ? Number(input.intervalMs)
      : defaultConfig.intervalMs,
    limitMb: memory.limitMb.includes(Number(input.limitMb))
      ? Number(input.limitMb)
      : defaultConfig.limitMb,
  };
}

class MemoryLab {
  constructor(profile = labProfile) {
    this.profile = profile;
    this.child = null;
    this.clients = new Set();
    this.stopTimer = null;
    this.snapshotPath = null;
    this.state = {
      status: 'idle',
      pid: null,
      config: null,
      latest: null,
      snapshot: { status: 'idle' },
      lastLog: 'Эксперимент ещё не запускался',
    };
  }

  cleanupSnapshot() {
    const previousPath = this.snapshotPath;
    this.snapshotPath = null;
    if (previousPath) void unlink(previousPath).catch(() => {});
  }

  snapshot() {
    return structuredClone(this.state);
  }

  broadcast(event, data) {
    const frame = `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`;

    for (const client of this.clients) {
      try {
        client.controller.enqueue(client.encoder.encode(frame));
      } catch {
        client.close();
      }
    }
  }

  createEventStream(signal) {
    if (this.clients.size >= this.profile.api.maxSseClients) {
      const error = new Error(
        'Достигнут лимит подключений к потоку memory-lab',
      );
      error.statusCode = 503;
      throw error;
    }

    const lab = this;
    let client;
    return new ReadableStream({
      start(controller) {
        const encoder = new TextEncoder();
        let closed = false;
        const close = () => {
          if (closed) return;
          closed = true;
          clearInterval(client.heartbeat);
          lab.clients.delete(client);
          try {
            controller.close();
          } catch {
            // Поток уже закрыт браузером.
          }
        };

        client = { controller, encoder, close, heartbeat: null };
        lab.clients.add(client);
        controller.enqueue(
          encoder.encode(
            `event: state\ndata: ${JSON.stringify(lab.snapshot())}\n\n`,
          ),
        );
        client.heartbeat = setInterval(() => {
          try {
            controller.enqueue(encoder.encode(': keep-alive\n\n'));
          } catch {
            close();
          }
        }, 15_000);
        client.heartbeat.unref?.();
        signal?.addEventListener('abort', close, { once: true });
      },
      cancel() {
        client?.close();
      },
    });
  }

  start(input) {
    if (this.child) {
      const error = new Error('Эксперимент уже запущен');
      error.statusCode = 409;
      throw error;
    }

    this.cleanupSnapshot();
    const config = safeConfig(input, this.profile);
    const safety = {
      retainedLimitMb: this.profile.memory.retainedLimitMb,
      hardRssLimitMb: this.profile.memory.hardRssLimitMb,
      maxDurationMs: this.profile.memory.maxDurationMs,
      deadlineAction: this.profile.memory.deadlineAction,
    };
    const child = fork(/* turbopackIgnore: true */ childPath, [], {
      execArgv: [
        '--expose-gc',
        `--max-old-space-size=${this.profile.memory.v8HeapLimitMb}`,
      ],
      stdio: ['ignore', 'pipe', 'pipe', 'ipc'],
      // Heap snapshots can contain strings reachable in the target isolate.
      // Do not let the synthetic lab child inherit application secrets.
      env: isolatedChildEnvironment(),
      windowsHide: true,
    });

    this.child = child;
    this.state = {
      status: 'starting',
      pid: child.pid,
      config,
      latest: null,
      snapshot: { status: 'idle' },
      lastLog: 'Запускаем изолированный процесс…',
    };
    this.broadcast('state', this.snapshot());

    child.on('message', (message) => {
      if (message.type === 'sample') {
        this.state.status = message.status;
        this.state.latest = {
          elapsedMs: message.elapsedMs,
          retainedBytes: message.retainedBytes,
          blocks: message.blocks,
          memory: message.memory,
          reason: message.reason,
        };
        this.broadcast('sample', this.snapshot());

        // Дочерний процесс также проверяет этот предел сам. Дублирование в
        // supervisor защищает лабораторию при ошибке учебного сценария.
        if (
          message.memory.rss >
          this.profile.memory.hardRssLimitMb * MB
        ) {
          this.state.lastLog =
            'Supervisor остановил процесс: превышен аварийный предел RSS';
          this.broadcast('log', {
            level: 'error',
            message: this.state.lastLog,
          });
          child.kill();
        }
      } else if (message.type === 'log') {
        this.state.lastLog = message.message;
        this.broadcast('log', message);
      } else if (message.type === 'snapshot') {
        if (message.status === 'ready') {
          this.cleanupSnapshot();
          this.snapshotPath = message.path;
          this.state.snapshot = {
            status: 'ready',
            fileName: message.fileName,
            size: message.size,
            createdAt: message.createdAt,
          };
        } else if (message.status === 'error') {
          this.state.snapshot = {
            status: 'error',
            error: message.error,
          };
        } else {
          this.state.snapshot = { status: 'creating' };
        }
        this.broadcast('state', this.snapshot());
      }
    });

    child.stderr.on('data', (chunk) => {
      const message = chunk.toString().trim();
      if (!message) return;
      this.state.lastLog = message;
      this.broadcast('log', { level: 'error', message });
    });

    child.on('error', (error) => {
      this.state.lastLog = error.message;
      this.broadcast('log', { level: 'error', message: error.message });
    });

    child.on('exit', (code, signal) => {
      clearTimeout(this.stopTimer);
      this.stopTimer = null;
      this.child = null;
      this.state.status = 'stopped';
      this.state.pid = null;
      this.state.lastLog =
        code === 0
          ? 'Изолированный процесс остановлен'
          : `Процесс завершился: code=${code ?? '—'}, signal=${signal ?? '—'}`;
      this.broadcast('state', this.snapshot());
      this.broadcast('log', {
        level: code === 0 ? 'info' : 'error',
        message: this.state.lastLog,
      });
    });

    child.send({ type: 'start', config: { ...config, safety } });
    return this.snapshot();
  }

  action(action) {
    const allowed = new Set([
      'pause',
      'resume',
      'release',
      'gc',
      'snapshot',
      'stop',
    ]);
    if (!allowed.has(action)) {
      const error = new Error('Неизвестное действие');
      error.statusCode = 400;
      throw error;
    }

    if (!this.child?.connected) {
      const error = new Error('Сначала запустите эксперимент');
      error.statusCode = 409;
      throw error;
    }

    if (action === 'snapshot') {
      if (this.state.snapshot?.status === 'creating') {
        const error = new Error('Heap snapshot уже создаётся');
        error.statusCode = 409;
        throw error;
      }
      const retainedMb = (this.state.latest?.retainedBytes ?? 0) / MB;
      const snapshotLimit = this.profile.memory.snapshotMaxRetainedMb;
      if (retainedMb > snapshotLimit) {
        const error = new Error(
          `Сначала уменьшите retained до ${snapshotLimit} MB или ниже: heap snapshot может временно удвоить потребление V8 heap`,
        );
        error.statusCode = 413;
        throw error;
      }
      this.state.snapshot = { status: 'creating' };
      this.broadcast('state', this.snapshot());
    }

    this.child.send({ type: 'action', action });

    if (action === 'stop') {
      clearTimeout(this.stopTimer);
      this.stopTimer = setTimeout(() => {
        if (this.child) this.child.kill();
      }, 1000);
      this.stopTimer.unref();
    }

    return this.snapshot();
  }

  snapshotDownload() {
    if (
      !this.snapshotPath ||
      this.state.snapshot?.status !== 'ready'
    ) {
      const error = new Error('Сначала создайте heap snapshot');
      error.statusCode = 404;
      throw error;
    }

    return {
      path: this.snapshotPath,
      ...this.state.snapshot,
    };
  }

  stopForShutdown() {
    if (this.child) {
      this.child.kill();
      this.child = null;
    }
    this.cleanupSnapshot();
  }
}

const memoryLabKey = Symbol.for('node-loop-lab.memory');
export const memoryLab =
  globalThis[memoryLabKey] ?? (globalThis[memoryLabKey] = new MemoryLab());
export { MemoryLab, safeConfig };

The application instruments its own live trace: rows and timestamps are recorded by real emit(...) calls, while the scenario supplies source and lane labels. This is not a V8/libuv profiler or a direct view of their internal queues. await and Promise keep the HTTP stream open until the scenario completes.

06 · PRODUCTION CASES

How a learning mistake becomes an incident

A realistic service: the original code, observable failure, corrected implementation, and why the correction works.

CASE 01

A retry timer retains multi-megabyte request payloads

When an external API fails, the service schedules a retry with setTimeout. The retry closure captures the parsed webhook payload and headers for several minutes.

INCIDENT CONTEXT

Each pending timer is a GC root path to the captured data. During a long upstream outage, thousands of retries retain far more memory than their small callback functions suggest.

BEFOREPROBLEMATIC IMPLEMENTATION
function scheduleRetry(webhook) {
  const parsedPayload = parseAndEnrich(webhook.body);

  setTimeout(async () => {
    await deliver({
      headers: webhook.headers,
      payload: parsedPayload,
    });
  }, retryDelay(webhook.attempt));
}

The Timer is a retaining path to the callback and its lexical environment, including the enriched payload. The retry has no durable owner.

AFTERCORRECTED IMPLEMENTATION
async function scheduleRetry(webhook) {
  const retry = await retryStore.insert({
    webhookId: webhook.id,
    attempt: webhook.attempt + 1,
    runAt: nextRetryAt(webhook.attempt),
  });

  await retryQueue.add(
    'deliver-webhook',
    { retryId: retry.id },
    {
      delay: retry.delayMs,
      jobId: 'retry-' + retry.id,
    },
  );
}

Durable storage owns the long-lived retry state, while process memory and the queue job retain only a small identifier. A restart does not discard pending work.

FUNCTIONS AND CONSTRUCTS

What the unfamiliar calls from both code samples actually do.

setTimeout(callback, delay)
Creates a Timer that retains its callback and every captured reference until it fires or is cancelled.
closure
A function together with access to outer variables. Here the callback continues to retain the webhook payload.
retryQueue.add(...)
Stores a retry as a durable job that survives process restarts and does not depend on a live Timer.
parseAndEnrich(...)
A representative parser and enrichment function whose large result should not be captured by a Timer.
retryStore.insert(...)
Writes retry state to durable storage and returns a small record containing its id and calculated delay.
delay / jobId
delay postpones the job, while a stable jobId helps prevent the same retry from being enqueued twice.
WHY THE CORRECTION WORKS

Use heap-snapshot comparison to find growing constructor counts and inspect their retaining paths. RSS alone tells you that memory grew, not which reference owns it.

WHAT PRODUCTION SHOWED
  • Heap grows in proportion to the number of scheduled retries.
  • Snapshots show Timeout objects retaining IncomingMessage instances.
  • Memory remains high after the upstream recovers until timers fire.
07 · DO NOT CONFUSE

Common misconceptions

The myth is on the left; the accurate model is on the right.

MYTH

Every closure is a memory leak.

ACTUALLY

It is a problem only when an unwanted payload stays reachable beyond its intended lifetime.

MYTH

The object with the largest shallow size is always the culprit.

ACTUALLY

A small Map or closure can dominate a huge subgraph and therefore have a large retained size.

MYTH

One snapshot proves a leak.

ACTUALLY

It shows state. Diagnosis usually compares snapshots under repeatable load and examines retaining paths.

MYTH

global.gc() fixes a production leak.

ACTUALLY

GC cannot remove reachable objects. A manual call is useful for a lab, not as an ownership fix.

MYTH

An unbounded cache is not a leak because the data is useful.

ACTUALLY

Without a defined lifetime and upper bound, it can exhaust memory just like an accidentally retained array.

08 · SELF-CHECK

Explain it in your own words

If you can explain the answer without quoting documentation, your mental model is starting to take shape.

  1. Which exact edge connects the payload to a GC root in Closure mode?
  2. Why can a snapshot of the main server not diagnose the child?
  3. How do TTL, LRU, and a hard max size protect a cache differently?
  4. Why can heapUsed fall while RSS remains above its original value?