JavaScript: Advanced Meta-Programming, Memory Management and Performance

Iterators, Generators and Lazy Evaluation

What is an iterator?

In JavaScript, an iterator is an object that defines a sequence, and potentially a return value upon its completion.
An iterator is any object that implements the iterator protocol by having a next() method.
The iterator protocol standardises how a sequence of values is produced.
An object is considered iterable if it implements the Symbol.iterator method.

Iterables vs iterators

Iterables and iterators can be easy to mix up. Here is an easy way to think about the difference:

Iterable: An object that has a sequence of data and wants to share it (like a book with pages, for instance).
Iterator: An object that does the work of getting the next item from that sequence (like a bookmark keeping track of where you are).

1. Iterables (the data source):

An iterable is any JavaScript object that implements the iterable protocol.
To be an iterable, an object must have a method with the special key Symbol.iterator.
When called, this method returns a new iterator.
Arrays, Strings, Maps, Sets and NodeLists are all built-in iterables.
Plain JavaScript objects ({}) are not iterable by default.

Example:

const numbers = [10, 20, 30]; // Array is an iterable

// You can access its Symbol.iterator method directly:
const iteratorMethod = numbers[Symbol.iterator];
console.log(typeof iteratorMethod); // "function"

2. Iterators (the traverser):

An iterator is an object that actually steps through the iterable.
An iterator must have a next() method.

The `next()` method returns an object with two properties:

value: The next value in the sequence.
done: A boolean that is true if the iterator has reached the end of the sequence or false if it has not.

When you call the next() method on an iterator, it steps through a collection one by one.

Example 1:

const numbers = [10, 20];

// 1. Get the iterator from the iterable
const iterator = numbers[Symbol.iterator]();

// 2. Step through manually using .next()
console.log(iterator.next()); // { value: 10, done: false }
console.log(iterator.next()); // { value: 20, done: false }
console.log(iterator.next()); // { value: undefined, done: true }

Example 2:

function makeIterator(array) {
  let nextIndex = 0;
  return {
    next: function() {
      if (nextIndex < array.length) {
        return { value: array[nextIndex++], done: false };
      } else {
        return { value: undefined, done: true };
      }
    }
  };
}

const iterator = makeIterator(['Apple', 'Banana', 'Cherry']);

console.log(iterator.next()); // { value: 'Apple', done: false }
console.log(iterator.next()); // { value: 'Banana', done: false }
console.log(iterator.next()); // { value: 'Cherry', done: false }
console.log(iterator.next()); // { value: undefined, done: true }

for...of loop

When you write a for...of loop, JavaScript automatically coordinates the iterable and iterator behind the scenes, so next() doesn't have to be called manually.

Example:

const colors = ["red", "blue"];

for (const color of colors) {
  console.log(color);
}

// What JavaScript is actually doing under the hood:
const iterator = colors[Symbol.iterator]();
let result = iterator.next();

while (!result.done) {
  const color = result.value;
  console.log(color);
  result = iterator.next();
}

Generator functions (`function*`) and bidirectional communication

Generator functions (declared with function*) simplify writing iterators.
Instead of manually creating a function with a next() method, you use the yield keyword to pause execution and emit a value.

Example:

function* numberGenerator() {
  yield 1;
  yield 2;
  yield 3;
}

const gen = numberGenerator();
console.log(gen.next()); // { value: 1, done: false }
console.log(gen.next()); // { value: 2, done: false }
console.log(gen.next()); // { value: 3, done: false }
console.log(gen.next()); // { value: undefined, done: true }

Generators don't just output data, they can receive data back into the yield expression through the arguments passed into next(value).

Example:

function* conversationalGenerator() {
  console.log("Generator started");
  
  const name = yield "What is your name?";
  console.log(`Received name: ${name}`);
  
  const age = yield `Hello ${name}, how old are you?`;
  console.log(`Received age: ${age}`);
  
  return `${name} is ${age} years old.`;
}

const convo = conversationalGenerator();

// 1. Start the generator until the first yield
console.log(convo.next()); 
// Output: "Generator started"
// Output: { value: "What is your name?", done: false }

// 2. Send data back into the first yield, run until the next yield
console.log(convo.next("Alice")); 
// Output: "Received name: Alice"
// Output: { value: "Hello Alice, how old are you?", done: false }

// 3. Send data back into the second yield, finish execution
console.log(convo.next(25)); 
// Output: "Received age: 25"
// Output: { value: "Alice is 25 years old.", done: true }

Lazy evaluation

Lazy evaluation means computing values on-demand rather than ahead of time. This enables powerful design patterns like infinite streams and memory-efficient data processing.

Infinite data streams

Because generators pause execution, you can define infinite loops without the program crashing. Values are only generated when requested.

Example:

function* infiniteSequence() {
  let i = 1;
  while (true) {
    yield i++;
  }
}

const infiniteNumbers = infiniteSequence();

// We can safely grab just what we need
console.log(infiniteNumbers.next().value); // 1
console.log(infiniteNumbers.next().value); // 2
console.log(infiniteNumbers.next().value); // 3
// ...and so on, indefinitely.

Custom data structures (e.g., tree traversal)

Generators make complex traversals clean because you can yield values recursively using yield*.

Example:

const tree = {
  value: 1,
  left: { value: 2, left: null, right: null },
  right: { value: 3, left: null, right: null }
};

function* traverseTree(node) {
  if (!node) return;
  yield node.value;
  yield* traverseTree(node.left);
  yield* traverseTree(node.right);
}

for (const val of traverseTree(tree)) {
  console.log(val); // Output: 1, 2, 3
}

Asynchronous generators

You can combine generators with async/await to stream data asynchronously (e.g., fetching pages of an API lazily).

async function* fetchPages() {
  let page = 1;
  let hasMore = true;

  while (hasMore) {
    // Simulating an asynchronous fetch request
    const data = await fakeApiFetch(page); 
    yield data.items;
    
    hasMore = data.hasMore;
    page++;
  }
}

// Consuming an async generator with for await...of
async function processPages() {
  for await (const items of fetchPages()) {
    console.log("Processed page items:", items);
  }
}

Meta-programming with proxies and the Reflect API

The `Proxy` object

A Proxy allows you to wrap an object and intercept or redefine fundamental operations (like getting a property, setting a property, validating data, or calling a function).

Example:

const target = { name: "Alice", age: 30 };

const handler = {
  // Intercept property access (e.g., target.age)
  get(obj, prop) {
    console.log(`Getting property "${prop}"`);
    return prop in obj ? obj[prop] : "Property does not exist";
  },
  
  // Intercept property assignment (e.g., target.age = 31)
  set(obj, prop, value) {
    if (prop === 'age' && typeof value !== 'number') {
      throw new TypeError("Age must be a number");
    }
    console.log(`Setting "${prop}" to ${value}`);
    obj[prop] = value;
    return true; // indicates success
  }
};

const proxyUser = new Proxy(target, handler);

console.log(proxyUser.name); // Logs interception, prints "Alice"
proxyUser.age = 35;          // Logs: Setting "age" to 35

The Reflect API is a built-in object that provides methods for interceptable JavaScript operations. It mirrors the traps of Proxy. Whenever you intercept an operation with a Proxy, it is common practise to use Reflect to perform the default behaviour.

Example:

const target = { x: 10, y: 20 };

const proxy = new Proxy(target, {
  get(target, prop, receiver) {
    console.log(`Tracking read for: ${prop}`);
    // Reflect handles the default lookup safely
    return Reflect.get(target, prop, receiver);
  }
});

console.log(proxy.x); // Tracks read, prints 10

Real-world applications

1. Observable state management (reactivity)

Modern front-end frameworks (like Vue 3) use Proxy under the hood to make application state "reactive". When a piece of data changes, the framework intercepts the write operation and automatically triggers DOM updates or re-renders components.

Example:

// A simple reactive state manager
function reactive(target, onChange) {
  return new Proxy(target, {
    set(obj, prop, value) {
      const oldValue = obj[prop];
      obj[prop] = value;
      
      // Only trigger if the value actually changed
      if (oldValue !== value) {
        onChange(prop, value, oldValue);
      }
      return true;
    }
  });
}

// Our application state
const state = reactive({ theme: 'light', count: 0 }, (prop, newValue, oldValue) => {
  console.log(`[UI Update] State property '${prop}' changed from '${oldValue}' to '${newValue}'`);
});

state.count = 1; // Triggers UI update log automatically!
state.theme = 'dark';

2. Strict data validation and sanitisation

Proxies allow you to enforce schemas, data types, or boundaries dynamically without needing to write custom getters and setters for every single property on a class or object.

Example:

const userValidatorHandler = {
  set(target, prop, value) {
    if (prop === 'age') {
      if (typeof value !== 'number' || value < 0 || value > 120) {
        throw new RangeError("Age must be a valid number between 0 and 120.");
      }
    }
    if (prop === 'email') {
      if (!value.includes('@')) {
        throw new TypeError("Invalid email format.");
      }
    }
    
    target[prop] = value;
    return true;
  }
};

const user = new Proxy({}, userValidatorHandler);

user.email = "test@example.com"; // Works fine
// user.age = 150;               // Throws RangeError: Age must be a valid number...

3. Auto-populating objects

Sometimes you want to automatically initialise missing keys when you try to read or write to them, preventing undefined errors. This is especially useful for nested structures, graphs, or caching layers.

Example:

function autoVivification(target = {}) {
  return new Proxy(target, {
    get(obj, prop) {
      // If it's a built-in symbol (like iterator), let it pass through normally
      if (typeof prop === 'symbol' || prop in obj) {
        return obj[prop];
      }
      
      // Automatically create a nested auto-populating object for missing properties
      obj[prop] = autoVivification();
      return obj[prop];
    }
  });
}

const nestedTree = autoVivification();

// Deeply nested assignment works seamlessly without checking if each level exists first
nestedTree.europe.france.capital = "Paris";
console.log(nestedTree.europe.france.capital); // "Paris"
console.log(nestedTree.asia.japan.capital);    // Returns a new proxy object instead of undefined!

Memory Management and Garbage Collection

Modern JavaScript engines (such as Google's V8) manage memory by dividing it into two primary areas:

The stack.
The heap.

The stack

A structured, fast memory region operating on "Last-In-First-Out".
It is responsible for static memory allocation, including execution contexts, pointer references and primitive values.
Allocating and clearing the stack is extremely fast because the size of primitives is known and fixed at compile time.

The heap

An unstructured, large memory pool used for dynamic memory allocation where data sizes can change or are unknown ahead of time (e.g., objects, arrays and functions).
Allocating memory takes longer because the engine as to search for a block of memory large enough to fit the data.

Primitive vs reference value storage

Primitive values (string, number, boolean, null, undefined, symbol, bigint): These are stored directly on the stack. When you assign a primitive to a new variable, a completely independent copy of that value is created.
Reference values (object, array, function): The actual data structure is stored on the heap, whereas a reference pointer to that heap address is stored on the stack. When an object is assigned to a new variable you are only copying the pointer. Both variables point to the exact same object in the heap. Modifying one will affect the other.

Garbage collection algorithms

The engine runs automated processes to reclaim memory from objects that are no longer needed.

Mark-and-sweep algorithm

This is the foundational algorithm used by V8.
The algorithm goes through two phases:

A mark phase.
A sweep phase.

The algorithm periodically starts from roots (global objects, currently active execution stacks/closures).
It traverses down the reference graph, marking every object it can reach.
During the sweep phase, any memory blocks that remain unmarked are deemed unreachable (garbage) and are reclaimed by the engine.

Generational collection

V8 optimises garbage collection based on the "weak generational hypothesis" - the observation that most objects die young.

The heap is split into:

The young generation (split further into "nursery" and "intermediate" generations): This is where new objects are born. It is collected frequently using a fast Scavenge algorithm (minor GC) that moves surviving objects between two semi-spaces.
The old generation: Objects that survive multiple young generation sweeps are placed here. It is collected less frequently using a Mark-Sweep-Compact algorithm (major GC).

Optimisation strategies

Engines use hidden classes (shapes) and inline caching to speed up property lookups on dynamic objects, reducing the overhead required during execution and memory traversal.

Memory leaks

Single Page Applications (SPAs) run for long periods of time in a browser session. This can make them prone to memory leaks if references to unused objects are unintentionally retained.

Common causes of memory leaks

Accidental globals: Forgetting to declare a variable with const, let, or var implicitly attaches it to the global window object - meaning it will never be garbage collected.
Forgotten event listeners: Attaching event listeners to global elements (like window, document or large parent containers) and failing to remove them (removeEventListener) when components unmount keeps the target elements and their outer scopes trapped in memory.
Detached DOM nodes: Storing a reference to a DOM element in a JavaScript array or object, and later removing that element from the document tree.
Closure retention: Inner functions that retain references to outer scope variables long after the outer function has returned can accidentally trap heavy data structures if not scoped properly.

Using Chrome DevTools memory profiler to address memory leaks

To address memory leaks you can:

Open Chrome DevTools and navigate to the Memory tab.
Take a Heap Snapshot before performing an action (e.g., opening and closing a modal).
Perform an action and take another snapshot.
Compare snapshots using the Comparison view or filter by Detached Elements to look for DOM nodes that should have been garbage collected.
Inspect the Retainer Tree to trace why an object is alive (which variable or listener is holding on to the reference), then you can clean up the reference in your code.

Analogy	The entire playlist or book	The play/bookmark pointer tracking your current spot
Key Property	`[Symbol.iterator]`	`next()`
Returns	An Iterator	`{ value, done }`
Examples	`Array`, `String`, `Set`, `Map`	The object returned by calling `array[Symbol.iterator]()`

JavaScript: Advanced Meta-Programming, Memory Management and Performance

JavaScript: Objects, Prototypes and Object-Oriented Programming

JavaScript: Mastering the Event Loop, Callbacks and Asynchronous Code

JavaScript: Demystifying Execution Context, the Call Stack and the 'this' Keyword

JavaScript: Understanding Variables and Scope

What Is the Document Object Model in Web Applications?

Iterators, Generators and Lazy Evaluation

What is an iterator?

Iterables vs iterators

1. Iterables (the data source):

2. Iterators (the traverser):

The `next()` method returns an object with two properties:

Comparison table

for...of loop

Generator functions (`function*`) and bidirectional communication

Lazy evaluation

Infinite data streams

Custom data structures (e.g., tree traversal)

Asynchronous generators

Meta-programming with proxies and the Reflect API

The `Proxy` object

The Reflect API

Real-world applications

1. Observable state management (reactivity)

2. Strict data validation and sanitisation

3. Auto-populating objects

Memory Management and Garbage Collection

The stack

The heap

Primitive vs reference value storage

Garbage collection algorithms

Mark-and-sweep algorithm

Generational collection

Optimisation strategies

Memory leaks

Common causes of memory leaks

Using Chrome DevTools memory profiler to address memory leaks

Stay in the loop

Comments (0)

JavaScript: Objects, Prototypes and Object-Oriented Programming

JavaScript: Mastering the Event Loop, Callbacks and Asynchronous Code

JavaScript: Demystifying Execution Context, the Call Stack and the 'this' Keyword

JavaScript: Understanding Variables and Scope

What Is the Document Object Model in Web Applications?

The next() method returns an object with two properties:

Generator functions (function*) and bidirectional communication

The Proxy object

Stay in the loop

Comments (0)

The `next()` method returns an object with two properties:

Generator functions (`function*`) and bidirectional communication

The `Proxy` object