Which one to choose? Worker Threads, Child Processes, and Cluster module

In Node.js, there are three common ways to handle parallelism and improve performance for CPU-intensive tasks or high-concurrency scenarios: Worker Threads, Child Processes, and the Cluster module. Each solves different problems and has specific use cases. Here’s a comparison of these tools with their respective use cases and code samples.

Quick Summary:

1. Worker Threads

Problem Solved:

• Node.js runs JavaScript code in a single thread. While it’s event-driven and handles I/O efficiently using the event loop, CPU-bound tasks (e.g., heavy computations) can block the main thread. Worker Threads allow running JavaScript code in parallel, taking advantage of multi-core processors to handle CPU-intensive tasks without blocking the main thread.

Use Case:

• CPU-intensive tasks like image processing, large mathematical computations, or encryption tasks where concurrency is needed, but you want to share memory between threads.

Code Example:

// main.js
const { Worker } = require('worker_threads');

function runWorker() {
  return new Promise((resolve, reject) => {
    const worker = new Worker('./worker-task.js');
    
    worker.on('message', resolve);
    worker.on('error', reject);
    worker.on('exit', (code) => {
      if (code !== 0) {
        reject(new Error(`Worker stopped with exit code ${code}`));
      }
    });
  });
}

runWorker().then((result) => {
  console.log(`Result from worker: ${result}`);
}).catch((err) => console.error(err));

// worker-task.js
const { parentPort } = require('worker_threads');

// Example heavy computation (factorial)
function factorial(n) {
  return n === 1 ? 1 : n * factorial(n - 1);
}

parentPort.postMessage(factorial(10));

Pros:

• Efficient for CPU-bound tasks.
• Can share memory between threads using SharedArrayBuffer.
• Threads run within the same process, reducing memory overhead.

Cons:

• Communication between threads requires serialization of messages.
• Not suitable for high-concurrency I/O-bound tasks.

2. Child Processes

Problem Solved:

• Sometimes you need to run a completely separate process for heavy tasks, especially when the task is written in a different language (e.g., Python). Child Processes let you run external programs or scripts without affecting the main Node.js process.

Use Case:

• Running external scripts (e.g., calling Python or Bash), performing CPU-intensive operations where process isolation is needed, or when using non-JS code (e.g., invoking FFMPEG, Python scripts).

Code Example:

// main.js
const { fork } = require('child_process');

// Fork a child process
const child = fork('./child-task.js');

// Send a message to child process
child.send({ num: 10 });

child.on('message', (result) => {
  console.log(`Result from child process: ${result}`);
  child.kill(); // End child process
});

// child-task.js
process.on('message', (message) => {
  const num = message.num;

  // Example heavy computation (factorial)
  function factorial(n) {
    return n === 1 ? 1 : n * factorial(n - 1);
  }

  const result = factorial(num);
  process.send(result);
});

Pros:

• Fully isolated processes, so memory leaks or crashes in the child process don’t affect the parent.
• Can spawn multiple processes to handle CPU-bound tasks or other languages.

Cons:

• Higher memory usage as each child process has its own memory space.
• Inter-process communication (IPC) can be slower compared to worker threads.

3. Cluster

Problem Solved:

• Node.js is single-threaded, meaning a single instance can only use one CPU core. Cluster module allows Node.js to take advantage of multi-core systems by spawning multiple instances (workers) of the same Node.js application. This improves concurrency for I/O-bound workloads, especially for web servers.

Use Case:

• Building scalable web servers that can handle many concurrent connections by distributing the load across multiple CPU cores.

Code Example:

const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
  // Fork workers for each CPU
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker ${worker.process.pid} died`);
  });
} else {
  // Workers can share any TCP connection, in this case, an HTTP server
  http.createServer((req, res) => {
    res.writeHead(200);
    res.end(`Worker ${process.pid} says Hello!\n`);
  }).listen(8000);

  console.log(`Worker ${process.pid} started`);
}

Pros:

• Automatically distributes incoming connections across multiple worker processes.
• Great for building scalable web servers using all available CPU cores.

Cons:

• Does not share memory between workers (they are separate processes).
• May require load balancing for stateful applications (e.g., using Redis for session management).

Each of these modules offers a solution depending on the problem. For CPU-intensive tasks, Worker Threads and Child Processes are excellent, while for scalable web servers, Cluster is a strong choice.