Node.js Child Process API: Executing Subprocesses

1. Creating Child Processes with spawn()

spawn() creates a child process asynchronously, establishing stdin, stdout, and stderr streams between parent and child.

const { spawn } = require('node:child_process');
const ls = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on('data', (data) => {
  console.error(`stderr: ${data}`);
});

ls.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});

2. Input/Output Management

• If output is not consumed, the child process may block.
• Use { stdio: 'ignore' } to discard output safely.

3. Alternatives: exec and execFile

• exec(): Runs a command inside a shell and returns buffered output via callback.
• execFile(): Runs a command directly without spawning a shell.
• fork(): Spawns a new Node.js process with an IPC channel for communication.
• Synchronous versions (execSync(), execFileSync()) block the event loop until completion.

4. Running .bat and .cmd Files on Windows

• On Windows, .bat and .cmd files require a shell.
• They can be run using spawn() with { shell: true }, or with exec().

const { exec } = require('node:child_process');
exec('my.bat', (err, stdout, stderr) => {
  if (err) console.error(err);
  console.log(stdout);
});

5. Using exec with Callback

const { exec } = require('node:child_process');
exec('cat *.js missing_file | wc -l', (error, stdout, stderr) => {
  if (error) {
    console.error(`exec error: ${error}`);
    return;
  }
  console.log(`stdout: ${stdout}`);
  console.error(`stderr: ${stderr}`);
});

6. Promise-based exec with util.promisify

const util = require('node:util');
const exec = util.promisify(require('node:child_process').exec);

async function lsExample() {
  const { stdout, stderr } = await exec('ls');
  console.log('stdout:', stdout);
  console.error('stderr:', stderr);
}
lsExample();

7. AbortController Integration

The signal option allows aborting a child process using AbortController.

const { exec } = require('node:child_process');
const controller = new AbortController();
const { signal } = controller;

const child = exec('grep ssh', { signal }, (error) => {
  console.error(error); // AbortError
});
controller.abort();

Conclusion

The child_process module in Node.js is a powerful tool for executing system commands and managing subprocesses. Asynchronous methods like spawn() and exec() provide efficient performance, while synchronous methods are useful for simpler scripts. Features like AbortController and IPC make process management flexible and safe.

1. execFile()

• Executes an executable file directly without a shell.
• More efficient than exec since no shell is created.
• Supports the same options as exec, but features like I/O redirection and globbing are not available.

const { execFile } = require('node:child_process');
execFile('node', ['--version'], (error, stdout, stderr) => {
  if (error) throw error;
  console.log(stdout);
});

A Promise-based version is available using util.promisify.

2. fork()

• A special case of spawn for creating new Node.js processes.
• Provides an IPC channel for message passing between parent and child.
• Each child Node.js process is independent, with its own memory and V8 instance.

const { fork } = require('node:child_process');
const child = fork('child.js');
child.on('message', (msg) => {
  console.log('Message from child:', msg);
});
child.send({ hello: 'world' });

3. spawn()

• Spawns a new process with a command and arguments.
• Options include cwd, env, stdio, and more.
• If the shell option is enabled, unsanitized user input must be avoided to prevent arbitrary command execution.

const { spawn } = require('node:child_process');
const ls = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on('data', (data) => {
  console.error(`stderr: ${data}`);
});

ls.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});

4. Error Handling and AbortController

• All three functions support the signal option.
• Using AbortController, processes can be safely terminated.

const controller = new AbortController();
const { signal } = controller;
const child = spawn('grep', ['ssh'], { signal });
controller.abort(); // Stops the process

Conclusion

The execFile, fork, and spawn functions are powerful tools for managing child processes in Node.js. By using them correctly, developers can build scalable and flexible applications that leverage system capabilities and inter-process communication.

1. Behavior on Windows

• Setting detached: true allows the child process to continue running after the parent exits.
• The child process will have its own console window.
• Once enabled, this option cannot be disabled.

2. Behavior on Unix-like Systems

• The child process becomes the leader of a new process group and session (setsid(2)).
• Child processes may continue running after the parent exits, regardless of whether they are detached.

3. Parent Waiting Behavior

• By default, the parent waits for the detached child process to exit.
• To prevent this, use subprocess.unref().
• This removes the child process from the parent’s event loop reference count, allowing the parent to exit independently.

4. stdio Configuration

• For long-running background processes, stdio must not be connected to the parent.
• If the parent’s stdio is inherited, the child remains attached to the controlling terminal.

5. Examples

Detached process ignoring stdio:

const { spawn } = require('node:child_process');
const subprocess = spawn(process.argv[0], ['child_program.js'], {
  detached: true,
  stdio: 'ignore',
});
subprocess.unref();

Detached process with output redirected to files:

const { openSync } = require('node:fs');
const { spawn } = require('node:child_process');
const out = openSync('./out.log', 'a');
const err = openSync('./out.log', 'a');

const subprocess = spawn('prg', [], {
  detached: true,
  stdio: [ 'ignore', out, err ],
});
subprocess.unref();

Conclusion

The detached option in the child_process module is a powerful tool for running independent, long-lived processes. By combining unref() with proper stdio configuration, developers can create background processes that continue running even after the parent exits.

1. The stdio Option

The stdio option controls how streams are connected between parent and child:

• 'pipe': Creates pipes between parent and child (default).
• 'overlapped': Same as pipe, but with FILE_FLAG_OVERLAPPED (Windows only).
• 'ignore': Connects fds to /dev/null, ignoring them.
• 'inherit': Passes through parent’s streams (stdin, stdout, stderr).
• 'ipc': Creates an IPC channel for message passing.
• <Stream>: Shares a stream (file, socket, pipe) with the child.
• Positive integer: Shares an existing fd from the parent.
• null/undefined: Defaults to pipe for fd 0–2, ignore for higher fds.

const { spawn } = require('node:child_process');
const process = require('node:process');

// Child inherits parent's stdio
spawn('prg', [], { stdio: 'inherit' });

// Child shares only stderr
spawn('prg', [], { stdio: ['pipe', 'pipe', process.stderr] });

// Extra fd=4 for custom interaction
spawn('prg', [], { stdio: ['pipe', null, null, null, 'pipe'] });

2. IPC Communication

• When an IPC channel is created and the child is a Node.js instance, process.send() and process.disconnect() become available.
• The IPC channel is unreferenced until the child registers a handler for 'message' or 'disconnect', allowing normal exit.

3. Synchronous Process Creation

• spawnSync(), execSync(), and execFileSync() run child processes synchronously.
• They block the event loop until the process exits, useful for scripting or configuration loading.

4. execFileSync()

• Identical to execFile() but synchronous.
• Does not return until the child process has fully closed.
• Throws an error if the process times out or exits with a non-zero code.

const { execFileSync } = require('node:child_process');

try {
  const stdout = execFileSync('my-script.sh', ['my-arg'], {
    stdio: 'pipe',
    encoding: 'utf8',
  });
  console.log(stdout);
} catch (err) {
  const { stdout, stderr } = err;
  console.error({ stdout, stderr });
}

Conclusion

The stdio option provides fine-grained control over communication between parent and child processes, from simple pipes to IPC channels. The execFileSync() method enables synchronous execution, making it ideal for tasks that require blocking behavior, such as configuration loading or scripting.

1. execSync()

• Runs a command inside a shell synchronously.
• Does not return until the child process has fully closed.
• Throws an error if the process times out or exits with a non-zero code.
• Returns the process output as a Buffer or string.

const { execSync } = require('node:child_process');
try {
  const output = execSync('ls -lh /usr', { encoding: 'utf8' });
  console.log(output);
} catch (err) {
  console.error('Error:', err);
}

2. spawnSync()

• Similar to spawn() but synchronous.
• Returns an object containing pid, stdout, stderr, status, and signal.
• If the process fails or times out, an error object is returned.

const { spawnSync } = require('node:child_process');
const result = spawnSync('ls', ['-lh', '/usr'], { encoding: 'utf8' });

if (result.error) {
  console.error('Error:', result.error);
} else {
  console.log('stdout:', result.stdout);
  console.log('stderr:', result.stderr);
  console.log('status:', result.status);
}

3. Class: ChildProcess

• Represents a spawned child process.
• Extends EventEmitter.
• Created using spawn(), exec(), execFile(), or fork().

Key Events:

• 'exit': Emitted when the process ends.
• 'close': Emitted after stdio streams are closed.
• 'error': Emitted if the process fails to spawn or execute.
• 'disconnect': Emitted when IPC communication is terminated.

const { spawn } = require('node:child_process');
const ls = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ls.on('close', (code) => {
  console.log(`child process closed with code ${code}`);
});

ls.on('exit', (code) => {
  console.log(`child process exited with code ${code}`);
});

ls.on('error', (err) => {
  console.error('Failed to start subprocess:', err);
});

Conclusion

The execSync() and spawnSync() methods are powerful for synchronous child process execution, ideal for simple scripts or configuration loading at startup. The ChildProcess class provides event-driven management of child processes, enabling developers to build scalable and flexible applications with robust process control.

1. Event: 'message'

• Triggered when a child process sends a message using process.send().
• The message is serialized and parsed, so it may differ from the original.
• With serialization: 'advanced', data beyond JSON can be transmitted.

2. Event: 'spawn'

• Emitted once the child process has successfully spawned.
• If spawning fails, the 'error' event is emitted instead.
• Occurs before any data is received via stdout or stderr.

3. IPC Channel

• subprocess.channel: Reference to the IPC channel.
• channel.ref(): Keeps the parent’s event loop active.
• channel.unref(): Allows the parent to exit even if the channel is open.

4. Connection Management

• subprocess.connected: Indicates if IPC communication is still possible.
• subprocess.disconnect(): Closes the IPC channel, allowing graceful exit.

5. Process Control

• subprocess.exitCode: Shows the exit code of the child process (null if still running).
• subprocess.kill([signal]): Sends a signal to the child process (default: SIGTERM).
• subprocess.killed: Indicates whether a signal was successfully sent.
• subprocess.pid: Returns the process identifier (PID).

const { spawn } = require('node:child_process');
const grep = spawn('grep', ['ssh']);

console.log(`Spawned child pid: ${grep.pid}`);
grep.kill('SIGHUP');

6. Reference Management

• subprocess.unref(): Removes the child process from the parent’s event loop reference count.
• subprocess.ref(): Restores the reference, forcing the parent to wait for the child.

const { spawn } = require('node:child_process');
const subprocess = spawn(process.argv[0], ['child_program.js'], {
  detached: true,
  stdio: 'ignore',
});

subprocess.unref();
subprocess.ref();

Conclusion

The ChildProcess class in Node.js provides a comprehensive set of events and properties for managing child processes. With events like 'message' and 'spawn', and methods such as kill() and disconnect(), developers can fully control child processes and manage IPC communication safely and effectively.

1. subprocess.send()

• Sends messages from the parent to the child via IPC.
• Messages are serialized and parsed, possibly differing from the original.
• With serialization: 'advanced', data beyond JSON can be transmitted.
• Returns false if the channel is closed or the message backlog is too large.

// Parent script
const { fork } = require('node:child_process');
const forked = fork(__dirname + '/sub.js');

forked.on('message', (msg) => {
  console.log('PARENT got message:', msg);
});

forked.send({ hello: 'world' });

// Child script (sub.js)
process.on('message', (msg) => {
  console.log('CHILD got message:', msg);
});
process.send({ foo: 'bar', baz: NaN });

2. Sending Server or Socket Handles

• sendHandle allows passing a TCP server or socket to the child.
• The child can then manage connections directly.

// Parent
const { fork } = require('node:child_process');
const { createServer } = require('node:net');
const child = fork('subprocess.js');

const server = createServer();
server.listen(1337, () => {
  child.send('server', server);
});

// Child
process.on('message', (m, server) => {
  if (m === 'server') {
    server.on('connection', (socket) => {
      socket.end('handled by child');
    });
  }
});

3. Related Properties

• subprocess.signalCode: The signal received by the child process, or null.
• subprocess.spawnargs: The full list of command-line arguments used to launch the process.
• subprocess.spawnfile: The executable file name of the child process.

4. Important Notes

• Messages with a NODE_ prefix in cmd are reserved for Node.js internals.
• Sending IPC sockets is not supported on Windows.
• An optional callback can be used for flow control after sending a message.

Conclusion

The subprocess.send() method and related properties of the ChildProcess class provide powerful tools for parent-child communication in Node.js. By sending messages, servers, or sockets, developers can build distributed and scalable applications with robust IPC capabilities.

1. subprocess.stderr

• A Readable Stream representing the child process’s error output.
• If stdio[2] is set to anything other than 'pipe', this will be null.
• Alias for subprocess.stdio[2].

2. subprocess.stdin

• A Writable Stream representing the child process’s input.
• If stdio[0] is set to anything other than 'pipe', this will be null.
• Alias for subprocess.stdio[0].

3. subprocess.stdout

• A Readable Stream representing the child process’s standard output.
• If stdio[1] is set to anything other than 'pipe', this will be null.
• Alias for subprocess.stdio[1].

const { spawn } = require('node:child_process');
const subprocess = spawn('ls');

subprocess.stdout.on('data', (data) => {
  console.log(`Received chunk ${data}`);
});

4. subprocess.stdio

• An array of pipes corresponding to the stdio option passed to spawn().
• Indices 0, 1, and 2 correspond to stdin, stdout, and stderr respectively.

const subprocess = child_process.spawn('ls', {
  stdio: [
    0,        // Use parent's stdin
    'pipe',   // Pipe stdout
    fs.openSync('err.out', 'w'), // Redirect stderr to file
  ],
});

5. subprocess.unref()

• Allows the parent process to exit without waiting for the child.
• Removes the child from the parent’s event loop reference count.

const subprocess = spawn(process.argv[0], ['child_program.js'], {
  detached: true,
  stdio: 'ignore',
});
subprocess.unref();

6. maxBuffer and Unicode

• Specifies the maximum number of bytes allowed on stdout or stderr.
• If exceeded, the child process is terminated.
• Impacts multibyte encodings like UTF-8 or UTF-16.

7. Advanced Serialization

• Introduced in v13.2.0 and v12.16.0.
• Based on the HTML structured clone algorithm.
• Supports advanced types like BigInt, Map, Set, ArrayBuffer, Buffer, Error, RegExp.
• Enabled by setting serialization: 'advanced' in spawn() or fork().

Conclusion

The stdin, stdout, stderr, and stdio properties are essential for managing communication between parent and child processes in Node.js. Combined with unref(), maxBuffer, and advanced serialization, they provide developers with powerful tools to build scalable and flexible applications.