File System Module in Node.js: Managing Files and Streams

1. Using Promise APIs

The Promise-based API performs file operations asynchronously and returns a Promise.

const { unlink } = require('node:fs/promises');
(async function(path) {
  try {
    await unlink(path);
    console.log(`successfully deleted ${path}`);
  } catch (error) {
    console.error('error:', error.message);
  }
})('/tmp/hello');

2. Using Callback APIs

The callback form executes operations asynchronously, with the first argument reserved for errors.

const { unlink } = require('node:fs');
unlink('/tmp/hello', (err) => {
  if (err) throw err;
  console.log('successfully deleted /tmp/hello');
});

3. Using Synchronous APIs

Synchronous APIs block the event loop until the operation completes. Errors are thrown immediately and handled with try...catch.

const { unlinkSync } = require('node:fs');
try {
  unlinkSync('/tmp/hello');
  console.log('successfully deleted /tmp/hello');
} catch (err) {
  console.error(err);
}

4. FileHandle

The FileHandle class is a wrapper for a numeric file descriptor created by fsPromises.open(). It is an EventEmitter and provides methods for file management.

• filehandle.close(): Closes the file handle.
• filehandle.appendFile(): Writes data to the file.
• filehandle.chmod(): Changes file permissions.
• filehandle.chown(): Changes file ownership.

5. Reading and Writing Streams

• filehandle.createReadStream(): Creates a readable stream with options like start and end.
• filehandle.createWriteStream(): Creates a writable stream with options like start and flush.

Streams allow processing large files without loading them entirely into memory.

6. Reading Methods

• filehandle.read(): Reads data into a buffer.
• filehandle.readFile(): Reads the entire file contents.
• filehandle.readLines(): Reads a file line by line using readline.
• filehandle.readableWebStream(): Returns a web-compatible ReadableStream.

7. Concurrency Considerations

Promise-based operations use Node.js’s internal threadpool and are not thread-safe. Care must be taken when performing concurrent modifications on the same file to avoid data corruption.

Conclusion

The fs module in Node.js provides powerful tools for file and stream management. Choosing between Promise, Callback, or Synchronous APIs depends on performance and application design. Proper use of FileHandle and streams improves efficiency and prevents resource leaks.

1. filehandle.readv()

Reads data from a file into multiple buffers or typed arrays.

• buffers: An array of Buffer or TypedArray objects.
• position: The file offset to start reading from.

Returns an object with bytesRead and references to the input buffers.

2. filehandle.stat()

Returns file statistics such as size, permissions, and timestamps. The bigint option allows numeric values to be returned as BigInt.

3. filehandle.sync()

Flushes all queued data for the file descriptor to the storage device. Equivalent to POSIX fsync(2).

4. filehandle.truncate(len)

Truncates the file to the specified length. If the file is longer, only the first len bytes remain; if shorter, it is extended with null bytes.

import { open } from 'node:fs/promises';
let fh = await open('temp.txt', 'r+');
await fh.truncate(4);
await fh.close();

5. filehandle.utimes(atime, mtime)

Updates the access (atime) and modification (mtime) timestamps of the file.

6. Writing Methods

• filehandle.write(buffer, offset, length, position): Writes data from a buffer.
• filehandle.write(buffer, options): Newer variant using an options object.
• filehandle.write(string, position, encoding): Writes a string with specified encoding.
• filehandle.writeFile(data, options): Writes or replaces the entire file with provided data.
• filehandle.writev(buffers, position): Writes multiple buffers or typed arrays to the file.

Note: Calling write methods multiple times without awaiting their Promises is unsafe. Use createWriteStream() for concurrent writes.

7. filehandle[Symbol.asyncDispose]()

Calls filehandle.close() and returns a Promise that resolves when the file handle is closed.

8. fsPromises.access()

Checks user permissions for a file or directory. The mode argument can be a combination of R_OK, W_OK, and X_OK.

import { access, constants } from 'node:fs/promises';
try {
  await access('/etc/passwd', constants.R_OK | constants.W_OK);
  console.log('can access');
} catch {
  console.error('cannot access');
}

Note: Using access() before open() is discouraged due to race conditions. Instead, attempt the operation directly and handle errors.

Conclusion

The FileHandle class and fs/promises APIs provide powerful tools for advanced file management in Node.js. They enable flexible reading and writing, file truncation, timestamp updates, syncing data, and access checks. Proper use of these methods improves efficiency and prevents concurrency issues.

1. fsPromises.appendFile()

Asynchronously appends data to a file, creating the file if it does not exist.

import { appendFile } from 'node:fs/promises';
await appendFile('log.txt', 'New entry\n');

2. fsPromises.chmod() and fsPromises.chown()

• chmod(path, mode): Changes file permissions.
• chown(path, uid, gid): Changes file ownership.

3. fsPromises.copyFile()

Copies a file to a destination. Options include COPYFILE_EXCL to prevent overwriting and COPYFILE_FICLONE for copy-on-write.

import { copyFile, constants } from 'node:fs/promises';
await copyFile('source.txt', 'dest.txt', constants.COPYFILE_EXCL);

4. fsPromises.cp()

Recursively copies directories and their contents. Options include recursive, preserveTimestamps, and filter.

5. fsPromises.glob()

Searches for files using glob patterns. Returns an AsyncIterator yielding matching paths.

import { glob } from 'node:fs/promises';
for await (const entry of glob('**/*.js')) {
  console.log(entry);
}

6. fsPromises.link()

Creates a hard link from an existing path to a new path.

7. fsPromises.lstat()

Returns file statistics for a symbolic link itself, rather than the file it points to.

8. fsPromises.mkdir()

Creates a new directory. The recursive option allows parent directories to be created automatically.

import { mkdir } from 'node:fs/promises';
await mkdir('project/data', { recursive: true });

9. fsPromises.mkdtemp()

Creates a unique temporary directory by appending six random characters to the provided prefix.

10. fsPromises.mkdtempDisposable()

Creates a temporary directory and returns a disposable object with remove() and [Symbol.asyncDispose] methods to delete it.

Conclusion

The advanced fsPromises methods in Node.js provide powerful tools for managing files and directories. They enable appending data, changing permissions, copying files and directories, searching with glob patterns, and creating temporary directories. Proper use of these methods improves efficiency and simplifies large-scale project development.

1. fsPromises.opendir()

Opens a directory asynchronously and returns an fs.Dir object, which can be iterated using AsyncIterator.

import { opendir } from 'node:fs/promises';
const dir = await opendir('./');
for await (const dirent of dir) {
  console.log(dirent.name);
}

2. fsPromises.readdir()

Reads the contents of a directory and returns an array of filenames. With withFileTypes set to true, it returns Dirent objects.

3. fsPromises.readFile()

Reads the entire contents of a file asynchronously. Returns a Buffer or string depending on the encoding. Supports AbortSignal for cancellation.

4. fsPromises.readlink()

Reads the contents of a symbolic link and returns the link path.

5. fsPromises.realpath()

Resolves the actual path of a file or directory.

6. fsPromises.rename()

Renames a file or directory.

7. fsPromises.rmdir() and fsPromises.rm()

rmdir() removes a directory. rm() removes files or directories with options like recursive and force.

8. fsPromises.stat() and fsPromises.statfs()

Returns file statistics (stat) or file system statistics (statfs).

9. fsPromises.symlink()

Creates a symbolic link. On Windows, the type can be file, dir, or junction.

10. fsPromises.truncate()

Truncates or extends a file to the specified length.

11. fsPromises.unlink()

Deletes a file or symbolic link.

12. fsPromises.utimes()

Updates the access (atime) and modification (mtime) timestamps of a file.

13. fsPromises.watch()

Watches for changes to a file or directory and returns an AsyncIterator of events.

import { watch } from 'node:fs/promises';
const watcher = watch('./file.txt');
for await (const event of watcher) {
  console.log(event);
}

14. fsPromises.writeFile()

Writes data to a file, replacing it if it already exists. Supports cancellation with AbortSignal.

Conclusion

The fs/promises module in Node.js provides powerful tools for advanced file and directory management. These methods enable developers to handle reading, writing, deleting, renaming, and monitoring files in a clean, asynchronous way. Proper use of these APIs improves efficiency and simplifies large-scale project development.

1. fs.appendFile()

Asynchronously appends data to a file. If the file does not exist, it is created.

import { appendFile } from 'node:fs';
appendFile('message.txt', 'data to append', (err) => {
  if (err) throw err;
  console.log('Data appended!');
});

2. fs.chmod()

Changes file permissions. The mode argument is a numeric bitmask constructed using fs.constants.

import { chmod } from 'node:fs';
chmod('my_file.txt', 0o775, (err) => {
  if (err) throw err;
  console.log('Permissions changed!');
});

3. fs.chown()

Changes the owner and group of a file.

4. fs.close()

Closes an open file descriptor. Calling fs.close() on a descriptor still in use may lead to undefined behavior.

5. fs.copyFile()

Copies a file to a destination. Options include COPYFILE_EXCL to prevent overwriting and COPYFILE_FICLONE for copy-on-write.

import { copyFile, constants } from 'node:fs';
copyFile('source.txt', 'dest.txt', constants.COPYFILE_EXCL, (err) => {
  if (err) throw err;
  console.log('File copied!');
});

6. fs.cp()

Recursively copies directories and their contents. Options include recursive and preserveTimestamps.

7. fs.createReadStream()

Creates a readable stream. You can specify start and end options to read a specific range of bytes.

import { createReadStream } from 'node:fs';
createReadStream('sample.txt', { start: 90, end: 99 });

8. fs.createWriteStream()

Creates a writable stream. Allows writing data at a specific position in the file. The autoClose option determines whether the file is closed automatically on error or finish.

Conclusion

The fs module in Node.js provides powerful callback-based methods for file and directory management. These APIs enable appending data, changing permissions, copying files, and working with streams. Proper use of these methods improves performance and prevents resource leaks in large-scale projects.

1. fs.exists()

Checks whether a file or directory exists. Deprecated due to inconsistent callback parameters. Use fs.access() or fs.stat() instead.

2. fs.fchmod() and fs.fchown()

• fchmod(fd, mode): Changes file permissions using a file descriptor.
• fchown(fd, uid, gid): Changes the owner and group of a file.

3. fs.fdatasync() and fs.fsync()

Flush queued data to disk. fdatasync() flushes only data, while fsync() flushes both data and metadata.

4. fs.fstat()

Retrieves file statistics based on a file descriptor. The bigint option allows numeric values to be returned as BigInt.

5. fs.ftruncate()

Truncates or extends a file to a specified length. If shorter, the file is padded with null bytes ('\0').

6. fs.futimes()

Updates the access (atime) and modification (mtime) timestamps of a file using its file descriptor.

7. fs.glob()

Searches for files matching glob patterns. Returns matching paths.

import { glob } from 'node:fs';
glob('**/*.js', (err, matches) => {
  if (err) throw err;
  console.log(matches);
});

8. fs.lchmod(), fs.lchown(), fs.lutimes()

• lchmod(): Changes permissions of a symbolic link (deprecated, macOS only).
• lchown(): Changes the owner of a symbolic link.
• lutimes(): Updates timestamps of a symbolic link without dereferencing it.

9. fs.link()

Creates a hard link from an existing path to a new path.

10. fs.lstat()

Retrieves statistics for a symbolic link itself, rather than the file it points to. Similar to stat() but specific to links.

Conclusion

The fs module in Node.js provides callback-based methods for advanced file management. While some methods are deprecated, modern alternatives ensure safer and more consistent behavior. Proper use of these APIs enhances performance and reliability in large-scale applications.

1. fs.mkdir()

Creates a directory asynchronously. The recursive option allows parent directories to be created automatically. If the directory already exists and recursive is false, an EEXIST error occurs.

import { mkdir } from 'node:fs';
mkdir('./tmp/a/apple', { recursive: true }, (err) => {
  if (err) throw err;
});

2. fs.mkdtemp()

Creates a unique temporary directory by appending six random characters to the provided prefix. The resulting path is returned to the callback.

import { mkdtemp } from 'node:fs';
import { join } from 'node:path';
import { tmpdir } from 'node:os';

mkdtemp(join(tmpdir(), 'foo-'), (err, directory) => {
  if (err) throw err;
  console.log(directory);
});

3. fs.open()

Opens a file asynchronously and returns a file descriptor. The flags option specifies how the file is opened (e.g., read, write). The mode option sets permissions if the file is created.

4. fs.openAsBlob()

Returns a Blob backed by the given file. The file must not be modified after the Blob is created. Useful for streaming file data in web-like contexts.

5. fs.opendir()

Opens a directory asynchronously and returns an fs.Dir object. This object can be used to iterate over directory entries and manage cleanup.

6. fs.read()

Reads data from a file descriptor into a buffer. The callback receives (err, bytesRead, buffer). It is important to check bytesRead to know how many bytes were actually read.

7. fs.readdir()

Reads the contents of a directory and returns an array of filenames. Options include withFileTypes to return Dirent objects and recursive to read subdirectories.

Conclusion

The fs module’s callback-based APIs provide powerful tools for directory and file management in Node.js. They enable developers to create directories, open files, read data, and iterate over directory contents efficiently. Proper use of these methods ensures reliable and scalable file system operations in large projects.

1. fs.readFile()

Reads the entire contents of a file asynchronously and passes the result to a callback.

• path: File path or file descriptor.
• options: Includes encoding and flag.
• callback(err, data): Returns either an error or the file contents.

import { readFile } from 'node:fs';
readFile('/etc/passwd', 'utf8', (err, data) => {
  if (err) throw err;
  console.log(data);
});

If no encoding is specified, the data is returned as a Buffer. An AbortSignal can be used to cancel an ongoing read operation.

Performance Notes

• For large files, prefer fs.createReadStream() to reduce memory usage.
• Regular files are read in 512 KiB chunks; pipes and special files are read in 64 KiB chunks.
• For maximum speed, use fs.read() directly and manage reading manually.

2. fs.readlink()

Reads the target path of a symbolic link.

• path: Path to the symbolic link.
• options: Encoding options.
• callback(err, linkString): Returns either an error or the link target.

import { readlink } from 'node:fs';
readlink('/path/to/symlink', (err, linkString) => {
  if (err) throw err;
  console.log('Link points to:', linkString);
});

If encoding is set to buffer, the link target is returned as a Buffer.

Conclusion

fs.readFile() and fs.readlink() are powerful tools for reading file contents and managing symbolic links in Node.js. Use readFile() when you need the full contents of a file, and readlink() when working with symbolic link structures. Proper use of these methods improves efficiency and simplifies file system operations in large projects.

1. fs.readv()

Reads data from a file descriptor into multiple buffers using readv(). The callback receives err, bytesRead, and buffers.

2. fs.realpath() and fs.realpath.native()

Resolves the canonical path of a file or directory, handling ., .., and symbolic links. Returns an error if the path does not exist.

3. fs.rename()

Renames a file or directory. If the destination exists, it is overwritten. If the destination is a directory, an error is raised.

4. fs.rmdir() and fs.rm()

rmdir() removes a directory (deprecated). rm() removes files or directories with options like recursive and force.

5. fs.stat() and fs.statfs()

Retrieves file statistics (fs.Stats) or file system statistics (fs.StatFs). The bigint option allows numeric values to be returned as BigInt.

6. fs.symlink()

Creates a symbolic link. On Windows, the type can be file, dir, or junction.

7. fs.truncate()

Truncates or extends a file to the specified length. If shorter, the file is padded with null bytes ('\0').

8. fs.unlink()

Deletes a file or symbolic link. To remove directories, use fs.rmdir() or fs.rm().

9. fs.unwatchFile(), fs.watch(), fs.watchFile()

These methods monitor file or directory changes. fs.watch() is more efficient than fs.watchFile(). fs.unwatchFile() stops monitoring.

10. fs.utimes()

Updates the access (atime) and modification (mtime) timestamps of a file.

11. fs.write()

Writes data to a file descriptor. The callback receives err, bytesWritten, and buffer. Using fs.write() multiple times without awaiting callbacks is unsafe; use fs.createWriteStream() instead.

Conclusion

The fs module in Node.js provides powerful callback-based methods for advanced file operations. These APIs enable developers to read, write, rename, delete, and monitor files efficiently. Proper use of these methods ensures reliable and scalable file system management in large projects.

1. fs.write()

Writes a string to a file specified by a file descriptor.

• fd: File descriptor.
• string: String data to write.
• position: Offset in the file where data should be written.
• encoding: String encoding (default: utf8).

The callback receives three arguments: err, written, and string.

2. fs.writeFile()

Writes data to a file, replacing it if it already exists.

• file: Path or file descriptor.
• data: String or Buffer.
• options: Includes encoding, mode, and flag.

import { writeFile } from 'node:fs';
writeFile('message.txt', 'Hello Node.js', 'utf8', (err) => {
  if (err) throw err;
  console.log('File saved!');
});

Supports AbortSignal for cancellation. For performance-sensitive code, prefer fs.createWriteStream().

3. fs.writev()

Writes multiple buffers or typed arrays to a file. The callback receives err, bytesWritten, and buffers.

4. Synchronous APIs

These methods perform operations synchronously, blocking the event loop until completion.

• fs.accessSync(): Tests file accessibility.
• fs.appendFileSync(): Appends data to a file.
• fs.chmodSync(): Changes file permissions.
• fs.chownSync(): Changes file ownership.
• fs.closeSync(): Closes a file descriptor.
• fs.copyFileSync(): Copies a file.
• fs.cpSync(): Recursively copies directories and their contents.

Conclusion

The fs module in Node.js provides powerful tools for writing data to files. Callback-based methods are ideal for asynchronous handling, while synchronous methods suit simple scripts or environments requiring immediate control. Choosing the right approach improves efficiency and prevents concurrency issues in large-scale projects.

1. fs.existsSync()

Checks whether a given path exists. Returns a boolean. Unlike the asynchronous version, this method is not deprecated.

2. fs.fchmodSync() and fs.fchownSync()

Change file permissions and ownership using a file descriptor.

3. fs.fdatasyncSync() and fs.fsyncSync()

Flush queued data to disk. fdatasyncSync() flushes only data, while fsyncSync() flushes both data and metadata.

4. fs.fstatSync()

Retrieves file statistics based on a file descriptor. The bigint option allows numeric values to be returned as BigInt.

5. fs.ftruncateSync()

Truncates or extends a file to the specified length. If shorter, the file is padded with null bytes ('\0').

6. fs.futimesSync()

Updates the access (atime) and modification (mtime) timestamps of a file using its file descriptor.

7. fs.globSync()

Searches for files matching glob patterns and returns their paths.

8. fs.lchmodSync(), fs.lchownSync(), fs.lutimesSync()

• lchmodSync(): Changes permissions of a symbolic link (deprecated, macOS only).
• lchownSync(): Changes the owner of a symbolic link.
• lutimesSync(): Updates timestamps of a symbolic link without dereferencing it.

9. fs.linkSync()

Creates a hard link from an existing path to a new path.

10. fs.lstatSync()

Retrieves statistics for a symbolic link itself, rather than the file it points to.

11. fs.mkdirSync(), fs.mkdtempSync(), fs.mkdtempDisposableSync()

Create directories synchronously. mkdtempSync() creates a unique temporary directory, while mkdtempDisposableSync() returns a disposable object that can remove the directory when disposed.

12. fs.opendirSync() and fs.openSync()

Open a directory or file synchronously, returning an fs.Dir object or a file descriptor.

13. fs.readdirSync()

Reads the contents of a directory. With withFileTypes set to true, returns Dirent objects.

14. fs.readFileSync()

Reads the entire contents of a file synchronously. Returns a string if encoding is specified, otherwise returns a Buffer. Behavior when reading directories is platform-specific.

Conclusion

The fs module in Node.js provides synchronous methods for file and directory management. These APIs are suitable for simple scripts or environments requiring immediate control, but in larger applications they can reduce performance. Proper use of these methods ensures reliability and simplicity in small to medium projects.

1. fs.readlinkSync()

Returns the target path of a symbolic link. If encoding is set to buffer, the result is returned as a Buffer.

2. fs.readSync() and fs.readvSync()

Read data from a file descriptor. readSync() writes data into a single buffer, while readvSync() writes into multiple buffers.

3. fs.realpathSync() and fs.realpathSync.native()

Resolve the canonical path of a file or directory, handling ., .., and symbolic links.

4. fs.renameSync()

Renames a file or directory. If the destination exists, it is overwritten.

5. fs.rmdirSync() and fs.rmSync()

rmdirSync() removes a directory (deprecated). rmSync() removes files or directories with options like recursive and force.

6. fs.statSync() and fs.statfsSync()

Retrieve file statistics (fs.Stats) or file system statistics (fs.StatFs).

7. fs.symlinkSync()

Creates a symbolic link. On Windows, the type can be file, dir, or junction.

8. fs.truncateSync()

Truncates or extends a file to the specified length.

9. fs.unlinkSync()

Deletes a file or symbolic link.

10. fs.utimesSync()

Updates the access (atime) and modification (mtime) timestamps of a file.

11. fs.writeFileSync, fs.writeSync, fs.writevSync

Write data to files synchronously. writeFileSync() saves entire data, writeSync() writes to a buffer or string, and writevSync() writes multiple buffers.

12. Common Objects

• fs.Dir: Represents a directory stream.
• fs.Dirent: Represents a directory entry (file or subdirectory).
• fs.FSWatcher: Monitors file and directory changes.
• fs.StatWatcher: Monitors file statistics changes.
• fs.ReadStream: Represents a readable file stream.
• fs.Stats: Provides file statistics.
• fs.StatFs: Provides file system statistics.
• fs.Utf8Stream: Optimized UTF-8 stream writer with advanced buffering.

Conclusion

The fs module in Node.js offers advanced synchronous APIs for file and directory management. These methods are suitable for scripts or environments requiring immediate control, but in larger applications they can reduce performance. Proper use of these APIs ensures reliability and simplicity in small to medium projects.