HTTP Module in Node.js: Low-Level Client and Server

1. Introduction to the HTTP Module

The http module can be imported using require('node:http') or import * as http from 'node:http'. It provides low-level tools for handling HTTP requests and responses.

2. Header Management

HTTP headers are represented as JavaScript objects with lowercase keys. Raw headers are preserved in the rawHeaders property, which stores them exactly as received.

3. Class: http.Agent

http.Agent manages connection persistence and reuse for HTTP clients. It pools sockets to optimize performance and reduce overhead.

keepAlive: Keeps sockets alive for reuse.
maxSockets: Maximum concurrent sockets per host.
maxFreeSockets: Maximum free sockets retained when keepAlive is enabled.
scheduling: FIFO or LIFO strategy for socket reuse.

4. Socket Management

The Agent maintains queues of sockets and reuses them when possible. Important methods include:

agent.createConnection(): Creates a new socket.
agent.keepSocketAlive(): Keeps a socket alive for reuse.
agent.reuseSocket(): Reattaches a socket to a new request.
agent.destroy(): Destroys all active sockets.

5. Agent Configuration Options

Custom agents can be configured with options such as:

keepAliveMsecs: Delay for TCP Keep-Alive packets.
agentKeepAliveTimeoutBuffer: Buffer time before socket expiration.
maxTotalSockets: Maximum sockets across all hosts.
proxyEnv: Proxy configuration via environment variables.

6. Example Usage

const http = require('node:http');
const keepAliveAgent = new http.Agent({ keepAlive: true });

const options = {
  hostname: 'localhost',
  port: 80,
  path: '/',
  agent: keepAliveAgent
};

http.request(options, (res) => {
  console.log('Response received');
});

Conclusion

The http module in Node.js provides powerful low-level tools for managing HTTP communication. By leveraging http.Agent and its configuration options, developers can optimize network performance and resource usage.

1. Introduction

The http.ClientRequest object is created by http.request(). It represents an in-progress request whose headers are queued but still mutable until request.end() is called or the first data chunk is sent.

2. Header Management

setHeader(name, value): Set a header.
getHeader(name): Retrieve a header value.
removeHeader(name): Remove a header.
getHeaderNames() and getHeaders(): View current headers.
getRawHeaderNames(): View raw header names with original casing.

3. Key Events

response: Fired when a response is received.
close: Indicates request completion or premature connection termination.
connect: Fired when a server responds to a CONNECT method.
continue: Fired when a 100 Continue response is received.
finish: Fired when the request has been fully sent.
information: Fired for 1xx informational responses.
upgrade: Fired when a server responds with a 101 Upgrade.
timeout: Fired when the underlying socket times out.

4. Core Methods

end([data[, encoding]][, callback]): Finish sending the request.
destroy([error]): Destroy the request and its socket.
flushHeaders(): Immediately send headers without waiting for data.
cork(): Buffer writes for efficiency.

5. Properties

finished: True if end() has been called (deprecated, use writableEnded).
destroyed: True if destroy() has been called.
path: Request path.
method: HTTP method.
host: Request host.
protocol: Request protocol.
reusedSocket: Indicates if the request used a reused socket.

6. Example

const http = require('node:http');

const options = {
  host: 'example.com',
  port: 80,
  path: '/',
  method: 'GET'
};

const req = http.request(options, (res) => {
  res.on('data', (chunk) => {
    console.log(`BODY: ${chunk}`);
  });
  res.on('end', () => {
    console.log('No more data.');
  });
});

req.on('error', (e) => {
  console.error(`Problem with request: ${e.message}`);
});

req.end();

Conclusion

The http.ClientRequest class is a powerful tool for managing outgoing HTTP requests in Node.js. By leveraging its events and methods, developers can fully control the request lifecycle, handle streaming data, and build robust network applications.

1. Header Management in ClientRequest

setHeader(name, value): Set an outgoing header.
getHeader(name): Retrieve a header value.
removeHeader(name): Remove a header.
getHeaderNames() and getHeaders(): View current headers.
getRawHeaderNames(): View raw header names with original casing.

2. Sending Data

write(chunk[, encoding][, callback]): Send part of the request body.
end([data[, encoding]][, callback]): Finish sending the request.
flushHeaders(): Immediately send headers.

3. Socket Management

The request.socket property references the underlying socket, typically an instance of net.Socket. It provides details such as localAddress and localPort.

4. Write State Properties

writableEnded: True after end() has been called.
writableFinished: True once all data has been flushed to the system.

5. ClientRequest Events

response: Fired when a response is received.
connect: Fired when a CONNECT response is received.
upgrade: Fired when a 101 Upgrade response is received.
timeout: Fired when the socket is idle.
close: Fired when the request ends or the connection closes.

6. Class: http.Server

http.Server extends net.Server and manages incoming HTTP requests.

7. Key Server Events

request: Fired for each incoming request.
clientError: Fired when a client error occurs.
connection: Fired when a new TCP connection is established.
upgrade: Fired when an upgrade request is received.
connect: Fired when a CONNECT request is received.
close: Fired when the server shuts down.
dropRequest: Fired when requests exceed maxRequestsPerSocket.

8. Server Closing Methods

server.close(): Stops accepting new connections and closes existing ones.
server.closeAllConnections(): Forcefully closes all connections.
server.closeIdleConnections(): Closes idle keep-alive connections.

9. Example

const http = require('node:http');

// Create a server
const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ data: 'Hello World!' }));
});

server.listen(8000);

// Create a request
const options = { host: 'localhost', port: 8000, path: '/', method: 'GET' };
const req = http.request(options, (res) => {
  res.on('data', (chunk) => console.log(`BODY: ${chunk}`));
});
req.end();

Conclusion

The http.ClientRequest and http.Server classes in Node.js provide powerful tools for managing the full lifecycle of HTTP communication. Together, they allow developers to precisely control headers, data, sockets, and events on both the client and server sides.

1. Timeout Management in http.Server

server.requestTimeout: Maximum time to receive the entire request from the client (default: 300000ms).
server.setTimeout(msecs[, callback]): Sets socket timeout and handles the 'timeout' event.
server.timeout: Inactivity timeout before a socket is closed.
server.keepAliveTimeout: Wait time for new data after a response is sent (default: 5 seconds).
server.keepAliveTimeoutBuffer: Extra buffer time to reduce ECONNRESET errors.

2. Connection Management

server.maxRequestsPerSocket: Maximum number of requests per socket before closing the keep-alive connection.
server.close(): Stops accepting new connections and closes existing ones.
server.closeAllConnections(): Forcefully closes all connections.
server.closeIdleConnections(): Closes idle keep-alive connections.

3. Class: http.ServerResponse

This class represents the server’s response and extends http.OutgoingMessage. It is created internally by the server and passed to the request event handler.

4. Header Management in ServerResponse

setHeader(name, value): Set an outgoing header.
getHeader(name): Retrieve a header value.
getHeaderNames() and getHeaders(): View current headers.
removeHeader(name): Remove a header.
headersSent: Check if headers have already been sent.

5. Sending Responses

write(chunk[, encoding][, callback]): Send part of the response body.
end([data[, encoding]][, callback]): Finish sending the response.
flushHeaders(): Immediately send headers.
addTrailers(headers): Add trailing headers in chunked responses.

6. Response Status

statusCode: HTTP status code (default: 200).
statusMessage: HTTP status message.
strictContentLength: Ensures Content-Length matches the body size.

7. Example

const http = require('node:http');

const server = http.createServer((req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.statusCode = 200;
  res.end(JSON.stringify({ message: 'Hello World!' }));
});

server.listen(8000, () => {
  console.log('Server running on port 8000');
});

Conclusion

The http.Server and http.ServerResponse classes in Node.js provide powerful tools for managing the full lifecycle of HTTP communication. They allow developers to precisely control timeouts, headers, response bodies, and connection management, enabling the creation of stable and secure servers.

1. Special Methods in ServerResponse

response.writeContinue(): Sends a 100 Continue message to indicate the request body should be sent.
response.writeEarlyHints(hints[, callback]): Sends a 103 Early Hints message with Link headers for preload/preconnect resources.
response.writeHead(statusCode[, statusMessage][, headers]): Sends response headers with a status code.
response.writeProcessing(): Sends a 102 Processing message to indicate the request is being processed.

2. Class: http.IncomingMessage

This class is created by http.Server or http.ClientRequest and represents incoming messages. It extends stream.Readable and parses headers and payloads independently of the underlying socket.

3. Events and Properties of IncomingMessage

aborted: Deprecated event for aborted requests.
close: Fired when the message is completed.
message.aborted: Deprecated property indicating if the message was aborted.
message.complete: True if the full message was received and parsed.
message.destroy([error]): Destroys the associated socket.

4. Header Management in IncomingMessage

message.headers: Lowercased headers object.
message.headersDistinct: Always returns arrays of strings, even for single headers.
message.rawHeaders: Raw headers exactly as received.
message.rawTrailers: Raw trailers received at the 'end' event.

5. Protocol and Status Information

message.httpVersion: HTTP version (e.g., 1.1).
message.method: Request method (GET, POST, DELETE).
message.statusCode: Response status code (e.g., 404).
message.statusMessage: Response status message (e.g., OK).

6. Socket and Timeout Management

message.socket: Associated net.Socket object.
message.setTimeout(msecs[, callback]): Sets socket timeout.

7. Trailer Management

message.trailers: Trailers object populated at the 'end' event.
message.trailersDistinct: Always returns arrays of strings for trailers.

8. Example

const http = require('node:http');

const server = http.createServer((req, res) => {
  res.writeContinue();
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello World');
});

server.listen(8000, () => {
  console.log('Server running on port 8000');
});

Conclusion

The advanced ServerResponse methods and the IncomingMessage class in Node.js provide powerful tools for managing HTTP communication. They allow developers to control headers, status codes, special response messages, and socket behavior, ensuring robust and flexible applications.

1. Property: message.url

The message.url property is valid only for requests obtained from http.Server. It contains the raw URL string from the HTTP request. Developers can parse it using the URL class for structured access to pathname, search parameters, and host information.

2. Class: http.OutgoingMessage

This class underpins ClientRequest and ServerResponse, offering methods such as:

addTrailers(headers): Adds trailing headers in chunked responses.
appendHeader(name, value): Appends a new value to an existing header.
setHeader(name, value) and setHeaders(headers): Set headers individually or in bulk.
getHeader(name), getHeaderNames(), getHeaders(): Retrieve current headers.
removeHeader(name): Remove a header.
flushHeaders(): Immediately send headers.
write(chunk[, encoding][, callback]): Send part of the body.
end(chunk[, encoding][, callback]): Finish sending the message.
destroy([error]): Destroy the message and its socket.

3. Write State Properties

writableEnded: True after end() has been called.
writableFinished: True once all data has been flushed to the system.
writableLength: Number of buffered bytes.
writableHighWaterMark: Buffer threshold for write operations.

4. HTTP Module Properties

http.METHODS: List of supported HTTP methods.
http.STATUS_CODES: Collection of standard HTTP status codes and their descriptions.

5. Core HTTP Methods

http.createServer([options][, requestListener]): Creates an HTTP server.
http.request(options[, callback]): Sends an HTTP request with full control.
http.get(options[, callback]): Simplified method for GET requests.

6. Example

const http = require('node:http');

// Create a server
const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ data: 'Hello World!' }));
});

server.listen(8000);

// Send a GET request
http.get('http://localhost:8000/', (res) => {
  let data = '';
  res.on('data', (chunk) => { data += chunk; });
  res.on('end', () => { console.log(data); });
});

Conclusion

The http.OutgoingMessage class and the HTTP module in Node.js provide powerful tools for low-level HTTP communication. They allow developers to precisely control headers, body content, sockets, and the request/response lifecycle, enabling robust and efficient applications.