modern-tar is a JavaScript library for creating and reading TAR archives with streaming support. It works in Node.js and browsers and has no dependencies.
Why?
The NPM ecosystem has a dependency problem. If you look at the most popular libraries in the registry, many hold legacy requirements with unnecessary dependencies that all developers have to pay the cost for. These huge dependency trees slow down applications, degrade install times, and drastically increase the surface area of supply chain attacks.
I started this project as my contribution to the e18e initiative, which works on cleaning up and modernising the JavaScript ecosystem. I wanted to see if we could build a TAR library with zero dependencies, better performance, and also natively support browsers from the start.
The Architecture
The TAR format stores entries sequentially, with 512-byte headers describing the data that follows. Extracting the archive also means preparing directories, opening files, and writing their contents. Across thousands of small files, all that surrounding work adds up.
The Parser
To support both Node.js and browsers, I separated the parsing logic from the I/O layer.
The parser is a synchronous state machine that consumes Uint8Array chunks. It tracks headers, contents, and padding without opening files or knowing which runtime is calling it. We then use wrappers to handle I/O and decide when the parser can make progress.
This lets the Node.js streams and Web Streams APIs share the same parser.
Ring Buffer
Incoming Uint8Array chunks don’t necessarily line up with TAR headers or file boundaries. For example, a simple approach to read a 512-byte header across multiple chunk boundaries is to concatenate them into one buffer, however, that can causes unnecessary copies and can contribute to memory fragmentation.
Instead, we can organize the chunks into a ring buffer, represented in JavaScript using a fixed-size array, and only store references to the chunk without copying its bytes. Head and tail indices then allow us to track where to read and append letting us efficiently reuse consumed slots. Copying does still happen, but only when a read needs a contiguous range that spans multiple chunks.
TAR headers and metadata need a contiguous range of bytes. If that range fits within one chunk, the queue returns the chunk or a .subarray() view. Otherwise, it copies the requested bytes into one array. File contents are streamed as separate chunks or views, so they don’t need to be copied even when they span multiple chunks. This design greatly improved performance and memory usage.
chunk-queue.ts is the implementation that encapsulates this logic.
The File Sink
For extraction, these chunks need to reach the filesystem. Our parser produces them synchronously, while filesystem writes complete asynchronously. A file sink lets us bridge the two components.
In Node.js, a typical way to write streamed data to a file is to connect a readable stream to a writable stream for that file using pipeline(). The readable stream produces chunks, and the writable stream consumes and queues them for writing. The pipeline coordinates the transfer, completion, and errors.
It also handles backpressure. If chunks arrive faster than they can be written, the write queue grows. Once it reaches its buffering threshold, the pipeline pauses the producer until the writer is ready for more. This prevents pending data from accumulating indefinitely.
Node streams work well as a general-purpose interface, but has a lot of underlying machinery we don’t need. For example, it uses EventEmitter to notify listeners about events such as completion, errors, and when writing can resume. Across thousands of small files, stream initialization, listener management, and lifecycle bookkeeping become work worth reducing.
Our file sink has just one consumer, allowing us to simplify our implementation. Instead of an event-based interface, the sink uses a small set of functions that share state through a closure. The sink passes in chunks, checks whether it needs to wait, and awaits completion. This keeps coordination explicit and gives us control over batching and buffering without bootstrapping an entirely separate stream pipeline for each file, improving performance and memory usage, especially for archives with many small files.
The sink collects the existing chunks and views in an array and relies on fs.writev() to batch several chunks in one system call without requiring us to concatenate their bytes.
Our file sink has a spare array to collect chunks. While one batch is being written, we already start collecting chunks in the spare array. When the write completes, we swap the arrays and start a new batch, allowing us to parse and write concurrently.
This optimization was inspired by node-tar which also uses a specialized filesystem writer, fs-minipass, although it is event-based and is a larger implementation with dependencies. Our implementation is in file-sink.ts.
Concurrency Model
Finally, concurrent operations are a key part to maximising performance, but due to TAR’s sequential nature conflicting with asynchronous filesystem operations, we need to coordinate them carefully.
Consider an archive containing a/b/first.txt and a/b/second.txt. Before opening either file, we need to prepare a, then a/b. Those operations must happen in order, but writing the two files doesn’t have to finish in order. The goal is to wait for dependencies without waiting for every outstanding filesystem operation.
To avoid repeating directory checks for every file, we keep a map of directory paths to preparation JavaScript promises. Each directory waits for its parent, forming a promise chain: prepare a, then a/b, then open the file. Files sharing a directory reuse the same preparation promise instead of repeating those checks.
Once the parser has passed a file’s contents to its sink, it can move to the next entry while the sink finishes writing.
Results
These key optimizations allowed modern-tar to become faster and lighter than existing libraries, while also being smaller and simpler. See benchmarks here.
modern-tar continues to be actively maintained, with zero runtime dependencies and support for Node.js and browsers.