-
-
Notifications
You must be signed in to change notification settings - Fork 1
Improve documentation with centralized docs/ folder and enhanced README #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
d27fb1c
Initial plan
Copilot 9014e59
Initial documentation improvement plan
Copilot 4a6b67f
Create docs/ folder with comprehensive codec documentation
Copilot 20da1b8
Include docs/ folder in npm package and finalize documentation
Copilot ff9a2de
Address PR feedback: revert yarn.lock, remove package-lock.json, moveβ¦
Copilot b4269a7
Update CBOR documentation with comprehensive guide and examples
Copilot 8bb0b9b
Remove docs/ folder and move codec documentation to src/<codec>/READMβ¦
Copilot 0753284
Merge branch 'master' into copilot/fix-13
Copilot 352b7c1
Resolve merge conflicts and add back detailed benchmarks to main README
Copilot 2e59483
docs: βοΈ update benchmarks in README
streamich File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
# json-pack Documentation | ||
|
||
Welcome to the `json-pack` documentation. This library provides high-performance implementations of various binary serialization formats for JavaScript. | ||
|
||
## Available Codecs | ||
|
||
### Binary Formats | ||
|
||
- **[MessagePack](./msgpack.md)** - Fast and efficient binary serialization format | ||
- Compact binary representation | ||
- Support for binary data and extensions | ||
- Fastest JavaScript implementation | ||
|
||
- **[CBOR](./cbor.md)** - Concise Binary Object Representation (RFC 7049) | ||
- Self-describing binary format | ||
- Rich data type support | ||
- DAG-CBOR variant available | ||
|
||
- **[UBJSON](./ubjson.md)** - Universal Binary JSON | ||
- Binary JSON with type information | ||
- Fixed-length encoding for better performance | ||
- Support for typed arrays | ||
|
||
- **[BSON](./bson.md)** - Binary JSON (MongoDB format) | ||
- Document-oriented binary format | ||
- Rich type system including dates and binary data | ||
- MongoDB-compatible implementation | ||
|
||
### Protocol Formats | ||
|
||
- **[RESP](./resp.md)** - Redis Serialization Protocol | ||
- RESP2 and RESP3 support | ||
- Streaming decoder available | ||
- Redis-compatible implementation | ||
|
||
- **[Amazon Ion](./ion.md)** - Amazon's data format | ||
- Self-describing with rich type system | ||
- Binary and text representations | ||
- Symbol table support | ||
|
||
### Network/Transfer Formats | ||
|
||
- **[Bencode](./bencode.md)** - BitTorrent encoding format | ||
- Simple binary encoding | ||
- Dictionary ordering for reproducible output | ||
- Extensions for additional types | ||
|
||
### Enhanced JSON | ||
|
||
- **[JSON](./json.md)** - High-performance JSON encoder/decoder | ||
- Faster than native JSON in many cases | ||
- Optimized for repeated operations | ||
- Binary-safe implementation | ||
|
||
- **[JSON Binary](./json-binary.md)** - JSON with binary data support | ||
- Uint8Array support via Base64 encoding | ||
- Drop-in replacement for JSON with binary data | ||
|
||
## Performance | ||
|
||
All codecs in this library are optimized for performance and typically outperform other JavaScript implementations. Each codec documentation includes detailed benchmarks comparing performance against popular alternatives. | ||
|
||
## Usage Patterns | ||
|
||
### Basic Usage | ||
|
||
```ts | ||
import {MessagePackEncoder, MessagePackDecoder} from 'json-pack/lib/msgpack'; | ||
|
||
const encoder = new MessagePackEncoder(); | ||
const decoder = new MessagePackDecoder(); | ||
|
||
const data = {hello: 'world', count: 42}; | ||
const encoded = encoder.encode(data); | ||
const decoded = decoder.decode(encoded); | ||
``` | ||
|
||
### Streaming Usage | ||
|
||
Many codecs support streaming operations for handling large datasets: | ||
|
||
```ts | ||
import {RespStreamingDecoder} from 'json-pack/lib/resp'; | ||
|
||
const decoder = new RespStreamingDecoder(); | ||
// Process streaming data... | ||
``` | ||
|
||
### Binary Data | ||
|
||
Several codecs support efficient binary data handling: | ||
|
||
```ts | ||
import {MessagePackEncoderFull} from 'json-pack/lib/msgpack'; | ||
|
||
const encoder = new MessagePackEncoderFull(); | ||
const data = { | ||
text: 'hello', | ||
binary: new Uint8Array([1, 2, 3, 4]) | ||
}; | ||
const encoded = encoder.encode(data); | ||
``` | ||
|
||
## Choosing a Format | ||
|
||
- **MessagePack**: Best general-purpose binary format, fastest performance | ||
- **CBOR**: Standards-compliant, rich type system, good for interoperability | ||
- **BSON**: When working with MongoDB or need document-oriented features | ||
- **RESP**: For Redis protocol compatibility | ||
- **Ion**: When you need Amazon Ion compatibility or rich type annotations | ||
- **JSON**: When you need text format with better performance than native JSON | ||
- **Bencode**: For BitTorrent protocol compatibility | ||
- **UBJSON**: When you need typed binary JSON | ||
|
||
Each format has its strengths - choose based on your specific requirements for performance, compatibility, and features. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# Bencode codecs | ||
|
||
Implements [Bencode][bencode] encoder and decoder. | ||
|
||
[bencode]: https://en.wikipedia.org/wiki/Bencode | ||
|
||
Type coercion: | ||
|
||
- Strings and `Uint8Array` are encoded as Bencode byte strings, decoded as `Uint8Array`. | ||
- `Object` and `Map` are encoded as Bencode dictionaries, decoded as `Object`. | ||
- `Array` and `Set` are encoded as Bencode lists, decoded as `Array`. | ||
- `number` and `bigint` are encoded as Bencode integers, decoded as `number`. | ||
- Float `number` are rounded and encoded as Bencode integers, decoded as `number`. | ||
|
||
|
||
## Extensions | ||
|
||
This codec extends the Bencode specification to support the following types: | ||
|
||
- `null` (encoded as `n`) | ||
- `undefined` (encoded as `u`) | ||
- `boolean` (encoded as `t` for `true` and `f` for `false`) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
# BSON | ||
|
||
Performant implementation of [BSON][bson] (Binary JSON) for JavaScript. | ||
|
||
[bson]: https://bsonspec.org/ | ||
|
||
## Overview | ||
|
||
BSON (Binary JSON) is a binary representation of JSON-like documents. It extends JSON's data model to provide additional data types, ordered fields, and efficient encoding and decoding. | ||
|
||
## Features | ||
|
||
- High-performance BSON encoding and decoding | ||
- Support for all BSON data types including: | ||
- ObjectId | ||
- Binary data | ||
- Dates | ||
- Regular expressions | ||
- JavaScript code | ||
- MongoDB-compatible implementation | ||
- Efficient binary representation | ||
|
||
## Usage | ||
|
||
```ts | ||
import {BsonEncoder, BsonDecoder} from 'json-pack/lib/bson'; | ||
|
||
const encoder = new BsonEncoder(); | ||
const decoder = new BsonDecoder(); | ||
|
||
const data = { | ||
name: 'example', | ||
created: new Date(), | ||
binary: new Uint8Array([1, 2, 3]) | ||
}; | ||
|
||
const encoded = encoder.encode(data); | ||
const decoded = decoder.decode(encoded); | ||
``` | ||
|
||
|
||
## Benchmarks | ||
|
||
``` | ||
npx ts-node benchmarks/json-pack/bench.bson.encoding.ts | ||
=============================================================================== Benchmark: Encoding | ||
Warmup: 1000x , Node.js: v20.4.0 , Arch: arm64 , CPU: Apple M1 | ||
----------------------------------------------------------------------------- Combined, 63374 bytes | ||
π json-pack JsonEncoder x 4,604 ops/sec Β±0.12% (100 runs sampled) | ||
π json-pack BsonEncoder x 3,962 ops/sec Β±0.18% (100 runs sampled) | ||
π bson BSON.serialize() x 1,439 ops/sec Β±0.19% (100 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 1,699 ops/sec Β±0.11% (100 runs sampled) | ||
Fastest is π json-pack JsonEncoder | ||
---------------------------------------------------------------------------- Small object, 53 bytes | ||
π json-pack JsonEncoder x 4,464,852 ops/sec Β±0.47% (96 runs sampled) | ||
π json-pack BsonEncoder x 3,684,236 ops/sec Β±0.18% (100 runs sampled) | ||
π bson BSON.serialize() x 884,917 ops/sec Β±0.14% (99 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 1,153,616 ops/sec Β±0.16% (98 runs sampled) | ||
Fastest is π json-pack JsonEncoder | ||
------------------------------------------------------------------------ Typical object, 1002 bytes | ||
π json-pack JsonEncoder x 306,241 ops/sec Β±0.22% (100 runs sampled) | ||
π json-pack BsonEncoder x 368,051 ops/sec Β±0.17% (100 runs sampled) | ||
π bson BSON.serialize() x 106,583 ops/sec Β±0.84% (99 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 126,497 ops/sec Β±0.12% (99 runs sampled) | ||
Fastest is π json-pack BsonEncoder | ||
-------------------------------------------------------------------------- Large object, 3750 bytes | ||
π json-pack JsonEncoder x 91,646 ops/sec Β±0.76% (100 runs sampled) | ||
π json-pack BsonEncoder x 109,402 ops/sec Β±0.17% (100 runs sampled) | ||
π bson BSON.serialize() x 35,037 ops/sec Β±0.19% (98 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 39,504 ops/sec Β±0.49% (101 runs sampled) | ||
Fastest is π json-pack BsonEncoder | ||
-------------------------------------------------------------------- Very large object, 45759 bytes | ||
π json-pack JsonEncoder x 6,234 ops/sec Β±0.47% (99 runs sampled) | ||
π json-pack BsonEncoder x 4,824 ops/sec Β±0.20% (99 runs sampled) | ||
π bson BSON.serialize() x 1,645 ops/sec Β±0.17% (101 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 2,696 ops/sec Β±0.66% (98 runs sampled) | ||
Fastest is π json-pack JsonEncoder | ||
------------------------------------------------------------------ Object with many keys, 978 bytes | ||
π json-pack JsonEncoder x 260,571 ops/sec Β±0.68% (96 runs sampled) | ||
π json-pack BsonEncoder x 243,776 ops/sec Β±0.42% (98 runs sampled) | ||
π bson BSON.serialize() x 86,641 ops/sec Β±0.29% (100 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 81,730 ops/sec Β±0.13% (99 runs sampled) | ||
Fastest is π json-pack JsonEncoder | ||
------------------------------------------------------------------------- String ladder, 4046 bytes | ||
π json-pack JsonEncoder x 92,381 ops/sec Β±0.13% (100 runs sampled) | ||
π json-pack BsonEncoder x 127,132 ops/sec Β±1.03% (90 runs sampled) | ||
π bson BSON.serialize() x 75,356 ops/sec Β±1.18% (94 runs sampled) | ||
π bson Buffer.from(EJSON.stringify()) x 47,308 ops/sec Β±0.08% (101 runs sampled) | ||
Fastest is π json-pack BsonEncoder | ||
``` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
docs/
. Link to it from main readme.json-pack
library is the fastest library in NPM ecosystem.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved benchmarks section to docs/benchmarks.md and added link to the benchmarking blog post in commit ff9a2de.