Sign in Subscribe

Boosting the JavaScript gRPC SDK with NaaE

Last updated on 
Boosting the JavaScript gRPC SDK with NaaE

TL;DR

  • The Bottleneck. Node.js is single-threaded. While excellent for I/O, @grpc/grpc-js blocks the event loop during heavy protobuf deserialisation, triggering HTTP/2 backpressure and slowing down data ingestion.
  • The Fix. We implemented NAPI-as-an-Engine (NaaE), utilising Rust for connection management and hot paths while maintaining the native Node.js event emitter syntax.
  • The Result. A 400% increase in data throughput without requiring users to rewrite their codebase.

→ View the SDK on GitHub

Node.js remains the dominant environment for many off-chain workloads, arbitrage bots, and indexers due to its massive ecosystem and ease of concurrency. However, on high-throughput chains like Solana, "ease of use" often comes with a hidden tax: the single-threaded event loop.

When you stream gigabytes of Solana data, the default @grpc/grpc-js library forces your application to choose between processing data or receiving it. It cannot do both efficiently. The result is artificial latency – not from the network, but from the library itself choking on deserialisation.

By rebuilding the engine in Rust while keeping the shell in Typescript, we created a client that offers superior performance with the same usability. This is a drop-in replacement and great @grpc-js alternative.

Why @grpc/grpc-js isn't good enough?

To understand the failure mode, you must look at the constraints of the default libraries.

The components and constraints of grpc-js

  • Client SDK: The @grpc/grpc-js JavaScript SDK is a pure JavaScript implementation of the gRPC client. It provides the necessary APIs for using services defined by protocol buffers (protobuf) and manages the underlying HTTP/2 connection and stream state.
  • The single-thread bottleneck: Node.js's event loop operates on a single thread. While Node.js is excellent for concurrent I/O (handling many connections), all JavaScript execution, including protobuf deserialization and calling the user's data callback function, happens on this single thread.

HTTP/2 flow control and backpressure in gRPC

  • gRPC/HTTP/2 flow control: gRPC, which is built on HTTP/2, uses a window-based flow control mechanism at both the stream level and the connection level. This is a critical feature designed to prevent a fast sender from overwhelming a slow receiver (known as backpressure).

    The "window size" defines the amount of data (in bytes) the receiver is prepared to accept without explicitly granting more credit.
  • The internal pause/resume cycle: When a large volume of data is quickly pushed from the network I/O buffer to the Node.js process:
    1. The @grpc/grpc-js library's internal buffer fills up because the single JavaScript thread is busy deserialising protobufs and executing the user's callback function for the received messages.
    2. As the buffer fills, the receiver's flow control window shrinks. When it hits zero, the gRPC layer internally pauses the stream by not sending a WINDOW_UPDATE frame to the server. This is a deliberate backpressure signal.
    3. The server is then halted from sending more data until it receives a new WINDOW_UPDATE frame.
    4. The stream resumes only after the JavaScript thread processes enough buffered messages to free up space, at which point the client sends a WINDOW_UPDATE frame to "refill" the window and signal the server to continue.

How does NAPI solve the problem?

N-API enables our Typescript SDK to leverage the performance and ecosystem of Rust underneath a JS access point.

Imagine the power of Rust with the ease and accessibility of JavaScript? BONKERS!

This way, connection configuration and management can be handled in Rust, deprecating the use of grpc-js for gRPC.

How we boosted our SDK while maintaining backwards compatibility using NaaE

After exploring different methods of interaction across Node and Rust, we have arrived at an optimal interaction between the two ecosystems that enables the following:

  1. Maximises compatibility with the greater Node ecosystem.
  2. Minimises boilerplate and glue code between Node and Rust.
  3. Maximises agile maintainability across high-performance teams by making changes and NAPI optimisations as decoupled as possible.

NAPI-as-an-Engine architecture

The core of NaaE is to deliberately optimise hot paths with Rust via napi-rs, while maintaining greater ecosystem compatibility in JS by utilising native interfaces as wrappers.

Implementation in the Typescript SDK

In our case, grpc-js was causing a slowdown on connection and runtime, bottlenecking the performance of our Typescript client.

We set out to improve this with the following key objectives:

  1. Maintain our previous client’s signatures to reduce migration friction as much as possible.
  2. Make the client performant by using Rust's asynchronous runtime.
  3. Make the code maintainable and easy to iterate on.

This was a heavy undertaking, given that performance and user experience rarely go together due to the nature of performance optimisations being a narrowing down of the solution domain, sacrificing generalizability, which leads to decreased user-friendliness.

However, with NaaE, maintaining user experience while squeezing out superior performance becomes possible.

For our TypeScript SDK, as an example, our main access point utilises Node’s standard event-driven pattern of emitter.on('event', callback) syntax.

Hence, our Rust implementation needed to adopt the same interfaces; otherwise, this would break code.

The trivial solution would be to mirror all expected interfaces in Rust; however, due to NAPI’s limitations, this will require a significant amount of code and will be difficult to maintain. The solution? NaaE.

Utilising Node’s native stream.Duplex() as a wrapper around our Rust Dragonsmouth gRPC client to replace the slow grpc-js connection management and runtime.

By doing so, the emitter.on('event', callback) syntax is maintained while the Rust implementation in napi-rs serves as the engine that enables our new SDK to be 400% faster than the old one.

Benchmarks and Verification

For our users, getting all the data as fast as possible is what matters. To measure that, we subscribe to all updates for all SubscribeRequest filters using the NAPI implementation and the older SDK, collect the responses over the same fixed testing window, and compare the total output size.

The NAPI implementation consistently returns more data within the same window, which is why it delivers higher throughput.

How this helps you

  1. The JS event-loop can now process your application logic faster, since it doesn't have to deal with gRPC at all. It gets served parsed JSON.
  2. Your applications won't be blocked by some internal SDK level halts. Above we mentioned how @grpc/grpc-js halts the gRPC stream to handle it's internal buffer; so while the one single thread is dealing with that full buffer:
      1. your application logic is not running
      2. new data is not being received
  3. This is no longer the case, now JS is always working on your application logic. Rust's async magic is now dealing with gRPC and giving parsed JSON to JS.

Getting started with the new SDK

NPM beta release: https://www.npmjs.com/package/@triton-one/yellowstone-grpc/v/5.0.1

Just bump the version in your package.json and you're all set to use it.

Changes you need to do

There's a code snippet at the end showing the changes. Here's an explanation

  1. Call client.connect() after client creation. Since gRPC connection is managed by Rust, we need to explicitly connect the client
  2. channel_options: ChannelOptions parameter are now camelCase and align with Rust params. For eg grpc.max_receive_message_length is now grpcMaxDecodingMessageSize. You can use IDE intellisense to see all the available options.
import Client from "@triton-one/yellowstone-grpc";
// OR
// import default as Client from "@triton-one/yellowstone-grpc";

async function main() {

  // Open connection.
  const client = new Client(args.endpoint, args.xToken, {
    // "grpc.max_receive_message_length": 64 * 1024 * 1024,
    grpcMaxDecodingMessageSize: 64 * 1024 * 1024
  });

  await client.connect(); // Add this line. Note that you need to await this

}

The new SDK code is open-source and available in the public repo.

Need more help? Open an issue here.