NodeJS: Parsing Binary from the TCP Layer

For a job recently, I was asked to create a bridge server that translated the protocol of an old legacy server to a new HTML5 client, and vice versa. The old protocol was built upon C# structs, and the server was communicating in binary on the TCP layer. The new client will be communicating in JSON messages via TCP over HTTP (aka web sockets).

NodeJS comes nicely equipped with a module, net, for commuinicating via the TCP layer. Simply require('net') and you are now equipped to open a TCP socket. The module comes with a class Socket, and new socket connections can be instantiated when needed:

var tcpSock = require('net');  
var client  = new tcpSock.Socket;  
var socket  = client.connect(PORT, HOST);  

NodeJS also features a Buffer class, that will assemble raw binary data packets into an array-like object that can contain only integers. One instance of the Buffer class represents a specific, fixed size memory allocation outside of the V8 heap. I use it to collect incoming data packets, as well as to properly assemble my outgoing binary data.

var buffer = new Buffer(0, 'hex');

// listen for incoming data
socket.on("data", function(data){

    // a custom function for logging more readable binary
    logDataStream(data)

    // pack incoming data into the buffer
    buffer = Buffer.concat([buffer, new Buffer(data, 'hex')]);
})

The incoming binary was organized as C structs, and I am able to parse the data fairly easily using a great npm library called node-struct. The node-struct library allows the developer to define the structs in terms of javascript objects with strictly typed, pre-allocated fields. There are types for 8bit words through 32 bit, signed and unsigned, big and little endian. There are also fields for ascii and other encoded characters, as well as fields for other structs within structs. For example:

var Struct = require('struct').Struct;

function makeAndParsePersonFromBinary(buffer){  
  var person = new Struct()
                    .('word8', 'Sex')     // 0 or 1 for instance
                    .('word32Ule', 'Age')
                    .('chars','Name', 64);

  person._setBuff(buffer);
  return person;
};

var incomingPerson = makeAndParsePersonFromBinary(buffer);  

Once the struct has been seeded with the binary buffer, you can now access the values from the binary packets, properly parsed as fields on the struct:

var personName = incomingPerson.get('Name');  

As you can probably see, this node-struct library is a very useful tool when working in NodeJS with binary streams that represent structs.

The data is parsed and I'm ready to start working with it in my app.

Sometimes I just want to see the raw ints come in and out. FWIW, here's how I'm logging my incoming and outgoing binary streams:

function logDataStream(data){  
  // log the binary data stream in rows of 8 bits
  var print = "";
  for (var i = 0; i < data.length; i++) {
    print += " " + data[i].toString(16);

    // apply proper format for bits with value < 16, observed as int tuples
    if (data[i] < 16) { print += "0"; }

    // insert a line break after every 8th bit
    if ((i + 1) % 8 === 0) {
      print += '\n';
    };
  }

  // log the stream
  console.log(print);
}