Node.js: Socket.IO and Buffer quirks

As of late, I’ve been bitten by the node.js bug and have become distracted by a side-project with a co-worker. I must admit this is my first time writing any Javascript outside of simple jQuery DOM manipulations, so it has been quite an experience for me.

While my co-worker has written a vast majority of the code, I finally found some free time to dive in, and one of the first things I began playing with was finding a way to stream the output of a unix process to the view. Of course the obvious choice was to use WebSockets and Socket.IO and I quickly found a couple examples that made it look like it wouldn’t be too hard.

In my evening-long codefest, I came across two interesting quirks with Socket.IO/node that took up a majority of my time, and I thought I’d detail them here in case anyone else runs into similar issues:

Node Buffer

We’re currently using a WebWorker object to handle an out-of-request process, the output of which I want to stream to the view. Inside the worker I created some code that looked like the following (just as an initial test for using spawn):

  var my_process = spawn('command/to/run'), var stdout = '', var stderr = '';

  my_process.stdout.on('data', function(data){
    stdout += data;

  my_process.stderr.on('data', function(data){
    stderr += data;

This isn’t quite exactly what it was doing, but suffice it to say that the output I got wasn’t what I expected; it instead was this:

  console.log('stderr: ' + data); // stderr: Expected output from my command
  // Logging the data in the view directly without string concat gave this:
  console.log(data); // { 0: 23, 1: 82:, 3: 15, ... 60: 10, length: 61 }

Needless to say, I was quite confused, until I logged the output on the server side and saw this:

  <Buffer 0a 1b 5b 31 6d 70 72 6f 6a 65 63 74 5f 74 65 73 74 1b 5b 32 32 6d 0a>

A short google trip later and I found this in the node.js docs so I updated my code to call data.toString('ascii') and was rewarded with the output I expected.


The next time-consuming bug came when trying to stream the output from the process to the view. The build worker that handles running the process passes the output back to the controller, which then broadcasts it via the socket to the clients. My initial code looked something along these lines:


However, I was unable to call stdout or stderr off the object passed in the broadcast, and when I logged to the browser console I found this odd output:

  ~j~{'stdout':'This would be stdout output.','stderr':null}

At first glance I was confused, it looked like a JSON object or an object literal, but I had no access to any of the keys inside. I thought to myself “The secret must lie in the ‘~j~’ at the beginning,” but Google returned no relevant results regarding that. Eventually, after reading several other examples, I found a simple solution:

  // Server-side
  socket.on('message', function(msg){
    var data = JSON.parse(msg);

By doing this, I was able to access stdout and stderr off the data variable. I’m still uncertain what the object originally being passed to the view was, as it was neither JSON nor an object literal, but the solution got me where I needed to be. If anyone can explain it further, please do so; I’d love to know what’s going on behind the scenes here.

I think it highly likely that I’ve made some rookie mistakes here, but, as I stated before, I’m very new to Javascript so much of this is foreign to me. I’m still trying to get past the Ruby/OO mindset and to wrap my brain around the JS way of doing things, but I’m getting there. I’ll soon be reading Javascript: The Good Parts, which I hope will get me over some of these newbie difficulties I’m running into. If you have any tips or suggestions, or if you can point me towards any good guides on the subject, I’d love to hear from you.

subscribe! reddit! hacker news!

blog comments powered by Disqus
Fork me on GitHub