Unit Testing Node.js HTTP Network Requests using PassThrough Streams

Do you really need unit testing?

I use the mocha, chai and sinon unit test stack for my Node.js and frontend Javascript projects. Its a very powerful and user-friendly stack. Sinon is great for stubbing and spying on your unit tested codes callbacks and promise resolves.

If you have had the need to unit test functions that also make network requests using Node.js http or https modules you are faced with some complex logic paths for sinon mocking. As it is a network request, you need to ask yourself "do I really need to hit this API and test it's valid response?" or "am I happy to simulate the network request but test the callback logic?"

In your quest to extend the coverage of functions that may also do network requests you ideally may just want to mock the network request but test the callback of the request for valid execution flow.

Since of late I have added nock to the above mentioned test stack as well. nock is a brilliant tool to completely mock out your Node.js network requests. It's very declarative and abstracts all the complexities of setting up stubs and spys.

I will write out a seperate post on how to use nock, but I wanted to show how I used to unit test network requests prior to nock. This will be useful for people who prefer not to use nock or keep their unit testing as "vanilla" as possible.

To simulate and mock network requests I implement the very useful steam.PassThrough class provided natively by Node.js. In their documentation they describe it as:

The stream.PassThrough class is a trivial implementation of a Transform stream that 
simply passes the input bytes across to the output. Its purpose is primarily for 
examples and testing, but there are some use cases where stream. 
PassThrough is useful as a building block for novel sorts of streams.

Here is an example implementation of Unit testing a function that incudes a https get request using the mocha, chai, sinon and PassThrough tools. I have provided detailed comments in the code so hope that helps explaining what is going on.

const https = require('https');

// An example function that has other logic you need unit tested
// ... but you also want to cover the https call as part of your coverage 
function functionWithHttpsCall(apiInput) { 
  // wrap this whole aync function to be Promise based 
  return new Promise((resolve, reject) => {
    // do somthing with apiInput, update logic if needed etc

    // make an api call
    const request = https.get(`https://dummyapi.com?giveme=${apiInput}`, (response) => {
      let body = '';

      // construct stream response
      response.on('data', (d) => {
        body += d;
      });

      // stream ended so resolve the promise 
      response.on('end', ()=> {
        resolve(JSON.parse(body));
      });
    });

    request.on('error', (e) => {
      reject(e);
    });
  });
}

We need to now write a unit test to test the functionWithHttpsCall function above. We want to test all execution flows in this function to improve out code coverage so we also want to test the https.get callback response (without not actually testing live API response)

Here is the unit test for this test case:

// using mocha, sinon
const chai = require('chai');
const sinon = require('sinon');
const expect = chai.expect; // use chai.expect assertions
chai.use(require('sinon-chai')); // extend chai with sinon assertions

const https = require('https'); // the core https Node.js module
const { PassThrough } = require('stream'); // PassThrough class

// Unit Tests
describe('My App Tests - functionWithHttpsCall', () => {
  // do this before each test
  beforeEach(() => {
    // stub out calls to https.get
    // due to how npm caches modules, calls to https.get
    // ... from this point onwards will use the sinon stub
    this.get = sinon.stub(https, 'get');
  });

  // clean up afte each test 
  afterEach(() => {
    // restore after each test
    https.get.restore();
  });

  // begin test config
  let mockedInput = 'authors';
  let expectedOutput = {'authorName': 'john doe'};
  let actualOutout;

  it('should return a valid response when we hit a https call', (done) => {
    // create a fake response stream using the PassThrough utility
    const response = new PassThrough();
    response.write(JSON.stringify(expectedOutput)); // inject the fake response payload
    response.end();

    // create a fake request stream as well (needed below)
    const request = new PassThrough();

    // when the test crawl below for functionWithHttpsCall hits the stub (https.get)
    // ... respond with the mock response Stream in the index 1 param of the callback of https.get
    this.get.callsArgWith(1, response)
            .return(request); // calls to https.get returns the request stream so we need this to as well

    // unit test the makeHttpCall function
    functionWithHttpsCall(mockedInput)
      .then((actualOutout) => {

        // actual output (actualOutout) will be same as the (expectedOutput) variable above
        // ... because we used (expectedOutput) in the response PassThrough above
        
        expect(actualOutout).to.deep.equal(expectedOutput); // this test will pass

        done();
      });
  });
});

Hope the above makes sense you have an idea now on how you can use PassThrough to unit test your network calls.

But PassThrough does have its limitations, I have not worked out how to simulate various https status codes (404, 500) and also simulate network timeouts. These are the main reasons why I moved over to nock.

Happy coding!