Node.js from the ground up

Node.js (Node) is an application runtime that enables you to develop JavaScript applications that run out of the browser.

To get started with Node, download and install it - you can find relevant instructions for your platform at https://nodejs.org. When it is installed on your computer, run the following command to check that it's working and that the node and npm commands are on your system's path.

$ node -v
$ npm -v

If all is well, these commands will output version numbers for the node and npm programs on your system.

Hello, world!

With Node installed, you're ready to start. Let's use the classic Hello, world example to build your first Node application. Create a new file using your favorite text editor and name the file hello.js. Add the code below to the file.

hello.js source code

console.log('Hello, world!');

Using your system's command prompt or terminal, navigate to the directory where you saved the file, and use the following command to run it.

$ node hello.js

You should see the following output

Hello, world!

A simple Web server

One of the primary reasons for Node's huge popularity is how easy it makes building network applications. The first example on the Node official website shows how you can build a simple Web server in just six lines of code.

simplewebserver.js source code

var http = require('http');
http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');

If you save this code in a file and run it like you did the previous example, you'll see the following output.

Server running at http://127.0.0.1:1337/

Open your browser and point it at http://127.0.0.1:1337/ and you should see a plain-text response "Hello World". Let's explore what's going on in this code a little deeper.

var http = require('http');

This line loads a core Node module named http, which makes building servers that respond to HTTP requests very simple. Node is an open source project, so you can read the source code behind the http module (and all other Core modules). The easiest way to do this is to view it on the Node project's GitHub repository at https://github.com/joyent/node/blob/master/lib/http.js.

http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
}).listen(1337, '127.0.0.1');

Here, the createServer method of the http module is being called with a callback function passed as the sole argument. This function will be executed any time a HTTP request is made to the server. Its contents simply writes a HTTP response, with the status code 200, which means OK, a MIME type of text/plain and a response body of the text Hello World followed by a newline character. Finally, method chaining is used to call the listen method on the newly created server, passing 1337 in as the port number and 127.0.0.1 as the address to listen for requests on.

The final line simply outputs a message to the console that the server is now running at the given address.

A simple TCP server

Although Node is commonly used for Web projects, you can also create servers for other protocols. The second example on the Node official website shows how to build a simple TCP echo server, again in just six lines of code.

simpletcpserver.js source code

var net = require('net');
var server = net.createServer(function (socket) {
  socket.write('Echo server\r\n');
  socket.pipe(socket);
});
server.listen(1337, '127.0.0.1');

Run this example with:

$ node simpletcpserver.js

but this time open a new terminal window and connect to the server using the telnet utility.

Tip

If you get an error when you try to run this example, check that you have killed the Web server you created previously. As both examples use the same port number, you won't be able to run them simulatenously on your computer.

$ telnet 127.0.0.1 1337

You should see output similar to the following:

Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Echo server

Enter some text and press the enter key. The server will echo the same message back to you. Not the most useful server ever conceived, but a functional one nonetheless. Let's dig into the code.

var net = require('net');

Similar to the Web server example, here you are using a core Node module named net to do some of the heavy lifting for you. You can find the source code for this module at https://github.com/joyent/node/blob/master/lib/net.js.

var server = net.createServer(function (socket) {
  socket.write('Echo server\r\n');
  socket.pipe(socket);
});

This code uses the createServer method of the net module to create a new TCP server. Its sole argument is a callback function which will be executed whenever a client connects to the server. This function receives the client socket as an argument, and writes out a message Echo server immediately on connection. It then uses the socket.pipe method to send any messages received from the socket back to it, hence producing the echo.

server.listen(1337, '127.0.0.1');

Finally, the listen method tells the server to listen for incoming connections on port 1337 at the address 127.0.0.1.

Working with I/O

If you have worked with JavaScript before, one of the parts of Node that will be new to you is working with input and output. Node can access the file system, accept input, command-line arguments and many other things that you won't have had access to while writing JavaScript that runs in a browser.

Let's begin this section by examining how you can use command-line arguments in a Node application. The code below should be added to a file named args.js.

args.js source code

var args = process.argv,
    len = args.length;

if(len < 3) {
  console.log('Missing argument: name');
  return;
} else if(len > 3) {
  console.log('Too many arguments: 1 argument expected');
  return;
}

console.log('Hello ' + args[2]);

To run this example, run it as before but add your first name to the end of the command.

$ node args.js Joe

This will produce the following output:

Hello Joe

If you leave out the command-line argument, you will see the following:

Missing argument: name

If you provide more than one argument, the output will be as follows:

Too many arguments: 1 argument expected

Let's explore what's going on in this code.

var args = process.argv,
    len = args.length;

The process object is available by default to all Node applications and is used to provide information about the current Node process. In this case, the code finds the argv property, which is an array that contains the arguments passed to the runtime. This is assigned to a variable named args, and a len variable is created that holds the size of this array.

if(len < 3) {
  console.log('Missing argument: name');
  return;
} else if(len > 3) {
  console.log('Too many arguments: 1 argument expected');
  return;
}

This code performs some simple validation on the arguments passed to the application. First it checks if there are less than three arguments, and second it checks if there are more than three. You're probably wondering why this code is expecting three arguments instead of one. This is because the argv property also contains the name of the runtime (node) and the path to the application executed (in my case /Users/joelennon/Code/node_basics/args.js) as the first two items in the array.

console.log('Hello ' + args[2]);

Here, the application outputs a message to the console. If you passed the value Joe as the argument then you will see the message Hello Joe.

For the next example, let's see how to read a file from the filesystem. Create a new text file named file.txt. Add some text to this file, such as:

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi viverra dolor quis odio 
mollis, eu posuere nisi finibus. Pellentesque non bibendum elit. Vivamus ullamcorper orci 
enim, eu auctor nisl commodo ut. Donec non ornare odio, vel vulputate nulla. Sed iaculis 
lobortis dolor, a blandit mi fermentum quis. Vivamus rutrum eu tortor ut vehicula. 
Suspendisse venenatis tincidunt convallis. Proin tortor nunc, congue eu dui vel, tempus 
pellentesque libero.

Next, create a file named readfile.js in the same directory you created file.txt. Add the code below to this file.

readfile.js source code

var fs = require('fs'),
    path = require('path'),
    file = path.resolve(__dirname, 'file.txt');

var buf = fs.readFileSync(file);
console.log(buf.toString());

When you run this application, the contents of the file.txt file will be output to the terminal. Let's take a closer look.

var fs = require('fs'),
    path = require('path'),
    file = path.resolve(__dirname, 'file.txt');

Here we are loading two core Node modules - fs (for file system) and path. We then create a new variable named file and use the path module to get a full path to the file.txt file using the __dirname global variable.

var buf = fs.readFileSync(file);
console.log(buf.toString());

This code uses the readFileSync method of the fs module to read the contents of the file.txt file. This reads the file synchronously, which will block the Node event loop. This means the application will not move on to the next line of source code until the file has been read successfully. It then saves the resulting Buffer object in a variable named buf. Finally, it outputs the string representation of this object to the console.

Asynchronous code

In the previous section, you learned how to read a file from the filesystem synchronously. One of Node's primary benefits is that it runs on an event loop in a single thread, making it excellent for applications that require high levels of concurrency. When you use synchronous functions, these cause the event loop to be blocked until the function has completed execution, preventing anything else from working in the process. To prevent this, you should use asynchronous functions whenever possible.

The fs core module includes both synchronous and asynchronous versions of its primary functions. Let's see how you can modify the previous example to use the asynchronous version. Create a file named asyncread.js and save it in the same location as the file.txt file you created in the previous section. Add the code below to this new file.

asyncread.js source code

var fs = require('fs'),
    path = require('path'),
    file = path.resolve(__dirname, 'file.txt');

fs.readFile(file, function(err, buf) {
  console.log(buf.toString());  
});

If you run this application, you should see the same result as before. This time, however, the code to read the file has been executed asynchronously. This means that it will not have blocked the event loop, and other code can continue to run while the file is being read. The main consequence of this is that the function no longer returns a value that can be used as it will return before the file has finished reading. Instead, it will execute a callback function when it has completed, and pass the result to this function as an argument.

The first three lines of this application are the same as the previous example, so ignoring those, let's take a look at what's changed.

fs.readFile(file, function(err, buf) {
  console.log(buf.toString());  
});

This time around, the readFile method of the fs module is executed. This is an asynchronous method, so we don't assign its return value to a variable. Instead, we pass a second argument, an anonymous callback function, which receives two arguments - err (for error) and buf for the Buffer that is returned as a result of the function. We're assuming that there is no error in this instance (never do this in production!) and simply logging the result to the console.

Asynchronous code can take a bit of getting used to. It can also get quite complex when you need to nest asynchronous method calls within one another. Thankfully there are some excellent modules that make working with asynchronous code much more manageable. I'll cover these in a future post.

Streams

One of the most important aspects of Node's event-driven I/O is the concept of streams. At a basic level, streams allow you to pipe readable streams to writable streams, much like how Unix pipes work. This allows you to chain commands together, where the data will flow through each command as it is emitted.

To understand the benefits of streams, let's consider a simple example. Suppose we have a CSV file that has been compressed using gzip and we want to write a program that takes this file, decompresses it and saves the output to another file.

One approach to this might be the following:

Synchronous approach

var fs = require('fs'),
    zlib = require('zlib');

var gzippedFile = fs.readFileSync('data.csv.gz');
var data = zlib.unzipSync(gzippedFile);
fs.writeFileSync('data.csv', data);

This approach synchronously reads the file, unzips the data and writes the output to a file. As we learned in the previous section, however, this will block the event loop while running, leading to poor concurrent performance. What does the code look like if we use asynchronous methods instead?

Asynchronous approach

var fs = require('fs'),
    zlib = require('zlib');

fs.readFile('data.csv.gz', function(err, buf) {
  zlib.unzip(buf, function(err, buf) {
    fs.writeFile('data.csv', buf, function() {});
  });
});

This approach is non-blocking, which is an imporvement, but there are still a couple of issues with it. You'll find that this code is quite difficult to read due to the nested callback functions. Also, this code will perform poorly with large files as each command will execute on the entire file at once.

Let's look at how we can resolve both of these issues using streams.

Streams approach

var fs = require('fs');
var zlib = require('zlib');

fs.createReadStream('data.csv.gz')
  .pipe(zlib.createGunzip())
  .pipe(fs.createWriteStream('data.csv'));

This program uses the fs module to read the file data.csv.gz as a readable stream. It then pipes the data emitted by this stream to the zlib.createGunzip method, which decompresses the data before piping it back to the fs module which writes the decompressed data to a file named data.csv.

This code is much easier to read than the nested callbacks from the prior approach. More importantly, it will perform much better with large files as the stream methods used will process the data in chunks rather than on the entire file. When each command is finished with a chunk, it will pass it along to the next command in the pipe until all chunks have been processed.

Modules

Node has an elegant module system that is both simple and powerful. There are two types of Node module:

Core modules - these are modules that are built-in to Node itself.
File modules - these are modules that are loaded from files.

It is very straightforward to import modules in Node. In fact, most of the prior exmaples in this post have loaded various core modules including fs, zlib and http. To import a module, you use the require method.

var http = require('http');

This method will load the http module into a local variable named http. Loading file modules works in a similar way. You simply specify the path of the module you wish to load. For example, if I had a file module named example.js in the same directory as my Node program, I can load it using:

var example = require('./example');

When you install Node, you'll also install a package manager for Node modules called npm. This makes is very easy to download third-party modules and use them in your program. When you use npm to download a module, it will be placed in a special subdirectory in your project named node_modules. When you specify a path for a module, Node will automatically check this subdirectory for the module you are looking for.

Let's use npm to download a module now and use it in a program. For the sake of simplicity, let's download the colors module, which allows your programs to output text to the terminal in color. First, create a new directory for this example, and navigate to this directory in your terminal or command prompt. The first thing we will do is initialise npm in our project directory:

$ npm init -f

This will create a file in your project directory named package.json, that npm uses to track various details about a module.

Tip: The -f flag passed to npm init tells it to skip asking various questions about your module and use the default values instead. Unless you are intending on publishing your program as a module in an npm repository, you don't need to worry too much about these values.

Next, to install the colors module, run the following command:

$ npm install colors

You should see a message similar to the following when it is done:

colors@1.1.2 node_modules/colors

Note: If you see WARN messages about package.json missing some fields, don't worry, you can safely ignore these.

If you explore your project directory, you'll notice that there is a node_modules subdirectory and in here there is a subdirectory named colors that contains the source code for the module.

Next, let's use the module in a program. Create a new file named rainbow.js in your project directory, and add the following to it.

rainbow.js source code

var colors = require('colors');

console.log(colors.rainbow('Check out the rainbow!'));

Run the program:

$ node rainbow.js

You'll see the message Check out the rainbow! displayed with a rainbow color effect. In spite of the contrived example, this module can actually be very useful if you want to stylise terminal output from your application such as logs and information messages.

This section only scratches the surface of modules. I'll go into more detail about modules including dependencies, versions, global installation and how to write your own later in a later post.

Summary

As you have seen in this post, Node.js is a powerful runtime that is easy to get started with. The goal of this post was to introduce to key Node concepts such as I/O, asynchronous callbacks, streams and modules in a way that is easy to understand. I'll dig into all of these concepts in much more detail later in a future post.