Table of contents
1.
Introduction 
2.
Types of Streams in Node.Js
3.
Buffers in Streams
4.
Stream and Event Emitters
5.
Flowing and Non-flowing 
6.
Buffer Management
6.1.
How to create a buffer?
7.
Frequently Asked Questions
7.1.
What are Stream and Buffers?
7.2.
What does the method Buffer.from() do?
7.3.
What are the different types of streams?
8.
Conclusion
Last Updated: Mar 27, 2024
Medium

Streams and Buffer in Node.Js

Author Rubleen Kaur
0 upvote
Career growth poll
Do you think IIT Guwahati certified course can help you in your career?

Introduction 

We require streams in Node Js to handle and manipulate the streaming data like a video, a large file, etc. The streams module in Node.js is used to manage all the streams. A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us. 

For instance, both are considered stream instances if we request an HTTP server and process.stdout. Streams can be readable, writable, or both. All streams are instances of EventEmitter

To access the stream module, the syntax to be used is:

const stream = require('stream'); 
You can also try this code with Online Javascript Compiler
Run Code

 

The stream module is used to create new types of stream instances. 

 

Node Js Logo

Types of Streams in Node.Js

In NodeJs, there are four different types of streams available for us to use. We have four main types of streams available. Let’s see each of them with one-liner explanations.

  1. Readable streams → This stream is used to create a stream of data for reading, for example: to read a large chunk of files.
    Example:
const fs = require('fs');

const readableStream = fs.createReadStream('./article.md', {
    highWaterMark: 10
});

readableStream.on('readable', () => {
    process.stdout.write(`[${readableStream.read()}]`);
});

readableStream.on('end', () => {
    console.log('DONE');
});
You can also try this code with Online Javascript Compiler
Run Code

 

  1. Writable streams → This stream is used to create a stream of data to write. For example: to write a large amount of data in a file.

    Example: 
const fs = require('fs'); 
const file = fs.createWriteStream('file.txt'); 
for (let i = 0; i < 10000; i++) 
{ 
file.write('Hello world ' + i); 
}
file.end(); 
You can also try this code with Online Javascript Compiler
Run Code
  1. Duplex streams → This stream is used to create a stream that is both readable and writable at the same time.

    Example:
const server = http.createServer((req, res) => {
    let body = '';
    req.setEncoding('utf8');
    req.on('data', (chunk) => {
        body += chunk;
    });
    req.on('end', () => {
        consol.log(body);
        try {
            // Send 'Hello World' to the user
            res.write('Hello World');
            res.end();
        } catch (er) {
            res.statusCode = 400;
            return res.end(`error: ${er.message}`);
        }
    });
}); 
You can also try this code with Online Javascript Compiler
Run Code
  1. Transform streams → This stream is used to create a readable and writable stream, but the data in the stream can be modified while reading and writing to the stream.

    Example: Creating a transform stream for calculating a value.
var crypto = require('crypto');
var stream = require('stream');
var util = require('util');
var Transform = stream.Transform ||
  require('readable-stream').Transform;

function codingninjas(options) {
    if (!(this instanceof codingninjas)) {
    return new codingninjas(options);
  }


  Transform.call(this, options);

  this.digester = crypto.createHash('val1');
}
util.inherits(codingninjas, Transform);

codingninjas.prototype._transform = function (chunk, enc, cb) {
 
  var buffer = (Buffer.isBuffer(chunk)) ?
    chunk :
    new Buffer(chunk, enc);
  this.digester.update(buffer); 

  cb();
};

codingninjas.prototype._flush = function (cb) {
  this.push(this.digester.digest('hex'));
  cb();
};


// try it out
var codingninjas = new ShaSum();
codingninjas.pipe(process.stdout); // output to stdout
codingninjas.write('hello world\n'); // input line 1
codingninjas.write('another line');  // input line 2
codingninjas.end();  // finish
You can also try this code with Online Javascript Compiler
Run Code

Buffers in Streams

In Node Js, buffers are used to store raw binary data. A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes. Now, the next question is, what a byte is correct? A byte is a sequence of eight bits. Bits are a basic storage unit for your computer; these bits are used to a  value that can be either 0 or 1. 

An example for buffer in Node.js looks like this: 

<Buffer 81 2e 71 3b 65 2e 31 2f 61 2e> 
You can also try this code with Online Javascript Compiler
Run Code


In the above example, we can see ten pairs of letters and numbers. Each pair is used to represent a byte that is stored in the buffer. The total size of the buffer is 10.

Stream and Event Emitters

EventEmitter is a class that helps us to create a publisher-subscriber pattern in NodeJS. Using the event emitter, we can simply raise a new event from a different part of an application, and a listener will listen to the raised event and have some action performed for the event.
Let’s consider an example for Event Emitters:

var emitter = require('events').EventEmitter; 
var em = new emitter(); 
//Subscribe FirstEvent 
em.addListener('FirstEvent', function (data) 
{
 console.log('First subscriber: ' + data); 
});
 //Subscribe SecondEvent 
em.on('SecondEvent', function (data) 
{ console.log('First subscriber: ' + data); 
}); 
// Raising FirstEvent 
em.emit('FirstEvent', 'This is my first Node.js event emitter example.');
 // Raising SecondEvent 
em.emit('SecondEvent', 'This is my second Node.js event emitter example.');
You can also try this code with Online Javascript Compiler
Run Code

Flowing and Non-flowing 

There are two types of readable streams in NodeJs: 

  • Flowing stream — A stream used to pass data from the system and provide this data to your programs. 
     
  • Non-flowing stream — The Non-flowing stream that does not push data automatically. Instead, the non-flowing stream stores the data in the buffer explicitly calls the read() method of the stream to read it. 

Buffer Management

Buffers are developed for developers who use binary data. Buffers were introduced where conventionally allocated with the strings rather than binaries. Buffers are connected deeply with the streams, and this means that when a stream processor accepts data faster than it can digest, it sends the data to a buffer.

How to create a buffer?

There are three different ways to allocate a buffer using the Buffer API, which is used to create a new buffer, methods like Buffer.from()Buffer.alloc(), and Buffer.allocUnsafe() are used. 

We can create a void buffer with a size of 12 bytes using the given command.

const buf1 = Buffer.alloc(12);
You can also try this code with Online Javascript Compiler
Run Code


We can also create a buffer using the UTF-8 encoded strings:

const buf2 = Buffer.from('Coding Ninjas!');
You can also try this code with Online Javascript Compiler
Run Code

There are the encodings accepted while creating a buffer. We have listed a few of them :

  • ASCII
  • UTF-8
  • Base64
  • Latin1
  • Binary
  • Hex

The last method that is used to create a buffer is given below:

const buf3 = Buffer.allocUnsafe(22);
You can also try this code with Online Javascript Compiler
Run Code

Frequently Asked Questions

What are Stream and Buffers?

A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us.  A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes.

What does the method Buffer.from() do?

The Buffer.form() method is used for creating a new buffer that contains a specified string, array, or buffer itself.

What are the different types of streams?

In NodeJs, there are four different types of streams available. 
→ Readable Streams
→ Writable Streams 
→ Duplex Streams
→ Transform Streams

Conclusion

Hey everyone, so let’s brief out the article. Let’s discuss in brief whatever we have discussed here. 

  •  In this article, we have briefed about what Streams and Buffers are. We require streams in Node Js to handle and manipulate the streaming data like a video, a large file, etc. The streams module in Node.js is used to manage all the streams. A stream is an abstract interface used to work along with the streaming data in Node.js. There are many stream objects,Node.js offers us.
     
  • We have discussed different types of the four available streams available for developers. Along with these types of streams, we have covered what buffers are and how to manage buffers in the NodeJs.In Node Js, buffers are used to store raw binary data. A buffer represents a chunk of memory that is allocated on our computer. The size of the buffer, once set, cannot be changed. A buffer is used to store bytes.
     

Isn’t Web Development engaging? Building new websites and using amazing animations and different APIs, don’t be scared if you cannot grasp the hold of this vast development. We have the perfect web development course for you to make you stand out from your fellow developers. 

Recommended Readings:


Do check out The Interview guide for Product Based Companies as well as some of the Popular Interview Problems from Top companies like Amazon, Adobe, Google, etc. on Coding Ninjas Studio.

Also check out some of the Guided Paths on topics such as Data Structure and Algorithms, Competitive Programming, Operating Systems, etc. as well as some Contests, Test Series, Interview Bundles, and some Interview Experiences curated by top Industry Experts only on Coding Ninjas Studio.

Happy Learning Ninjas! 

Live masterclass