Working with Pipeline Concept in MongoDB

In the previous article we have discussed about aggregation framework. In the aggregation framework a certain concept named pipeline concept has a very important role to play. In this article we are going to discuss about that same pipeline concept in mongodb.

What is pipeline concept in mongodb?

In the UNIX command shell, pipeline means the possibility to execute an operation on some input and use the output as the input for the next command. So , we can say that pipeline concept is more or less a cyclic process of input and output. Mongodb also supports this concept in the aggregation framework. As per mongodb.org states pipeline is

The aggregation pipeline is a framework for data aggregation modeled on the concept of data processing pipelines. Documents enter a multi-stage pipeline that transforms the documents into an aggregated results.

How the pipeline works?

As we all know mongodb aggregation is a series of special operators applied to a collection. Whereas an operator is a JavaScript object with a single property, the operator name. Let us see a basic example:


{ $name: { /* options */ } }

Supported operator names are: $project, $match, $limit, $skip, $unwind, $group, and $sort, each with their own set of options. A series of operators is called a pipeline. In the below we have a basic example of the pipeline concept:


[{ $project: { /* options */ } },  { $group: { /* options */ } }]

Mongodb pipes operators into each other in time of executing a pipeline. “Pipe” means the output of an operator becomes the input of the following operator. The result of each operator is a new collection of documents. So, mongo executes the previous pipeline as follows:


collection | $project | $group => result

In pipeline concept we have many operators as we have displayed above. Let us look on them more thoroughly:

$project:
It selects some specific fields from a collection.
$match:
This is a filtering operation and thus this can reduce the amount of documents that are given as input to the next stage.
$group:
It is also somewhat same with the $match. This operator can also put the inputs in a certain group for executing all at once.
$sort:
It is used to sort the documents.
$skip:
It is possible to skip forward in the list of documents for a given amount of documents.
$limit:
It limits the amount of documents to look at by the given number starting from the current position.
$unwind:
This operator unwinds document that are using arrays. when using an array the data is kind of pre-joined and this operation will be undone with this to have individual documents again.

The $match and $sort pipeline operators can take advantage of an index when they occur at the beginning of the pipeline. The $geoNear pipeline operator takes advantage of a geospatial index.

The mongodb aggregation pipeline consists of stages. Each stage transforms the documents as they pass through the pipeline. Pipeline stages do not need to produce single output document for every input document.Pipeline stages can appear multiple times in the pipeline. Mongodb provides the db.collection.aggregate() method in the mongo shell and the aggregate command for aggregation pipeline.

Expressions in pipeline:

Some pipeline stage takes a pipeline expression as its operand. Pipeline expressions specify the transformation to apply to the input documents. Expressions have a document structure and can contain other expression. Pipeline expressions can only operate on the current document in the pipeline and cannot refer to data from other documents. Expression operations provide in-memory transformation of documents.In this article we are just mentioning about this topic.

Related Links:

1> How to Get Started with MongoDB Database?
2> How to Get Started with MongoDB?
3> How to Import and Export Through Mongodb?
4> How to Use Projection in MongoDB?
5> Using sort method in mongodb
6> Map-Reduce in MongoDB
7> Introduction to Replication in MongoDB
8> Deploying a Replica Set in MongoDB
9> Discussing Replication Lag in MongoDB
10> Replica Set Members in Mongodb
11> Working with Sharding in MongoDB
12> Working with Index in MongoDB
13> Working with Aggregation in MongoDB
14> How to Work with Aggregation Framework in MongoDB?
15> Discussing about Pipeline Expression in MongoDB

If you find this article helpful, you can connect us in Google+ and Twitter.

1 thought on “Working with Pipeline Concept in MongoDB

Leave a Reply

Your email address will not be published. Required fields are marked *