Issue with executing nested iteration through cursors in MongoDB shell

Question

Issue with executing nested iteration through cursors in MongoDB shell

When faced with two nested cursor.forEach() functions or a while loop, the second one doesn't seem to be executed. The issue arises when attempting to remove duplicates from a vast collection by transferring documents to another collection and checking for existing duplicates using the following code in the mongo shell:

var fromColl = db.from.find(),
    toColl;

fromColl.forEach(function(fromObj){
    toColl = db.to.find({name: fromObj.name});
    if (toColl.length() == 0) {
        //no duplicates found in the target collection, insert
        db.to.insert(fromObj);
    } else {
        //possible duplicates found in the target collection
        print('possible duplicates: ' + toColl.length());
        toColl.forEach(function(toObj){
            if (equal(fromObj.data, toObj.data)) {
                //duplicate...
            }
        });
    }
});

Although toColl.length() is printed in the else block, the second forEach loop fails to execute. Any idea why this might be happening?

javascript mongodb

Answer 1

Answer №1

--- ALTERNATIVE SOLUTION ---

After some experimentation, I devised a workaround by creating an array of the second cursor like so:

toColl = db.to.find({name: fromObj.name}).toArray();

. I then proceeded to iterate through this array using a basic JavaScript for loop:

var fromColl = db.from.find(),
    toColl,
    toObj;

fromColl.forEach(function(fromObj){
    toColl = db.to.find({name: fromObj.name}).toArray();
    if (toColl.length == 0) {
        // No duplicates found in the target collection, insert
        db.to.insert(fromObj);
    } else {
        // Possible duplicates found in the target collection
        print('Possible duplicates: ' + toColl.length());
        for (var i = 0; i < toColl.length; i++) {
            toObj = toColl[i];
            if (equal(fromObj.data, toObj.data)) {
                // Duplicate...
            }
        });
    }
});

--- UPDATE ---

In light of feedback from Stephen Steneker:

The mongo shell offers shortcuts for data manipulation within the shell environment. More information on these shortcuts can be found in the MongoDB documentation: Iterating Through a Cursor in MongoDB Shell.

One key point highlighted is that when the returned cursor isn't assigned to a variable using the var keyword, it automatically iterates up to 20 times to display the first 20 documents in the results.

In the provided code sample, the declaration of toColl with var preceded the execution of find().

Although iterating all results with toArray() is an option, it necessitates loading all cursor documents into memory. Manually iterating over the cursor is deemed a more scalable approach.

-- RESOLUTION --

A critical issue arose due to using toColl.length() instead of toColl.count().

This was because toColl.length() causes the cursor to reset.

Special thanks to Rhys Campbell and Stephen Steneker from the MongoDB user group for their valuable assistance in resolving this setback.

Answer 2

--- ALTERNATIVE SOLUTION ---

After some experimentation, I devised a workaround by creating an array of the second cursor like so:

toColl = db.to.find({name: fromObj.name}).toArray();

. I then proceeded to iterate through this array using a basic JavaScript for loop:

var fromColl = db.from.find(),
    toColl,
    toObj;

fromColl.forEach(function(fromObj){
    toColl = db.to.find({name: fromObj.name}).toArray();
    if (toColl.length == 0) {
        // No duplicates found in the target collection, insert
        db.to.insert(fromObj);
    } else {
        // Possible duplicates found in the target collection
        print('Possible duplicates: ' + toColl.length());
        for (var i = 0; i < toColl.length; i++) {
            toObj = toColl[i];
            if (equal(fromObj.data, toObj.data)) {
                // Duplicate...
            }
        });
    }
});

--- UPDATE ---

In light of feedback from Stephen Steneker:

The mongo shell offers shortcuts for data manipulation within the shell environment. More information on these shortcuts can be found in the MongoDB documentation: Iterating Through a Cursor in MongoDB Shell.

One key point highlighted is that when the returned cursor isn't assigned to a variable using the var keyword, it automatically iterates up to 20 times to display the first 20 documents in the results.

In the provided code sample, the declaration of toColl with var preceded the execution of find().

Although iterating all results with toArray() is an option, it necessitates loading all cursor documents into memory. Manually iterating over the cursor is deemed a more scalable approach.

-- RESOLUTION --

A critical issue arose due to using toColl.length() instead of toColl.count().

This was because toColl.length() causes the cursor to reset.

Special thanks to Rhys Campbell and Stephen Steneker from the MongoDB user group for their valuable assistance in resolving this setback.

Issue with executing nested iteration through cursors in MongoDB shell

Answer №1

Similar questions

Learn how to easily incorporate a drop-down list into the material-UI Search component within the navbar to enhance the search results

Can WebDriverJS be compiled without code minimization using Google Closure Compiler?

Tips for locating 2 geopoints in mongodb with the help of mongoose

What is the best way to serialize the file from a form for uploading and storing in a MongoDB database, utilizing Backbone.js, jQuery, and Node.js?

Promise not resolving when image onload event is not being triggered

How to create a karma-jasmine test case in Angular 2 to trigger a modal opening when a link is clicked

Having trouble with PHP Storm and Vue component not working? Or encountering issues with unresolved function or method in your

The mark-compacts were not efficient enough, they approached the heap limit and as a result, the allocation failed. The JavaScript

Using Javascript or Typescript constants along with WebPack being invoked twice

Employing the identical directive within a directive [angularjs]

The behavior of the Ionic checkbox in version 5 seems to be quite delayed

How to Insert Key-Value Pairs into CKEditor Selection Element

Find the mean of two fields by taking into account the entries from the previous 5 minutes on the specified datetime in MongoDB

Vue's beforeRouteEnter hook patiently awaits for the child component to complete its request

Setting up multiple base tags in AngularJS for different modules

Encountering a type error while trying to create a document within a nested collection in Firebase Firestore

What is the best way to send HTML content from a controller using ajax?

using angular.js to assign a value to an <input> field

Customizing response headers in vanilla Node.js

Retrieve all items pertaining to a specific week in the calendar