Searching for 10 random data records within a collection of 100 in MongoDB

Can MongoDB retrieve a specific number of random documents in one query?

For example, I currently load all the documents in the collection on the JavaScript side, which is inefficient. I'm wondering if there's a better way to achieve this with a single database query.

Here's the approach I'm using on the JavaScript side:

  • Retrieve all data
  • Create an array of IDs
  • Shuffle the array of IDs (random order)
  • Take the desired number of documents from the shuffled array
  • Select the documents based on the IDs remaining in the array from the collection

The major drawbacks of this approach are that I'm either loading all the data or making multiple queries.

Any suggestions would be greatly appreciated.

Answer №1

In the past, this question was answered, but MongoDB has since undergone significant advancements in its capabilities.

As mentioned in another response, MongoDB now includes sampling within the Aggregation Framework starting from version 3.2:

To achieve this, you can use the following code:

db.products.aggregate([{$sample: {size: 5}}]); // Select 5 documents

Or:

db.products.aggregate([
  {$match: {category:"Electronic Devices"}}, // Filter the results
  {$sample: {size: 5}} // Select 5 documents
]);

However, there are some cautions regarding the $sample operator:

(as of Nov, 6th 2017, with the latest version being 3.4) => If any of the following conditions are not met:

  • $sample is the first stage of the pipeline
  • N is less than 5% of the total documents in the collection
  • The collection contains more than 100 documents

If any of the above conditions are NOT met, $sample performs a collection scan followed by a random sort to select N documents.

Similar to the previous example with the $match

OUTDATED ANSWER

You could previously run:

db.products.find({category:"Electronic Devices"}).skip(Math.random()*YOUR_COLLECTION_SIZE)

However, the order won't be random, and you will require two queries (one for counting YOUR_COLLECTION_SIZE) or estimating its size (around 100 records, 1000 records, 10000 records, etc.)

Another approach would be to add a field with a random number to all documents and query based on that number. The drawback here is that you'll get the same results every time you run the query. To address this, you can play with limit, skip, sort, or update the random numbers when fetching a record (leading to additional queries).

--It's uncertain whether you're using Mongoose, Mondoid, or directly the Mongo Driver for a specific language, so I'll focus on the mongo shell.

For instance, your product record might appear as follows:

{
 _id: ObjectId("..."),
 name: "Awesome Product",
 category: "Electronic Devices",
}

I recommend using:

{
 _id: ObjectId("..."),
 name: "Awesome Product",
 category: "Electronic Devices",
 _random_sample: Math.random()
}

Then you could execute:

db.products.find({category:"Electronic Devices",_random_sample:{$gte:Math.random()}})

Periodically, you could update the _random_sample field in documents like this:

var your_query = {} // This may impact performance with many records
your_query = {category: "Electronic Devices"} // Update 
// Upsert = false, multi = true
db.products.update(your_query,{$set:{_random_sample::Math.random()}},false,true)

Alternatively, you could update all retrieved records or only a few after fetching them, depending on the number of records:

for(var i = 0; i < records.length; i++){
   var query = {_id: records[i]._id};
   // Upsert = false, multi = false
   db.products.update(query,{$set:{_random_sample::Math.random()}},false,false);
}

EDIT

Keep in mind that

db.products.update(your_query,{$set:{_random_sample::Math.random()}},false,true)

may not function optimally as it would update all products matching your query with the same random number. The last approach works more efficiently (updating documents as you retrieve them).

Answer №2

Starting from version 3.2, there is a simpler method for obtaining a random sample of documents from a collection:

$sample Introducing in the latest 3.2 release.

It randomly picks the specified number of documents from its input.

The syntax for the $sample stage is as follows:

{ $sample: { size: <positive integer> } }

Reference: MongoDB Documentation

In this scenario:

db.products.aggregate([{$sample: {size: 10}}]);

Answer №3

Here is the final outcome of my work:

let totalItems = 10;


// FETCH LIST OF ALL ID's
SchemaNameHere.find({}, { '_id': 1 }, function(error, fetchedData) {

    if (error) res.send(error);

    // Shuffling the array, reference: https://github.com/coolaj86/knuth-shuffle
    let randomizedArray = shuffle(fetchedData.slice(0));

    // Selecting only the first 'totalItems' from the shuffled array
    randomizedArray.splice(totalItems, randomizedArray.length - totalItems);

    // Creating a new array to store all items
    let newItemsArray = [];

    // Utilizing async each, guide: http://justinklemm.com/node-js-async-tutorial/
    async.each(randomizedArray, function(selectedItem, callback) {

        // Retrieving items one by one and adding them to the newItemsArray
        SchemaNameHere.findById(selectedItem._id, function(error, retrievedData) {

            if (error) res.send(error);
            newItemsArray.push(retrievedData);

            // Proceed to the next item, or to the subsequent function upon completion
            callback();

        });

    }, function(error) {

        // Executed upon looping through all items in the array
        res.json(newItemsArray);

    });

});

Answer №4

Unfortunately, the skip method did not yield the desired outcome for me. Instead, I ended up using the following code snippet:

let randomDocument = db.getCollection("collectionName").aggregate([ {
    $match : {
// specify criteria for filtering matches
    }
}, {
    $sample : {
        size : 1
    }
} ]).result[0];

This code snippet retrieves a single random result that meets the specified criteria.

Answer №5

Using a random sample may not always yield the desired outcome. One alternative approach is to implement a function on the server side that randomizes the data before returning it. This way, you can ensure a more diverse set of results compared to a direct fetch from the database.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Is there a way to modify Style Properties in JavaScript by targeting elements with a specific class name using document.getElementsByClassName("Class_Name")?

I am seeking a solution to change the background color of multiple div boxes with the class name "flex-items" using JavaScript. Below is my current code: function changeColor(){ document.getElementsByClassName("flex-items").style.backgroundColor = "bl ...

AngularJS Error: Attempting to Access Undefined Object - Jasmine Testing

Encountering an error while running Jasmine tests when trying to mock/spy on an API call in the tests. Below is the code snippet responsible for calling the API: UploadedReleasesController.$inject = ['$log', '$scope', '$filter&apo ...

Differences between count() and length() methods in Protractor

When it comes to determining the number of elements inside the ElementArrayFinder (which is the result of calling element.all()), you have two options according to the documentation: $$(".myclass").length, detailed here: This approach involves using ...

Component in Next Js fails to pass props when using getInitialProps

I am facing some challenges with Next Js and could really use some assistance. We have a component where I am utilizing "getInitialProps" to pass props to it during the server-side rendering process. However, no matter what I try, the props do not seem to ...

How can we utilize a loop to continuously sum up numbers until we reach a multiple of another number, let's say when the total is divisible by 4?

I am trying to create a function in JavaScript that will detect when a given number is not a multiple of 4. If the number is not a multiple of 4, I want to add numbers incrementally until it reaches the closest multiple of 4. Here’s what I have so far: ...

Experience the advanced NgPrime p-dropdown template with templating, filtering, and a clear icon that collapses when wrapped within a form element

Check out this link for a demo Whenever I enclose the code below in a </form> element, the drop-down menu collapses. <h5>Advanced with Templating, Filtering and Clear Icon</h5> <p-dropdown [options]="countries" [(ngModel)]=& ...

Utilizing Vue CLI plugin to dynamically pass JS variables as props

I am currently using version 4.5 of the Vue CLI plugin. I have created a component that takes in a title as a prop. This component has been converted into a web component and added to an HTML webpage (which is not a Vue JS project) Now, I am attempting to ...

$set { "array.variable.value" :"newvalue"} utilize a different term besides "variable" or implement a new variable

When working with mongoose to store user data, one of the attributes is an array called items. In my additems.js file: const User = require('../models/User'); var array = user.items; array.indexOfObject = function (property, value) { ...

What are the steps to start using the Intersection Observer API right away?

Utilizing the Intersection Observer API, I can accurately determine whether an element is within the viewport or not. Is there a way to utilize the Intersection Observer API to detect if an element is in the viewport without relying on a callback function ...

Discover content within nested arrays - angularJS

I have a function written in angularJS that utilizes find to verify the presence of an item in an array; function checkCartItem(item) { return $scope.cart[0].cart_items.find(function(itm) { return itm.item_id === item.item_id }); } The fu ...

Implement the addition of a class upon hovering using AngularJS

My goal is to include a new class when hovering over the li element using Angular in the code snippet below. <li ng-mouseenter="cola-selected=true" class="pull-left" ng-class="{'selected' : cola-selected}"> <a href="interna.html"> ...

The issue persists with the ajax.reload() function in DataTables

It's been driving me crazy that my datatables table won't refresh, despite using ajax.reload. I've spent weeks on this code but still can't get it to work. DataTablesDraw = (selector, order, pages, file, sort, column, template, data_se ...

Disable Vue click event based on a condition

One issue I encountered was that even though I successfully disabled the document body scroll when my mobile nav was open, the overflow hidden feature was not being removed when a user clicked a link to another route or page. To address this, I implement ...

What is the process for generating a dynamic array in typescript?

Looking to create a TypeScript dynamic array with the desired format: const display = [ { id: 1, displayName: "Abc1" }, { id: 2, displayName: "Abc2" }, { id: 3, displayName: "Abc3" } ] Attempted the following code ...

Updating HTML5 localStorage value upon a click

Currently, I have implemented a countdown timer using the provided code snippet. To prevent the count from resetting upon page refresh or reload, I am utilizing local storage. However, if there is an alternative solution to achieve this functionality, I wo ...

What causes the mounted hook in Vue to be triggered multiple times when used within a plugin or mixin?

How can I prevent repetitive behavior in my code? Is this a bug that needs fixing? Take a look at the plugin below: const globala = { install(Vue) { Vue.mixin({ mounted() { console.log('hi') } }) } } And here&apos ...

Execute sequential animations on numerous elements without using timeouts

I'm currently working on developing a code learning application that allows users to write code for creating games and animations, similar to scratch but not block-based. I've provided users with a set of commands that they can use in any order t ...

Updating Hidden Field Value to JSON Format Using jQuery and JavaScript

var jsonData = [{"Id":40,"Action":null,"Card":"0484"}]; $('#hidJson', window.parent.document).val(jsonData); alert($('#hidJson', window.parent.document).val()); // displays [object Object] alert($('#hidJson', window.parent.doc ...

Using NodeJS with the Express framework to send a GET request

Can express be used as a client-module for making http-requests to another server? Currently, I'm handling requests like this: var req = http.get({host, path}, function(res) { res.on('data', function(chunk) { .... } } This ...

How can I update the image source using Angular?

<div class="float-right"> <span class="language dashboard" data-toggle="dropdown"> <img class="current" src="us-flag.png" /> </span> <div class="dropdown dashboar ...