Retrieving data simultaneously from two MongoDB collections

I want to fetch data from 2 collections and combine them into a single array with merged data.

The solution that worked for me was:

const bothValues = await ValueA.aggregate([
      { $unionWith: { coll: 'valueB' } },
      { $sort: { rank: -1, _id: -1 } },
      {
        $match: {
          isAvailable: true,
        },
      },
      { $skip: skip },
      { $limit: 30 },
    ]);

This solution worked perfectly. However, the $unionWith operator is not supported in my MongoDB version (4.0.X), so I can't use it.

const bothValues = await ValueA.aggregate(
      [
        { $limit: 1 },
        {
          $lookup: {
            from: 'valueB',
            pipeline: [{ $limit: 15 }],
            as: 'valueB',
          },
        },
        {
          $lookup: {
            from: 'ValueA',
            pipeline: [{ $limit: 15 }, { $sort: { rank: -1, _id: -1 } }],
            as: 'ValueA',
          },
        },
        {
          $project:
          {
            Union: { $concatArrays: ['$valueB', '$ValueA'] },
          },
        },
        { $unwind: '$Union' },
        { $replaceRoot: { newRoot: '$Union' } },
      ],
    );

However, I now face two problems:

  • I am unable to use $skip, which is crucial. How do I incorporate it?
  • How do I utilize $match?

Thank you

javascriptarraysmongodbmongooseunion

Answer №1

Question

  • Your query has been modified to align with the requirements of the first query.
  • Both pipelines now include matching, sorting, and limiting (with specific values for limit and skip).
  • We extract 70 sorted documents from each pipeline to ensure we have enough data for the final sort/skip/limit after combining them.
  • Operations such as concatenation, unwinding, and replacing root are performed similar to your original query.
  • The resulting union is then sorted again, skipped, and limited accordingly.
  • This example assumes a scenario where skip=40 and limit=30, hence the adjustment in the first two pipelines to limit=70.
db.ValueA.aggregate([
  {
    "$limit": 1
  },
  {
    "$lookup": {
      "from": "valueB",
      "pipeline": [
        {
          "$match": {
            "isAvailable": true
          }
        },
        {
          "$sort": {
            "rank": -1,
            "_id": -1
          }
        },
        {
          "$limit": 70
        }
      ],
      "as": "valueB"
    }
  },
  {
    "$lookup": {
      "from": "valueA",
      "pipeline": [
        {
          "$match": {
            "isAvailable": true
          }
        },
        {
          "$sort": {
            "rank": -1,
            "_id": -1
          }
        },
        {
          "$limit": 70
        }
      ],
      "as": "valueA"
    }
  },
  {
    "$project": {
      "union": {
        "$concatArrays": [
          "$valueA",
          "$valueB"
        ]
      }
    }
  },
  {
    "$unwind": {
      "path": "$union"
    }
  },
  {
    "$replaceRoot": {
      "newRoot": "$union"
    }
  },
  {
    "$sort": {
      "rank": -1,
      "_id": -1
    }
  },
  {
    "$skip": 40
  },
  {
    "$limit": 30
  }
])

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Unable to perform the 'setSelectionRange' function on the 'HTMLInputElement' due to the input element's type being 'number', which does not allow selection

My attempt to enable text selection in an input box upon user click by using the code snippet below was unsuccessful: <input type="number" onclick="this.setSelectionRange(0, this.value.length)" name="quantity" /> Instead of achieving the desired ef ...

javascripthtmljquerycsshtml-input

error detection in AJAX response handler

My web-application was created using PHP, AJAX, and jQuery, and the development process went smoothly. The majority of the requests to the application are made via AJAX for operations such as insert, update, delete, and select. I have already implemented ...

javascriptphpjqueryajaxerror-handling

What could be causing the React text input to constantly lose focus with every keystroke?

In my React project using Material-UI library, I have a component called GuestSignup with various input fields. const GuestSignup = (props: GuestSignupProps) => { // Component code goes here } The component receives input props defined by an ...

javascriptreactjstypescriptmaterial-ui

Retrieve the specified time period with a fixed length that meets certain criteria in a MongoDB database using daily data

My goal is to compile a list of countries that have successfully reached 2 consecutive weeks with 0 new cases after surpassing 50 accumulated cases. For instance, a country has achieved 2 weeks in a row with no new cases reported and had more than 50 tota ...

mongodb

Tips on passing a variable into a controller using jQuery AJAX from the URL

Can you help me figure out how to pass an id to a function in the controller? I keep getting an error saying it cannot find the method in the controller. Here's my view: function Delete1(id) { if (confirm("Are you sure?")) { $. ...

javascriptc#jquery.netajax

Arranging Multiple Files in Sequence Using HTML5 File API Instead of Uploading All Simultaneously

I am currently working on a BackboneJS/Marionette App and I want to enable users to upload multiple files. Right now, the functionality works when users select multiple files simultaneously, but I would like to give them the option to select one file init ...

javascriptjqueryhtmlfile

Encountered an error: Unable to retrieve properties of an undefined value (reading '_id')

After logging in, I encounter a white page displaying the error ChatLogics.js:2 Uncaught TypeError: Cannot read properties of undefined (reading '_id'). Oddly enough, upon refreshing the page once, the error vanishes. Why is the _id undefined rig ...

node.jsreactjsmongodbexpressmern

Retrieving the value associated with a rank vector in Matlab

In my current project, I am working on assigning ranks to indices within an array (not the array itself) and then creating a second vector to store these ranks. For instance: data: x=[ 3 7 2 0 5 2] ranks: x'=[ 3 5 2 1 4 2] Once the ranks are assign ...

arraysmatlabsortingvectorranking

What methods are available to display Excel files in a web browser without any cost? (Specifically looking for solutions involving JavaScript and Python within a Django

When you click on the file, it automatically downloads. However, I would prefer it to open in a new tab, similar to how SharePoint works. Unfortunately, I cannot use SharePoint due to security concerns, as it would require access to all company files. htt ...

javascriptpythonhtmljqueryazure

Tips for managing asynchronous REST errors in unit tests with Jest

Below is the code snippet for my Node.js test file. This unit test case is failing, here are the details of the code and error message: jest.unmock('./utils.js'); describe('test', () => { it('test', async (done) => ...

javascriptnode.jsunit-testingjestjs

I am looking to choose the branch ID and location where the total sum of items exceeds 11 in a MongoDB database

entire_collection[{ "_id" : ObjectId("571ca6076e589915f893ee8c"), "cbill_details" : [ { "c_branch_id" : "b25", "c_branch_location" : "Kolkata", "c_total_item" : NumberInt(5) ...

mongodbaggregation-framework

Unable to reset session with JavaScript on JSP page

Created a session from the login.jsp page using a servlet String msg = ""; HttpSession sess = request.getSession(); // if(sess != null) //sess.invalidate(); if (sess.getId() != null) { sess.setAttribute("uname", ...

javascriptjavajspsessionservlets

employ the inverse Euler application

var a = new THREE.Euler( 0, 1, 1.57, 'YXZ' ); var b = new THREE.Vector3( 1, 0, 1 ); var c = b.applyEuler(a); In order to get the value of c using b.applyEuler(a), I am looking for the reverse operation involving applyEuler. Given that the value ...

javascriptthree.js

Expanding a container component with React TypeScript

I'm in the process of developing a foundational class, encapsulating it within a container, and extending it in my various components. Here's a basic example: let state = new State(); class Base extends React.Component<{}, {}> { } const ...

javascriptreactjstypescript

How to dynamically watch elements in an array using Vue.js whenever they change

I am currently developing a text editor that includes fields for name and address. <ckeditor :editor="editor" v-model="data[index].name"> <ckeditor :editor="editor" v-model="data[index].address.1"> <ck ...

arraysvuejs2

Encrypting and decrypting data using RSA in TypeScript

Currently, I am utilizing Angular 4 to develop the front end of my application. For authentication, I have integrated OAuth2 on the backend (which is created using Spring in Java), ensuring that only authorized individuals can access my app. However, ther ...

javascriptangulartypescriptencryptionrsa

What is the best way to create a loop that is constantly changing?

Below is a PHP array that I am working with: echo "<pre>"; print_r($notifications); /* output: Array ( [0] => Array ( [score] => 120 [type] => 5 [post_id] => 1 [subject] => ...

phphtmlarrays

serving JavaScript and various other files to HTML through a Node.js HTTP server

My web server setup is quite basic: var http = require('http'); var fs = require('fs'); http.createServer(function (req, res) { res.writeHead(200, { 'Content-Type': 'text/html' }); fs.readFile('./index.ht ...

javascriptnode.jshttp

The Object filter is experiencing a delay with processing 10,000 items

When an API returns over 10,000 objects in the format of {firstName:'john',lastName:'Cena'}, I am faced with a performance issue. In my parent React component, I make the API call in componentDidMount and pass this object to child compo ...

javascripttypescript

React JS: How to prevent Yup and Formik error messages from being displayed multiple times upon submission

I have implemented Yup and Formik in my Sign up form to handle validation. My goal is to display specific errors based on the Yup validation rules I've set up. Take a look at the code snippet below: import React from 'react'; import { ...

javascriptreactjsvalidationformikyup