Exploring MongoDB - when we need to peel back nested subdocuments

Consider the following dataset:

lists

{ _id: 1, included_lists: [ 2 ], items: [ "i1" ]}
{ _id: 2, included_lists: [], items: [ "i2", "i3" ]}

items

{ _id: "i1", details: [{}, {}, {}]}
{ _id: "i2", details: [{}, {}, {}]}
{ _id: "i3", details: [{}, {}, {}]}

I am trying to retrieve all the items for a list, including those attached to the included_lists

For instance, if we are examining list with _id 1, we should get items i1, i2, i3

My approach involves using populate or $lookup, but I'm uncertain about how to unwind the nested items inside the included_lists and connect them with the original list's items.

Ultimately, I want to work with a dataset where I can utilize limit, skip, and match.

I am utilizing mongoose, however raw mongodb code would also suffice.

Update

My current strategy is to first fetch all the list ids in one query like so:

List.find({ _id: id}, { included_lists: 1})

Then, create an array of all those ids:

var all_ids = [id, ...included_lists]

Subsequently, find the items and unwind them

Pseudo-code:

List
    .aggregate([
        {
            $match: {
                _id: {
                    $in: all_ids
                }
            }
        },
        { $lookup: {} }
        {
            $unwind: "$items"
        },
        {
            $project: {
                "list.name": 1,
                "list._id": 1,
                "items": 1
            }
        }
    ])

However, I wish to avoid having to execute an initial query to retrieve all the list_ids, rather I should be able to fetch all related items through just one _id which will then enable me to access the remaining items via included_lists

Answer №1

If you are using MongoDB version 3.6 or higher, you can experiment with the following aggregation query:

List.aggregate([
  { "$match": { "_id": id }},
  { "$lookup": {
    "from": Items.collection.name,
    "let": { "items": "$items" },
    "pipeline": [
      { "$match": { "$expr": { "$in": [ "$_id", "$$items" ] } } }
    ],
    "as": "items"
  }},
  { "$lookup": {
    "from": Lists.collection.name,
    "let": { "included_lists": "$included_lists", "items": "$items" },
    "pipeline": [
      { "$match": { "$expr": { "$in": [ "$_id", "$$included_lists" ] } } },
      { "$lookup": {
        "from": Items.collection.name,
        "let": { "items": "$items" },
        "pipeline": [
          { "$match": { "$expr": { "$in": [ "$_id", "$$items" ] } } }
        ],
        "as": "items"
      }},
      { "$project": { "allItems": { "$concatArrays": [ "$$items", "$items" ]}}}
    ],
    "as": "included_lists"
  }},
  { "$unwind": "$included_lists" },
  { "$replaceRoot": { "newRoot": "$included_lists" }}
])

Answer №2

Here is an aggregation method you can experiment with in version 3.4.

To start, utilize the $lookup operation to fetch values from items in included_lists. Then, use $concatArrays to merge these looked up items with existing ones.

Follow up with a second $lookup to retrieve item details, and apply $unwind to flatten the results.

List.aggregate([
{"$lookup":{
  "from":name of the list collection,
  "localField":"included_lists",
  "foreignField":"_id",
  "as":"included_items"
}},
{"$unwind":"$included_items"},
{"$project":{"allItems":{"$concatArrays":["$items","$included_items.items"]}}},
{"$lookup":{
  "from":name of the item collection,
  "localField":"allItems",
  "foreignField":"_id",
  "as":"lookedup_items"
}},
{"$unwind":"$lookedup_items"},
{"$skip": some number},
{"$limit": some number}
])

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Leveraging mongodb's $nin in conjunction with and

My app is designed for managing hackathon events and this is the mongodb schema structure { "_id": "cZpRg4Qa8JPRAJojE", "lat": "-1.981388262628872", "lng": "36.29487934112549", "eventownernumber": "0800123456", "eventownernames": "Krau ...

It is not possible to nest a <Router> within another <Router>. It is recommended to only have one <Router> in your application at a time

Below is the snippet of my code from index.js file: import React from "react"; import { render } from "react-dom"; import "./index.css"; import App from "./App"; import * as serviceWorker from "./serviceWorker&q ...

Transforming SQL queries into MongoDB queries using a join operation with id-1

I want to modify the following SQL query: select a1.opis_ruchu as poprzednik, a2.opis_ruchu as nastepnik, count(a1.opis_ruchu) as ilosc_wystapien from public.informacje a1 join public.informacje a2 on a1.id = a2.id-1 group by a1.opis_ru ...

What is the best way to show HTML code from an HTML file in a Vue template?

I need help showcasing the HTML code from an external file in a Vue component template. <template> <div class="content"> <pre> <code> {{fetchCode('./code/code.html')}} & ...

Create independent SVG files using Meteor and iron-router

My goal is to use Meteor and Iron-Router to serve dynamic SVG files with templating capabilities. To start, I create a new route: @route 'svg', { path: '/svg/:name' data: -> { name: this.params.name } # sample data layoutT ...

How can you effectively connect an image with a mongodb entry for optimal performance?

This is my first real project that involves working with Express and MongoDB. Transitioning from my Udemy course to this backend-heavy project has raised several questions for me. The goal of my project is to create a mock online store where items I' ...

Tips for adjusting image hues in Internet Explorer?

I have successfully altered the colors of PNG images in Chrome and Firefox using CSS3. Here is my code: #second_image{ -webkit-filter: hue-rotate(59deg); filter: hue-rotate(59deg); } <img src='http://i.im ...

Iterate over the key-value pairs in a loop

How can I iterate through a key-value pair array? This is how I declare mine: products!: {[key: string] : ProductDTO}[]; Here's my loop: for (let product of this.products) { category.products.push((product as ProductDTO).serialize()); } However, ...

How to pass a function parameter as a property in MongoDB and Typescript

I've been working on a project that involves using Mongoose to write, update, and perform other operations in a MongoDB database. Currently, I am utilizing the updateOne() function within my own custom function. However, I am facing an issue where if ...

Vue: Customize data based on userAgent

As a newcomer to VUE, I am attempting to dynamically modify the disabled value based on the userAgent in order to display or hide the paymentMethod: data() { return { paymentMothods: [ { name: 'Visa che ...

Is there an equivalent of gsub in JavaScript?

Is there a way to achieve functionality similar to Ruby's gsub in JavaScript? I am working with a local HTML file and need to replace specific template variables with content, but I am struggling to find a solution. The HTML code includes elements lik ...

Highlighting neighboring components with the same marker when hovering in Slate JS: a step-by-step guide

// custom mouse over hook function useMouseOver() { const [mouseIsOver, setMouseIsOver] = useState(false); return { mouseIsOver, triggers: { onMouseEnter: () => setMouseIsOver(true), onMouseLeave: () => setMouseIsOver(false), ...

Activate mouseover function automatically for each feature on the leaflet

I'm facing a challenging situation where I need to develop a dashboard application that is similar to the functionality of ChoroplethExample. However, the catch is that I have to loop through all states (Features in geoJSON) and pause for 3 seconds at ...

Error encountered during the execution of the store method in Ajax CRUD operation

Greetings, I'm encountering an error in my AJAX code every time I try to execute the store function https://i.sstatic.net/SW86I.jpg Below is my controller: public function store_batch(Request $request) { $rules = array( 'batch_name& ...

Creating a Button with Icon and Text in TypeScript: A step-by-step guide

I attempted to create a button with both text and an icon. Initially, I tried doing it in HTML. <button> <img src="img/favicon.png" alt="Image" width="30px" height="30px" > Button Text ...

Node.js: Implementing Mongoose to filter data from an array of ObjectIDs within a collection

My goal is to search using node.js, ejs, and mongoose. So far, all the filter parameters are functioning correctly except for categoryIds. This specific parameter (stored as a collection of ObjectIDs in the MongoDB document) always returns an empty record ...

The Mean.js platform seems to be experiencing issues as it is unable to establish a connection

SCENARIO: I have downloaded the repository from this link: https://github.com/meanjs/mean After following and executing all instructions, I ran the command $ npm start and encountered the following error: ERROR: Could not connect to MongoDB! { Mon ...

Getting the ajax response in PHP while editing a popup is a common requirement in web development

When the edit button is clicked, I want a popup to appear with all selected values displayed using ajax. My response is sent in this format: $data = array('cdid' => $model->cdid, 'cid' => $model->cid, 'icdcode' = ...

Effective approach for managing a series of lengthy API requests

I am developing a user interface for uploading a list of users including their email and name into my database. After the upload process is complete, each user will also receive an email notification. The backend API responsible for this task is created u ...

Increase the CSS integer by a set value at regular intervals with the help of JavaScript

I am looking to create a simulated loading icon that gives the illusion of loading for some calculations, but in reality, it loads quickly. My plan is to increment the animation every second until it appears "complete". I am using CSS3 animations, with a h ...