Analyzing two disorganized sets in MongoDB

I need to compare a large number of documents stored in two collections. In total, there are approximately 1300 documents in each of these collections.

My goal is to create a diff comparison report after comparing the two collections. My main focus is on identifying the differences between the documents rather than pinpointing specific additions or deletions. Each document has a unique identifier other than Mongo's ObjectId ("_id").

It's worth mentioning that my database is structured using a denormalized data model, which includes embedded documents (documents within documents).

What would be the most effective approach to implement a solution for this task?

Thank you in advance to all who are willing to assist!

javascriptmongodb

Answer №1

For optimal results, it is recommended to implement the $lookup function along with $eq across all relevant fields.

db.collection1.aggregate([
   {
      $lookup:
         {
           from: "collection2",
           let: { unique_id: "$unique_id", field1: "$field", field2: "$field", ... },
           pipeline: [
              { $match:
                 { $expr:
                    { $and:
                       [
                         { $eq: [ "$unique_id_in_2",  "$$unique_id" ] }
                         { $eq: [ "$field_to_match",  "$$field1" ] },
                         { $eq: [ "$field_to_match.2",  "$$field2" ] }
                       ]
                    }
                 }
              },
           ],
           as: "matches"
         }
    },
   {
     $match: {
         'matches.0': {$exists: false}
      }
   }
])

** This syntax is applicable for mongo 3.6 and above for executing lookup operations.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

JavaScript failing to accurately measure the length

Currently experiencing a Javascript issue where the length of an element is not displayed correctly when using .length, even though it shows up in Chrome console. Here is what it looks like in Chrome console <html xmlns="http://www.w3.o ...

javascripthtmlcssconsole.logvariable-length

Filtering HTML tables to display only one instance of each value

I have a 300x11(rowXcolumn) html table that I wanted to filter like Excel or Google Sheets's filter. Upon researching, I came across the code below on a website. While it works as desired, there is an issue where it displays the same value multiple ti ...

javascripthtmlfilterhtml-table

What is the process of transforming async/await code into synchronous code in JavaScript?

Blocking the event loop is generally considered bad practice due to its consequences. However, even the native fs module includes some synchronous functions for specific purposes, such as CLIs using fs.readFileSync. I am interested in converting the follo ...

javascriptnode.jsasynchronous

The module script failed to load due to an unexpected response from the server, which was MIME type of text/jsx instead of a javascript module script

I have recently set up an express server and created an API, as well as installed React using Vite for my frontend. However, when I attempt to connect or load my main HTML file to the server, an error is displayed in the console. This is all new to me as I ...

javascriptnode.jsreactjsmongodbexpress

Error: You cannot implement an import statement beyond a module while utilizing reactjs CDN Links

I am developing a Reactjs app using react CDN Links instead of 'npx create-react-app'. I have set up an index.html, index.js, and App.js files. My goal is to import the App.js component into the Index.js file using import App from '../compon ...

javascriptreactjsimportfrontend

Adding a background image in javascript using data from a MySQL database

My current tech stack includes CodeIgniter, vanilla JavaScript, AJAX, CSS, and MySQL. I am trying to figure out how to set the background of an image that is stored in a MySQL database. While the following code is error-free and working perfectly, my cha ...

javascriptphpcssajaxcodeigniter

Utilizing Prototype in Node.js Modules

Currently, I am working on a project involving multiple vendor-specific files in node. These files all follow a similar controller pattern, so it would be more efficient for me to extract them and consolidate them into a single common file. If you're ...

javascriptnode.jsprototypecommonjs

Combining round brackets and square brackets when initializing an array

In the snippet below, values are assigned with a mix of parentheses and square brackets without any errors. However, most other combinations (such as parentheses inside square brackets) do not work at all. var myItems = []; myItems[5] = ("A1", "B1", ["C1" ...

javascript

Querying MongoDb to calculate the time interval between switching on and off alternately

{ metadata:{ eventcode:100 } power:on // the power status can vary between on and off time:1667984669//unix timestamp } My document is structured in a way that indicates whether the power is on or off, and I need to calculate the total number of hours it ...

mongodb

Is it advisable to send a response in Express.js or not?

When working with Express.js 4.x, I'm unsure whether to return the response (or next function) or not. So, which is preferred: Option A: app.get('/url', (req, res) => { res.send(200, { message: 'ok' }); }); Or Option B: ...

javascriptnode.jsexpresstypescript

Manipulating HTML output with JavaScript

I am in the process of creating a website that retrieves and displays data from a mySQL database using PHP. When I toggle a switch, I want the displayed data to be refreshed with a new search query. Below is the Javascript function for the switch: functi ...

javascriptphpjqueryajax

Incorporate Stripe Elements into your Nuxt Js application

I recently managed to integrate Stripe into my React + Spring Boot application by following the guidelines provided in this documentation: https://stripe.com/docs/stripe-js/react. I used it in my React class component. Now, I am transitioning to Nuxt from ...

javascriptvue.jsnuxt.js

Building a Dynamic Checkbox Validation Feature in Angular Using Data retrieved from an API

Currently, I have a function that retrieves and displays a list obtained from an API: displayEventTicketDetails() { this.Service .getEventTicketDetails().subscribe((data: any) => { this.eventTicketDetails = data.map(ticket => ticket. ...

javascripthtmlangulartypescript

AngularJS module dependencies configuration functions smoothly during initialization

The structure of my modules looks like this: angular.module('mainModule',["cityModule", "countryModule"]); angular.module('mapModule',[]); angular.module('cityModule',["mapModule"]); angular.module('countryModule',[ ...

javascriptangularjsangularjs-directive

JavaScript - combining elements of an array of time

I've encountered a challenge where I need to calculate the total time duration from an array that contains time durations like ['00:30', '01:30', '03:00', '04:30']. The code snippet I'm using for this task ...

javascriptarraystime

Tips for avoiding background color interference with raycaster

In my current three js scene, I have a ground, sky, and various objects. I want specific objects to change color to red when the mouse hovers over them, but not all objects should do this. Currently, everything I touch turns red, which is not what I want. ...

javascriptthree.jsraycasting

There seems to be a problem with the Chart property getter as it

I'm in the process of creating an object that corresponds to chartJS's line-chart model <line-chart :data="{'2017-05-13': 2, '2017-05-14': 5}"></line-chart> There is an API I utilize which returns a standard arra ...

javascriptchartsvue.js

Ensure that both textarea and pre elements have equal dimensions and line wrapping behavior in Internet Explorer

I am in the process of implementing a dynamic textarea that resizes automatically, based on a technique discussed in this article: Alistapart: Expanding Textareas (2011). The idea is quite straightforward: by using a pre element to mirror the input in the ...

javascripthtmlcssinternet-explorer

Error encountered during file download with AXIOS: TypeError - 'validateStatus' in blob cannot be searched using 'in' operator

I recently encountered an issue with a Vue app when attempting to download a CSV file using Axios, and I am unsure of how to resolve it. downloadFile: function(res = "") { axios .get('/api/file/' + res, 'blob') ...

javascriptvue.jsaxiosblob

Tips for maintaining an active class on parent nav items in NextJS when navigating dynamic routes

In my nextjs-application, I've implemented a Navbar showcasing several navitems: Navbar.tsx: const Navbar = ({ navitems }) => { return ( <div> {navitems?.map((navitem, idx) => ( <NavItem key={idx} navitem={nav ...

javascriptreactjsnext.js