Eliminating repeated entries in MongoDB

Hey guys, I've been struggling for days trying to eliminate all the duplicates from my database. I attempted a solution I came across on this Link, but encountered an error message stating that forEach is not a function. I'm puzzled as to why it works for others and not for me, even though the code is very similar. Here's the code I've been experimenting with:

exports = function(payload, response) {
  const mongodb = context.services.get("mongodb-atlas");
  var obj = EJSON.parse(payload.body.text());
  var inserimentoDB = mongodb.db("test").collection("test0").insertMany(obj);
  var duplicates = [];
  mongodb.db("test").collection("test0").aggregate([
    { "$group": {
        "_id": { "Val": "$Val" },
        "dups": { "$push": "$_id" },
        "count": { "$sum": 1 }
    }},
    { "$match": { "count": { "$gt": 1 } }}
]).toArray().forEach(function(doc) {
    doc.dups.shift();
    mongodb.db("test").collection("test0").remove({ "_id": { "$in": doc.dups } });
});
  }

I'm running this code within a stitch function

Answer №1

Just wanted to share my experience with a similar issue that I encountered. When utilizing the MongoDB Atlas platform, I found that testing the aggregation in the aggregation widget beforehand helped in ensuring the expected outcomes. Additionally, I made some adjustments to the code and opted to utilize .then() instead of .forEach().

For your situation, the revised code snippet would appear as follows:

collection.aggregate([
    { "$group": {
        "_id": { "Val": "$Val" },
        "dups": { "$push": "$_id" },
        "count": { "$sum": 1 }
    }},
    { "$match": { "count": { "$gt": 1 } }}
]).toArray().then(data => {
    var dr = data.map(d => d.name);
    console.log("duplicate Records:: ", data);
    
    collection.remove({ id: { $in: dr } }).then(removedD => {
        console.log("Removed duplicate Data:: ", removedD);
    })
});

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Control the scope value using an AngularJS directive

I have a model with values that I need to modify before presenting them to the user. I have checked the documentation but might be overlooking something. For example, I would like to format my variable in this way: <span decode-directive>{{variable ...

Exploring the comparison of information across two sets of databases using Node.js and MongoDB

In the realm of Node.js and MongoDB, I have set up two databases (1-btech2014 & 2-btechre2014) on a single server. My objective is to compare the data in btech2014 with that in btechre2014. If they match, I should retrieve the data from btech2014 as outpu ...

Dealing with cascading jQuery deferred callsHere are some guidelines on

When I have a function that needs to retrieve data and return a promise, I often find myself making multiple requests one after another. This results in nested deffered calls where the last call resolves on the deferred object that the function ultimatel ...

Ways in which the user can modify the city name within this inquiry

I am just beginning to learn JavaScript and I am struggling to figure out how to allow the user to change the city name in this request. Currently, it works when I manually input the city name in the code (e.g., askWeather.open("GET", "url.../london")), bu ...

An error occurred while trying to load the image due to an unexpected character appearing at the beginning of the JSON

Currently, I am facing an issue with storing an image in MongoDB and then displaying it back in Angular. Uploading the image is successful, however, I encounter an error when attempting to display it. The API, which works perfectly fine when tested in POST ...

Sending output from javascript back to html

<head> function displayProductName(name) { } </head> <body> <img src="...images/car.jpg" onclick="displayProductName('car')"> </body> How can I modify this javascript function to display the value received from t ...

Verifying user login on NodeJS through connection from an IIS-hosted website

I am currently upgrading an outdated CMS system and looking to implement a real-time chat feature. The existing CMS operates on IIS, MSSQL, and PHP. The chat feature will be hosted on a separate Linux box running Node.js and Socket.io After successfully ...

Injecting a service into an Angular constant is a straightforward process that involves

Is it possible to define a constant that utilizes the $locale service? Since constants are objects, injecting them as parameters like in controllers is not an option. How can this be achieved? angular.module('app').constant('SOME_CONSTANT&a ...

Pressing the reset button on the search box triggers a JQuery and HTML function

I am new to this, so bear with me. I've simplified my styling for easy pasting here, but the concepts still apply. I have a list of items, a working search field, and functioning filter buttons. What I want is for the filter buttons to reset to &apos ...

Using jQuery .css({}) is not causing negative margin to function as expected

$('#thankYouMessage').css({"height": textHeight, "margin-top:": "-52px", "padding-left": "19px"}); The CSS property 'padding-left:' will be applied as expected, but the negative margin will not take effect. The 'margin-top:' ...

Preventing JQuery from interrupting asynchronous initialization

I am currently developing an AngularJS service for a SignalR hub. Below is the factory code for my service: .factory('gameManager', [function () { $.connection.hub.start(); var manager = $.connection.gameManager; return ...

A guide to obtaining a single access token/refresh token for unlimited use with oauth2 on the Spotify API

I'm working on a project that will exclusively rely on one Spotify account, but I'm having trouble grasping the concept of the refresh token in oauth2. Ideally, I want to generate an access token and refresh token through the Spotify API console ...

Creating dynamic templates for table rows in AngularJS directives

Is it possible to dynamically load an AngularJS Directive templateUrl while working within a table? In my scenario, I have the following HTML structure where I am repeating a tr element with a fw-rule directive: <tbody> <tr ng-repeat="rule in ...

Using a combination of jQuery and JavaScript to switch image tags between different classes

I'm really struggling with this issue and could use some guidance. Essentially, I have a code that should change the class of an img tag when a user clicks on a div element. Let's take a look at the HTML structure: <li><img src ...

Combining both the Javascript and PHP components of the Facebook SDK

I recently integrated the JavaScript version of Facebook SDK, but now I'm unsure how to also integrate the PHP version without causing any conflicts. Here are my questions: If I implement the PHP SDK from Facebook, is there a way to detect if the J ...

A guide on how to effectively simulate a commonJS module that has been imported in

How can I successfully mock a commonJS style module in Jest that I am importing for an external connection? The module is called 'ganblib.js' and it's causing some trouble when trying to mock it with Jest. Any advice or suggestions would be ...

Having trouble adding a property to an object, any solutions?

I've been struggling with adding a property to a nested Object in my JavaScript code. Despite setting the property name and value, the property doesn't seem to persist. populated_post.comments[i].comment.end = true console.log(typeof(populated_po ...

VueJS emits a warning when filtering an array inside a for loop

I'm encountering an issue with my filtering function in VueJS. While it works fine, I am seeing a warning message in the console. You can check out this example to see the problem. The problem arises when I need to filter relational data from a separ ...

How to message someone privately in a public Discord channel using discord.js

Can someone help me figure out how to create a message in discord.js version 12.5.3 that only I can see? I know how to send messages to channels using message.channel.send, but I'm not sure how to make a message visible only to myself. Thank you! ...

Struggle arising from the clash between a customized image service and the built-in image element

Dealing with a custom Angular service called Image, I realized that using this name poses a problem as it clashes with Javascript's native Image element constructor, also named Image. This conflict arises when attempting to utilize both the custom Im ...