"MongoDB scripting issue: Some documents within an array not being generated without any error messages

After creating a long array of documents, I am encountering an issue where not all the documents are being successfully created.

const docsJson = [an array of JSON documents to create]
 
const orders = await MySchema.create(ordersJSON);
// The number of documents in 'orders' matches the number of documents in 'docsJson'

However, upon searching for the newly created documents, I am finding that only some of them have actually been created.


const actualOrdersCreated = await MySchema.find({ _id: { $in: orders.map((p) => p._id) } });
// The length of 'actualOrdersCreated' is less than the number of documents in 'docsJson'

What could be causing this issue?

Answer №2

The reason for the issue I encountered was a difference in the index settings between two MongoDB databases. When copying documents from one database to another, I noticed that the first database had a TTL (time to live) index, while the second database did not have this feature.

$ mongo "mongodb+srv://....database-1-url"
>> db.myschema.getIndexes()
[
        {
                "v" : 2,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_"
        },
        {
                "v" : 2,
                "key" : {
                        "paidOn" : 1
                },
                "name" : "paidOn_1",
                "background" : true
        }
]

In contrast, the database I was working with had an expireAfterSeconds index setting.

$ mongo "mongodb+srv://....database-2-url"
>> db.myschema.getIndexes()
[
        {
                "v" : 2,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_"
        },
        {
                "v" : 2,
                "key" : {
                        "expireAt" : 1
                },
                "name" : "expireAt_1",
                "background" : true,
                "expireAfterSeconds" : 86400
        },
        {
                "v" : 2,
                "key" : {
                        "paidOn" : 1
                },
                "name" : "paidOn_1",
                "background" : true
        }
]

This resulted in MongoDB deleting new documents where the expireAt field contained an old date.

To resolve the issue, I executed await Order.syncIndexes(); in a script, resetting the index setting to

[ { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } ]
. While this solved my immediate problem, it meant that the index structure differed from the first database as the paidOn key was no longer indexed.

My Initial Misconception

Initially, I attributed the problem to the large size of the jsonDocs.

Within these objects were fields containing extensive base64 strings for images, placeholders awaiting replacement with HTTP URLs for the images.

Although removing the base64 strings appeared to assist in uploading documents, in reality, it only accelerated the process. MongoDB still required one minute to check for expired documents.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Javascript animations failing to run sequentially

I am working on a circular design made up of 12 arc segments and would like to create a smooth transition from the start pattern to the end pattern for the user to view. There will be multiple start and end patterns to showcase. Check out my current code ...

PLSQL and HTML combine in Oracle Apex to create dynamic regions

Using Oracle Apex 4.2 with Oracle 12c For example: In my PL/SQL region in Apex, I have the following code to create a drop-down list: <button id="Clickme" onclick="myFunction()">Click me</button> <br> <select name="F01" class="my ...

Creating a star-based rating feature through directive implementation

Can anyone help me figure out why my static star rating system using angularjs/ionic is not showing up on the screen? I've been struggling with it and would appreciate some guidance. service.html <ion-list> <ion-item ng-repeat="busine ...

Adjust the dimensions of a Three.js-generated 3D Cylinder in real-time

Is it possible to dynamically change the dimensions of a 3D cylinder created using Three.js, similar to how we can adjust the size of a cube in this live example: http://jsfiddle.net/EtSf3/4/ Below is the code I have for the cylinder: HTML: <script s ...

PhoneGap 3.5.0 FileTransfer onprogress issue unresolved

I can't seem to get the onprogress event handler to work when downloading a file. The success callback is triggered and the download goes through successfully, but for some reason, the progress events are not firing. Does anyone see any issues with my ...

Searching for duplicated packages in npm

npm dedupe is known to simplify the folder structure, but I want to have a clear view of any duplicate packages before proceeding. Is there a way to generate a list of duplicate packages before running the command? If not, are there any scripts available ...

What is the best way to trigger a function with a bootstrap toggle?

A toggle switch has been implemented using bootstrap with the following code snippet: <label > Automatic Refresh: </label> <input class="pull-right" data-size="mini" type="checkbox" data- toggle="toggle"> When the toggle button is in ...

There was a TypeError encountered when attempting to read the length property of a null value in the Google Map JSON data

I keep getting an Uncaught TypeError: Cannot read property 'length' of null error when I attempt to use JSON data as markers on my Google map. However, when I check console.log(data);, it shows that the data is fine. What could be causing this is ...

Having trouble getting StencilJS Bottomsheet to work properly? Looking for a way to smoothly slide up your content?

I've been working on creating a Bottomsheet in Stencil, but I'm facing an issue where it shows up suddenly when activated. My goal is to display the overlay when the active property is set, and then smoothly slide up the content. Below is my comp ...

Node.js error: Unable to find address when making an HTTPS request to an API

I am in the process of creating a dashboard, and I need to connect to an API to retrieve an Auth Token in order to send information using HTTPS protocol. I am utilizing Node.js for this task, but when I run my code, I encounter the following error on the p ...

Guidelines for integrating historyAPI with AngularJS

I am currently working on developing a prototype similar to qz.com. Here is the strategy I am following: Create a function that loads the next piece of content when scrolling to the bottom of the page. This function will also update the URL using either ...

Creating a custom pipe that converts seconds to hours and minutes retrieved from an API can be achieved by implementing a transformation function

Can someone please provide guidance on creating a custom pipe in Angular 8 that converts seconds to hours and minutes? Thank you. <div class="col-2" *ngFor="let movie of moviesList"> <div class="movie"> {{ movie.attributes.title }} ...

What's the best method for securely handling user input containing iframes from a WYSIWYG editor?

I have implemented a feature using the TinyMCE WYSIWYG editor that allows users to input rich text content. Users have the ability to paste links to rich media in the editor, which automatically detects and creates an iframe display. This means that pastin ...

Is there a way to show a 'Refresh' icon in HTML without the need to download an image through HTTP?

In my HTML/JavaScript app project, I am looking to incorporate a 'refresh' symbol without having to load an image through HTTP requests. Are there reliable methods that can achieve this across all major browsers? I came across the Unicode value: ...

Clearing cookies in Express.js can be a challenging task

I've tried multiple options, but I can't seem to delete the cookie. Can anyone help me understand this? I have attempted using set maxage, expires, setheaders, but nothing seems to delete the cookie. It's impossible. res.cookie("refres ...

In what way does jQuery enable its array-like collection objects to be visualized as arrays in the browser console?

I am curious about the inner workings of jQuery and how it constructs its array-like object. I am particularly intrigued by how it tricks the console into interpreting it as an array and displaying it accordingly. While I understand that this may not offer ...

What are some strategies for creating a website that caters to both web crawlers and users of single page applications?

Let's get down to the specifics. I've got a frontend website at (don't worry, it's not porn). If you have javascript enabled, the URL becomes [this uses jquery/ajax load]. If you're without javascript, the URL remains as . To ...

Utilize React.js to embed an iFrame video using YouTube's JavaScript API

Despite scouring for examples, all I could find were youtube-react and npm packages that I'm not interested in using. My intention is to learn and implement it on my own. The issue lies with the guidance provided in the documentation found here: http ...

Should I generate a new object or utilize an already existing one?

When working with JavaScript and needing to replace existing data with new data, what is considered a good or "correct" practice? I currently utilize Dojo's Memory and dGrid to showcase my data. The information is refreshed each time the user clicks ...

Tips for managing and loading data into a dataGrid or table with the help of ReactJS and ReactHooks

Struggling to retrieve user input data from the form and display it in the table/Datagrid below, without success. Follow the process flow outlined below Once the user submits the form and clicks the send now button, the {handleSubmit} function is trigger ...