What is the most effective approach to updating data in MongoDB using an array of JSON objects?

My JSON file contains the following data:

[
   {"person_id": "3455666", "person_app": "bjjiu877y"},
   {"person_id": "5633444", "person_app": "rh5556ggg"},
   {"person_id": "9866654", "person_app": "66he4455"},
   ......
]
// The array length could range from 100k to 200k

In my person_collection, there is no field for person_app, and the identifier is person_id

I want to update the person_collection document to include the person_app

I attempted to achieve this using a JavaScript script in a separate file:

for( let {person_id, person_app} of dataFromJson) {
  db.person_collection.update({person_id}, {"set": {person_app}})
}

The updating process took a long time, and I am unsure if all the data was updated. How can I efficiently update this large dataset while ensuring that all the records are successfully updated? Thank you.

javascriptmongodbmongodb-query

Answer №1

If you want to efficiently update a large number of documents in MongoDB, consider using a Bulk operation like the following example:

let bulkOperations = [];
let count = 0;

for (let { personId, personData } of jsonData) {
   bulkOperations.push({ updateOne: { filter: { _id: personId }, update: { $set: personData } } });
   count++;
   if (count % 10000 === 0) {
      db.person_collection.bulkWrite(bulkOperations, { ordered: false });
      bulkOperations = [];
   }
}
if (bulkOperations.length > 0)
   db.person_collection.bulkWrite(bulkOperations, { ordered: false });

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Give Jquery a quick breather

My goal is to have my program pause for 3 seconds before continuing with the rest of the code. I've been researching online, but all I can find are methods that delay specific lines of code, which is not what I need. What I would like to achieve look ...

javascriptjquerysleepbreakpause

Mongoose encounters an issue: MissingSchemaError is thrown because the Schema has not been registered for the model

Encountering a perplexing error that I can't seem to resolve. I HAVE NAMED THE REF EXACTLY AS THE MODEL: MissingSchemaError: Schema hasn't been registered for model "ParticipantStatus". Here are the models in question: ParticipantStatus model: c ...

javascriptnode.jsmongodbmongoose

Troubleshooting a problem with scrolling functionality while keeping the header in place and ensuring the correct tab is highlighted

I've successfully created a page with a fixed menu that appears when scrolling, and the tabs activate based on their corresponding section IDs. However, I'm encountering some issues: The scrolling behavior is not precise; when scrolling to sec ...

javascriptjqueryhtml

Issue #98123 encountered during execution of `npm run develop` command, related to WEBPACK

I want to start a brand new Gatsby site following the instructions provided on . Here's what I did: npm init gatsby # see note below cd my-gatsby-site npm run develop Note: I didn't make any changes to the configuration, so I'm using JavaS ...

javascriptreactjswebpackgatsby

Unique events catered to specific devices: mobile vs. desktop

When a user clicks on the login icon from a desktop, I want a modal dialog to appear. However, when using smaller devices like mobile or tablets, I prefer that a reactjs route is triggered to display in the main content area instead of using a modal dialog ...

javascripthtmlcssreactjsresponsive-design

Ensuring consistency between TypeScript .d.ts and .js files

When working with these definitions: https://github.com/borisyankov/DefinitelyTyped If I am using angularJS 1.3.14, how can I be certain that there is a correct definition for that specific version of Angular? How can I ensure that the DefinitelyTyped *. ...

javascriptangularjstypescript

Obtain the native element of a React child component passed through props, along with its dimensions including height and width

Is there a way in React to access the native HTML element or retrieve the boundaries of an element passed as a child in props? Consider the following component: class SomeComp extends Component { componentDidMount() { this.props.children.forEa ...

javascriptreactjs

The ReactJS input box is stubbornly rejecting all input

Struggling with this code and can't seem to figure out why the input lines aren't accepting anything. After searching extensively, I decided it was time to ask for help. P.S. I am new to react class App extends React.Component { state = { inp ...

javascriptreactjsinputdata-binding

Retrieve the highest value indexes from a three-dimensional array with JavaScript

In my dataset, I have an array structured like this: [34, 12, 56] [100,125,19] [30,50,69] The highest value in this array is 125, so the index [1,1] is returned. This means that 125, the highest value, can be found in row 1, column 1. To determine the i ...

javascriptarrays

Exploring the potential of utilizing the "wait" function in Selenium WebDriver for

I want to automate a test on my website using the Selenium WebDriver for JavaScript. How can I approach running tests here with content that may not be ready when the page loads, such as data coming from an external API? In my case, the content is loaded ...

javascripthtmlseleniumselenium-webdriverauth0

Sending data with React using POST request

Currently in my React application, I have a form that includes fields for username and password (with plans to add "confirm password" as well). When submitting the form, I need it to send JSON data containing the email and password in its body. The passwo ...

javascriptnode.jsreactjspostaxios

Is there a versatile spell-checking tool available for HTML text boxes?

When designing a text box in HTML, I want to provide real-time input validation and spell check for the user's text. My goal is to underline any spelling mistakes as they type. This seems like a basic feature, but I've yet to find a plugin or AP ...

javascriptjqueryhtmlvalidationjquery-ui

Is there a way to create a password-like input in HTML that can also trigger keydown events?

Is there a way to record keyboard events on masked password input elements without compromising security? I want to assure everyone that my intentions are not malicious. I am simply looking for a way to analyze user behavior after informing them and ensur ...

javascripthtmlpasswordskeyboard-eventsmaskedtextbox

Transferring properties from React Router to child components on the server side

For my Isomorphic app using React, react-router v3, and material-ui, it's crucial to pass the client's user agent to the theme for server-side rendering. This is necessary for MUI to properly prefix inline styles. Initially, the root component o ...

javascriptreactjsreact-routermaterial-ui

Server not recognizing nested routes in React-Router

I am currently using Express.js to render a React application on the server side, but I have encountered an issue where some of my routes are getting skipped over unexpectedly. Here is my server.js: app.get('*', (req, res) => { conso ...

javascriptreactjsexpressreact-router

AngularJS closes when clicked outside

I've been attempting to add a close on outside click functionality to my code, similar to this example: http://plnkr.co/edit/ybYmHtFavHnN1oD8vsuw?p=preview However, I seem to be overlooking something as it's not working in my implementation. HT ...

javascriptangularjs

Verify the presence of plain ASCII text

I need to verify if the file uploaded is in ascii plain text format. Can you suggest a method? $("#my_file").change(function(){ //alert if file is not ascii plain text }); <input type="file" name="my_file" id="my_file" /> ...

javascriptjqueryhtml

The React array map function seems to be creating a duplicate of the array within an array of objects

When creating a movie application, the challenge arises when showtimes are embedded in an array of objects inside an array of movie objects. The issue is that the times are duplicating, causing the movie to render multiple times based on the number of show ...

javascriptreactjsarraysjsonobject

unable to bypass mongoose nested documents

I'm currently working with mongoose and node in an effort to paginate data from sub-documents. I've managed to limit the subdocuments, but skipping them seems to be a challenge. The specific versions I'm using are: Mongo 3.0.0 Node 0.10.3 ...

javascriptnode.jsmongodbmongoose

How can I conceal the contents of a webpage until the entire page has finished loading before revealing them?

Is there a way to delay displaying my login template until all elements are fully loaded? Currently, when I visit the site, the template appears first followed by fonts and other components. I want to ensure that nothing is shown to the user until the enti ...

javascripthtmlfunctionclassvue.js