I manage a group of individuals that require regular updates from various APIs, each with its own unique rate limits.
To handle this, I have multiple cron jobs set up with a similar structure:
const cursor_i = db.collection('users').find();
while(await cursor_i.hasNext()){
let user = await cursor_i.next();
user.field_1 = 'New Information';
const write_result = await db.collection('newusers').save(user);
if(!write_result.result.ok){
console.log(write_result);
}
}
However, a problem arises when multiple updaters are modifying the same document simultaneously, as only the last save() call takes effect.
Consider the following scenario:
const cursor_1 = db.collection('users').find();
const cursor_2 = db.collection('users').find();
let user_cursor_1 = await cursor_1.next();
let user_cursor_2 = await cursor_2.next();
user_cursor_1.new_field_1 = 'ok';
const write_result = await db.collection('users').save(user_cursor_1);
if(!write_result.result.ok){
console.log(write_result);
}
// The first user now has a new_field_1 with the value 'ok'
user_cursor_2.new_field_2 = 'ok';
const write_result_2 = await db.collection('newusers').save(user_cursor_2);
if(!write_result_2.result.ok){
console.log(write_result);
}
// The first user now has a new_field_2 with the value 'ok', but new_field_1 is no longer present
As a result, despite receiving two updates, the first user only retains the changes from the second update.
I could implement my own locking mechanisms, but I suspect MongoDB has built-in solutions for handling these situations.