Using express, neat-csv, and mongoose has led me to a scenario where I need to verify the existence of a series of users when a batch register call is made from a .csv file. The current process works smoothly as long as all information is input correctly, but it only catches errors inconsistently. To test this, I intentionally uploaded a .csv file that does not meet the requirements (some users already exist), resulting in most executions running fine and returning the list of existing users.
However, there are instances where it randomly returns the completed array of users anyway. Upon inspecting my code, I noticed that the issue lies in the following section:
//data consists of the data fetched from neat-csv, allUsers represents an array of user IDs, and I check the database for existing users before proceeding further
let Failed = false;
allUsers.forEach(async (id, index) => {
await User.findById(id)
.then(async users => {
console.log('User', users);
console.log('indexes', index, allPartners.length - 1)
if (users === null && index !== allUsers.length - 1) return Failed = true;
else if (users && Failed === false && index === allUsers.length - 1) {
console.log('submitted indexes', index, allPartners.length - 1)
data.forEach((user, i) => {
console.log('Failed', Failed);
if (i !== data.length - 1) user.addValue = 'newValue';
else res.json(data).status(200);
})
} else if (index === allUsers.length - 1) {
// process a failed search
} else return Failed = true;
})
.catch(err => res.json({message: 'Failed to send a compatible .csv file.'}).status(401));
});
The issue is specifically with
else if (users && Failed === false && index === allUsers.length - 1) {[0] User {object}
[0] Failed false
[0] indexes 2 2
[0] submitted indexes 2 2
[0] User null
[0] Failed false
[0] indexes 0 2
[0] User null
[0] Failed true
[0] indexes 1 2
The root cause seems to be related to the asynchronous nature of JavaScript when dealing with nested loops. I attempted to incorporate some async/await methods, but they did not produce the desired outcome. Is there a way to ensure that JavaScript waits for the first loop to finish before proceeding? This would avoid the need for such deeply nested loops. Alternatively, is there a method to guarantee synchronous execution of the loops?
I am open to suggestions using ES6+ or Bluebird, but any advice would be valuable at this point.