I have a model called Posts in my mongoDB database, which contains an array of references to a model called Comments.
One of the get requests in my Express application requires me to respond with an array of all Posts in the database, where each Post includes an array of all its associated comments.
To achieve this, I first fetched all the Posts using the find method, resulting in an array of posts, each containing references to their respective comments as per the model structure.
To resolve these references, I utilized the populate method provided by mongoose, which replaced the references with actual comment objects from the database for a single post instance.
await post.populate("comments").execPopulate();Here, "post" represents a single post instance after resolving all its comment references into comment objects.
However, to perform this resolution for every post individually, I experimented with two methods. In both cases, allPosts refers to the array of retrieved posts from the initial find query.
- Initially, I used the execPopulate operation to create an array of promises for each post without waiting for their resolution. Subsequently, I used
Promise.all()to proceed with further operations once all promises were resolved.
let promiseArray = allPosts.map((post) => {
return post.populate("comments").execPopulate();
});
Promise.all(promiseArray).then((posts) => {
let responseArray = posts.map((post) => {
return {
comments: filterComments(post.comments),
_id: post._id,
title: post.title,
commentcount: countComments(post.comments),
};
});
return res.send(responseArray);
});
- Alternatively, I implemented a while loop where I waited for each post to be resolved individually in every iteration before formatting and pushing the post object into a response array. Once all posts were processed, I sent the response array back.
let responseArray = [];
let length = allPosts.length;
let counter = allPosts.length;
while (counter) {
let post = allPosts[length - counter]
await post.populate('comments').execPopulate();
responseArray.push({
comments: filterComments(post.comments),
_id: post._id,
title: post.title,
commentcount: countComments(post.comments),
})
counter--;
}
return res.send(responseArray);
When comparing the performance between these two approaches with around 150 posts, the process using the while loop took almost three times longer than the one utilizing Promise.all(). I am uncertain about the nuances of the JavaScript event loop; hence, I am seeking insights on what might be causing this difference in execution time.
TL;DR :
I have several async operations to execute on each item in an array; I tested:
Iterating through the array and storing unresolved promises in an array, then proceeding with further actions after using
Promise.all()on the promises arrayGoing through the array with a while loop, executing the async task for each element on every cycle
The while loop approach significantly increases processing time compared to resolving promises with Promise.all().