Cloud function for Firestore to recursively update a subcollection or collection group

I've been working on this cloud function:

import pLimit from "p-limit";


const syncNotificationsAvatar = async (
  userId: string,
  change: Change<DocumentSnapshot>
) => {
  if (!change.before.get("published") || !change.after.exists) {
    return;
  }

  const before: Profile = change.before.data() as any;
  const after: Profile = change.after.data() as any;
  const keysToCompare: (keyof Profile)[] = ["avatar"];
  if (
    arraysEqual(
      keysToCompare.map((k) => before[k]),
      keysToCompare.map((k) => after[k])
    )
  ) {
    return;
  }

  const limit = pLimit(1000);

  const input = [
    limit(async () => {
      const notifications = await admin
        .firestore()
        .collectionGroup("notifications")
        .where("userId", "==", userId)
        .limit(1000)
        .get()

      await Promise.all(
        chunk(notifications.docs, 500).map(
          async (docs: admin.firestore.QueryDocumentSnapshot[]) => {
            const batch = admin.firestore().batch();
            for (const doc of docs) {
              batch.update(doc.ref, {
                avatar: after.avatar
              });
            }
            await batch.commit();
          }
        )
      );
    })
  ];

  return await Promise.all(input);
};


Is there a way to recursively update the notifications collection while limiting the query to 1,000 documents at a time (repeating until all documents have been processed) and then perform a batch.update? I'm concerned about potential timeouts due to the collection's potential growth over time.

Answer №1

Sharing a solution that I've implemented, slightly deviating from the question's context but can be easily adapted to fit. Hopefully, this will benefit others.

import * as admin from "firebase-admin";

const onQueryResults = async (
  query: admin.firestore.Query,
  action: (batchNumber: number, documentSnapshots: admin.firestore.QueryDocumentSnapshot[]) => Promise<void>
) => {
  let batchNumber = 0;
  const recursiveFunction = async (startingDoc?: admin.firestore.DocumentSnapshot) => {
    const { documents, isEmpty } = await (startingDoc == null
      ? query.get()
      : query.startAfter(startingDoc).get());
    if (isEmpty) {
      return;
    }
    batchNumber++;
    await action(
      batchNumber,
      documents.filter((document) => document.exists)
    ).catch((error) => console.error(error));
    await recursiveFunction(documents[documents.length - 1]);
  };
  await recursiveFunction();
};

const fetchMessages = async () => {
  const messagesQuery = admin
    .firestore()
    .collection("messages")
    .where("createdAt", ">", new Date("2020-05-04T00:00:00Z"))
    .limit(200);

  const messagesArray: FirebaseFirestore.DocumentData[] = [];

  await onQueryResults(messagesQuery, async (batchNumber, documentSnapshots) => {
    console.log(`Retrieving Message: ${batchNumber * 200}`);
    documentSnapshots.forEach((documentSnapshot) => {
       messagesArray.push(documentSnapshot.data());
    });
  });
  return messagesArray;
};

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

The jQuery ajax function is failing to return any results

Here is the code snippet I am working with: $("#MainContent_btnSave").click(function () { if (($("#MainContent_txtFunc").val() == "") || ($("#MainContent_cmbLoc").val() == "")) { alert("Please make sure to fill in all required ...

Having trouble retrieving AJAX response data using jQuery

I have been searching and attempting for hours without success. On my current page, I am displaying basic data from a database using PHP in an HTML table. However, I now want to incorporate AJAX functionality to refresh the data without reloading the page ...

Discovering the clicked button in a partial view during an onSuccess Ajax call

Within my partial view, I have implemented two buttons: Save and Preview. Both buttons are functioning as expected, with Preview enabling widget preview and Save saving data to the database. However, I am encountering two issues: I am unable to determine ...

Node.js Multer encountering undefined req.file issue when handling multiple file uploads

FIXED: ( NO req.file ) ( YES req.files ) My project requires the ability to upload multiple files. If single image uploads are working but multiple image uploads aren't (uploading to files), I need req.file.filename in order to write the image path ...

Resolve the issue with automatically generating SCSS type definitions (style.d.ts) for Preact within a TypeScript webpack setup

Utilizing webpack with Preact 10.x (nearly identical to React) and TypeScript in the VSCode environment. Following an update from Node version 12 to version 14, there seems to be a problem where *.scss files no longer automatically generate their correspo ...

Learn the process of adding values to HTML forms within ExpressJS

I am facing a challenge in injecting Javascript variable values into HTML forms on an Expression JS server. I am unsure about how to solve this issue. All I want to do is insert the values of x, y, and res into the forms with the IDs 'firstvalue&apos ...

How to incorporate "selectAllow" when dealing with dates in FullCalendar

I've been attempting to restrict user selection between two specific dates, but so far I haven't been able to make it work. The only way I have managed to do it is by following the routine specified in the businessHours documentation. Is there a ...

During testing, the Vuetify expansion panel body is hidden with a display none style

Greetings! I am currently facing an issue while debugging my testing site. The problem is that the expansion panels are not displaying due to a style attribute attached to the div element of v-expansion-panel__body. Strangely, this issue does not occur on ...

How can I redirect after the back button is pressed?

I am currently working with a trio of apps and scripts. The first app allows the user to choose a value, which is then passed on to the second script. This script fetches data from a database and generates XML, which is subsequently posted to an Eclipse/J ...

What could be the reason for the absence of an option in the navbar

As I work on creating a navbar menu that functions as an accordion on desktop and mobile, I encountered an issue. When I click on the dropdown on mobile, it displays one less item than intended. This seems to be related to a design error where the first it ...

Is there a way to modify an npm command script while it is running?

Within my package.json file, I currently have the following script: "scripts": { "test": "react-scripts test --watchAll=false" }, I am looking to modify this script command dynamically so it becomes: "test&qu ...

Is there a way to gradually reveal JSON data without continuously re-parsing and displaying it on a webpage?

Currently, I am working with a log file that is constantly updated by a running script in real-time. My goal is to effectively monitor the status of this script on a web page using HTML and JavaScript. To achieve this, I have utilized JavaScript to dynamic ...

When using Mongoose and Socket.IO, an empty string will not be printed if the result is empty

I'm facing an issue with a filter I created using socket.io and mongoose. Specifically, I'm trying to display a message if there are no results returned when querying, but it seems to not be working as expected. Restaurant.find({ $text : { $sear ...

AngularJS error: Uncaught MinError Object

Recently, I started a new AngularJS project and successfully set it up. The installation of angular and angular-resource using bower went smoothly. However, upon installing another service that I have used previously - https://github.com/Fundoo-Solutions/a ...

Sloped Divider on the Upper Edge of the Page

I'm currently in the process of developing a new website, and I'm looking to create a unique design for the main navigation bar on the homepage. Here is the ideal layout that I have in mind: https://i.stack.imgur.com/pc8z4.png While I understan ...

Guide on converting the <br> tag within a string to an HTML tag in VUE.js

When working with Vue.js, I often use {{}} to display my data on the HTML page. However, I recently encountered a situation where my data includes a string with tags that I would like to be rendered as actual HTML tags when displayed. data(){ return ...

Adding an operation to a function that is linked to an event

On one of my pages, I have a button that triggers some ajax calls with data binding. The binding code is generic and used in various parts of the application, so altering its behavior could impact other users. However, I now have a specific requirement fo ...

What could be causing the props to appear empty in a Child component within a Quasar framework and Vue 3 application?

I am facing an issue while passing props to a Child component table in my Quasar Vue3 app. The content is not being rendered, and I can't figure out why. Strangely, the console is clear of any errors. In the parent component, I am creating an object w ...

Utilize JSON to create a dictionary populated with objects following a complex grouping operation

I am faced with a JSON query that contains the Date, Value, Country, and Number fields. My goal is to create two separate JSON dictionaries based on unique dates (there will be two of them). The desired output can be seen in the code snippet below along wi ...

Troubles with Express JS POST Requests

I'm facing an issue while attempting to write code that creates a MongoDB entry using express.js. Every time I test my code with a cURL request, I receive an error message stating "empty response from server". Below is the snippet of my express.js co ...