Restrict the amount of data stored in Firebase

Every minute, a script automatically adds a new record to my Firebase database.

I am trying to figure out how to delete the oldest records once the list reaches a certain length.

After researching through documentation and other posts, I have come across the following solution:

// Define the maximum number of records to keep in the chat history.
const MAX_ARDUINO = 10;

exports.arduinoResponseLength = functions.database.ref('/arduinoResponse/{res}').onWrite(event => {
  const parentRef = event.data.ref.parent;
  return parentRef.once('value').then(snapshot => {
    if (snapshot.numChildren() >= MAX_ARDUINO) {
      let childCount = 0;
      let updates = {};
      snapshot.forEach(function(child) {
        if (++childCount <= snapshot.numChildren() - MAX_ARDUINO) {
          updates[child.key] = null;
        }
      });
      // Update the parent to remove extra children.
      return parentRef.update(updates);
    }
  });
});

The issue with this approach is that onWrite seems to download all related data each time it is triggered.

While this may work well for smaller lists, I run into problems with my 4000 records as it impacts my Firebase download quota every month.

Is there a more efficient way to handle this situation?

Answer №1

After much deliberation, I have devised 3 functions to handle the arduino records. One function updates the number of arduino records, another completely recounts them if the counter is missing. The final function utilizes the counter to execute a query using the limitToFirst filter to retrieve only the relevant data for deletion.

This approach is an amalgamation of the examples provided by Firebase at: https://github.com/firebase/functions-samples/tree/master/limit-children https://github.com/firebase/functions-samples/tree/master/child-count

Below is the finalized code:

const MAX_ARDUINO = 1500;

exports.deleteOldArduino = functions.database.ref('/arduinoResponse/{resId}/timestamp').onWrite(event => {
    const collectionRef = event.data.ref.parent.parent;
    const countRef = collectionRef.parent.child('arduinoResCount');

    return countRef.once('value').then(snapCount => {
        return collectionRef.limitToFirst(snapCount.val() - MAX_ARDUINO).transaction(snapshot => {
            snapshot = null;
            return snapshot;
        })
    });
});



exports.trackArduinoLength = functions.database.ref('/arduinoResponse/{resId}/timestamp').onWrite(event => {
    const collectionRef = event.data.ref.parent.parent;
    const countRef = collectionRef.parent.child('arduinoResCount');
    
    return countRef.transaction(current => {
        if (event.data.exists() && !event.data.previous.exists()) {
            return (current || 0) + 1;
        } else if (!event.data.exists() && event.data.previous.exists()) {
            return (current || 0) - 1;
        }
    }).then(() => {
        console.log('Counter updated.');
    });
});


exports.recountArduino = functions.database.ref('/arduinoResCount').onWrite(event => {
    if (!event.data.exists()) {
        const counterRef = event.data.ref;
        const collectionRef = counterRef.parent.child('arduinoResponse');

        return collectionRef.once('value')
            .then(arduinoRes => counterRef.set(arduinoRes.numChildren()));
    }
});

I have yet to test this implementation, but I will share my results soon!

On a side note, I've heard rumors that Firebase may introduce a "size" query in the future, which is something that is sorely missed in the current setup.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Reasoning behind splitting this solution into a function utilizing Javascript

Today, while working on a coderbyte problem, I managed to solve it on my own without any outside help, and it felt fantastic. However, upon completion, I realized that I solved it without creating any functions. Rather, I kept referring back to a previous ...

What is the best way to toggle the visibility of fields on a modal in a Ruby on Rails

I am working on an application that utilizes Rails 4.1 and Ruby 2.1.2. The show.html.erb view for the Assignment Model in this application is as follows: <h1><%= "#{'Source URL'}"%> <div class="source-url"><%= @assignment.so ...

Update data in PHP dynamically without reloading the page

It would be great if they are doing well! I have encountered a minor issue - everything seems to work perfectly, but it doesn't quite meet my requirements. I am referring to this particular function: <script> var previous = null; var current = ...

What sets Protractor apart from Grunt?

According to the Protractor website (http://www.protractortest.org/#/infrastructure), Protractor utilizes Selenium for browser automation. However, when browsing through the Grunt website (http://gruntjs.com/), it's mentioned that Grunt is also used f ...

Unable to retrieve an array in SAPUI5 and assign it to a JSONModel

I encountered an issue with accessing an array where I stored data from a model read operation. The array, named aData, contains row data for a table and everything seems to be working correctly as all the data is present. This code snippet is executed af ...

"Implementing a dependent dropdown feature within a CodeIgniter application using data from

I am working with a table called "drivers" that has fields for "id", "name", and "mob". Currently, I am facing an issue with 2 dropdowns on my website: The first dropdown should display the names of drivers fetched from the database The second dropdown ...

media query not displaying properly on iOS browsers such as Chrome, Safari, and Mozilla

I have encountered an issue with the @media print not functioning properly on iOS devices, while it works without any problems on android. I have implemented a feature to hide content that should not be included in the printed version. body.printing *{dis ...

Six Material-UI TextFields sharing a single label

Is there a way to create 6 MUI TextField components for entering 6 numbers separated by dots, all enclosed within one common label 'Code Number' inside a single FormControl? The issue here is that the label currently appears only in the first tex ...

What would be the optimal type for the second argument of the `simulate` method?

When using the simulate function, I am familiar with code like this: simulate("change", { target: { value: '7' } }); However, if my onChange function requires an object as a parameter, what should I pass in the second argument? interface myObj ...

How can I efficiently retrieve the server time in JavaScript using a web browser?

Is there a faster way to synchronize client-side time with server time in JavaScript? I found this solution on Stack Overflow, which seems good, but are there any other ideas that might be quicker? ...

`In TypeScript Angular, encountering challenges with accessing object properties`

My TypeScript object looks like this const playlist: { tracks: Array<Track> } = { tracks: new Array<Track>() }; This is the Track interface I am working with interface Track { title?: string; album?: string; artists?: string; duration? ...

Clicking on the Primary Division

Experimenting with the onclick event on a parent div led me to realize that making the entire div clickable at 100% width is not what I intended. I only wanted the sub div to trigger a specific function. Here's an example: <div id="loginContain ...

React - Triggering the onSubmit function of the parent component when a form is submitted

I am curious to understand the behavior I encountered. My React component consists of a Form (React Bootstrap) enclosed within a Modal (React Bootstrap). By mistake, I linked the onSubmit action to the outer Modal, and this function runs when the form is s ...

I am experiencing an issue with the checkbox in my React app where the state is not updating after it has been

I am currently building a todo app using React, but I'm encountering an issue where nothing happens when I click the checkbox. I've provided my code below: App.js import './App.css'; import React from 'react' import TodoItem ...

Issues with posting form data in AngularJS are occurring

Here is the code I am currently using: On the Angular side vm.onSubmit = function(){ var person = vm.formData.slice(0, 1)[0]; //This line extracts the required fields from the model object which is nested in an array. $http({ ...

Guide on inserting a new column into an array of objects in Vue

Below is the fetch method I have defined to retrieve recordings from the database. However, I need assistance in adding a new column to each record specifically for frontend purposes. Can someone help me with this problem? <script> export default { ...

Persistent vertical menu dropdown that remains expanded on sub menu pages

I am struggling to understand how to keep my menu sub items open when on the active page. Although I have tried similar solutions, I have not been successful in implementing them. I apologize if this question has been asked before. My approach involves usi ...

Problem with flags series in Highcharts/Highstock

Can anyone help me figure out why not all the flags in the withdrawals series are displaying? For reference, you can view the following JS fiddle: https://jsfiddle.net/lucianpurcarea/5zxa0jsm/13/ The snippet of code below is responsible for creating the d ...

I need help figuring out the right way to define the scope for ng-model within a directive

I found a straightforward directive to automate sliders: app.directive('slider', function() { return { restrict: 'AE', link: function(scope, element, attrs) { element.slider({ value: scop ...

Select value not updating as expected

I have a select box that is linked to a switch statement in PHP, which changes the order of the results returned from the database based on the value selected in the select box. However, I am facing an issue with my JavaScript not working correctly. Can an ...