Remove data from Firebase that is older than two hours

I need to implement a solution to automatically delete data that is older than two hours. Currently, I am iterating through all the data on the client-side and deleting outdated entries. However, this approach triggers the db.on('value') function each time an entry is deleted. Additionally, the deletion process only occurs when a client connects, raising concerns about what happens if multiple clients connect simultaneously.

Where is the best place to establish a mechanism for removing old data? Each object contains a timestamp generated using JavaScript's Date.now() method.

Answer №1

Firebase doesn't have support for queries that involve dynamic parameters like "two hours ago". However, it is capable of executing a query for a specific value, such as "after August 14th, 2015 at 7:27:32 AM".

This means you can periodically run a block of code to tidy up items that are older than 2 hours at that particular time:

var reference = firebase.database().ref('/path/to/items/');
var currentTime = Date.now();
var timeLimit = currentTime - 2 * 60 * 60 * 1000;
var oldItems = reference.orderByChild('timestamp').endAt(timeLimit).limitToLast(1);
var eventListener = oldItems.on('child_added', function(snapshot) {
    snapshot.ref.remove();
});

In this snippet, I utilize child_added instead of value, and apply limitToLast(1). As each child is deleted, Firebase will trigger a child_added event for the new "last" item until no more items exist after the cutoff point.

Update: To execute this code in Cloud Functions for Firebase:

exports.deleteOldItems = functions.database.ref('/path/to/items/{pushId}')
.onWrite((change, context) => {
  var reference = change.after.ref.parent; // reference to the items
  var currentTime = Date.now();
  var timeLimit = currentTime - 2 * 60 * 60 * 1000;
  var oldItemsQuery = reference.orderByChild('timestamp').endAt(timeLimit);
  return oldItemsQuery.once('value', function(snapshot) {
    // create a map with all children that need to be removed
    var updates = {};
    snapshot.forEach(function(child) {
      updates[child.key] = null
    });
    // execute all updates in one go and return the result to end the function
    return reference.update(updates);
  });
});

This function gets triggered whenever data is written under /path/to/items, meaning child nodes will only be deleted during data modifications.

You can also find this code in the functions-samples repository.

Answer №2

A cloud function triggered by an HTTP request is responsible for deleting nodes based on their creation date and expiration.

Whenever a node is added to the database, it must include two fields: timestamp indicating its creation time, and duration specifying the expiry time.

https://i.sstatic.net/jjqrL.png

The following HTTP-triggered cloud function is used:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();

/**
 * @function HTTP trigger to check and delete expired messages from the database upon request.
 * @type {HttpsFunction}
 */
exports.removeOldMessages = functions.https.onRequest((req, res) => {
    const timeNow = Date.now();
    const messagesRef = admin.database().ref('/messages');
    messagesRef.once('value', (snapshot) => {
        snapshot.forEach((child) => {
            if ((Number(child.val()['timestamp']) + Number(child.val()['duration'])) <= timeNow) {
                child.ref.set(null);
            }
        });
    });
    return res.status(200).end();
});

You can schedule a cron job to periodically send requests to the function URL at set intervals using .

Alternatively, you can run a custom script that triggers a request every 10 seconds:

watch -n10 curl -X GET https://(your-zone)-(your-project-id).cloudfunctions.net/removeOldMessages

Answer №3

The most recent update to Firebase API has replaced ref() with simply ref

var reference = new Firebase('https://yours.firebaseio.com/path/to/items/');
var currentTimestamp = Date.now();
var cutoffTime = currentTimestamp - 2 * 60 * 60 * 1000;
var oldData = reference.orderByChild('timestamp').endAt(cutoffTime).limitToLast(1);
var dataListener = oldData.on('child_added', function(snapshot) {
    snapshot.ref.remove();
});

Answer №4

If anyone encounters a similar issue with Firestore, I devised a script to address it. The script first reads documents to the console and then proceeds to delete messages older than 24 hours from a collection. I utilized to schedule website refresh every 24 hours. Check out the code snippet below.

var yesterday = firebase.firestore.Timestamp.now();
  yesterday.seconds = yesterday.seconds - (24 * 60 * 60);
  console.log("Test");
  db.collection("messages").where("date",">",yesterday)
      .get().then(function(querySnapshot) {
        querySnapshot.forEach(function(doc) {
          console.log(doc.id," => ",doc.data());
        });
      })
  .catch(function(error) {
        console.log("Error getting documents: ", error);
  });

  db.collection("messages").where("date","<",yesterday)
    .get().then(function(querySnapshot) {
      querySnapshot.forEach(element => {
        element.ref.delete();
      });
    })

Answer №5

If you're interested in automating tasks with Firebase functions, check out this helpful guide on Scheduling Firebase Functions with Cron Jobs. This resource explains how to set up a schedule for running Firebase Cloud Functions at specific intervals. You can then use the suggestions provided in this discussion to utilize those scheduled functions for handling old data removal.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Guide on uploading images to a NodeJS server from an Angular 2+ application

I have a NodeJS rest-API that can handle both images and text content in the same function. Currently, I am using multer in my NodeJS server to manage image uploads along with text content. Below is an example code snippet showcasing how I am handling this ...

Error Encountered When Using SQLite Ionic Plugin

For my Ionic application, I am in the process of creating a database test using information from the original documentation found here. However, upon creating and running the app, I encountered the following error on the terminal: **Uncaught TypeError: Ca ...

The ASP.NET Core MVC controller encounters a null input parameter when called by an ajax request

Here is the scenario involving an ajax call: ... $("#testBtn").click(function (e) { e.preventDefault(); $.ajax({ url: "/profile/GuestList", data: {"keytype": "1"}, method: "POST&q ...

Navigating to a new URL using window.location.href will seamlessly embed the new page within

I am new to JavaScript and I am facing an issue with my AJAX function. The function is supposed to navigate to a new URL, but instead of loading the new page separately as if the user had typed the URL, it gets inserted inside the same div where the button ...

How to Identify CanActivate in Angular 2 Deprecated Routing?

One issue I am facing involves a component that has been decorated with @CanActivate. @Component({ // ... }) @CanActivate(() => false) export class UserManagementComponent { // ... } My dilemma lies in the fact that I want to disable or hide t ...

Setting an Element's attribute is only visible in the console

Just dipping my toes into the world of Web Development. While playing around with some JavaScript within HTML, I decided to try updating the content of an element. Upon running the code below, "Updated Content" appears in the console as intended. However, ...

What could be causing my Chrome extension's AJAX calls to behave synchronously?

Recently, I've been experimenting with implementing AJAX in a Chrome extension. Within my content script, the goal is to inject some HTML into an existing page. I've attempted using JQuery.get, JQuery.load, and ng-include. To my surprise, all ...

The MUI Grid design features information displayed on the left side, accompanied by an image placed directly to the right

In the React MUI V5 sample code below, I am creating a profile page layout with details of a person displayed on the left side of the page in label/value format. My question is how to position an "IMAGE" entry (assume it's a 300px x 300px profile ima ...

The impact of array splicing on data manipulation

I have a $scope array variable that I'm using to generate tabs on the front end, but I'm struggling with implementing a remove tab function. The code for removing tabs is as follows: function removeTab(index) { $scope.tabs.splice(index, 1); ...

Simple Authentication in Node.js with Express Sessions

Scenario - After successfully creating a basic website using Node.js, the next step is to implement a simple form of local authentication to ensure that only authorized users have access to the content. While researching various options like JSON Web Token ...

Encountering a surprising token error while running a node.js application with a classic example

I downloaded node.js from its official website and followed the instructions provided here. I attempted to run the example code snippet from the "JavaScript - The Good Parts" textbook: var myObject = { value: 0; increment: function (inc) { this.value ...

Flashing images with jQuery Flickrush - Encasing the image within <li> tags

I have a div with an unordered list inside: <div id="flickr"> <ul></ul> </div> Currently, I am utilizing a plugin found at this link. To make it functional, I've included the following Javascript code: $(function() { $( ...

concealing the AngularJS scope

I'm facing an issue where I need to hide search results until the user starts typing in the search bar. The module I'm using is a DotNetNuke plugin, so I didn't create it myself. I believe I need to work with the scopes '.angrid-search& ...

Node.js application experiences a crash upon entering incorrect credentials

Would you mind reviewing my Login Function? After entering incorrect credentials, a response is sent but unfortunately the app crashes The versions I am using are Express ^4.17.2 and Node.js v16.14.0 router.post( "/login", [ body(" ...

The function you are trying to call is not callable. The type 'Promise<void>' does not have any call signatures. This issue is related to Mongodb and Nodejs

Currently, I am attempting to establish a connection between MongoDB and Node (ts). However, during the connection process, I encountered an error stating: "This expression is not callable. Type 'Promise<void>' has no call signatures" da ...

I am trying to include the Css Baseline from @mui/material in my project. However, even though it is present in my JSON file, I am encountering an error stating that '@mui/material' needs to be included in the project

Struggling to import Css Baseline from @mui/material, it's listed in my json but I keep getting an error saying '@mui/material' should be included in the project's dependencies. I've been stuck on this issue for a while now! { &q ...

Using Vue.js to filter a list based on index matches

I need help with synchronizing two lists. The first list is displayed in a carousel format, and the second list contains details corresponding to each card in the carousel. I have successfully implemented logic to track the current index of the displayed c ...

Handling Posts and Gets with Jquery/Javascript

Working on a web application that involves generating numerous post and get requests. I am able to monitor all the requests using Developer Tools in Mozilla/Chrome, as well as with Charles (an app). Is it feasible to develop a jQuery or JavaScript functi ...

Unable to reset HighCharts Speedometer value to zero

I am currently experimenting with implementing the highcharts speedometer. The goal is to simulate a connection meter - when there is a good connection, the meter should display values between 30 and 100. If the connection result from an Ajax call is any ...

Utilizing Angular JS for Globalization

Just diving into the world of Angular JS and currently exploring directives. My first challenge is Internationalization, and I've been studying the Angular JS i18n documentation. However, I would greatly appreciate it if someone could provide a more a ...