Tips for refreshing data in BigQuery following an onUpdate occurrence

Currently, I am successfully importing data from Firebase to BigQuery using the onWrite event and the table.insert function. However, I am facing an issue when trying to update data in BigQuery on the onUpdate event as the table.update function is not available or working. Can anyone suggest an alternative solution for updating data in this scenario? Below is the code snippet I have been using:

exports.updatetobigquery = 
functions.database.ref('/mn_users/{userId}/').onUpdate(event => {
  const dataset = 
bigquery.dataset('KHUSHUApp');//functions.config().bigquery.datasetname);

  const table = 
dataset.table('mn_users');//functions.config().bigquery.tablename);
      console.log('Uppercasing', event.data.val());
  return table.update({
    'id': event.data.key,
    'name': event.data.val().name,
    'email': event.data.val().email
  });

});

Answer №1

Top recommendation for managing this situation:

  1. While BigQuery does support data updates, it is important to note that BigQuery primarily functions as an analytical database rather than one optimized for constant updates.
  2. To enhance efficiency, strive to avoid modifying data directly in BigQuery and explore alternative methods to accomplish your objectives.
  3. Instead of updating existing records, consider sending fresh rows to BigQuery and then consolidating and eliminating duplicates when it is time to analyze the most recent information. This approach proves beneficial if you require the ability to reference each individual state change throughout the process of analysis.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Decode the JSON serialized format generated by CircularJSON

I have a JSON object in my node.js code that contains circular references. To send this information to the browser, I utilized the NPM package circular-json to stringify the object and serialize the circular references: var CircularJSON = require("circula ...

Passing values from Ruby on Rails to JavaScript: a comprehensive guide

Is there a way to pass the value from <%= controller.controller_name %> <%= controller.action_name %> to JavaScript in the following manner? return (document.body.className += " load-done '<%= controller.controller_ ...

Learn how to use Bootstrap to position my component below another component on the webpage

Understanding the basics of Bootstrap and how to assign sizes to columns is one thing, but I seem to be stuck on a simple use case. Here is the scenario: I have two components for a Todo list application: Component 'New Todo' (Form Input field ...

Tips for adding information to a PHP file

Previously, I used PHP to append form data to a JSON file. Currently, I am working on a PHP file instead of a JSON file with two parameters, as shown to users in the figure below. Image URL Link My Form: <form action="process.php" method="POST"> ...

Invoking a function within a loop

In order to complete this assignment with a pop art theme, I have chosen to incorporate pokeballs into the design. The task at hand involves using two for loops, one for the top row and one for the bottom row, while also creating a function. The main cha ...

Dispose the inputpicker filter after setting the value

I am currently utilizing the "Jquery inputpicker plugin" for creating dropdown menus. More information about this plugin can be found here. To initialize my dropdown, I use the following code: $('#test').inputpicker({ data:[ {value:"1 ...

Display a modal before leaving the page?

I need to display a modal message when a user navigates away from the page, triggered by a successful AJAX call. The issue is that if the message loads too quickly, the user may not have enough time to read it before being directed to another page. This is ...

Use jQuery to save a list of URLs in an array, then trigger the opening of each URL when a button is

Explanation of my approach, described in two different ways to assist understanding First Approach Gather all elements structured like this: <a href="etc">url1</a> within the <div class=".resources"></div> and place them in an a ...

Making Mat-Tab Table Headers Sticky in Angular

I'm facing an issue in my application where I have a screen with 3 tabs. One of these tabs contains a table with a large number of rows, and I want to make the headers of this table sticky so that they remain visible when the user scrolls down. Despit ...

Having trouble with your mobile dropdown menu not responding to clicks?

I'm having trouble getting a dropdown menu to work on the mobile version of my website. When I click on the dropdown menu image, it's supposed to appear, but it's not working as expected. JSFiddle: https://jsfiddle.net/xfvjv184/ Included ...

Applying the Directive/Isolated Scope as Embedded HTML within an Angular-Bootstrap Popover

If you're using the angular-ui popover, you have the ability to include HTML content within it. However, I encountered some difficulties in placing a directive called sampleDirective inside the popover. Despite my attempts with the $sce.trustAsHtml an ...

Is it possible to achieve efficient Autocompletion features within VS Code?

Just had my first experience with PhpStorm and I must say, I'm impressed, especially with the completion feature. Pressing Ctrl + Space on my Mac gives me relevant suggestions, like when I'm in a JS file and my project includes jQuery, I get Ele ...

Importing external scripts using requirejs without access to the configuration

I'm currently facing an issue with loading the datatables javascript library within a plugin I'm developing. The problem arises when attempting to load the external resource, as there seems to be a conflict with datatables when I trigger the requ ...

What is the most efficient way to retrieve the current user's ID within Loopback?

Given the instability and deprecation of loopback.getCurrentContext(), what strategies can I use to accurately identify users when they interact with API endpoints? Is it feasible to include the token ID in the request payload for verification purposes? H ...

When attempting to access a Google Apps Script URL for a deployed WebApp, a 404 error is encountered

This issue is very similar to others (like the problem with Google Drive mentioned on this Stack Overflow post) that have been discussed here. Although not identical, I believe it stems from the same core issue highlighted in that thread. The issue arises ...

Arrange links in a list using Protractor and select one to click at random

How can I create a list of all 100 search engine result weblinks and then use a random number generator to click on one randomly? Here is the code I've been working with that doesn't seem to be functioning properly: for (var iterate=1; iterate& ...

Exploring the methods of a class within Node.js modules

I have a basic module that contains a class: function PusherCom(options) { } PusherCom.prototype.lobbyEvent = function(event, message) { }; module.exports = PusherCom; In my main file ./index.js, I import the module: var PusherCom = require('../c ...

Error: Unable to Navigate to Present Location ("/search") as It is Duplicated

I keep encountering the error message NavigationDuplicated when I attempt to perform multiple searches. My search function is located in the navbar, and it's configured to capture the input value using a model. Subsequently, this value is passed as a ...

JavaScript: Different ways to utilize the reduce method

Creating an array for search purposes based on a string like BBC Sport, the desired output array will be: [ 'BBC', 'BB', 'B', 'Sport', 'Spor', 'Spo', 'Sp', 'S' ] An implement ...

What is the most effective method for creating and removing a component just once in a React application? (Is it acceptable to use unused setters from useState?)

On my canvas, I have a large game object that should only be created once when the component is rendered and should persist until the component is removed. (I want to avoid making changes to the game structure by adding storage in a Context or Redux. I pr ...