Calculating the total of integer array elements in BigQuery with a custom JavaScript UDF

My issue revolves around a peculiar behavior I encountered while working with a table that includes a repeated integer field named coord1:

https://i.sstatic.net/B5I6N.png

Initially, when attempting to compute the sum of the integer array coord1 in BigQuery, my code unexpectedly resulted in concatenating the values as strings instead of adding them up. Below is the code snippet along with its output:

CREATE TEMP FUNCTION func(array_values ARRAY<INT64>)
RETURNS INT64
LANGUAGE js AS """
  var divisor = 0;
  for(var i = 0; i < array_values.length; i++){
    divisor = divisor + array_values[i];
  };
  return divisor;
""";

with
  `project.dataset.table` as
  (
    SELECT 1 id1, [1, 2, 3, 4] coord1 UNION ALL
    SELECT 2, [2, 0, 1, 1, 0, 2, 1, 1]
  )
SELECT
  id1,
  coord1,
  func(coord1) as array_sum
from
 `project.dataset.table`

https://i.sstatic.net/jTMpe.png

Upon further investigation, I found that incorporating the parseInt function before executing the addition operation on the array elements produced the desired result:

CREATE TEMP FUNCTION func(array_values ARRAY<INT64>)
RETURNS INT64
LANGUAGE js AS """
  var divisor = 0;
  for(var i = 0; i < array_values.length; i++){
    divisor = divisor + parseInt(array_values[i]);
  };
  return divisor;
""";

with
  `project.dataset.table` as
  (
    SELECT 1 id1, [1, 2, 3, 4] coord1 UNION ALL
    SELECT 2, [2, 0, 1, 1, 0, 2, 1, 1]
  )
SELECT
  id1,
  coord1,
  func(coord1) as array_sum
from
 `project.dataset.table`

https://i.sstatic.net/Ze6NP.png

The question that arises now is: what led to the initial code treating array values as strings and performing string concatenation instead of summation?

Answer №1

According to the documentation:

The documentation explains that JavaScript does not have a native 64-bit integer type, so using INT64 as an input type for JavaScript UDFs is not supported. Instead, it suggests using FLOAT64 for representing integer values as numbers or STRING for representing integer values as strings.

Despite the lack of official support, BigQuery utilizes an encoding of JavaScript's String type to handle INT64 in order to maintain precision when passing through values of that type. If there is a need to convert to Number, the recommended method is to use parseInt.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Creating a program to automate block placement

I’m attempting to program my MineFlayer bot to automatically position a sand block on the face of a piston whenever it detects one within a five-block radius. Despite my efforts using bot.placeblock(33, vec(0,0,1), cb), I keep encountering an error mess ...

Using JSON with PHP to send data and receiving the response back with JavaScript

Despite exploring numerous questions and attempting various methods, I have been unable to find a solution that works. Here is my JavaScript file: function sendData() { var jsondata; var j = {"pub_id":"'+pid+'","a_type":"'+a_t&ap ...

How to Retrieve the Selected Value from an AngularJS UI-Select

At the moment, I am facing an issue with multiple dropdown directives that are using the same array of values. Whenever a selection is made in one dropdown, it updates the selected value in all other dropdowns. How can I ensure that each dropdown has a uni ...

What is the best way to incorporate a fade out transition into the current tab-pane?

I am currently implementing Bootstrap 5.3 with a basic pills and tab structure, and I am looking for a way to smoothly add a fadeOut effect to the active tab before the fadeIn of the selected tab. It appears that simply adding the "fade" class only introd ...

I encountered an issue with adding a console log in the getStaticProps() function that resulted in the error: SyntaxError: Invalid Unicode escape sequence at eval (<anonymous>

Welcome to my users.js page! import React from 'react' const displayUsers = ({users}) => { return ( <> { users.map(user => { return <div key={user.id}> {user.name} </div> }) } </&g ...

Inserting an item into a list

Looking for assistance with this particular scenario: { "EekvB3cnwEzE":{ "name":"hi", }, "Brv1R4C6bZnD":{ "name":"yup", }, "kRwRXju6ALJZ":{ "name":"okay", } } I'm attempting to store each of these objects in an array. Howe ...

Creating a PHP form with multiple buttons

I'm facing a problem with my form that contains 2 buttons. Depending on which button is selected, the data in the database should be either deleted or edited. I have separate pages for editing and deleting using SQL statements (questionedit and questi ...

Need to configure popper.js functionality for my Squarespace website

I'm having trouble implementing popper.js on the pricing page of my Squarespace site. I followed the steps outlined in a Squarespace tutorial video, adding the following code to the <head> section of my page under page settings -> advanced -& ...

When integrating react-hook-form with Material-UI TextField in a form, an error occurs stating that "TypeError: e.target is undefined" when

Hey there, I stumbled upon something fascinating and could really use some assistance. Every time I attempt to perform an onChange, I run into the following error: TypeError: e.target is undefined. Here's a snippet of my setup: import React, { useE ...

StorageLimit: A new condition implemented to avoid saving repetitive values in the localStorage

Is there a way to store the text of an li element as a localStorage value only once when clicked? If it already exists in localStorage, a second click should have no effect other than triggering an alert. I'm currently using an if statement inside a ...

Storing external API requests in create-react-app's service worker for faster retrieval

I'm in the process of transforming a React web application into a PWA (Progressive Web App). I've made the necessary change in the index.js file - serviceWorker.register();. Everything is functioning properly as I can access the home page and as ...

Obtaining the value of an object with a key containing a space and special character in both the key and object name using Node.js

Here is the value stored in a variable named publicIdentifier. By using console.log("publicIdentifier", publicIdentifier);, I am able to retrieve the following value. Now, my task is to extract the value of MSISDN-MSISDN into a different variable ...

unable to detect image input type

My dilemma lies in trying to submit a form using an image input type. The ajax request works as expected, and I receive a response from my php script. However, I encountered an issue where the image button was not changing its image upon submission. Accord ...

The default promise library in the MEAN stack is outdated and no longer supported for Mongoose

Starting a MEAN-stack server, but encountering an error message: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html After searching for solution ...

What is the method to determine if a date is larger or smaller than another using Javascript?

Using the inputText function retrieves the value entered in the textbox, while the hidden field value returns the current value. This is my current code snippet: if (inputText.value.length != 0) { if (inputText.value < document.getElementById(&apo ...

Issue with Vuejs Bootstrap select not refreshing its text value

I'm currently developing an application where I have implemented a custom element for a dropdown feature. The dropdowns are rendered using Bootstrap select. However, when I try to update the values in the select component, everything changes as expect ...

Tips for Taking a Screenshot of an iFrame Using html2canvas

Can anyone help me figure out how to save an iframe screen as a picture and share it? The issue I'm facing is that when I click on the code I created, the video within the iframe doesn't display. Is there a solution for this problem or another wa ...

Vue: The async Apollo mixin function successfully logs a value, however it ultimately returns as undefined

I've encountered numerous async/return undefined queries on this platform, but despite trying various solutions, I'm unable to make any progress. My apologies if I overlooked something obvious. In an attempt to improve reusability, I extracted a ...

What is the process for cancelling a pending request using jQuery in JavaScript Selenium?

My website has a high volume of users clicking on one button. Waiting for long-pending ajax requests to get responses, sometimes over a minute, seems nonsensical. I understand how to wait for the response, but how can I cancel it? How do I cancel all pend ...

Angular $http.get: How to handle errors effectively?

When sending a form to Node.js for authentication, I am using $http.get in the following function and adding a promise with .then. Will this handle all possible errors that may occur from the server in production? Is there anything else I need to include i ...