JSON-formatted time-series data

When it comes to modeling over 1,000,000 data points in JSON, there are two potential paths to consider:

a) Utilizing an array of objects: 

[{time:123456789,value:1432423},{time:123456790,value:1432424},....]

or

b) Implementing nested arrays 

[[123456789,1432423],[123456790,1432424],....]

On initial comparison, the latter option may seem faster due to its minimized character usage, although it sacrifices some descriptive detail. Is the speed difference significant between these two methods? Which one would be your preference and for what reasons?

Could there potentially be a third approach worth exploring?

javascriptjsonstreamingtime-series

Answer №1

{time:[123456789,123456790,...], value:[1432423,1432424,...]}

What is the reason behind this?

  1. Going through a simple array is more efficient.
  2. It is similar to "JSON size" in option b), but it retains the "column" data.

Check out this npm package: https://github.com/michaelwittig/fliptable

Answer №2

If you have time series data that follows a continuous function, particularly with regular time intervals, there is potential for a more efficient representation using delta compression. Even when utilizing JSON:

[
    {time:10001,value:12345},
    {time:10002,value:12354},
    {time:10003,value:12354},
    {time:10010,value:12352}
]

It can be transformed into:

[[10001,1,1,7],[12345,9,,-2]]

Which results in a representation that is 4 times shorter.

The original data could then be reconstructed by:

[{time:a[0][0],value:a[1][0]},{time:a[0][0] + a[0][1]||1, value: a[1][0] + a[1][1]||0 ...

Answer №3

To illustrate this concept further (idea: 'time is a key'):

timestamp1 = {123456789: 1432423, 123456790: 1432424}

Consider this alternative approach:

timestamp2 = {"2017-01-01": {x: 2, y: 3}, "2017-02-01": {x: 1, y: 5}}

This notation is quite concise.

If you need to access the keys, utilize Object.keys:

Object.keys(timestamp2) // ["2017-01-01", "2017-02-01"]

You can then retrieve the values by iterating through these keys or experiment with Object.values:

Object.values(timestamp2) // [{x: 2, y: 3}, {x: 1, y: 5}]

In terms of efficiency: Testing with an array of 10,000,000 items produced these results:

object3 = {}; 
for(var i=0; i < 10000000; i++) {object3[i] = Math.random()};
console.time("values() test");
Object.values(object3); 
console.timeEnd("values() test");
console.time("keys() test");
Object.keys(object3); 
console.timeEnd("keys() test");

Results on my system (Chrome, 3.2Ghz Xeon):

  • values() test: 181.77978515625ms
  • keys() test: 1230.604736328125ms

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Traversing through nested arrays within nested objects using AngularJS

In my project, I am utilizing angularjs 1 and dealing with a complex json object that has multiple levels of nesting. My goal is to use ng-repeat in order to access a nested array within the json structure. [{ "information": { "name": "simdi jinki ...

javascriptangularjs

Submitting form by clicking a link on the page

To submit a POST request with "amount=1" without displaying it in the URL, I need the site to send this request when any link on the site is clicked. This JavaScript code achieves that with a GET request: window.onload = function () { document.body.oncli ...

javascripthtmlpostget

Incorporate the block-input feature from sanity.io into your next.js blog for enhanced functionality

Currently, I'm in the process of creating a blog using next.js with sanity.io platform. However, I am facing some difficulties when it comes to utilizing the code-input plugin. What's working: I have successfully implemented the code component b ...

javascriptreactjsgraphqlnext.jssanity

Executing PHP scripts using Ajax

Check out the code snippet below: <?php //echo $this->Html->css(array('bootstrap', 'mark', 'style')); echo $this->Html->script(array('timer','swfobject','bootstrap.min.js')); // ...

javascriptphpjqueryajaxcakephp

Experiencing a lack of defined or no outcome when outputting the API callback

I am troubleshooting an issue with my script that searches for movie titles from a database. While I receive results in the console without any errors, the renderMovies function is not storing the API movie title, plot, and other details properly in my var ...

javascriptjqueryajaxapi

Implementing hover-over tooltips with JavaScript and HTML

New JS function: function DisplayToolTip() { let x = event.clientX; let y = event.clientY; document.getElementById('divToolTip').style.left = x + 'px'; document.getElementById('divToolTip').style.top = y + &ap ...

javascriptjqueryhtmltooltip

Ways to retrieve elements from a JavaScript array?

I have made an AJAX call that retrieves data in the form of an array structured as shown below: { “periodStart” : “2016-10-09T06:00:00Z", “periodEnd":"2016-10-16T06:00:00Z", “nextPageStart":null, “prevPageStart":"2016-10-02T00:00:00Z", “per ...

javascriptarraysjsonajaxget

What are the best practices for integrating RxJS into Angular 2 projects?

How can I write code like this in Angular 2? var closeButton1 = document.querySelector('.close1'); var close1ClickStream = Rx.Observable.fromEvent(closeButton1, 'click'); I have attempted various methods to incorporate this into an An ...

javascriptangular

Sending data from a Javascript variable to Java in JSP

I need help with passing a Javascript value to a Java function in JSP. What is the best way to achieve this? The ID I want to pass comes from a combobox in JSP through Javascript. My goal is to extract the ID from the combobox and then pass it as a paramet ...

javajavascriptajaxjspparameters

Transferring JSON information to the primary activity

My current project involves connecting my app to a MySQL database using JSON and PHP. The PHP script I have created returns a JSON array that determines the status of the light (either true or false). To make things more organized, I decided to put the JS ...

androidjson

The Bootstrap image slider is limited to showing a single item at a time

I've been attempting to implement a carousel in Bootstrap 4 that displays multiple items at once and allows scrolling one item at a time, but I can't seem to get it right. Despite going through various tutorials and forum posts, the carousel alwa ...

javascriptjquerymodel-view-controllerbootstrap-4carousel

Calculate the number of elements shared between two arrays using JavaScript

I am working with two arrays of objects and I need to determine how many different types of cars we have. The first array contains the IDs of all the cars, while the second array contains information about the types of cars. Here is the data: var arr = ...

javascriptarraysjson

Implementing Dual Submit Buttons in Node.js using Express Framework

Struggling with implementing a like and dislike function in my node js app. Currently, I can only do one at a time. Below is the HTML code snippet: <form method="post" name="ratings"> <input type="submit" name="vote" value="like"> < ...

javascriptnode.jsexpresstypescript

Tips for validating a string in a URL with Selenium IDE

When I click on a tab on my website, it triggers an AJAX service call where the URL contains parameters related to the data being loaded after the tab is clicked. The data is displayed as horizontal tiles one below the other, with 4 tiles being loaded pe ...

javascriptajaxseleniumintegration-testingselenium-ide

Output the 'script' using the `document.write`

Revised inquiry I am endeavoring to construct two distinct HTML files: main.html and sufler.html. The concept is to manage the sufler.html page from main.html. To achieve this, I have devised a solution where I write the code for sufler.html as a string e ...

javascriptdocument.write

Retrieve the height of a div element in an AngularJS framework, and assign this value to a corresponding attribute of another element

Is it possible to retrieve the height of an element using AngularJS and assign it to another div's attribute? In my Bootstrap website, I have a fixed sidebar that needs to stop before reaching the footer. To achieve this, I currently have to manually ...

javascripthtmlangularjs

CordovaJS encountered an error due to a failed manifest merger: the minimum SDK version cannot be lower than version 19 as declared in the code

For the past 48 hours, I have been struggling to build my Cordova app for Android. Every time I attempt to open my project in Android Studio, I encounter an error during the gradle sync process in the run tasks phase: ERROR: Manifest merger failed : uses- ...

javascriptandroidcordova

Node.js fails to send a list of all clients

js. I have conducted a series of tests successfully so far. Hello world Server-client communication with events Multiple rooms However, I am facing an issue in sending messages to the clients using socket.io The main reason for this is to maintain a li ...

javascriptnode.jssocket.io

What could be the reason behind the sudden inability of JavaScript prompt newlines to copy-paste into windows applications?

This function grabs the text from a single row in a table and shows it, alongside the text from the header row, in a prompt for users to copy and paste. It was functioning properly earlier, but now the newlines are not being included when pasting from the ...

javascriptjquerynewline

Strategies for analyzing Authorize.net payment responses in Ionic and JavaScript

Despite numerous attempts, I am struggling to parse the response from the authorize.net payment gateway. Below is the primary response from authorize.net: "{"transactionResponse":{"responseCode":"1","authCode" ...

javascriptangularjsangularionic-framework