Converting large JSON data (approximately 100MB) into an Excel file using JavaScript

Question

Converting large JSON data (approximately 100MB) into an Excel file using JavaScript

As a beginner in the realm of REST technology, I find myself navigating through a JSON response received from a server and showcasing the data on the client side.

Currently, I am faced with handling approximately 22MB of JSON data that needs to be exported to an Excel sheet.

The issue at hand:

During the process of iterating through the JSON, my browser becomes unresponsive. Where could I potentially be making a mistake?

javascript

Answer 1

Answer №1

It is not recommended to export large excel files for download. It's best to break the data into smaller chunks for the user to download in batches. However, if you still need to handle a big excel file download, refer to the solution below.

Consider downloading the json data in batches and combining them into a single excel file for user download!

When a script becomes unresponsive, it means a Javascript thread is taking too long to complete. Modifying the registry could be a solution, but it would need to be done on all client machines. One way to address this is by using a "recursive closure", which allows for a long-running loop to yield control to the browser intermittently, preventing the unresponsive script dialog.

To implement this, add the Utility Class RepeatingOperation to your javascript file. You can use the following code:

RepeatingOperation = function(op, yieldEveryIteration) {

  var count = 0;   

  this.step = function() {
    if (++count >= yieldEveryIteration) {
      count = 0;
      setTimeout(function() { op(); }, 1, [])
      return;
    }
    op();
  };
};

The problematic code causing the 'stop running this script' dialog can be fixed using the following approach:

process10000HeavyTasks = function() {
  var len = 10000;  
  for (var i = len - 1; i >= 0; i--) {
    heavytask(); 
  }
}

Here is the solution for the problematic code:

process10000HeavyTasks = function() {

  var global_i = 10000;

  var repeater = new this.RepeatingOperation(function() {

    heavytask();

    if (--global_i >= 0){
      repeater.step();
    }
    else {
       alert("Task completed");
    }
  }, 10);

  repeater.step();

};

Modified from the following source:

Answer 2

It is not recommended to export large excel files for download. It's best to break the data into smaller chunks for the user to download in batches. However, if you still need to handle a big excel file download, refer to the solution below.

Consider downloading the json data in batches and combining them into a single excel file for user download!

When a script becomes unresponsive, it means a Javascript thread is taking too long to complete. Modifying the registry could be a solution, but it would need to be done on all client machines. One way to address this is by using a "recursive closure", which allows for a long-running loop to yield control to the browser intermittently, preventing the unresponsive script dialog.

To implement this, add the Utility Class RepeatingOperation to your javascript file. You can use the following code:

RepeatingOperation = function(op, yieldEveryIteration) {

  var count = 0;   

  this.step = function() {
    if (++count >= yieldEveryIteration) {
      count = 0;
      setTimeout(function() { op(); }, 1, [])
      return;
    }
    op();
  };
};

The problematic code causing the 'stop running this script' dialog can be fixed using the following approach:

process10000HeavyTasks = function() {
  var len = 10000;  
  for (var i = len - 1; i >= 0; i--) {
    heavytask(); 
  }
}

Here is the solution for the problematic code:

process10000HeavyTasks = function() {

  var global_i = 10000;

  var repeater = new this.RepeatingOperation(function() {

    heavytask();

    if (--global_i >= 0){
      repeater.step();
    }
    else {
       alert("Task completed");
    }
  }, 10);

  repeater.step();

};

Modified from the following source:

Converting large JSON data (approximately 100MB) into an Excel file using JavaScript

Answer №1

Similar questions

Retrieve Backbone data immediately following a successful Save operation

Tips for retrieving data from a database with just one key press

Creating an AJAX request in Play 2.x by using a shared button

Is JavaScript Promise Chaining Allowed?

[Vue alert]: "Maximum" property or method is not declared in the instance but is being referenced during the rendering process

I have discovered some amazing jQuery image hover effects that are simply breathtaking –

What is the reason for the maximum alias number in the npm YAML library?

The elusive Yeoman composeWith module remains elusive and cannot be found

What is the best method to set one div as active by default in jQuery UI Accordion?

Create a copy of an element without altering the original

Make a JSONP request with the MIME Type set to text/plain

Display the input text value when the button is clicked

Acquire the jQuery cookie with the power of AngularJS

Is it possible for me to save external CDN JavaScript files to my local system?

Ways to conceal a div element when the JSON data does not contain a value

New in the world of JavaScript: A method that replicates the functionality of jQuery's

Updating text color with Ajax response value

There are critical vulnerabilities in preact-cli, and trying to run npm audit fix leads to a never-ending loop between versions 3.0.5 and 2.2.1

Tips for updating the display by fetching data from a database through a websocket

Challenges arise when utilizing CSS3 animations in conjunction with transitions triggered by toggling a JavaScript class