What is the most effective way to transmit a sizeable JSON file to a server?

In my web.config file, I set the maxrequestlength to 10MB. However, there is a feature in my system that allows clients to import a .csv file that can potentially be larger than 10MB. As a result, I am looking for an efficient way to send a large json file (the imported data).

I am considering either compressing the data with zip or sending the json file in smaller parts. Which approach would be best?

Are there any other more effective methods to handle this situation?

Answer №1

By utilizing the File APIs, we have the ability to simplify the process of uploading large files. The strategy involves breaking the upload into smaller chunks, initiating an XHR for each segment, and assembling the file on the server side. This method mirrors how GMail handles large attachments with speed and efficiency. It could also serve as a workaround for Google App Engine's 32MB HTTP request limitation.

function upload(blobOrFile) {
  var xhr = new XMLHttpRequest();
  xhr.open('POST', '/server', true);
  xhr.onload = function(e) { ... };
  xhr.send(blobOrFile);
}

document.querySelector('input[type="file"]').addEventListener('change', function(e) {
  var blob = this.files[0];

  const BYTES_PER_CHUNK = 1024 * 1024; // Chunk size set to 1MB.
  const SIZE = blob.size;

  var start = 0;
  var end = BYTES_PER_CHUNK;

  while(start < SIZE) {
    upload(blob.slice(start, end));

    start = end;
    end = start + BYTES_PER_CHUNK;
  }
}, false);

})();

The code snippet does not include instructions on reassembling the file on the server side.

Furthermore, functions like http://underscorejs.org/#zip can also be leveraged in this context.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

The networking feature stops functioning on Android devices after upgrading from Ionic 1.5.0 to 1.6.3

After upgrading from ionic 1.5.0 to 1.6.3 (the latest version), I noticed that networking ajax calls were no longer working on Android. I had to remove and re-add the android platform, and there seemed to be a change in the apk names from MainActivity-debu ...

When an empty list is returned on Google App Engine, the behavior varies between the development server and the deployed server

After setting up an endpoint to retrieve a list of items, I noticed that when the list is empty, the JSON response does not include the list field as expected. This inconsistency between the dev server and the deployed server is causing some confusion. Fo ...

Issues with the code: $("input[type=checkbox]:checked").each(function() { var checkboxId = $(this).attr("id"); });

When submitting a form, I need to retrieve the IDs of all checked checkboxes. Currently, $(this).id() is causing an error. What is the correct code to obtain the IDs of all checked checkboxes? $("#pmhxform input:checkbox:checked").each(function() { ...

How to search for property values within an array of objects using MongoDB with C#

Within my documents, there is an array property known as arrayProperty structured like this: { _id: mongoObjectIdThingy, arrayProperty: [ {string1: "aString", otherProperty:"somethingelse"}, {string1: "aString2", otherProperty:"somethingelse" ...

Decapitalizing URL string in jQuery GET request

Check out my code below: $.get('top secret url and stuff',function(data){ console.log($("[style='color:white']", data.results_html)[0].innerHTML); window.html = document.createElement('d ...

Resolving the 'unexpected token u' error in ASP.NET MVC

Having some trouble implementing autocomplete in my asp.net mvc 4 project. I keep getting an error: unexpected token u. From what I understand, it means there's an unidentified object but I'm not sure how to fix it. Any suggestions? <link rel ...

Encountering problems when attempting to effectively filter a list of products using data

<div id="prod"> <div class="content" data-brand="Andrew" data-price="1000" data-store="JCPenny">Andrew</div><br /> <div class="content" data-brand="Hill" d ...

Unable to capture screenshot of hovered element using Cypress

Having an issue with taking a screenshot of an element with a hover effect. The screenshots always come out without the hover effect applied. tableListMaps.lineWithText('Hello world', 'myLine'); cy.get('@myLine').realH ...

Tips for including a footer element in React:How can a footer division be

I need help with implementing a footer in my React web app. Currently, the body section in index.html looks like this: <body> <div id="root"></div> <script src="index.js"></script> </body> Inside index.js, the rend ...

Group HTML Tables According to Specific Attributes

Let's cut to the chase. My table is functioning well, printing all the necessary information without any issues. However, I'm facing a challenge when it comes to grouping the data rows under Program 1 together. Instead of having Program 1 print, ...

Updating a single .jshintrc option for a folder

My project has a .jshintrc file at the root, containing the following settings: { "node": true, "smarttabs": true, "undef": true, "unused": true } While these settings work well for node-related code in my project, they are not suitable for brows ...

Issues stemming from cross-domain AJAX have resulted in errors with Karma and Jasmine

I've been working on configuring tests that utilize Jasmine and Karma to test my JavaScript code. Karma operates in Node.js and initiates a Chrome browser for testing purposes. Unfortunately, I keep encountering an error message that reads "Chrome 28 ...

Looking for a vertical scrollbar solution for your idangerous vertical swiper?

Incorporating idangerous vertical swiper with two slides in a mobile app through jquery mobile has proven challenging. Specifically, I am struggling to implement a vertical scroll bar in the second slide due to its fixed height and the potential collision ...

Listening to events on the iterative variable of NgFor directive in Angular 2

Angular2 has been my latest exploration in solving a unique data binding challenge. In my UI, I've presented a javascript array of objects like a database recordset in an HTML table. Each row contains menus and inputs allowing users to modify the rec ...

Adding event listeners to modal buttons in a Handlebars template is a simple process that involves utilizing the `

I've been working on developing a web application that interacts with a news API to display articles. Each article is presented in a card format, and I have also incorporated modals using handlebars. My goal is for each card's button to trigger ...

What is the best way to transfer GitHub OAuth information to a client?

I am in the process of implementing Github authorization and then sending received JSON data to the client. After doing some research, I came across a helpful tutorial at The tutorial suggests following this path: "/" -> "/login" -> "/redirect" -&g ...

User authentication on AngularJs is only initiated on the second interaction

My personally built AngularJs user authentication system is experiencing an unusual issue. While everything seems to be working fine - I can login, check access for specific pages, etc. - there is a strange behavior with the token authentication. It seems ...

Duplicate text content from a mirrored textarea and save to clipboard

I came across some code snippets here that are perfect for a tool I'm currently developing. The codes help in copying the value of the previous textarea to the clipboard, but it doesn't work as expected when dealing with cloned textareas. Any sug ...

What are the various undisclosed schema types in A-Frame?

I've been exploring different examples of property types in the official documentation and various Github repositories (though now I can't remember which ones). The latter introduced me to unique properties like "min" and "max" for numbers, as we ...

Mastering VSCode IntelliSense: Unleashing the Power of Type Declarations

In my JavaScript projects, I aim to include TypeScript types sparingly to leverage IntelliSense for better code completions and receive warnings about type-related issues. To set up typechecking in JS, I created a jsconfig.json file and rely mostly on JSD ...