Error encountered while parsing Japanese characters using the express body-parser resulting in a bad control character issue

Currently, I am sending a large JSON string to a node express endpoint that is set up like this:

import bodyParser from 'body-parser';
const app = express();
const jsonParser = bodyParser.json({ limit: '4mb' });

const databaseUri = '<db connection string>';
const databaseClient = new MongoClient(databaseUri);

app.use(cors());

app.post('/fillDatabase', jsonParser, async (request, response) => {
    const subjects = request.body;
    console.log(`Filling database with ${subjects.length} subjects`);
    const database = databaseClient.db('wanikani_db');
    const subjectsTable = database.collection('subjects');
    const result = await subjectsTable.insertMany(subjects).then((response) => {
        console.log('inserted ' + response.insertedCount + ' subjects');
    }).catch(() => {
        console.log('failed to insert subjects');
    }).finally(() => {
        console.log('finally after inserting subjects');
    });
});

After posting the JSON string, the function in the endpoint runs successfully to completion but the request does not finish. A stack trace is displayed with an error message:

SyntaxError: Bad control character in string literal in JSON at position 289
    at JSON.parse (<anonymous>)
    at parse (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/body-parser/lib/types/json.js:92:19)
    at /Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/body-parser/lib/read.js:128:18
    at AsyncResource.runInAsyncScope (node:async_hooks:206:9)
    at invokeCallback (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:238:16)
    at done (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:227:7)
    at IncomingMessage.onEnd (/Users/maxc/Documents/repos/wanikani-flashcards/backend/node_modules/raw-body/index.js:287:7)
    at IncomingMessage.emit (node:events:514:28)
    at endReadableNT (node:internal/streams/readable:1376:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

The JSON object causing the issue can be found here. The problematic character is located at data -> characters.

The hexadecimal representation of that character is as follows:

000001c0: 2020 2020 2020 2020 2020 2022 6368 6172             "char
000001d0: 6163 7465 7273 223a 2022 e4b8 8022 2c0a  acters": "...",.

Position 289 contains the Japanese character 一. Any suggestions on how to handle this specific character and others like it?

Answer №1

After dealing with a large JSON dataset, I encountered an issue where only six out of ten requests to the database were successful. Initially, I believed it was a JSON problem due to Bad Control Character errors. However, upon closer inspection, I realized that the root cause was not including response.sendStatus() at the end of the /fillDatabase endpoint. Once I added this, all requests went through successfully. The Bad Control Character errors still appeared, but it seems like all the data is being inserted correctly.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Passing objects to an AngularJS form using UIB TypeAhead without two-way binding

In my current situation, I am encountering an issue with a typeahead feature on a repeating form element that consists of 5 input fields. While everything functions correctly when selecting results and populating the input fields, the model does not get up ...

A step-by-step guide on transferring data from an HTML file to MongoDB using Python Flask

I am currently developing a web application that involves uploading multiple CSV files and transferring them to MongoDB. To build this application, I have utilized Python Flask. To test out different concepts for the application, I have created a sample f ...

JS: Modifying this function to manage a click on a hyperlink

After following the guide provided here I have successfully implemented a drop down list that sends the value to an external PHP script, retrieves HTML output, and displays it in a "div" on the same page. It works flawlessly. My next objective is to send ...

What is the best way to obtain a distinct collection from two arrays that eliminates the second appearance of an element based on a key's value, rather than the first as in the Lodash uniqueBy function?

Let's say I have two arrays... const arr1 = [ { id: 1: newBid: true } ]; const arr2 = [ { id: 1, newBid: false }, { id: 2, newBid: false } ]; My goal is to end up with an array that looks like this [ { id: 1, newBid: false }, { id: 2, newBid: fals ...

Accessing a model's field within an Ember.js each loop

Here is the code for a route that I am working on: Calendar.DateIndexRoute = Ember.Route.extend({ model: function(data) { return {arr:getCalendar(data), activeYear: data.year, activeMonthNumber: data.month, activeDay: data.da ...

Exploring jQuery functionality through data attributes

I'm having trouble implementing the search functionality using the data-find attribute. Currently, my function is simply matching the input string with any text inside the container. You can test this by adding more words to the HTML and then enterin ...

What steps do I need to take to ensure my token remains active following the creation of a new user account

I am currently working on an API using Express.js and MongoDB. Users are able to login, but only admin users have access to the /register endpoint in order to create new user accounts. The goal is to restrict the creation of new employees and users to on ...

Jest - Silence greets the test results

Struggling with Jest has been a common theme for me ever since I first attempted to use it. Regardless of the tests I run or the options I try to pass to Jest, I never seem to get the expected 'Pass' or 'Fail' results in the console. In ...

Employing Ajax.Updater to retrieve a javascript file (prototype.js)

My ajax request is set up as follows: new Ajax.Updater({ success: 'footer' }, '/dyn/actions/checkSystemMessage', { insertion: 'after', evalScripts: true }); The content found at /dyn/actions/checkSystemMessag ...

Improving efficiency of basic image carousel in Angular 8

In my Angular 8 app, I am developing a basic carousel without relying on external libraries like jQuery or NgB. Instead, I opted to use pure JS for the task. However, the code seems quite cumbersome and I believe there must be a more efficient way to achie ...

Uploading multiple files to MongoDB's GridFS system

app.post("/", upload.single("avatar"), function(req, res, next){ var writestream1 = gfs.createWriteStream({ filename: req.file.filename }); var writestream2 = gfs.createWriteStream({ filename: req.file.filename }); fs ...

Experiencing difficulties in installing opensea-js using NPM

While working on my project, I encountered an issue when trying to install opensea-js as a dependency. My Node version is v14.16.0 and NPM version is v7.7.5. Every time I run the command npm install --save opensea-js, it results in an error. I attempted c ...

Customizing the styling of buttons in Highcharts is disabled when in full screen mode

I've integrated highcharts into my Angular application and included a custom button inside the chart to navigate users to another page. However, I encountered an issue when trying to fullscreen the chart using the export menu. The position of the cus ...

Unable to locate the Chart object within the chartjs-plugin-labels.js file

Hello there, I am currently working on an Angular project where I want to incorporate a chart plugin. To achieve this, I executed the following commands: npm install angular2-chartjs npm install chartjs-plugin-labels Following that, I imported it into my ...

What could be the reason for the malfunction of Bootstrap Js in my template?

This question focuses on understanding how something works rather than just fixing it. I really enjoy learning about this. Currently, I am involved in a project that combines VueJS and Symfony. One of the things I would like to achieve is using Bootstrap ...

Difficulty encountered with document.querySelectorAll retrieving paginated elements

I am currently developing a project called STEEP. Our implementation involves utilizing infinite scroll to load 8 videos at a time as the user scrolls through the page. However, we are facing an issue with the script responsible for handling video playbac ...

Tips for persisting objects created in PHP while utilizing XMLHttpRequest

Currently, I am working on a web page with the index.php file structured as shown below: include('UserClass.php'); include('html/top.php'); $user = new User(); if (isset($_POST['user'], $_POST['pass'])) { $user-& ...

Puppeteer does not display the TikTok comment input, whereas it is visible on the browser

Why can't I see the TikTok comment input on Puppeteer, but it's visible in the browser? Browser screenshot Puppeteer screenshot I've tried zooming out and adjusting the viewport with no success. I've written the code, but can't ...

Organizing requirejs and angularjs code structure into separate files

Looking to organize my Angular application with requirejs by separating controllers, services, and directives into different files. Hoping to achieve this structure: src/ components/ Navigation/ index.js module.js NavigationCon ...

Testing URL Parameters in JEST with an API

Seeking integration tests using Jest for an API endpoint. Here's the specific endpoint: http://localhost/universities/ucla/class/2013/studentalis/johndoe. When tested with a put request, it returns 201 on Postman. However, the testing encountered a t ...