How to filter out corrupt JSON data from an object?

Question

How to filter out corrupt JSON data from an object?

There are times when retrieving building geolocation data from a service results in missing properties within the data.

Example of Good Data:

[{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Winterfell",
    "Latitude": "52.900619",
    "Longitude": "-1.434602",
    "Region": "The North",
}, {
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Water Gardens",
    "Latitude": "51.818051",
    "Longitude": "-.354871",
    "Region": "Dorne"
}]

Example of Bad Data:

Missing Region or Building Name

{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Kings Landing",
    "Latitude": "23.818051",
    "Longitude": "-.154871",
}

Upon receiving the JSON response through an AJAX request, it is stored in an object named _regionAndBuildings.

The goal is to eliminate any bad data entries, so the following code was attempted:

console.log("Initial size of building data : " + _regionAndBuildings.length);
// Clean the JSON by setting objects to undefined
for (var i = 0; i < _regionAndBuildings.length; i++) {
    if (_regionAndBuildings[i].BuildingName === undefined && _regionAndBuildings[i].Region === undefined) {
        console.log("Deleting");
        delete _regionAndBuildings[i];
    }
}
console.log("Final size of building data after cleanup : " + _regionAndBuildings.length);

Output:

Initial size : 209

Deleted x 17

Final size : 209

The issue arises when trying to remove the bad elements from the JSON object using the provided code. Why is this happening?

javascript arrays json data-cleaning

Answer 1

Answer №1

Your code has a couple of issues that need to be addressed:

Firstly, regarding the condition in your code snippet:

The issue lies in handling the scenario where either the Region or Building Name is missing.

The current condition you have written does not accurately cater to this requirement:
```
if (_regionAndBuildings[i].BuildingName === undefined && 
    _regionAndBuildings[i].Region === undefined)
```
By using the logical AND operator, both conditions need to be met for the code inside the if statement to execute. In order to fix this, you should use the logical OR operator like so:
```
if (_regionAndBuildings[i].BuildingName === undefined ||
    _regionAndBuildings[i].Region === undefined)
```
The second problem pertains to utilizing the delete operator on an array:

When delete is used on an array element, it removes the element itself but leaves behind an empty space at the index without changing the length of the array. For better management, consider using Array.prototype.splice instead:
```
_regionAndBuildings.splice(i, 1);
```

A more efficient approach is reconstructing the array by filtering out unwanted elements using Array.prototype.filter method:

var filteredBuildings = _regionAndBuildings.filter(function (currentObject) {
    return currentObject.hasOwnProperty("BuildingName") &&
        currentObject.hasOwnProperty("Region");
});

This function will iterate through all items in the array and only include objects that pass the specified criteria. If an object possesses both BuildingName and Region properties, it will make it into the filtered result.

Note: It's recommended to utilize Object.hasOwnProperty over comparing values against undefined when checking for property existence within an object. This is due to the distinction between non-existing properties and properties with an actual value of undefined.

Answer 2

Your code has a couple of issues that need to be addressed:

Firstly, regarding the condition in your code snippet:

The issue lies in handling the scenario where either the Region or Building Name is missing.

The current condition you have written does not accurately cater to this requirement:
```
if (_regionAndBuildings[i].BuildingName === undefined && 
    _regionAndBuildings[i].Region === undefined)
```
By using the logical AND operator, both conditions need to be met for the code inside the if statement to execute. In order to fix this, you should use the logical OR operator like so:
```
if (_regionAndBuildings[i].BuildingName === undefined ||
    _regionAndBuildings[i].Region === undefined)
```
The second problem pertains to utilizing the delete operator on an array:

When delete is used on an array element, it removes the element itself but leaves behind an empty space at the index without changing the length of the array. For better management, consider using Array.prototype.splice instead:
```
_regionAndBuildings.splice(i, 1);
```

A more efficient approach is reconstructing the array by filtering out unwanted elements using Array.prototype.filter method:

var filteredBuildings = _regionAndBuildings.filter(function (currentObject) {
    return currentObject.hasOwnProperty("BuildingName") &&
        currentObject.hasOwnProperty("Region");
});

This function will iterate through all items in the array and only include objects that pass the specified criteria. If an object possesses both BuildingName and Region properties, it will make it into the filtered result.

Note: It's recommended to utilize Object.hasOwnProperty over comparing values against undefined when checking for property existence within an object. This is due to the distinction between non-existing properties and properties with an actual value of undefined.

Answer 3

Answer №2

One of the main issues is that the operator && is being used instead of ||:

_regionAndBuildings[i].BuildingName === undefined && _regionAndBuildings[i].Region === undefined

This conflicts with the statement:

Bad data

Missing Region or Building Name

Additionally, removing elements from an array using delete leaves behind gaps of undefined:

[<elem>, undefined, <elem>]

While the length remains the same, you can eliminate these undefined values by using filter:

var newArray = yourArray.filter(function(item) { return item});

A better alternative is to use slice rather than delete:

_regionAndBuildings.splice(i, 1);

Answer 4

One of the main issues is that the operator && is being used instead of ||:

_regionAndBuildings[i].BuildingName === undefined && _regionAndBuildings[i].Region === undefined

This conflicts with the statement:

Bad data

Missing Region or Building Name

Additionally, removing elements from an array using delete leaves behind gaps of undefined:

[<elem>, undefined, <elem>]

While the length remains the same, you can eliminate these undefined values by using filter:

var newArray = yourArray.filter(function(item) { return item});

A better alternative is to use slice rather than delete:

_regionAndBuildings.splice(i, 1);

Answer 5

Answer №3

Could you attempt this:

if (typeof _regionAndBuildings[i].BuildingName === "undefined" 
       && typeof _regionAndBuildings[i].Region === "undefined")
            {
                console.log("Removing");
               _regionAndBuildings.splice(i, 1);
}

Answer 6

Could you attempt this:

if (typeof _regionAndBuildings[i].BuildingName === "undefined" 
       && typeof _regionAndBuildings[i].Region === "undefined")
            {
                console.log("Removing");
               _regionAndBuildings.splice(i, 1);
}

Answer 7

Answer №4

Looking at the example you provided

{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Kings Landing",
    "Latitude": "23.818051",
    "Longitude": "-.154871",
}

The BuildingName or Region property is not present. It might be better to use hasOwnProperty instead of checking explicitly for

_regionAndBuildings[i].BuildingName === undefined

for (var i = 0; i < _regionAndBuildings.length; i++){
    if (!(_regionAndBuildings[i].hasOwnProperty('BuildingName')) && !(_regionAndBuildings[i].hasOwnProperty('Region'))) {
        console.log("Deleting");
        _regionAndBuildings.splice(i, 1);
     }
}

Check out the DEMO here

Answer 8

Looking at the example you provided

{
    "ModifiedOn": "2015-04-29 11:17:28.0",
    "BuildingName": "Kings Landing",
    "Latitude": "23.818051",
    "Longitude": "-.154871",
}

The BuildingName or Region property is not present. It might be better to use hasOwnProperty instead of checking explicitly for

_regionAndBuildings[i].BuildingName === undefined

for (var i = 0; i < _regionAndBuildings.length; i++){
    if (!(_regionAndBuildings[i].hasOwnProperty('BuildingName')) && !(_regionAndBuildings[i].hasOwnProperty('Region'))) {
        console.log("Deleting");
        _regionAndBuildings.splice(i, 1);
     }
}

Check out the DEMO here

Answer 9

Answer №5

If you want to filter out the bad values from an array, you can do it in a different way by using the filter method:

var cleanRegionAndBuildings = _regionAndBuildings.filter(function(item) {
    // only keep items with non-empty strings
    return (
        item.ModifiedOn !== "" &&
        item.BuildingName !== "" &&
        item.Latitude !== "" &&
        item.Longitude !== "" &&
        item.Region !== ""
    );
});

Answer 10

If you want to filter out the bad values from an array, you can do it in a different way by using the filter method:

var cleanRegionAndBuildings = _regionAndBuildings.filter(function(item) {
    // only keep items with non-empty strings
    return (
        item.ModifiedOn !== "" &&
        item.BuildingName !== "" &&
        item.Latitude !== "" &&
        item.Longitude !== "" &&
        item.Region !== ""
    );
});

How to filter out corrupt JSON data from an object?

Answer №1

Answer №2

Answer №3

Answer №4

Answer №5

Similar questions

Interacting with Elements in three.js when the canvas is not in full screen mode version 74

One way to achieve the same functionality as onclick in an Ajax call using

Trouble in Angular JS: Syntax glitches

Transforming BSON to JSON and delivering it as an API response using Haskell

The implementation of various asynchronous Html.BeginForm scripts

Encountering an issue while attempting to send a POST response through Node Express, specifically an error related

The second parameter within the ReactJS function is not defined

Dealing with models in Vue.js is proving to be quite a challenge

Tips for populating a textfield with data from a <select> dropdown using javascript

A guide on creating extruded shapes using Three.JS

The array is not receiving the objects as intended

Why did the Mongoose fail to cast to ObjectID for this value?

Checking for null properties in Typescript objectsorHow to verify if a

Struggling to style a nested table using CSS

Performing math calculations on data inputted through a form in React Native

Is memory being corrupted during a single-line function call?

Leveraging AngularJS $filter in conjunction with ng-disabled

techniques for extracting object values and transferring them into an array

Strange response received from $http GET request on Android device running Crosswalk

There was an issue encountered while parsing the JSON Array