Exploring the presence of duplicates in JavaScript and/or AngularJS

Question

Exploring the presence of duplicates in JavaScript and/or AngularJS

While working on my AngularJS app, I found myself iterating through a .csv file to perform element level validation. Since I couldn't find a suitable library in Angular, I decided to write custom JavaScript code to handle this scenario. Below is a sample of the data I'm working with (specifically interested in the first column CN and the 5th column NAME). My approach so far involves using if conditions to check the index of i, j, and store the values. Any suggestions or improvements would be greatly appreciated.

CN  N   ACTIVE  TYPE    NAME       NO   COM
USA 45  Engin   Fed     Anderson   #10  NA
USA 46  Sports  BB      Kobe       #1   NA
USA 32  Sports  Soccer  Kobe       #17  NA
GER 30  Sports  Soccer  Gotze      #12  NA
GER 27  Sports  Soccer  Ozil       #18  NA
ITA 38  Sports  Soccer  Buffon     #2   NA

Here's a snippet of the code:

for (var i = 0; i < file.length; i++ ){
    var singleRowData = file[i].split(',');
    singleRowData = singleRowData.filter(function(n){ return n != "" });
    for (var j = 0; j < singleRowData.length; j++){
        duplicateFunction(singleRowData, singleRowData[j], j, i);
    } 
}  

function duplicateFunction (singleRowData, singleRowDataElement, singleRowDataElementIndex,  singleRowDataIndex){
/*
  handle the duplicates
*/
}

If I come across the same values in the CN column for consecutive rows, then I want to check if there are duplicate values in the NAME column for those rows.
If there are no duplicate NAME values for different rows with the same CN value, then no error should be thrown.

In the example data provided, an exception should be flagged on the 3rd row where CN=USA and NAME=Kobe, while the rest of the data should proceed without issues.

javascript angularjs arrays multidimensional-array

Answer 1

Answer №1

To easily identify duplicates, you can concatenate the key pairs (CN + NAME) and store them in a keys object. If a new record contains the same concatenated key, it means there is a duplicate:

var file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

var keys = {}; // store processed keys
var result = []; // array with accepted rows
for ( var i=0; i< file.length; i++ ){
    var singleRowData = file[i].split(',');
    singleRowData = singleRowData.filter(function(n){ return n != "" });
    var key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    if (key in keys) {
        console.log("Duplicate at " + i + " is ignored");
    } else {
        keys[key] = 1; // register key
        result.push(singleRowData);
    }        
}  
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

For a more concise ES6 approach using an ES6 Map and reduce, consider the following implementation:

const file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

const result = [...file.reduce( (result, line) => {
    const singleRowData = line.split(',').filter(n => n != ""),
          key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    return result.has(key) ? result : result.set(key, singleRowData);
}, new Map).values()];
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

Answer 2

To easily identify duplicates, you can concatenate the key pairs (CN + NAME) and store them in a keys object. If a new record contains the same concatenated key, it means there is a duplicate:

var file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

var keys = {}; // store processed keys
var result = []; // array with accepted rows
for ( var i=0; i< file.length; i++ ){
    var singleRowData = file[i].split(',');
    singleRowData = singleRowData.filter(function(n){ return n != "" });
    var key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    if (key in keys) {
        console.log("Duplicate at " + i + " is ignored");
    } else {
        keys[key] = 1; // register key
        result.push(singleRowData);
    }        
}  
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

For a more concise ES6 approach using an ES6 Map and reduce, consider the following implementation:

const file = [
    'USA,45,Engin,Fed,Anderson,#10,NA',
    'USA,46,Sports,BB,Kobe,#1,NA',
    'USA,32,Sports,Soccer,Kobe,#17,NA',
    'GER,30,Sports,Soccer,Gotze,#12,NA',
    'GER,27,Sports,Soccer,Ozil,#18,NA',
    'ITA,38,Sports,Soccer,Buffon,#2,NA'
];

const result = [...file.reduce( (result, line) => {
    const singleRowData = line.split(',').filter(n => n != ""),
          key = singleRowData[0] + '|' + singleRowData[4]; // CN + NAME
    return result.has(key) ? result : result.set(key, singleRowData);
}, new Map).values()];
console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

Exploring the presence of duplicates in JavaScript and/or AngularJS

Answer №1

Similar questions

When making a $http request in Angular, the passport req.isAuthenticated method consistently returns false

Encountering a glitch while utilizing BreezeJS alongside AngularJS

How can I retrieve the selected items from a Listbox control?

JavaScript Drag Events in Microsoft Edge (Including IE)

Using regular expressions to replace all special characters, excluding dots

Invoke a function within a distinct context using a loop

Encountered a JavaScriptException while trying to click on an element using Selenium and Python: "arguments[0].click is not a function"

Deactivate a form on the page while another form is being used

Maximizing search engine optimization for an AngularJS website

What is the best way to modify the state of a particular element in an array when using useState in React?

Uploading a large number of images to GCP Bucket results in a failure to connect to the oauth2 token

Sending data from Django's render() method to React.js

Integrate SVG directly into the three.js module to eliminate the need for an additional HTTP request

Customizing Images using a JSON array of images

Syntax Error Unearthed: Identifier Surprise Discovered in (Javascript, ASP.NET MVC, CSHTML)

The wrapAll() method can be used to wrap list items within two columns

The callback in Jquery getJSON does not execute even when a valid JSON response is received

How can we implement a select-all and deselect-all feature in Vue Element UI for a multi-select with filtering capability?

What is the method for incorporating PHP's header() function within PayPal JavaScript code?

Step-by-step guide on repairing a countdown clock using JavaScript, HTML, and