Eliminate redundant items within an array by comparing their unique identifiers and dates

I am looking to eliminate duplicate object entries based on id, while displaying the most recent object entry based on pubDate. Take a look at my example array below:

var arr = [
  {fileID: "1234", pubDate: "04/13/2018", id: "4979146", jobID: "9146", downloadURL: "", title: null},
  {fileID: "1235", pubDate: "04/13/2020", id: "4979147", jobID: "9147", downloadURL: "", title: null},
  {fileID: "1236", pubDate: "02/23/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1237", pubDate: "01/15/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1238", pubDate: "05/17/2019", id: "4979146", jobID: "9146", downloadURL: "", title: null}
];

Desired Output:

[
    {fileID: "1236", pubDate: "02/23/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null}, 
    {fileID: "1235", pubDate: "04/13/2020", id: "4979147", jobID: "9147", downloadURL: "", title: null},
    {fileID: "1238", pubDate: "05/17/2019", id: "4979146", jobID: "9146", downloadURL: "", title: null}
];

This is how I attempted to solve it:

var result = arr.reduce((unique, o) => {
  if (!unique.some(obj => obj.id === o.id)) {
    unique.push(o);
  } 
  return unique;
}, []);

... however, this method appears to be missing the logic to retain only the latest record.

javascriptarrayssortingecmascript-6filter

Answer №1

Is this solution effective for your needs?

var array = [
  {fileID: "1234", pubDate: "04/13/2018", id: "4979146", jobID: "9146", downloadURL: "", title: null},
  {fileID: "1235", pubDate: "04/13/2020", id: "4979147", jobID: "9147", downloadURL: "", title: null},
  {fileID: "1236", pubDate: "02/23/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1237", pubDate: "01/15/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1238", pubDate: "05/17/2019", id: "4979146", jobID: "9146", downloadURL: "", title: null}
];


function removeDuplicates(array) {
    const uniqueHash = array.reduce((acc,obj)=>{
        if (!acc[obj.id]) {
            acc[obj.id] = [];
        }
        acc[obj.id].push(obj);
        return acc;
    }
    , {});
    return Object.values(uniqueHash).reduce((acc,list)=>{
        list.sort((a,b)=>new Date(b.pubDate) - new Date(a.pubDate));
        acc.push(list[0]);
        return acc;
    }
    , []);
}

console.log(removeDuplicates(array));

Answer №2

You're on the right path, but make sure to include the comparison when searching for the result to override. For instance:

var arr = [
  {fileID: "1234", pubDate: "04/13/2018", id: "4979146", jobID: "9146", downloadURL: "", title: null},
  {fileID: "1235", pubDate: "04/13/2020", id: "4979147", jobID: "9147", downloadURL: "", title: null},
  {fileID: "1236", pubDate: "02/23/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1237", pubDate: "01/15/2021", id: "4979148", jobID: "9148", downloadURL: "", title: null},
  {fileID: "1238", pubDate: "05/17/2019", id: "4979146", jobID: "9146", downloadURL: "", title: null}
];

function parse(pubDateStr = '') {
  const parts = pubDateStr.match(/\d+/g);
  if (!parts || parts.length !== 3) throw new Error(`Unexpected date format: [ ${pubDateStr} ]`); 
  return `${parts[2]}-${parts[0]}-${parts[1]}`;
}

const results = Object.values(arr.reduce((hash, curr) => {
  const prev = hash[curr.jobID];
  if (!prev || parse(prev.pubDate) < parse(curr.pubDate)) {
    hash[curr.jobID] = curr;
  }
  return hash;
}, {}));

// added sorting here as it seems to be sorted in your example, 
// albeit not mentioned in OP
console.log(results.sort((a, b) => b.jobID - a.jobID));

Avoid iterating through the list twice (as seen in an alternative solution), and there's no need to store all records and then sort them. Simply perform the comparison when deciding whether a record should be included.

Using a specific function to parse MM/DD/YYYY instead of new Date has its benefits: it guards against browser differences and inconsistencies, as mentioned on MDN page, and handles missing or incorrect dates more effectively (compared to new Date(null) which gives 1970-01-01, for instance).

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the most efficient way to use map-reduce in TypeScript to filter a list based on the maximum value of an attribute?

Recently, I came across a list that looked something like this: let scores = [{name: "A", skills: 50, result: 80}, {name: "B", skills: 40, result: 90}, {name: "C", skills: 60, result: 60}, {name: "D", skills: 60, ...

javascriptangulartypescript

What is the mechanism Angular utilizes to determine the exact location of directives within a webpage?

Can you explain how Angular is able to locate the directives on a webpage and establish connections with or observe those elements? I've searched through the DOM reference, but it seems like methods like getElementbySomething and querySelectorAll may ...

javascriptangularjs

Adding data from a database into an object in PHP for temporary use during the loading process can be achieved by following

I'm a beginner in PHP and I have some code that retrieves category type data from a database. I want to temporarily store this data in a PHP object while the page is loading. Initially, I need to load all predefined data and then use it when a certain ...

javascriptphpjquerymysqljson

What is the best way to transform this string into a Luxon Datetime object using Typescript?

Here is a snippet of Javascript/Typescript code that I have for converting a string into a Luxon DateTime: import { DateTime } from 'luxon'; const x = '2023-10-27T01:00:57.830+00:00' const y = DateTime.fromFormat(x, 'yyyy-MM-dd ...

javascripttypescriptdatetimeluxon

What causes the 'then' method of my angular service to return a resolved promise instead of the expected value?

I am perplexed as to why the "result" in this code snippet is a resolved promise instead of the actual value: searchService.getLink(vm.queryObject).then(function (result) { console.log(result); }); The implementation for the getLink() function is pro ...

javascriptangularjsangular-promise

Validate selected checkbox using JavaScript

Is there a way for me to achieve real-time updates when checking and unchecking multiple checkboxes in a list? Here's the code snippet I currently have: window.onload = function () { var input = document.getElementById('listTaxi'); fu ...

javascriptjquerycheckbox

Vuetify's data table now displays the previous and next page buttons on the left side if the items-per-page option is hidden

I need help hiding the items-per-page choices in a table without affecting the next/previous functionality. To achieve this, I've set the following props: :footer-props="{ 'items-per-page-options':[10], &apo ...

javascripthtmlcssvue.jsvuetify.js

"Optimizing Pytables CArray read performance for faster data retrieval

I have successfully created a chunked array using the following code: import tables FILTERS = tables.Filters(complib='lzo', complevel=1) h5file = tables.openFile('file.h5', mode='w', filters=FILTERS) x = h5file.createCArray(h ...

arrayspytableschunked

A guide on properly formatting an Array of Objects in JavaScript based on the contained data

My goal is to create an Array of Object that groups the same weekday data into one object. For example, if we have weekday:1 in five objects, the desired output would be: { ..., weekDay: 1, repeated: 5 } While I can achieve this by hard coding a solution ...

javascriptarraysreactjssortingformatting

Issues with Ajax calls not functioning properly within CakePHP

I'm attempting to make an AJAX request in CakePHP. The submit button is marked as #enviar and the action as pages/contato. This is the code for my AJAX request: $(document).ready(function() { $('#enviar').click(function(){ $. ...

phpjavascriptjqueryajaxcakephp

using configureStore instead of createStore

After updating my packages, I received a notification from VScode indicating that the createStore function was deprecated. This prompted me to go ahead and replace it with the recommended alternative. In my store file, I left the original line as a commen ...

javascriptreactjsreduxreact-reduxnext.js

AngularJS filters not functioning properly after applying additional filters

I have a page where I am filtering data based on multiple values. For each block in the list, I am using the following code snippet: data-ng-repeat="c in vm.competencies | filter : c.competencyTypeID = <number>" While some of the filters are workin ...

javascriptangularjs

The function given to requestAnimationFrame is not being triggered as expected

I have successfully implemented an infinite loop animation using setInterval. However, I want to enhance the performance by switching to requestAnimationFrame(). Unfortunately, the function supplied to requestAnimationFrame() is not being called for some r ...

javascripthtmlrequestanimationframe

HTML element resizing unexpectedly due to browser interactions

I recently created a sleek HTML transparent header bar using the following CSS: #head-bar{ position:fixed; top: 0px; width:100%; left:0px; min-height:25%; z-index:2; background-color:#000; background: rgba(0, 0, 0, 0.5); } ...

javascriptjqueryhtmlcss

Production deployment may encounter issues as crypto/webcrypto is not defined

When working with a Next.js API route, I start by importing webcrypto like this: import { webcrypto } from 'crypto' Later on, I utilize it in the following manner: const random = webcrypto.getRandomValues(new Uint8Array(8)) While testing on loc ...

javascriptnext.jswebcrypto-api

Guide on displaying a real-time "Last Refreshed" message on a webpage that automatically updates to show the time passed since the last API request

Hey all, I recently started my journey into web development and I'm working on a feature to display "Last Refreshed ago" on the webpage. I came across this website which inspired me. What I aim to achieve is to show text like "Last Refreshed 1 sec ago ...

javascripthtmlexpressnpm-package

Why am I getting the error "TypeError: Object(...) is not a function" when attempting to integrate Vanilla Tilt with React?

Looking for assistance on implementing Vanilla Tilt in my React application, I have referenced the following example: https://codesandbox.io/s/vanilla-tilt-with-react-n5ptm The code snippet I am working with is as follows: import React, { Component, useEf ...

javascriptreactjstypeerror

Efficient React search feature with pagination that avoids redundant setState calls

Currently in the process of incorporating search functionality with pagination in React. After reviewing numerous examples, it appears they all involve using setState() twice, both before and after making an AJAX call to the backend. Here is a snippet of m ...

javascriptreactjspaginationstate

Incorporate a custom style sheet into your Express application

Utilizing ejs, express, and parse.com for my web application backend has presented a challenge when it comes to adding stylesheets. Despite searching for solutions, I have not been able to resolve the issue. Sharing my code in hopes of finding a solution. ...

javascriptexpressparse-platformejs

Read the characters from the stream

Is there a specific class available for stream reading that allows me to read only a certain number of char from a string or byte from a byte[]? For example, when reading a string: string chunk = streamReader.ReadChars(5); // Read next 5 chars And when ...

c#streamarraysreader