What is the most effective method for identifying the initial timestamp for each day, week, or month within a collection of timestamps?

Question

What is the most effective method for identifying the initial timestamp for each day, week, or month within a collection of timestamps?

I am dealing with a lengthy array of sorted timestamps representing stock price quotes. These timestamps have minimal resolution, meaning that each timestamp is at least 1 minute bigger than the previous one. However, there can be gaps during the day, especially in pre/after-hours sessions, and no data exists for weekends and holidays. My goal is to efficiently find the timestamps that mark the start of new intervals (e.g., daily, weekly, or monthly) for charting purposes.

Currently, my approach involves looping through the array, creating a Date object from each timestamp, and comparing the .getDay() values between the previous and current array elements.

let rangeSecs = data[data.length-1] - data[0];
let rangeDays = rangeSecs / 86400;  
let spt=[]; 
const minInterval = data.slice(0,9).reduce((ac,x,i,ar)=>Math.min(ac,x-ar[i-1]))

let prevdate = new Date(data[0]*1000)

for(let i=1; i < data.length; i++){
    const curDate = new Date(data[i]*1000)
    if(rangeDays > 70 && prevdate.getMonth() != curDate.getMonth()){  
        spt.push(i)
    } else if(rangeDays <= 70 && rangeDays > 14 && prevdate.getDay() > curDate.getDay()){
        spt.push(i)
                                
    } else if(rangeDays <= 14 && prevdate.getDay() != curDate.getDay()){
        spt.push(i)
    }
    prevdate = curDate;

}

The current implementation works but is slow. I am looking for suggestions on how to optimize its performance.

I have tried skipping a certain number of "safe" timestamps based on assumptions about daily trading hours. However, this optimization is not efficient as I cannot accurately predict the number of data points per day beyond regular trading hours.

SAMPLE DATA:

[1625572800,1625573700, ... , 1627676100]

javascript arrays performance time-series timestamp

Answer 1

Answer №1

Forget about converting each timestamp to a Date object. Instead, pinpoint the next important date by calculating the timestamp value for it (the current date plus a week, month, or day) and avoid checking timestamps that are before that value. Make sure to check the comments on the question for more insights.

In the code snippet below, I've included some assumptions that you can modify as needed — such as ensuring the next minimum timestamp aligns with a Monday when the interval is weekly, and always starting at midnight regardless of the time increment.

For detailed insight, refer to the comments:

// Function to identify indexes of significant dates in the given data
function findIndexes(data) {
    let rangeSecs = data[data.length-1] - data[0];
    let rangeDays = rangeSecs / 86400;  // timestamps in seconds, not milliseconds
    let spt = []; // array of indexes of "timestamps of interest"

    const scaleMin = 0;
    const scaleMax = data.length - 1;

    const interval = rangeDays > 70 ? "month" : rangeDays <= 14 ? "day" : "week";
    
    /* 
    console.log(`${displayString(data[0])} to ${displayString(data[data.length-1])}, rangeDays = ${rangeDays}, interval = ${interval}`);
    */

    spt.push(scaleMin);
    let prevdate = new Date(data[0] * 1000);
    
    let minNext = getMinNext(interval, prevdate);

    for (let i = scaleMin + 1; i <= scaleMax; ++i) {
        const value = data[i];
        
        if (value >= minNext) {
            spt.push(i);
            prevdate = new Date(value * 1000);
            minNext = getMinNext(interval, prevdate);
        }
    }

    return spt;
}

// Subroutine of `findIndexes`
function getMinNext(interval, prevdate) {
    let nextTime;

    if (interval === "month") {
        nextTime = new Date(+prevdate);
        nextTime.setMonth(nextTime.getMonth() + 1);
        nextTime.setHours(0, 0, 0, 0);
    } else {
        const intervalLength = interval === "day" ? 86400 : 7 * 86400;
        nextTime = new Date(+prevdate + (intervalLength * 1000));
        nextTime.setHours(0, 0, 0, 0);
        
        if (interval === "week") {
            while (nextTime.getDay() !== 1) {
                nextTime.setDate(nextTime.getDate() - 1);
            }
        }
    }
    
    const minNext = Math.floor(nextTime / 1000);
    
    /*
    console.log(`minNext for ${displayString(prevdate)} is ${displayString(minNext)}`);
    */
    
    return minNext;
}

Try out the functionality with random data:

// Implement the examples with different intervals
let indexes;
console.log("Days:");
const daysData = getRandomData(3 * 60 * 60); // Up to 4 hour intervals
indexes = findIndexes(daysData);
console.log(indexes.map(index => new Date(daysData[index] * 1000).toLocaleString()));
/* Continue with other intervals like weeks and months... */

Answer 2

Forget about converting each timestamp to a Date object. Instead, pinpoint the next important date by calculating the timestamp value for it (the current date plus a week, month, or day) and avoid checking timestamps that are before that value. Make sure to check the comments on the question for more insights.

In the code snippet below, I've included some assumptions that you can modify as needed — such as ensuring the next minimum timestamp aligns with a Monday when the interval is weekly, and always starting at midnight regardless of the time increment.

For detailed insight, refer to the comments:

// Function to identify indexes of significant dates in the given data
function findIndexes(data) {
    let rangeSecs = data[data.length-1] - data[0];
    let rangeDays = rangeSecs / 86400;  // timestamps in seconds, not milliseconds
    let spt = []; // array of indexes of "timestamps of interest"

    const scaleMin = 0;
    const scaleMax = data.length - 1;

    const interval = rangeDays > 70 ? "month" : rangeDays <= 14 ? "day" : "week";
    
    /* 
    console.log(`${displayString(data[0])} to ${displayString(data[data.length-1])}, rangeDays = ${rangeDays}, interval = ${interval}`);
    */

    spt.push(scaleMin);
    let prevdate = new Date(data[0] * 1000);
    
    let minNext = getMinNext(interval, prevdate);

    for (let i = scaleMin + 1; i <= scaleMax; ++i) {
        const value = data[i];
        
        if (value >= minNext) {
            spt.push(i);
            prevdate = new Date(value * 1000);
            minNext = getMinNext(interval, prevdate);
        }
    }

    return spt;
}

// Subroutine of `findIndexes`
function getMinNext(interval, prevdate) {
    let nextTime;

    if (interval === "month") {
        nextTime = new Date(+prevdate);
        nextTime.setMonth(nextTime.getMonth() + 1);
        nextTime.setHours(0, 0, 0, 0);
    } else {
        const intervalLength = interval === "day" ? 86400 : 7 * 86400;
        nextTime = new Date(+prevdate + (intervalLength * 1000));
        nextTime.setHours(0, 0, 0, 0);
        
        if (interval === "week") {
            while (nextTime.getDay() !== 1) {
                nextTime.setDate(nextTime.getDate() - 1);
            }
        }
    }
    
    const minNext = Math.floor(nextTime / 1000);
    
    /*
    console.log(`minNext for ${displayString(prevdate)} is ${displayString(minNext)}`);
    */
    
    return minNext;
}

Try out the functionality with random data:

// Implement the examples with different intervals
let indexes;
console.log("Days:");
const daysData = getRandomData(3 * 60 * 60); // Up to 4 hour intervals
indexes = findIndexes(daysData);
console.log(indexes.map(index => new Date(daysData[index] * 1000).toLocaleString()));
/* Continue with other intervals like weeks and months... */

What is the most effective method for identifying the initial timestamp for each day, week, or month within a collection of timestamps?

Answer №1

Similar questions

Javascript - Could anyone provide a detailed explanation of the functionality of this code snippet?

Modifying the size of an element with D3 when hovering with the mouse

Using jQuery UI to dynamically add a widget based on a specific class, rather than relying on

How can React Components be imported into a website that is not built with React?

Creating a form with multiple checkboxes using Material-UI components where only a single checkbox can be selected

What's the best way to implement a conditional header in React?

"Streamlining data entry with an uncomplicated HTML form input that accepts multiple values from a

Tabbed horizontal slider

The value stored in $_POST['valuename'] is not being retrieved

How to Set Up a Simple Gulp Uglify Configuration

Update the content of a div and refresh it when a key on the keyboard is pressed

Is there a way to position one DIV behind another?

Ensure that the three.js script remains in a fixed position on a page that can be

Issue with Mouse Hover not functioning properly across various 3D objects

What are the additional costs associated with pthread mutexes?

Basic AngularJS application, however I am receiving {{this is supposed to be the information}}

Can Javascript (PWA) be used to detect fake GPS or mock GPS in applications?

Steps to select an option from a drop-down menu that does not have an ID assigned

Tips on how to connect the scope from a controller to a custom directive in Angular

What is the best way to use ReactJS to remove the last 3 characters from a selected text?