Transform milliseconds into an ISODate representation

I have a MongoDB collection where records are stored with timestamp in milliseconds. I need to aggregate these records by hour and convert the timestamp to ISODate so that I can use MongoDB's built-in date operators ($hour, $month, etc.)

Here is an example of how records are stored:

{ 
"data" : { "UserId" : "abc", "ProjId" : "xyz"}, 
"time" : NumberLong("1395140780706"),
"_id" : ObjectId("532828ac338ed9c33aa8eca7") 
} 

I am attempting to run the following aggregate query:

db.events.aggregate(
    { 
       $match : { 
         "time" : { $gte : 1395186209804, $lte : 1395192902825 } 
       } 
    }, 
    { 
       $project : {
         _id : "$_id", 
         dt : {$concat : (Date("$time")).toString()} // need to project as ISODate
       } 
    },
    // further processing in $project or $group clause
)

The results produced look like this:

{
    "result" : [
        { 
            "_id" : ObjectId("5328da21fd207d9c3567d3ec"), 
            "dt" : "Fri Mar 21 2014 17:35:46 GMT-0400 (EDT)" 
        }, 
        { 
            "_id" : ObjectId("5328da21fd207d9c3567d3ed"), 
            "dt" : "Fri Mar 21 2014 17:35:46 GMT-0400 (EDT)" 
        }, 
            ... 
} 

My issue now is that I want to extract hour, day, month, and year from the date, but since it is projected as a string, I cannot use MongoDB's built-in date operators. How do I convert the timestamp from milliseconds to ISO date for operations like the following:

db.events.aggregate(
    {
        $match : { 
            "time" : { $gte : 1395186209804, $lte : 1395192902825 } 
        }
    },
    {
        $project : {
            _id : "$_id",
            dt : <ISO date from "$time">
        }
    },
    { 
        $project : {
            _id : "$_id",
            date : { 
                hour : {$hour : "$dt"} 
            }
        }
    }
)

Answer №1

Yes, it can be done by adding your milliseconds time to a Date object that starts with zero-milliseconds. Use the following syntax:

dt : {$add: [new Date(0), "$time"]}

I made modifications to your previous aggregation in order to achieve the desired result:

db.events.aggregate(
    {
        $project : {
            _id : "$_id",
            dt : {$add: [new Date(0), "$time"]}
        }
    },
    { 
        $project : {
            _id : "$_id",
            date : { 
                hour : {$hour : "$dt"} 
            }
        }
    }
);

The output (based on one entry of your example data) is as follows:

{
  "result": [
    {
      "_id": ObjectId("532828ac338ed9c33aa8eca7"),
      "date": {
        "hour": 11
      }
    }
  ],
  "ok": 1
}

Answer №2

In my opinion, there may not be a straightforward way to achieve this task. The aggregation framework is designed with native code and does not rely on the V8 engine for execution. Therefore, traditional JavaScript functions may not work seamlessly within the framework, contributing to its faster performance compared to other methods such as Map/Reduce.

If you are considering Map/Reduce as an alternative solution, it is worth exploring the performance implications. You can refer to this thread for more insights.

One workaround could involve obtaining a "raw" result from the aggregation framework and converting it into a JSON array. Subsequently, you can manipulate the data using JavaScript functions. For example:

var results = db.events.aggregate(...);
results.forEach(function(data) {
    data.date = new Date(data.dateInMillionSeconds);
    // The date is now stored in the "date" property
}

Answer №3

To generate a valid BSON date, you can utilize some simple date calculations with the help of the $add operator. By adding new Date(0) to the timestamp, you can achieve this. The new Date(0) represents the number of milliseconds since the Unix epoch (January 1, 1970) and is essentially equivalent to new Date("1970-01-01").

db.events.aggregate([
    { "$match": { "time": { "$gte" : 1395136209804, "$lte" : 1395192902825 } } },
    { "$project": { 
        "hour": { "$hour": { "$add": [ new Date(0), "$time" ] } }, 
        "day": { "$dayOfMonth":  { "$add": [ new Date(0), "$time" ] } },
        "month": { "$month": { "$add": [ new Date(0), "$time" ] } },
        "year": { "$year":  { "$add": [ new Date(0), "$time" ] } } 
    }} 
])

This operation will output:

{
    "_id" : ObjectId("532828ac338ed9c33aa8eca7"),
    "hour" : 11,
    "day" : 18,
    "month" : 3,
    "year" : 2014
}

Answer №4

With the introduction of Mongo 4.0, a new way has been added to convert different data types to dates using the $toDate aggregation operator (for example, converting from a long):

// { time: NumberLong("1395140780706") }
db.collection.aggregate({ $set: { time: { $toDate: "$time" } } })
// { time: ISODate("2014-03-18T11:06:20.706Z") }

To extract the hour from this:

// { time: NumberLong("1395140780706") }
db.collection.aggregate({ $project: { hour: { $hour: { $toDate: "$time" } } } })
// { hour: 11 }

Answer №5

If you're encountering a situation where the function {$add: [new Date(0), "$time"]} is returning a string type instead of an ISO date type, use the following solution:

Even after trying all available options, I was still facing failure because the new date generated from the $project was in string type format like '2000-11-2:xxxxxxx' instead of being in date type like ISO('2000-11-2:xxxxxxx'). For those experiencing the same issue as me, try using this approach.

db.events.aggregate(
    {
        $project : {
            _id : "$_id",
            dt : {$add: [new Date(0), "$time"]}
        }
    },
    { 
        $project : {
            _id : "$_id",
            "year": { $substr: [ "$dt", 0, 4 ] },
            "month": { $substr: [ "$dt", 5, 2] },
            "day": { $substr: [ "$dt", 8, 2 ] }
        }
    }
);

The result will be:

{ 
    _id: '59f940eaea87453b30f42cf5',
    year: '2017',
    month: '07',
    day: '04' 
},

You can extract hours or minutes depending on the specific string portion you want to subset and then proceed to group them based on the same date, month, or year.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Creating a new variable and then trying to access its value (which has not been defined)

I'm encountering an issue while trying to define a const variable called filteredRecipes and use it to setState. The problem is that the console is showing an error message saying that filteredRecipes is undefined, Uncaught ReferenceError: filteredRe ...

What steps can I take to ensure that my React child components will render successfully, even if a prop is absent

TLDR; Seeking solution to render child components in React even if a property of this.props is missing. My React app utilizes Yahoo's Fluxible and fetches data from a Wordpress site using WP REST API. Sometimes, the API may return incomplete data cau ...

Problem with validation in jQuery not being compatible with Kendo Button (sample code provided in jsfiddle)

It took me some time to figure out that the reason jquery-validate wasn't functioning in my Kendo Mobile application was because my submit button was a Kendo Button. Check out this jsfiddle for illustration: DEMO <div id="phoneApp" style="displa ...

My Next.js application does not display an error message when the input field is not valid

I've recently developed a currency converter application. It functions properly when a number is provided as input. However, in cases where the user enters anything other than a number or leaves the input field empty, the app fails to respond. I aim t ...

Managing the result efficiently when asp.net mvc ModelState IsValid is false

My colleagues and I are currently working on a CRUD application using .net mvc4. The project involves rendering dynamic content through jQuery based on customer choices. One challenge we face is the need to create multiple hidden inputs to pass additional ...

Struggling to make the fancybox feature function with html/php

I've been trying to find a solution to this problem repeatedly, but I just can't seem to crack it. All I want to do is use fancybox to display an image. Since this is my first time using fancybox, I'm sure someone with more experience will ...

Objects within an array are not sorted based on their properties

I'm currently struggling with sorting an array of objects based on a specific property. Despite my efforts, I can't seem to figure out what's causing the issue as it remains unsorted. Would appreciate some assistance. You can take a look at ...

AJAX not showing validation error message

For the past two days, I've been grappling with an issue and can't seem to pinpoint where it's coming from. After leaving the textbox, the Ajax call functions correctly and returns results as either true or false, triggering the success fun ...

Methods for organizing consecutive elements within an array in Javascript/Typescript

Let's explore this collection of objects: [ { key1: "AAA", key2: "BBB" }, { key1: "BBB", key2: "CCC" }, { key1: "CCC", key2: "DD ...

Unable to render Google map on Vue CLI component

I am currently using the google map api to develop a basic application with vue.js. Interestingly, when I utilize a simple html and javascript setup with the api key, everything runs smoothly. However, once I transition the same process to vue, the map fai ...

New Exploit Allows for Arbitrary Code Execution by Bypassing Sandboxes - The popular web template engine, Jade, has

I'm looking to create a todo app using node.js, express.js, and mongoDb. I've installed express and followed these steps: npm install -g express-generator cd todo_api npm install But encountered an error while running npm install. After runni ...

Give it a little time before uploading a widget onto the page

As a newcomer to programming, I recently came across this code from an open source project. I am in the process of loading a widget onto a website. Instead of having the widget load instantly, I would like it to wait 10 seconds before displaying. Below i ...

The issue of not being able to add data to the client is encountered in a local setup involving Socket.io, MSSQL

I have a large dataset that needs to be retrieved from a SQL server using Node.js and then broadcasted to clients in real-time using Socket.IO. I've been successful in fetching the data, but the UI on the client side isn't updating. Although I c ...

Managing media file transfers using multer and mongodb in a Node.js environment

As I work on developing a blog-style application for a business website, I have successfully implemented a login system and a basic blog structure. My current focus is on enabling users to upload images along with their blog posts. At the moment, I am trou ...

Calculate the total amount based on the selected value from the radio button

For example, radiobutton A = value X, radiobutton B = value Y. Below is the code snippet I am utilizing: Javascript file: <script type="text/javascript" $(document).ready(function () { $("div[data-role='footer']").prepend(' ...

Does the user need to download the scripts from Google API multiple times during the insertion process?

Concerned about the potential need for multiple downloads of a script from an unknown location on the user's computer, I would like to explore options to mitigate this issue. Specifically, I am considering creating a condition to check the download st ...

Tips for displaying a jQuery error message when a key is pressed

I am working with a textarea that has a word count limit of 500. I need to display an error message below the textarea if the word count exceeds 500. I have successfully calculated the word count, but I am unsure how to display the error message and preve ...

The received URL from the POST request in React is now being opened

After completing an API call, I successfully received the correct response in my console. Is there a way to redirect my React app from the local host to the URL provided (in this case, the one labeled GatewayUrl under data)? Any assistance would be greatly ...

Issue with Vue2's v-text functionality

Recently, I've delved into Vue2 and encountered a problem with form submission and validation on a single page. The issue lies in the error display process – I want errors to be shown beneath each form input as soon as they occur, rather than waitin ...

Express file response problems affecting React front-end communication

I am currently developing a basic questionnaire on a React full stack website that uses React, Express, and SQL. Right now, my main goal is to return an error message from React and have it displayed on the front end. This is the code for my express endp ...