Executing numerous queries in mongoDB

I need to query multiple collections in MongoDB. Here is an example of the data structure:

Collection stops {
    { stop_id : 1, stop_name: 'a'},
    { stop_id : 2, stop_name: 'b'}, ...

Collection stop_time {
    { stop_id : 1, trip_id: 40},
    { stop_id : 2, trip_id: 41}, ...

Collection trips {
    { trip_id : 40, route_id: 400},
    { trip_id : 41, route_id: 401}, ...

Collection route {
    { route_id : 400, route_name: 'foo'},
    { route_id : 401, route_name: 'bar'}, ...

I am looking for a way to determine the route_name that matches each stop_name. This data structure follows GTFS format.

Is there a straightforward way to perform queries that address this issue?

Thank you

javascriptmongodbgtfs

Answer №1

When working with MongoDB, it's important to consider an approach that involves creating one collection containing all the necessary data and executing just a single query.

There are several reasons why this is crucial:

  1. Transactions are not supported in MongoDB
  2. Joins are not available
  3. Only atomic operations on one document can be performed
  4. The aggregation framework cannot be utilized

If transitioning to a single collection schema is not feasible, an alternative solution may involve the following code snippet:

    var _ = require("underscore")._;
    function get_all(cb) {
        stops.find({}, function(err, stps, cb) {

            stop_time.find({}, function(err, stptime) {

                var all = _.map(stps, function(e) {
                    e.trip_id = stptime[e.stop_id].trip_id;
                    return e;
                });

                trips.find({}, function(err, trips) {

                    all = _.map(all, function(e) {
                        e.route_id = trips[e.trip_id].route_id;
                        return e;
                    });

                    route.find({}, function(err, routes) {

                        all = _.map(all, function(e) {
                            e.route_name = routes[e.route_id].route_name;
                            return e;
                        });

                        cb(all)
                    });
                });

            });
        });
    }

It's advisable to exercise caution when dealing with large amounts of data in the database by considering the use of Cursors. http://mongoosejs.com/docs/api.html#querystream_QueryStream

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Elegant Decline of Javascript Functionality - Imported Web Assets

Looking for assistance from experienced JS coders. I'm currently working on a Wordpress website that utilizes jQuery AJAX methods to reload either the entire content area or just the main content section based on different navigation clicks. My aim i ...

javascriptajaxjquery

Ways to stop two JavaScript files from running at the same time

After combining two JavaScript files, I am facing some issues. The first file handles validation while the second one has an ajax plugin for form submission after validation. When I include these files in the header section, they both run simultaneously. H ...

javascriptjqueryhtmlajaxvalidation

Transferring $scope information to resolve in $stateProvider.state

In the app.teams.show parent state, "team" is stored in $scope.data.team. From within a controller, I can access $scope.data.team and thus $scope.data.team.organization_id. The question is: How can I retrieve $scope.data.team.organization_id from inside t ...

javascriptajaxangularjsangularjs-scopeangularjs-routing

The function of the 'Class' attribute is not functioning properly

I am currently working on a search field with a drop-down menu that determines the type of data being searched. This, in turn, dictates the input type for the search field. For example, if I am searching by "name" or "ID number", the input type is set to t ...

javascriptjqueryhtml

Utilizing winston to generate multiple log files with set maximum sizes and daily rotation

Currently, I am utilizing winston for logging with a maximum size and daily rotation. I am interested in having this functionality with one file per API endpoint to define multiple log files. Is there a way to achieve this? Displayed below is my winston ...

javascriptnode.jstypescriptloggingwinston

Node.js configuration for setting the maximum size of old space

As someone who is new to working with nodejs and mongodb, I encountered an issue while attempting to read about 100000 records from my mongodb using a nodejs application. Upon trying to retrieve the 100000 records, I came across the following error message ...

node.jsmongodbout-of-memory

"What is the best way to connect a md-input to the value of a md-slider

I'm in the process of developing an application using Angular 2.0/meteor, and I'm facing a challenge with binding an input to md-slider. Below is the HTML code for the component: <div class="panel-body"> <form [formGroup]="filtreFor ...

javascriptjqueryangularmaterial-design

The Authorization header in POST and PATCH fetch requests is stripped by Typescript

I have developed an API in C# that utilizes JWT tokens for authorization. On the frontend, I store these tokens in local storage and retrieve them when making a request. GET or DELETE requests work seamlessly, as I can verify through console.log() that t ...

javascriptc#typescript

What is the best method for importing a single module within a monorepo project using JavaScript and NPM?

I've organized my codebase into a monorepo with the following structure: ➜ yw git:(master) tree . ├── package.json ├── packages │ ├── common │ │ ├── package.json │ │ ├── src │ │ │ ├─ ...

javascriptnode.jsnpmmonorepoyarn-workspaces

Distinguishing Between Two Arrays Using MongoDB

Hello there, I'm wondering if it's possible to find the variance between two arrays in mongoDB. Collection1 ID Name 1 Emp1 2 Emp2 3 Emp3 Collection2 ID Name 2 Emp2 EmployeeDetails I ...

c#mongodb

Where did my HTML5 Canvas Text disappear to?

I am encountering a common issue with my code and could use some guidance. Despite numerous attempts, I can't seem to figure out why it isn't functioning properly. To better illustrate the problem, here is a link to the troublesome code snippet o ...

javascripthtmlcanvas

Exploring the wonders of Vue.js and Laravel's type casting techniques

I have implemented DB2-style IDs for my database records in my Laravel 5.7 application, like this example: 201402241121000000000000. When trying to use it in my Vue component, I used the following syntax: <mycomponent v-bind:listing-key="{{ $listing-&g ...

javascriptlaravelvue.jsvuejs2

Utilizing IISNode and/or nodemon for efficient node.js development on Windows platform

For my node.js application running on Windows, I currently utilize IISNode both locally during development and on production hosting. Would incorporating nodemon (or a comparable module that monitors file changes and restarts node.exe when necessary) pro ...

javascriptwindowsnode.jsiisnodenodemon

Utilizing ng-disabled with a custom directive

Is it possible to achieve the following: <directiveName parameter1=value1 parameter2=value2 ng-disabled="true"> </directiveName> I tried this but couldn't get it to work and didn't find many examples of its use. However, I can togg ...

javascriptangularjs

Calculating the mean value of a multidimensional array that has been sorted in JavaScript

Check out the structure of my JSON File below. { "questions": ["Question1", "Question2"], "orgs": ["Org1", "Org2", "Org3"], "dates": ["Q1", "Q2", "Q3"], "values": [ [ [5, 88, 18], [50, 83, 10], ...

javascriptarrays

Displaying a collection of objects in HTML by iterating through an array

As someone new to coding, I am eager to tackle the following challenge: I have designed 3 distinct classes. The primary class is the Place class, followed by a restaurant class and an events class. Both the restaurant class and events class inherit core p ...

javascriptarraystypescriptloops

Leveraging AngularJS to Connect Controller Functions with Service Attributes

Just dipping my toes into the world of ngJS. I've managed to successfully bind the service objects to controllers. Is this the best way, or is there a more recommended approach? I'm also curious why this functionality seems limited to objects ...

javascriptangularjsangularjs-scope

Displaying HTML with AngularJS dynamic data-binding

Here is a sample view: <div ng-show=""> <div style='background-color: #13a4d6; border-color: #0F82A8'> {{headerdescription}} <div style='float: right'>{{price}} $</div> </div> <div style=&apos ...

javascriptangularjshttp

Is there a way to automate the distribution of tasks to users in order to ensure that each user receives an equal number of assignments?

I'm in the process of developing an automated task manager that assigns tasks to users based on their role. Currently, I'm randomly selecting a user with the same role as the task from the list of users and assigning the task to them using math.r ...

javascriptnode.jsmongodbexpress

A guide to handling deep updates with subdocuments in Mongodb/Mongoose

In this scenario, I am looking to utilize mongoose. Consider a Schema structured like the following: const userSchema = new Schema({ name: { first: { type: String, required: true }, last: { type: String, required: true }, }, email: { type: S ...

javascriptnode.jsmongodbmongoose