What is the best way to handle processing large amounts of data stored in a file using JavaScript within the

Suppose my file contains the following data and is located at /home/usr1/Documents/companyNames.txt

Name1

Name 2

Name 3

Countless names...

I attempted to use this code:

$> var string = cat('home/usr1/Documents/companyNames.txt');
$> string = string.split('\n');
$> db.records.find({field: {$in: string}});

According to the code in the link Can I read a csv file inside of a Mongo Shell Javascript file?

This method works fine for small files, but when dealing with files containing millions of lines, it crashes as all the lines try to fit into memory. Is there an alternative way to process large files within the Mongo shell using JavaScript?

javascriptmongodbmongo-shell

Answer №1

In handling extensive queries, Mongo may not be the best choice.

One alternative could be utilizing Javascript in the following manner:

var text = fetchFile('home/usr1/Documents/companyNames.txt');
text = text.split('\n');
let output = [];
text.forEach(line => output.push(db.records.find({field: {$eq: line}})));

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Flickity remains in plain sight on desktop devices

I am trying to hide the flickity slider on desktop and larger devices. I have followed the instructions in the documentation, but for some reason, it's not working as expected. Here is how the div looks: <div class="w-full flex pl-4 pb-16 overflo ...

javascriptcssvue.jsflickityvue-flickity

Updating is not happening with ng-repeat trackBy in the case of one-time binding

In an attempt to reduce the number of watchers in my AngularJS application, I am using both "track by" in ngRepeat and one-time bindings. For instance: Here is an example of my view: <div ng-repeat="item in items track by trackingId(item)"> {{ : ...

javascriptangularjs

Incorporating dynamic numerical values into image names within a Vue JS application

I have linked an image with the following code <img title="head" :src="availableParts.heads[selectNextHeadIndex].src"/> This image is called from a JSON file: { id: 1, description: 'A robot head with an ...

javascriptvue.jsvuejs3

Every time I attempt to execute route.js in my API folder, I encounter a persistent 404 error

Project Structure: https://i.stack.imgur.com/qMF6T.png While working on my project, I encountered an issue with a component that is supposed to call an API function. However, it seems like the file is not being found. I have double-checked the directory ...

javascriptreactjsnext.js

Steps for performing position by position sorting within an array of arrays of numbers using the Lodash library

My task involves sorting an array of strings: ['1.2.3', '1.5.2', '1.23', '1.20.31'] I am looking for a way to sort the array by splitting each string separated by dots, such as 1.2.3 into ['1','2&apo ...

javascriptlodash

Develop a feature within a standard plugin that allows users to add, remove, or refresh content easily

I have developed a simple plugin that builds tables: ; (function ($, window, document, undefined) { // Define the plugin name and default options var pluginName = "tableBuilder", defaults = { }; // Plugin constructor func ...

javascriptjquery

Tips for organizing data when parsing JSON in Javascript

I am facing a challenge with maintaining the order of JSON data that I am parsing using Javascript and displaying in an HTML SELECT element. The incoming data is already sorted, but I am encountering issues sustaining this order after decoding the JSON str ...

javascriptjson

Show a notification if the MongoDB collection is devoid of any data

One of the features of my website is a notices section, which retrieves data from my Mongo Database. In case there are no new notices in the collection, I want to display a message saying "No new notices." The following code snippet shows how I am impleme ...

mongodbexpressmongooseejs

Issue encountered during installation of mongojs in nodejs is as follows

Encountering an error while attempting to download MongoDB. Any advice you could offer? https://i.stack.imgur.com/UwvgF.png Managed to fix some Python setup errors, but still facing issues with Kerberos. Assistance would be greatly appreciated. https:// ...

node.jsmongodbnpmmongojs

The AJAX request encountered an error due to an Unexpected End of JSON Input

My AJAX code is encountering an error message. parsererror (index):75 SyntaxError: Unexpected end of JSON input at parse (<anonymous>) at Nb (jquery.min.js:4) at A (jquery.min.js:4) at XMLHttpRequest.<anonymous> (jquery.min.js: ...

javascriptphpjqueryjsonajax

Increase the placeholder's line height and font size for the InputBase component in Material UI

Hello, I am new to material UI and currently using it for my website development. I am trying to customize the placeholder of the inputbase in material ui by increasing their lineHeight and fontSize. However, I am having trouble accessing the placeholder A ...

javascriptreactjsmaterial-uifrontendplaceholder

Is SWR failing to provide outdated data?

My understanding was that SWR should display the cached data upon page load before refreshing with new information from the API. However, in my Next.js app with a simple API timeout, the "loading" message appears every time due to the 5-second delay I adde ...

javascriptreactjsnext.jsswr

Steps for enabling a feature flag via API in specific environments

Within my project, I am working with three separate environments and I am looking to activate a feature flag only for a specific environment. Is it feasible to toggle an unleash feature flag using the API for just the development environment? The code snip ...

javascriptnode.js

Executing a controller function in AngularJS from an event

I am facing an issue with my AngularJS code. I have defined a function within my controller, and I am trying to call it inside an event listener, but I keep getting an 'undefined' error. Here is how the controller code looks like: inputApp.cont ...

javascriptangularjsevents

Using the Vue.js Compositions API to handle multiple API requests with a promise when the component is mounted

I have a task that requires me to make requests to 4 different places in the onmounted function using the composition api. I want to send these requests simultaneously with promises for better performance. Can anyone guide me on how to achieve this effic ...

javascripttypescriptvue.js

Is it possible to establish role-based access permissions once logged in using Angular 6?

Upon logging in, the system should verify the admin type and redirect them to a specific component. For example, an HOD should access the admi dashboard, CICT should access admin2 dashboard, etc. Below is my mongoose schema: const mongoose = require(&apo ...

javascriptangulartypescript

Discovering the differences between input values in HTML when using scripts

I have a code snippet here for an HTML project. The code includes an input field for username and password. I am looking to compare the user's input with a specific value using JavaScript. My question is, what code should be included in the button cli ...

javascripthtmlif-statement

What is the reason the child component is not being displayed?

The code within the APP.js component looks like this: import React from "react"; import Exam from "./exam.js"; export default function App() { return ( <Exam> <h1>hashemi</h1> </Exam> ); } Similarly, the ...

javascripthtmlreactjsreact-nativejsx

Communication between Laravel and controller using AJAX for exchanging information

I have a specific AJAX function being called from a view: function gatherProductData() { var productIds = []; $('#compare-widget tbody tr').each(function(i, ele) { productIds[i] = $(ele).data('product-id'); }); ...

javascriptphpjqueryajaxlaravel

Is there a way to sequentially execute requests in a loop?

My goal is to extract a list of URLs from the request body, pass them to a request function (using the request module) to retrieve data from each URL, and then save that data to MongoDB. The response should be sent only after all requests are completed, in ...

javascriptnode.jsmongodbexpressrequest