What is the best way to handle processing large amounts of data stored in a file using JavaScript within the

Suppose my file contains the following data and is located at /home/usr1/Documents/companyNames.txt

Name1

Name 2

Name 3

Countless names...

I attempted to use this code:

$> var string = cat('home/usr1/Documents/companyNames.txt');
$> string = string.split('\n');
$> db.records.find({field: {$in: string}});

According to the code in the link Can I read a csv file inside of a Mongo Shell Javascript file?

This method works fine for small files, but when dealing with files containing millions of lines, it crashes as all the lines try to fit into memory. Is there an alternative way to process large files within the Mongo shell using JavaScript?

javascriptmongodbmongo-shell

Answer №1

In handling extensive queries, Mongo may not be the best choice.

One alternative could be utilizing Javascript in the following manner:

var text = fetchFile('home/usr1/Documents/companyNames.txt');
text = text.split('\n');
let output = [];
text.forEach(line => output.push(db.records.find({field: {$eq: line}})));

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

creating intricate services using ngTagsInput

Integrating ngTagsInput into my blog allows users to add existing or custom tags to new posts. The blog utilizes a firebase datasource accessible through a factory: servicesModule.factory("postsDB", function($resource){ return $resource("https://data ...

javascriptangularjsangular-servicesng-tags-input

My Node setup is not displaying my scene with the THREE.js Software Renderer

Struggling to incorporate 3d graphics into Node.js environment, I stumbled upon the Software Renderer after exhaustive research. The renderer is up and running, but I am facing difficulties in rendering my scene. The issue lies in the fact that my 3d obje ...

javascriptnode.jsexpressthree.jswebgl

The execution of ajax within a script being called by another ajax request is not functioning as expected in PHP

I am currently working on a project that involves three files, each serving a specific purpose as outlined below: //File1.php $('button.button1').click(function(e){ $.ajax({ type: "POST", url: "file2.php ...

javascriptphpjqueryajax

Node.js and MySQL: Troubles with closing connections - Dealing with asynchronous complexities

I am currently working on a Node program to populate my MySQL database with data from files stored on disk. While the method I'm using seems to be effective, I am facing challenges in ensuring that asynchronous functions complete before ending the con ...

javascriptmysqlnode.jsasynchronousnode-mysql

Update the content within a document and implement jQuery to make it clickable

Within my webpage, there is a random occurrence of the word -FORM-. I am looking to replace this word with another text that includes dashes for creating a clickable div. Despite having some code that successfully replaces the text, it lacks the function ...

javascriptjquery

Looking to show an image when a checkbox is ticked and hide it when it is unticked in HTML?

How do I make a specific image appear when a user clicks on a checkbox? I'm unsure if I should use jQuery, Ajax, or another method. What is the best approach for this task? Your assistance would be greatly appreciated. I have successfully created the ...

javascriptjqueryajaxhtml

Oops! Module './api/routers' not found

Hello, I hope everyone is doing well... Currently, I am working on developing a NextJS single-page web application. To create a custom server for NextJs, I am utilizing Express, MongoDB, and nodemon for hot reload functionality. Upon starting the server, ...

javascriptnode.jsreactjsexpressnext.js

Tips for customizing MUI PaperProps using styled components

I am trying to customize the width of the menu using styled components in MUI. Initially, I attempted the following: const StyledMenu = styled(Menu)` && { width: 100%; } `; However, this did not have any effect. After further research, I ...

javascriptcssreactjsmaterial-ui

Implement the addition of a class upon hovering using AngularJS

My goal is to include a new class when hovering over the li element using Angular in the code snippet below. <li ng-mouseenter="cola-selected=true" class="pull-left" ng-class="{'selected' : cola-selected}"> <a href="interna.html"> ...

javascriptcssangularjs

Utilize NodeJS API to convert a base64 data string into a downloadable PDF file

My NodeJS API is set up to communicate with another API and fetch a data object in the form of a Base64 string. The client making the API call needs to be able to download a PDF file generated from this base64 data. What is the best way to return the dat ...

javascriptnode.jspdfbase64

Facebook has broadened the scope of permissions for canvas applications

I am in the process of developing a Facebook canvas application that requires extended permissions for managing images (creating galleries and uploading images) as well as posting to a user's news feed. I am currently facing challenges with obtaining ...

phpjavascriptfacebook

An issue occurred during the project compilation using npm

My project installation process is giving me some trouble. Initially, when I run npm install, it successfully installs all the dependencies. However, when I proceed to execute npm run compile, I encounter an error. Below is the log file for a better under ...

javascripttypescriptnpmlerna

What is the process for transmitting information via a Link in Next.js?

In my upcoming 13.1.5 version, I am faced with a challenge of sending the value of contact.name through a Link in my component. I am looking for the most effective approach to achieve this. The initial code block for the Link represents my original code. ...

javascriptreactjsnext.jsnext-link

Continuously improving the form as they evolve

I am interested in creating a form that automatically sends data when users make changes to it. For instance: Imagine a scenario where a moderator is editing an article and changes values in a select field. As soon as the change is made, an ajax request ...

jqueryajaxjavascript

Enhancing the accessibility of Material UI Autocomplete through a Custom ListboxComponent

I have developed a custom ListboxComponent for the MUI Autocomplete component in MUI v4. How can I ensure that it meets the necessary accessibility requirements, such as navigating through options using the arrow keys? <Autocomplete ListboxComponent ...

javascriptreactjsmaterial-ui

What is the best way to authenticate an admin in the front-end using backend technologies like Node.js, Angular, and MongoDB?

Within the user model, there is a property named isAdmin with a default value of false. In MongoDB, I have manually created an admin account with the isAdmin property set to true. When logging in as an admin, the program verifies this and displays "admin ...

node.jsangularmongodbadminmean-stack

Why is the defaultDate property not functioning properly in Material-UI's <DatePicker/> component with ReactJS?

I recently implemented the <DatePicker/> component from Material-UI ( http://www.material-ui.com/#/components/date-picker ) in my project. I encountered an issue while trying to set the defaultDate property to the current date on my computer, as it r ...

javascriptcssreactjsdatepickermaterial-ui

angucomplete-alto automatically fills in data based on another input

Having two autocomplete select boxes with a unique feature has been quite interesting. The first input accepts a code that is related to a label, autofilling the second input with the corresponding object once the code is selected in the first input. Howev ...

javascriptangularjsautocomplete

Encountering problems with createMediaElementSource in TypeScript/React when using the Web Audio API

Currently, I am following a Web Audio API tutorial from MDN, but with a twist - I am using TypeScript and React instead of vanilla JavaScript. In my React project created with create-react-app, I am utilizing the useRef hook to reference the audio element ...

javascriptreactjstypescriptweb-audio-api

Implementing Multiple HTML Files Loading in QUnit

Currently, I am utilizing QUnit for unit testing JavaScript and jQuery. The structure of my HTML document is as follows: <!DOCTYPE html> <html> <head> <title>QUnit Test Suite</title> <script src="../lib/jquery.js">< ...

javascriptjqueryqunit