File reading and processing must be completed before attempting to write to a new file. This is a promise-related stipulation

Feeling completely lost and stuck in a rut. Promises seemed straightforward until I actually tried to implement them. Sharing my basic code here without any promise attempts for simplicity's sake. After taking a few months hiatus from this, I returned to find myself struggling with a script that reads files line by line from a directory and adds those lines to an array. The ultimate goal is to then write all those lines to a new file.
The issue lies in the fact that the function writeAllLinesToFile is being called before the allTheLines array is populated with lines read from the directory's files. As a result, the array remains empty and nothing gets written to the new file. I've made multiple failed attempts at solving this problem and ended up complicating my actual code further. Can anyone help me get past this obstacle?

var allTheLines = [];

function processFile(inputFile) {
    var fs = require('fs');
    var readline = require('readline');
    var instream = fs.createReadStream(inputFile);
    var outstream = new (require('stream'))();
    var rl = readline.createInterface(instream, outstream);

    // action to take when a line is read 
    rl.on('line', function (line) {
        // console.log("pushing: ", line);
        allTheLines.push(line);
    });

    // action to take when there are no more lines to read
    rl.on('close', function (line) {
        console.log('...end of a file');
    });
}

function writeAllLinesToFile() {
    var fs = require('fs');
    for (var i = 0; i < allTheLines.length; i++) {
        console.log("line: : ", allTheLines[i]);
        fs.appendFile('./allTheLines.html', allTheLines[i] + '\n', function (err) {
            if (err) throw err;
        });
    };
    console.log("Done writing.");
}

// ***************************************
// Execution Starts Here
// ***************************************
var fs = require('fs');
fs.readdir('./filesToRead', (err, files) => {
    for (var i = 0; i < files.length; i++) {
        processFile('./filesToRead/' + files[i]);
        console.log('file#: ', i);
    };
    console.log('Done processing files.');
    writeAllLinesToFile();
});

Answer №1

Here's a solution I came up with quickly, but feel free to enhance it further according to your needs.

An important aspect you may have overlooked is that event emitters operate asynchronously rather than synchronously. This caused the writeAllLinesToFile function to receive an empty array because it was invoked before rl.on('line', ....) had completed its task - quite tricky to keep up with, isn't it? ;)

I hope this explanation sheds some light on how Promises work and reminds you to handle asynchronous code with care.

var fs = require('fs');
var readline = require('readline');
var path = require('path');
var stream = require('stream');

function processFile(inputFile) {
    var instream = fs.createReadStream(inputFile);
    var outstream = new stream();
    var rl = readline.createInterface(instream, outstream);
    var allTheLines = [];

    // Perform actions when a line is received
    rl.on('line', function (line) {
        allTheLines.push(line);
    }).on('close', function () {
        writeAllLinesToFile(allTheLines);
    });
}

function writeAllLinesToFile(lines) {
    for (var i = 0; i < lines.length; i++) {
        fs.appendFile(path.join(__dirname, 'allTheLines.html'), lines[i] + '\n', function (err) {
            if (err) throw err;
        });
    };
  
    console.log("Finished writing.");
}

// ***************************************
// Start Execution Here
// ***************************************

fs.readdir(path.join(__dirname, 'filesToRead'), (err, files) => {
  if (err) throw err;

    for (var i = 0; i < files.length; i++) {
        processFile(path.join(__dirname, 'filesToRead', files[i]));
    };

    console.log('Completed processing files.');
});

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Utilizing JavascriptExecutor in Powershell

I've been working on a series of Powershell scripts that utilize the Selenium Webdriver. Now, I am looking to incorporate some JavaScript functionality into one of them. However, I am struggling to understand how to get the syntax right. I tried ada ...

Creating a dynamic JSTree that loads data on demand using Stored Procedures

Currently in my SQL Server database, I have two Stored Procedures that are responsible for extracting data from a tree structure: One procedure retrieves all nodes at a specific level based on the provided level number. The other procedure retrieves th ...

Creating a clickable button within an image container using Bootstrap 5

I am attempting to create a button inside an img element, specifically in the center of that img element within Bootstrap 5. The image is not being used as a background on the grid and I am applying some hover animations to zoom in. I am curious if there ...

Loading Objects with Material Textures Ahead in Three.js

I am faced with the challenge of preloading multiple obj+mtl files using Three.js, each with distinct file paths, and I need to trigger another function once all the objects have finished loading. Initially, I attempted to use a boolean variable that woul ...

Top method for removing quotation marks from form input using jquery

Here is a form input that I need to handle: <tr class="formulaRow"> <input type="text" class="ingredient required" name="ingredient"> </tr> Currently, the value from this input is stored as follows: var ingredient = $(".formulaRow").fi ...

Generate an array of objects using the values from a different array of objects

I have an array of objects: const users = [ { name: 'Alice' }, { name: 'Eve' } ] However, I want to transform it into an array like this: const updatedUsers = [ { key: 'Alice', value: 'Alice', text: 'Al ...

Streamline the process of renaming or remapping keys in a collection of JavaScript/JSON objects

Structured JSON data is essential. Imagine using JSON.parse(): [ { "title": "pineapple", "uid": "ab982d34c98f" }, { "title": "carrots", "uid": "6f12e6ba45ec" } ] The goal is to transform it by changing titl ...

The functionality of the button for selecting and deselecting all checkboxes is currently malfunctioning

I have a button in my ng-grid that successfully selects all checkboxes. However, I am also trying to implement functionality to deselect the checkboxes using the same button. Here is the initial code: app.directive('selectAll',function(){ ret ...

How to retrieve a JSON value without a key using jQuery

I am struggling to fetch JSON using jQuery. If anyone can help me figure out the mistake in my code that would be greatly appreciated. The data.json file contains { "value1", "value2", "value3", "value4" } Here is my jQuery code $.getJSON( "data.js ...

Revising the input value by updating the properties

After clicking on the Reset link, I encounter an issue where the input value fails to update properly. class DiscountEditor extends Component { render() { <div className="inline field"> <a className="ui reset" onClick={thi ...

Automatically choosing a radio button in a carousel using Angular

My Angular application includes the npm-hm-carousel. I am looking to automatically select the item in the center of the carousel, similar to the image provided. However, I also need to bind one of the ids to the selected item as I scroll through the carous ...

Substitute the main node with the subordinate node within a json file

Here is a JSON object structure: { "coach": { "Errors": { "items": [ { "errorMessage": "You must select a date that is in the future.", "errorBOPath": "twl.date" ...

Trying to assign a value to a property that is not defined

I'm attempting to initiate the loading and exhibition of a .stl file through three.js by implementing the following code: var stlLoader = new THREE.STLLoader(); stlLoader.load('assets/Cap.stl', function (object){ object.position.y = - 1 ...

Is there a discrepancy in performance when running a function on an individual element versus a group of elements within jQuery?

Imagine having the choice between applying a function to an individual DOM element or a list of them: For Individual Elements: $('#element1').click(function () { $(this).hide(); return false; }); $('#element2').click(functi ...

What is preventing me from being able to effectively transmit the array using the POST method in Express?

As a newcomer trying to learn Express, I believe I am on the right path, but I am currently facing some challenges with the POST method. The issue I'm encountering is as follows: Whenever I send a POST request to an HTTP file, I receive an empty ob ...

Updating Angular UI-Router to version 1.0 causes issues with resolving data inside views

After upgrading my Angular UI-Router to version 1.0, I came across an interesting statement in the migration guide: We no longer process resolve blocks that are declared inside a views While it makes sense to move all resolve blocks to the parent state ...

JavaScript: Implementing a for loop to dynamically adjust CSS properties

How can I adjust the margin-top property of the ball element when clicking a button? Below is my current code, but it's not working as expected. Can you help? let ballMargin = ball.style.marginTop; moveButton.addEventListener('click', funct ...

Explain the mechanics of the calculator code operating by inputting numerical values in a string format

Recently, I attempted to create a calculator in JavaScript and encountered a requirement to place the button values within single quotes as if they were strings. It's fascinating how these string values can work alongside operators to produce the desi ...

Exploring html select and input elements in a loop

Currently, I am in the process of developing an application that calculates the total price of selected items based on user input. Each row consists of two dropdown menus (ID/item) and a quantity input field. Additionally, users have the option to add rows ...

AngularJS $http.post() response function not executing in the correct sequence

When calling a function from my angular controller to make a $http.post() request, the code below the function call is executing before the successFunction(), preventing the code inside the if block from running. How can I ensure the if block executes wi ...