What steps should I take to guarantee that two separate asynchronous tasks are added to a file in a sequential

Below is the code snippet I'm working with:

const fs = require("fs");

const saveFile = (fileName, data) => {
  return new Promise((resolve) => {
    fs.writeFile(fileName, data, (err) => {
      resolve(true);
    });
  });
};

const readFile = (fileName) => {
  return new Promise((resolve) => {
    fs.readFile(fileName, "utf8", (err, data) => {
      resolve(data);
    });
  });
};

const filename = "test.txt";

saveFile(filename, "first");

readFile(filename).then((contents) => {
  saveFile(filename, contents + " second");
});

readFile(filename).then((contents) => {
  saveFile(filename, contents + " third");
});

The expected content in 'test.txt' is:

first second third

However, the actual output observed is:

first thirdd

The objective here is to append more text to the file every time a specific post request is received.

If anyone has a solution for this issue, it would be greatly appreciated!

Edit:

The limitation faced while using async/await or a chain of .then() methods is that I need to add more text to the file whenever a certain post request is received. This means not having control over what gets written or when. The goal is to prevent any overwriting even if multiple post requests arrive simultaneously.

I will discuss a linked list-based solution I devised yesterday. However, I am open to better alternatives from others as well.

const saveFile = (fileName, data) => {
  return new Promise((resolve) => {
    fs.writeFile(fileName, data, (err) => {
      resolve(true);
    });
  });
};

const readFile = (fileName) => {
  return new Promise((resolve) => {
    fs.readFile(fileName, "utf8", (err, data) => {
      resolve(data);
    });
  });
};

class LinkedCommands {
  constructor(head = null) {
    this.head = head;
  }

  getLast() {
    let lastNode = this.head;
    if (lastNode) {
      while (lastNode.next) {
        lastNode = lastNode.next;
      }
    }
    return lastNode;
  }

  addCommand(command, description) {
    let lastNode = this.getLast();
    const newNode = new CommandNode(command, description);
    if (lastNode) {
      return (lastNode.next = newNode);
    }
    this.head = newNode;
    this.startCommandChain();
  }

  startCommandChain() {
    if (!this.head) return;
    this.head
      .command()
      .then(() => {
        this.pop();
        this.startCommandChain();
      })
      .catch((e) => {
        console.log("Error in linked command\n", e);
        console.log("command description:", this.head.description);
        throw e;
      });
  }

  pop() {
    if (!this.head) return;
    this.head = this.head.next;
  }
}

class CommandNode {
  constructor(command, description = null) {
    this.command = command;
    this.description = description;
    this.next = null;
  }
}

const linkedCommands = new LinkedCommands();

const filename = "test.txt";

linkedCommands.addCommand(() => saveFile(filename, "first"));

linkedCommands.addCommand(() =>
  readFile(filename).then((contents) =>
    saveFile(filename, contents + " second")
  )
);

linkedCommands.addCommand(() =>
  readFile(filename).then((contents) => saveFile(filename, contents + " third"))
);

Answer №1

Since these functions are asynchronous, they signal the completion of work in the then function.
To properly handle this behavior, you should create a chain of then statements (or an async function) like the example below:

readFile(filename).then((contents) => {
  return saveFile(filename, contents + " second");
}).then(() => {
  return readFile(filename)
}).then((contents) => {
  saveFile(filename, contents + " third");
});

Answer №2

If you're looking to implement a solution where functions returning promises are queued up in a FIFO manner, here's a possible approach.

const { readFile, writeFile } = require("fs/promises");

let queue = [];
let lock = false;

async function flush() {
  lock = true;

  let promise;
  do {
    promise = queue.shift();
    if (promise) await promise();
  } while (promise);

  lock = false;
}

function createAppendPromise(filename, segment) {
  return async function append() {
    const contents = await readFile(filename, "utf-8");
    await writeFile(
      filename,
      [contents.toString("utf-8"), segment].filter((s) => s).join(" ")
    );
  };
}

async function sequentialWrite(filename, segment) {
  queue.push(createAppendPromise(filename, segment));
  if (!lock) await flush();
}

async function start() {
  const filename = "test.txt";

  // Create all three promises right away
  await Promise.all(
    ["first", "second", "third"].map((segment) =>
      sequentialWrite(filename, segment)
    )
  );
}

start();

This method operates by enqueuing promise functions as they arrive. When new requests come in, the corresponding functions are created and added to the queue.

Each time a function is added to the array, an attempt is made to flush it out. If a locking mechanism is already active, indicating ongoing flushing, the process is abandoned for the time being.

During flushing, the first function in the queue is retrieved, removed from the queue, executed, and waited upon for its promise to resolve. This cycle continues until the queue is empty. Asynchronous operations allow the queue to be replenished even while flushing is ongoing.

Remember that if this system is deployed on a server with multiple processes accessing the same file simultaneously (e.g., through horizontal scaling), there is a risk of data loss. To mitigate this, consider implementing a distributed mutex. A recommended option is utilizing Redis and redlock for this purpose.

Hope this explanation proves helpful!

Edit: For verification of functionality, you can introduce a completely random setTimeout within the createAppendPromise function.

function createAppendPromise(filename, segment) {
  const randomTime = () =>
    new Promise((resolve) => setTimeout(resolve, Math.random() * 1000));

  return async function append() {
    await randomTime();
    const contents = await readFile(filename, "utf-8");
    await writeFile(
      filename,
      [contents.toString("utf-8"), segment].filter((s) => s).join(" ")
    );
  };
}

Answer №3

It's perfectly okay to chain promises without knowing in advance how many will be created. Just make sure to keep the end of the chain accessible and link new promises to it as needed...

// context persistence is essential
let appendChain = Promise.resolve();
const targetFile = "test.txt";

// assuming functions like readFile and saveFile are functional...
const appendToTargetFile = (filename, data) =>
  return readFile(filename).then(contents => {
    return saveFile(filename, contents + data);
  });
}

function processNewData(input) {
  return appendChain = appendChain.then(() => {
    return appendToTargetFile(targetFile, input);
  });
}

What follows is a demonstration utilizing fake async read, write, and append operations. The <p> tag represents the fabricated file content. Click the button to simulate adding new data to the virtual file.

The button serves as an external event trigger for appending. Note that there's a 1-second delay in the append function so you can press the button multiple times before all append operations complete.

function pretendReadFile() {
  return new Promise(resolve => {
    const fileContent = document.getElementById('the-file');
    resolve(fileContent.innerText);
  })
}

function pretendWriteFile(data) {
  return new Promise(resolve => {
    const fileContent = document.getElementById('the-file');
    fileContent.innerText = data;
    resolve();
  })
}

function pretendDelay(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

function appendToFile(data) {
  return pretendDelay(1000).then(() => {
    return pretendReadFile()
  }).then(result => {
    return pretendWriteFile(result + data);
  });
}

document.getElementById("my-button").addEventListener("click", () => click());

let operationChain = Promise.resolve();
let counter = 0
function click() {
  operationChain = operationChain.then(() => appendToFile(` ${counter++}`));
}
<button id="my-button">Click Rapidly At Varying Intervals</button>
<h3>Fake file contents:</h3>
<p id="the-file">empty</p>

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Is there a way to automatically render Katex without the need for double dollar signs?

I've been facing difficulties grasping the concept of rendering Katex without relying on $$ before and after the math expression. According to Katex's repository on GitHub, I should use the following code: <script> renderMathInElement( ...

How to Efficiently Remove Array Elements by Index in Typescript

What is the best way to remove an item by its index using Typescript? For example: let myArray = ['apple', 'banana', 'cherry', 'date']; // How can I delete the item at index 2? ...

Is there a way to update the href attribute within the script section in vue.js?

I need to dynamically set the href attribute of a link based on data retrieved from a database and rendered in the methods section. <template v-if="header.value == 'ApplicationName'"> <a id="url" href="#" target="_blan ...

Sending assets with invalid MIME type via express

After deploying a Vue.js webpack app with Express serving it, I encountered the following error: https://i.sstatic.net/KLgr1.png Could the code handling the app deployment be causing this issue? app.use(helmet()) app.use(express.static(path.resolve(__dir ...

Track every click on any hyperlink throughout the entire webpage

I am attempting to capture any click event on a link throughout the entire page. For example, when a user clicks on the following link: <a href="/abc">abc<a/> I want to be able to retrieve the anchor tag like so: <span hre="/abc">abc& ...

Most effective method for including and excluding items from an array using personalized checkboxes

Is there a way to dynamically add and remove objects from an array based on the selection of custom checkboxes, along with a trigger or flag indicating if the checkbox is checked? Using the onClick function const handleFilterValues = (meta_name, meta_valu ...

Sharing various data from Express to Jade is quite simple and can be done

I have a series of variables that are checking against a condition to determine if they return true or false. While I've been able to print these results using console.log, I now need to display them in a jade template. However, I'm encountering ...

Javascript recursive method for fetching data entries

Seeking a solution to retrieve interconnected records based on a parent column, where the relation can be one or many on both ends. After attempting a recursive function without success, I found my code became overly complex and ineffective. Is there a st ...

Split the word inside the <td> element

Is there a way to display the full text inside the <td> element? You can test it here: $(document).ready(function(){ var users = [], shuffled = [], loadout = $("#loadout"), insert_times = 30, duration_time = 10000; $("#roll").click(functio ...

The Node application seems to be having trouble locating the JavaScript files that are referenced in

I am in the process of creating a basic chat application using node and socket.io. I'm following a tutorial that can be found at this link: http://socket.io/get-started/chat/ The problem I am encountering is that the tutorial includes some javascript ...

Having trouble retrieving the API URL id from a different API data source

For a small React project I was working on, I encountered a scenario where I needed to utilize an ID from one API call in subsequent API calls. Although I had access to the data from the initial call, I struggled with incorporating it into the second call. ...

Error encountered while attempting to build Ionic 5 using the --prod flag: The property 'translate' does not exist on the specified type

I encountered an error while building my Ionic 5 project with the --prod flag. The error message I received is as follows: Error: src/app/pages/edit-profile/edit-profile.page.html(8,142): Property 'translate' does not exist on type 'EditProf ...

Exploring deep within JSON data using jQuery or Javascript

I have a substantial JSON file with nested data that I utilize to populate a treeview. My goal is to search through this treeview's data using text input and retrieve all matching nodes along with their parent nodes to maintain the structure of the tr ...

Bringing in the component's individual module

I encountered the error message in my Angular application - Can't bind to 'formGroup' since it isn't a known property of 'form' - and managed to resolve it by including the import import { AddEditModule } from './add.edit ...

AngularJS slider featuring ng-change functionality

One of my current tasks involves working with an input slider that looks like this: <input ng-model="value" type="text" id="mySlider1" slider ng-change="fun()"/> The slider has values ranging from 1 to 10. I am looking to retrieve the value using ...

Troubleshooting a formatting problem with JSON retrieved from an AJAX POST request

Upon receiving a POST request via an ajax function, my PHP script returns JSON formatted data as follows: //json return $return["answers"] = json_encode($result); echo json_encode($return); The string returned looks like this: answers: "[{"aa":"Purple", ...

Plumage reaching out to personalized API function

To set up my API, I typically write something like the following: class MyFeathersApi { feathersClient: any; accountsAPI: any; productsAPI: any; constructor(app) { var port: number = app.get('port'); this.accountsAPI = app.serv ...

React: asynchronous setState causes delays in updates

Currently learning React.js, I am working on creating a simple combat game. In my code snippet below, I attempt to update the state to determine whose turn it is to strike: this.setState({ whoseRound: rand }, () => { console.log(this.state.whoseRo ...

Combining arrays to create a single object: A step-by-step guide

Here is the current output I have, which contains multiple arrays: ["{"486|575":2,"484|568":4}", "{"486|575":2,"484|568":4}", "{"481|570":1,"482|564":1}"] My goal is to combine these arrays into an object with the following output using JavaScript/jQuery ...

ng-repeat is malfunctioning in AngularJS

Retrieve an array from the server in this format: https://i.sstatic.net/rsG6T.jpg It should be displayed as follows: <div class="object1"> <div class="content"></div> <div class="content"></div> <div class="content">& ...