Develop an asynchronous function that can fetch a result at a later time within an Express route

Question

Develop an asynchronous function that can fetch a result at a later time within an Express route

I need to retrieve an Excel file from Box.com using their API and then convert the Excel data into JSON format. Afterwards, I plan to display this JSON data using Express and Handlebars.

Below is the function I've created for fetching the Excel file and converting it to JSON:

let getExternalData = async () => {
  const stream = await client.files.getReadStream('123456789');
  const externalFile = await fs.createWriteStream('/external.xlsx');
  
  stream.pipe(externalFile);
  
  let finished = await externalFile.on('finish', async () => {
    const excelData = await excelToJson({
      source: fs.readFileSync(externalFile.path),
      header: {
        rows: 1
      },
      columnToKey: {
        A: "edate",
        B: "esource"
      },
      sheets: ['Sheet1']
    });
    
    console.log(excelData);
    return excelData;
  })
}

My current issue is that I am unsure how to handle the pending Promise when calling the function like this:

let testData = getExternalData();
console.log(testData); // Promise { <pending> }

I intend to use this function to download multiple files and store them in variables to be passed to my express route later on.

app.get('/', (req, res) => {
    res.render('stories.ejs', {
        msg: "Welcome to stories.ejs", 
        odata: odata, 
        edata: edata,
        ...
    });
});

Would wrapping it in an anonymous async function be a solution?

javascript express async-await

Answer 1

Answer №1

async/await is not a one-size-fits-all solution for handling asynchronous operations. These keywords were specifically designed to work with Promises and do not function properly with event emitters. To correctly implement the getExternalFile function, follow this updated code snippet:

let getExternalFile = async () => {
  const stream = client.files.getReadStream('123456789'); // no await here
  const external = fs.createWriteStream('/external.xlsx'); // no await here
  stream.pipe(external);

  let finished = new Promise ((resolve, reject) => {
    external.on('finish', async () => { // cannot use await here
      const excel = await excelToJson({
        source: fs.readFileSync(external.path),
        header: {
          rows: 1
        },
        columnToKey: {
          A: "edate",
          B: "esource"
        },
        sheets: ['Sheet1']
      });
      console.log(excel);

      resolve(excel);
    })
  });

  return finished; // return the promise so you can await it later
}

Functions that are not Promises are incompatible with await, including event emitters like x.on('some_event' ... ). In such cases, you must wrap them in new Promise() to convert them into Promises.

Once the above function has been rewritten, you can easily await the result:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testData = await getExternalFile(); // use await here!!

    // ...
});

If multiple awaits slow down your function, consider executing async functions in parallel using Promise.all():

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testDataArray = await Promise.all([
        getExternalFile(), getExternalFile2()  // FAST!
    ]);

    // ...
});

Epilogue: Dealing with Errors

In real-world scenarios, always handle errors on awaited code to prevent server crashes:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    try {
        let testData = await getExternalFile(); // use await here!!

        // ...
    }
    catch (err) {

        // ...
    }
});

Alternatively, utilize middleware like express-async-handler to manage async errors effectively.

Answer 2

async/await is not a one-size-fits-all solution for handling asynchronous operations. These keywords were specifically designed to work with Promises and do not function properly with event emitters. To correctly implement the getExternalFile function, follow this updated code snippet:

let getExternalFile = async () => {
  const stream = client.files.getReadStream('123456789'); // no await here
  const external = fs.createWriteStream('/external.xlsx'); // no await here
  stream.pipe(external);

  let finished = new Promise ((resolve, reject) => {
    external.on('finish', async () => { // cannot use await here
      const excel = await excelToJson({
        source: fs.readFileSync(external.path),
        header: {
          rows: 1
        },
        columnToKey: {
          A: "edate",
          B: "esource"
        },
        sheets: ['Sheet1']
      });
      console.log(excel);

      resolve(excel);
    })
  });

  return finished; // return the promise so you can await it later
}

Functions that are not Promises are incompatible with await, including event emitters like x.on('some_event' ... ). In such cases, you must wrap them in new Promise() to convert them into Promises.

Once the above function has been rewritten, you can easily await the result:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testData = await getExternalFile(); // use await here!!

    // ...
});

If multiple awaits slow down your function, consider executing async functions in parallel using Promise.all():

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testDataArray = await Promise.all([
        getExternalFile(), getExternalFile2()  // FAST!
    ]);

    // ...
});

Epilogue: Dealing with Errors

In real-world scenarios, always handle errors on awaited code to prevent server crashes:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    try {
        let testData = await getExternalFile(); // use await here!!

        // ...
    }
    catch (err) {

        // ...
    }
});

Alternatively, utilize middleware like express-async-handler to manage async errors effectively.

Develop an asynchronous function that can fetch a result at a later time within an Express route

Answer №1

Epilogue: Dealing with Errors

Similar questions

Guide on sending AJAX requests from Javascript/React to Python REST API and receiving data

Disappearing Facebook share button: A common occurrence when navigating in Nuxt.js (Vue.js)

Is Passport.js' serializeUser and deserializeUser functions never triggering?

The image loads successfully from an AJAX request in CodePen, but fails to load in the local

Challenges with loading times in extensive AngularJS applications

What is causing the fs.readFile function to give back undefined instead of the expected result?

Executing a raw query within an express.js route using sequelize

Show a variety of pictures in a random arrangement?

Working with jQuery, CSS, and functions to modify the current tag's class

Error in Displaying Vuetify Child Router View

React- hiding div with hover effect not functioning as expected

Send JSON data that has been refined utilizing jQuery

Modal's ng-click not triggering component's function

JWT triggers a JsonWebTokenError with the message "token is not valid"

Durandal attempts to retrieve a view located within the viewmodel directory

The success function is failing to display the Ajax response

Move the option from one box to another in jQuery and retain its value

Customized style sheets created from JSON data for individual elements

Transitioning the style code from inline to the head of the document disrupts the straightforward JavaScript intended to

Extract reference value from an HTML element