Develop an asynchronous function that can fetch a result at a later time within an Express route

Question

Develop an asynchronous function that can fetch a result at a later time within an Express route

I need to retrieve an Excel file from Box.com using their API and then convert the Excel data into JSON format. Afterwards, I plan to display this JSON data using Express and Handlebars.

Below is the function I've created for fetching the Excel file and converting it to JSON:

let getExternalData = async () => {
  const stream = await client.files.getReadStream('123456789');
  const externalFile = await fs.createWriteStream('/external.xlsx');
  
  stream.pipe(externalFile);
  
  let finished = await externalFile.on('finish', async () => {
    const excelData = await excelToJson({
      source: fs.readFileSync(externalFile.path),
      header: {
        rows: 1
      },
      columnToKey: {
        A: "edate",
        B: "esource"
      },
      sheets: ['Sheet1']
    });
    
    console.log(excelData);
    return excelData;
  })
}

My current issue is that I am unsure how to handle the pending Promise when calling the function like this:

let testData = getExternalData();
console.log(testData); // Promise { <pending> }

I intend to use this function to download multiple files and store them in variables to be passed to my express route later on.

app.get('/', (req, res) => {
    res.render('stories.ejs', {
        msg: "Welcome to stories.ejs", 
        odata: odata, 
        edata: edata,
        ...
    });
});

Would wrapping it in an anonymous async function be a solution?

javascript express async-await

Answer 1

Answer №1

async/await is not a one-size-fits-all solution for handling asynchronous operations. These keywords were specifically designed to work with Promises and do not function properly with event emitters. To correctly implement the getExternalFile function, follow this updated code snippet:

let getExternalFile = async () => {
  const stream = client.files.getReadStream('123456789'); // no await here
  const external = fs.createWriteStream('/external.xlsx'); // no await here
  stream.pipe(external);

  let finished = new Promise ((resolve, reject) => {
    external.on('finish', async () => { // cannot use await here
      const excel = await excelToJson({
        source: fs.readFileSync(external.path),
        header: {
          rows: 1
        },
        columnToKey: {
          A: "edate",
          B: "esource"
        },
        sheets: ['Sheet1']
      });
      console.log(excel);

      resolve(excel);
    })
  });

  return finished; // return the promise so you can await it later
}

Functions that are not Promises are incompatible with await, including event emitters like x.on('some_event' ... ). In such cases, you must wrap them in new Promise() to convert them into Promises.

Once the above function has been rewritten, you can easily await the result:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testData = await getExternalFile(); // use await here!!

    // ...
});

If multiple awaits slow down your function, consider executing async functions in parallel using Promise.all():

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testDataArray = await Promise.all([
        getExternalFile(), getExternalFile2()  // FAST!
    ]);

    // ...
});

Epilogue: Dealing with Errors

In real-world scenarios, always handle errors on awaited code to prevent server crashes:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    try {
        let testData = await getExternalFile(); // use await here!!

        // ...
    }
    catch (err) {

        // ...
    }
});

Alternatively, utilize middleware like express-async-handler to manage async errors effectively.

Answer 2

async/await is not a one-size-fits-all solution for handling asynchronous operations. These keywords were specifically designed to work with Promises and do not function properly with event emitters. To correctly implement the getExternalFile function, follow this updated code snippet:

let getExternalFile = async () => {
  const stream = client.files.getReadStream('123456789'); // no await here
  const external = fs.createWriteStream('/external.xlsx'); // no await here
  stream.pipe(external);

  let finished = new Promise ((resolve, reject) => {
    external.on('finish', async () => { // cannot use await here
      const excel = await excelToJson({
        source: fs.readFileSync(external.path),
        header: {
          rows: 1
        },
        columnToKey: {
          A: "edate",
          B: "esource"
        },
        sheets: ['Sheet1']
      });
      console.log(excel);

      resolve(excel);
    })
  });

  return finished; // return the promise so you can await it later
}

Functions that are not Promises are incompatible with await, including event emitters like x.on('some_event' ... ). In such cases, you must wrap them in new Promise() to convert them into Promises.

Once the above function has been rewritten, you can easily await the result:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testData = await getExternalFile(); // use await here!!

    // ...
});

If multiple awaits slow down your function, consider executing async functions in parallel using Promise.all():

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    let testDataArray = await Promise.all([
        getExternalFile(), getExternalFile2()  // FAST!
    ]);

    // ...
});

Epilogue: Dealing with Errors

In real-world scenarios, always handle errors on awaited code to prevent server crashes:

app.get('/', async /* <--IMPORTANT */ (req, res) => {
    try {
        let testData = await getExternalFile(); // use await here!!

        // ...
    }
    catch (err) {

        // ...
    }
});

Alternatively, utilize middleware like express-async-handler to manage async errors effectively.

Develop an asynchronous function that can fetch a result at a later time within an Express route

Answer №1

Epilogue: Dealing with Errors

Similar questions

JavaScript - Need to automatically scroll to a different div when scrolling occurs

Despite the value being true, the if statement does not work when used with a Mongoose object in Node.js

Is it possible to use the Stop Button on the HTML5 Audio Tag to halt a live MP3 stream

Exploring the Depths of the DOM: Enhancing User Experience with Jquery

Implementing Basic Authentication with Express and Mongoose

To collapse a div in an HTML Angular environment, the button must be clicked twice

choosing a section within a table cell

Modify the values of an object by utilizing the setter function

An issue has been identified where the Export to Excel and PDF functionality is not functioning properly within a Datatable after applying a

Error: The build process for Next.js using the command `npm run build`

Tips for generating a single sitemap in Next.js

There was an error in reading the 'nativeElement' property in Angular's waveform library, resulting in a TypeError

What is the reason behind the NodeJS domains documentation code attempting to end the process?

Error: Cannot locate 'import-resolver-typescript/lib' in jsconfig.json file

AngularJS form submission with and without page refresh

The npm command encountered a permission issue when trying to install node-sass

Leveraging the combination of Express JS and Vue.js within the Electron framework

Guide on creating a project for developing an npm package

What is the process of transforming a JSON string into a JavaScript date object?

Disregarding text within an express route