The loading on the Express endpoint does not cease, despite configuring the response status (Encountering problems with Multer and Express)

Currently, I am in the process of developing a filter that evaluates a file's signature obtained through a file's buffer provided by Multer. While Multer supplies the MIME type, additional validation methods are required to confirm if the file matches that MIME type.

My approach involves the following steps:

  1. Temporarily storing the request's files in Multer's memoryStorage() to access the buffer
  2. Filtering out the valid files
  3. Appending these valid files and other necessary information to create a new FormData object
  4. Initiating a POST request to another endpoint ("/upload") with the new FormData
  5. Storing the files in the cloud using Multer

THE ISSUE: Whenever an error is deliberately triggered at the second endpoint, it results in an infinite loop. Although the error is identified, I am unable to provide any HTTP response code back to the initial endpoint!

Initial endpoint:

// NOTE: storeInMemory.array("files") stores the files' buffer information in memory,
// which is essential for validating the files.
app.post("/filesUpload", storeInMemory.array("files"), async (req, res) => {
    const files = req.files;

    // Array to hold items requiring upload
    let uploadArray = [];

    // Filtering only the necessary items for uploading
    for (const item of files) {
        // -- add filter logic here --
        uploadArray.push(item);
    }

    // Creating a new FormData to contain the required files as multer operates over http
    let form = new FormData();
    form.append("someData", req.body["someData"]);

    // Attaching files to the form for subsequent upload
    uploadArray.forEach((item) => {
        form.append("files", item.buffer, item.originalname);
    });

    try {
        const postUrl = process.env.MY_URL + "/upload";
        const myHeaders = { headers: { 'Content-Type': `multipart/form-data; boundary=${form._boundary}` } };
        // Performing a request to another endpoint
        let uploadResult = await axios.post(postUrl, form, myHeaders);

        if (uploadResult.status == 200) return res.send(uploadResult).status(200);
        // ISSUE HERE: This part is never reached, but it needs to be
        else {
            return res.send(uploadResult).status(500);
        }
    }
    catch (error) {
        console.error("Unexpected error occurred: ", error);
        return res.send(error)
    }

});

Second endpoint:

app.post("/upload", async (req, res) => {
    try {
        await new Promise((resolve, reject) => {
            // Uploading the files to the cloud
            uploadOnCloud.array("files")(req, res, (err) => {
                if (err) {
                    console.error("An error occurred successfully!", err);
                    return reject(err);
                }
                else
                    return resolve();
            });
        });
        // Respond with status 200 if no errors detected
        res.sendStatus(200);
    }
    catch (error) {
        // PROBLEM: Unable to set status or anything else
        res.status(500).send(error.message);
    }
});

What could possibly be wrong with this setup?

Answer №1

After testing the code you provided, I identified an error in the line

form.append("files", item.buffer, item.originalname)
. This line was incorrect and should be replaced with the following corrected version. Additionally, there is no need to explicitly set the Content-Type.

form.append("files", 
   new File([file_buffer], file_name, {
        type: file_mimetype,
   })
);

Furthermore, I did not encounter the issue of the loading getting stuck. My analysis suggests that this might be due to Multer waiting for data on the request body, which was missing since the request body was attached without any files!

The Solution:

Variables

const cloudStorage = multer.memoryStorage();
const localStorage = multer.memoryStorage();

const cloudUploader = multer({
  limits: {
    fileSize: 1024 ** 3, // 1GB
  },
  storage: cloudStorage,
});

const localUploader = multer({
  limits: {
    fileSize: 100 * 1024 ** 2, // 100MB
  },
  storage: localStorage,
});

Route for uploading files & its controller

app.post("/upload-files", localUploader.array("files"), async (req, res) => {
  /** @type {Express.Multer.File[]} */ const files = req.files;

  const form = new FormData();
  form.append("someData", req.body["someData"]);

  // You can implement your filtering logic here for files

  // Directly using append function on files 
  files.forEach((file) => {
    form.append(
      "files",
      new File([file.buffer], file.originalname, {
        type: file.mimetype,
      })
    );
  });

  try {
    const postUrl = `${process.env.ORIGIN}/upload`;
    
    // Using node-fetch without needing to manually set Content-Type headers as form-data handles it
    const uploadResult = await fetch(postUrl, {
      method: "POST",
      body: form,
    });

    if (uploadResult.ok) { 
      return res.status(200).json({
        error: false,
        message: "FILES UPLOADED SUCCESSFULLY!"
      });
    } else {
      return res.status(500).json({
        error: true,
        message: "Something went wrong",
      });
    }
  } catch (error) {
    console.error(error);
    return res.status(500).json(error);
  }
});

Cloud file upload route & controller

app.post("/upload", async (req, res) => {
  try {
    await new Promise((resolve, reject) => {
      /* upload the files to cloud */
      cloudUploader.array("files")(req, res, (err) => {
        /** @type {Express.Multer.File[]} */ const files = req.files;
        // Save files locally as a check
        files.forEach((file) => {
          fs.writeFileSync(`public/uploads/${file.originalname}`, file.buffer, {
            encoding: "utf-8",
          });
        });

        if (err) {
          console.error("Error occurred during upload!", err);
          return reject(err);
        } else return resolve();
      });
    });

    return res.json({ error: false, message: "Files uploaded" });
  } catch (e) {
    console.error(e);
    return res.json({ error: true, message: "Something went wrong" });
  }
});

Answer №2

Upon receiving a status 500 from your second endpoint, the first endpoint automatically triggers the catch block assuming an error has occurred. To address this issue, you should explicitly set the status within the catch block like so:

catch (error) {
    res.status(500).send(error.message);
}

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Utilize a variable within a Jade include statement

I'm currently using Jade and Express and I want to include a variable in my include statement. Here is an example of what I am trying to achieve: app.js app.get('/admin', function (req, res) { var Admin = require('./routes/admin/app ...

Develop a JavaScript application that contains a collection of strings, and efficiently sorts them with a time complexity of O(nlog n)

Seeking assistance in developing a JavaScript program that contains an array of strings and must sort them efficiently in O(nlog n) time. Grateful for any guidance... ...

Uncertain about how to transfer data between server and client in Next.js?

I'm currently grappling with understanding the proper method of exchanging data between server-side and client-side components in NextJS 13. To simplify my learning process, I have created a basic scenario. In this setup, I have two functions that pe ...

What are the steps for accessing a server-side WebControl from the client side?

Issue Overview: I am facing a problem with managing different types of input values in my dialog box. Depending on the selection from a dropdown list, the required input could be simple text, a date, or specific data from a database. I have successfully i ...

Learn how to display months on a countdown timer and then customize the format with basic JavaScript for a website

Looking to create a countdown page for the upcoming ICC cricket world cup event that displays the remaining days in two different formats: Format #1: 01 months 10 days 10 hours Format 2: 01 hours 20 minutes 10 seconds (If less than 2 days remain) I curr ...

Repeated Values Issue in Material Ui List Function

Struggling to display only the newly added todo item in the list? Utilizing the material ui library for list creation, I have successfully displayed the new item. However, instead of showing just the specific value that was added, the entire array is being ...

What causes the "This page isn't responding" error to pop up in Edge and Chrome browsers while attempting to perform consecutive tasks in a web application built with Angular 8?

Trouble with Page Loading Whenever this error occurs, I find myself unable to perform any activities on that page. The only solution is to close the tab and open a new one. My current code allows me to navigate through an array list (Next and Previous) us ...

The difference between using document.getElementsByTagName("img") and document.getElementById('testimg')

My HTML code is as follows: <a href="index.php"><img id="testimg" src="images/logo.png"/></a> This is my JavaScript code: function getW(){ var theImg = document.getElementById('testimg'); return theImg; } theImg = ...

I am having trouble resolving 'otp-input-react' in my project directory at D:projectappsrc

I have been troubleshooting this issue but haven't been able to find a solution yet. I even tried uninstalling and reinstalling the package, but it still isn't working as expected. Here are some images for better clarity: https://i.stack.imgur.c ...

Formatting dates in JavaScript

When using the jQuery datepicker, my goal is to set a minimum and maximum date range for users to select from. I also need to ensure that the date format is based on the user's locale. In this case, there are two date fields: Start date and End date. ...

Find the difference between the sum of diagonals in a 2D matrix using JavaScript

I'm currently practicing on hackerrank and came across a challenge involving a two-dimensional matrix. Unfortunately, I encountered an error in my code implementation. 11 2 4 4 5 6 10 8 -12 The task at hand is to calculate the sum along the primary ...

The final condition of the ternary operator fails to render if the specified condition is satisfied

Having trouble identifying the issue with this ternary statement. The last element (blurredscreen) is not appearing when authStatus !== "authenticated". return ( <> <div key={"key-" + id}> {isO ...

Electronic circuit embedded within a material-textured text field offering multiline functionality

While experimenting with TagsInput, I came across this helpful snippet on codesandbox that you can check out here. The challenge I encountered is when there are numerous chips, they extend beyond the boundaries of the text field. My goal is to implement ...

"Object.entries seems to be returning only the initial object in the list

This is my object var obj = { "first_obj": { "a":1, "status": 1 }, "second_obj": { "a":2, "status": 3 } } I'm struggling to loop through this object using foreach and Object.entries, as the latter only returns the first object. How ...

Can one track the moment when a user clicks on an advertisement within a YouTube video?

Can we track when a user clicks on an advertisement within a YouTube video? We have integrated the YouTube API v2 into our website, allowing users to watch videos. The issue is that when a user clicks on an advertisement, the YouTube video pauses automat ...

Leveraging arrays generated from two separate MySQL queries for dual selection functionality with JavaScript

I have successfully populated the first HTML select with results from the first query. Now, I would like to store the results from the second query in either a Json file or XML and then use plain javascript (not jQuery) to populate the second HTML select ...

What could be causing me to receive 'undefined' and an empty array[] when using Promise.all with JavaScript async operations making calls to Azure APIs?

In my personal project utilizing Azure AI APIs and Node.js/Express,, I am working on handling a get request to a /viewText route by extracting text and key phrases from an uploaded image/document. Below is the code snippet that should log this data to the ...

Styling <Link> component with styled-components: A step-by-step guide

Utilizing the Link component from @material-ui/core/Link in my TypeScript code was initially successful: <Link href="#" variant="body2"> Forgot? </Link> However, I am exploring the transition to styled-components located in a separate file. ...

I'm struggling to find a way to showcase my JSON data in HTML. Update: I have a clear vision of how I want it to look, but I'm struggling to display it in any format other than raw JSON

I've been trying to figure this out for hours, scouring every resource I can find, but I'm stuck. I've left the API URL in there, so feel free to take a look (it's public). If my message doesn't make sense due to exhaustion, please ...

Typescript - unexpected behavior when using imported JavaScript types:

I am struggling with headaches trying to integrate an automatically generated JavaScript library into TypeScript... I have packaged the JavaScript library and d.ts file into an npm package, installed the npm package, and the typings modules in the TypeScr ...