Best practices for utilizing page.waitForNavigation in Puppeteer

I'm encountering some confusion with the page.waitForNavigation function. Although I understand its purpose and functionality, I seem to get varying results based on internet speed (which I suspect is a contributing factor).

Consider this code snippet:

await page.$eval('#username', (el , setting ) => el.value = setting.username , setting );
await page.$eval('#password', (el , setting ) => el.value = setting.password , setting );
await page.$eval('form', form => form.submit());
await page.waitForNavigation();
await page.goto( 'http://example.com/dashboard'  );

This piece of code fills out a login form, submits it, waits for the submission to complete, and then redirects to the dashboard.

Interestingly, this functions smoothly on my localhost, which has slower internet connectivity compared to the server. However, when deployed on the server, I encounter the following error:

Error: Navigation Timeout Exceeded: 30000ms exceeded 

On the server, removing the line await page.waitForNavigation(); allows the redirection to the dashboard without issues.

Conversely, on localhost, removing await page.waitForNavigation(); leads to premature redirection to the dashboard before the form submission is completed, resulting in a message like "you can't view the dashboard, you are not logged in."

My assumption is that internet speed plays a crucial role in this behavior.

With high-speed connectivity on the server, the form submission occurs instantly, surpassing the await page.waitForNavigation() line and triggering a navigation timeout error.

In contrast, on localhost with slower speed, the form requires more time to submit, necessitating the presence of await page.waitForNavigation() after the submission to prevent premature redirection to the dashboard.

I seek guidance from experienced individuals working with Puppeteer on how to address such scenarios. Currently, I am constantly modifying my code for server or localhost execution, which proves effective but cumbersome!

When implementing the following function:

async function open_tab(setting) {
  const page = await global_browser.newPage();
  await page.setViewport({
    width: 1000,
    height: 768
  });

  return await new Promise(async(resolve, reject) => {
      await page.$eval('#username', (el, setting) => el.value = setting.username, setting);
      await page.$eval('#password', (el, setting) => el.value = setting.password, setting);
      await Promise.all(
        page.$eval('form', form => form.submit()),
        page.waitForNavigation()
      )
      await page.goto('http://example.com/dashboard');
      resolve();
    }).then(() => {
      console.log(' -> done!  ');
      page.close();
    })
    .catch(() => {
      console.log(' -> something went wrong!');
      page.close();
    })
}

The encountered issue reads as:

(node:14812) UnhandledPromiseRejectionWarning: TypeError: undefined is not a function
    at Function.all (<anonymous>)
    at Promise (D:\wamp\www\gatewayCard\robot\server.js:287:23)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
(node:14812) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
(node:14812) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Answer №1

One reason this occurs is because the navigation may be triggered upon submission before waiting for it to complete.

To address this, combine the submission and waitForNavigation within a single promise using Promise.all, ensuring that both actions are awaited simultaneously.

await Promise.all([
 page.waitForNavigation(),
 page.$eval('form', form => form.submit())
])

Alternatively,

await Promise.all([
 page.$eval('form', form => form.submit()),
 page.waitForNavigation()
])

Either approach should resolve the issue.

EDIT 1:

Your main query aside, there's an unrelated edit regarding proper use of async...await in your code. Here's an enhanced version:

The advantage of async...await functions lies in writing concise, readable, maintainable code. Though demanding, they offer great benefits when applied correctly.

async function open_tab(setting) {
  try {
    const page = await global_browser.newPage();
    await page.setViewport({
      width: 1000,
      height: 768
    });
    await page.goto('http://example.com/dashboard');
    await page.$eval('#username', (el, setting) => el.value = setting.username, setting);
    await page.$eval('#password', (el, setting) => el.value = setting.password, setting);
    await Promise.all(page.$eval('form', form => form.submit()), page.waitForNavigation());
    console.log(' -> done! ');
    await page.close();
  } catch (error) {
    console.log(' -> something went wrong!', error);
    await page.close();
  }
}

EDIT 2:

In your code snippet,

return await new Promise(async (resolve, reject ) => {

Several issues arise here. As async functions inherently return a promise, nesting a promise within another without handling it properly leads to complications. It's crucial to rectify this promptly to prevent potential challenges.

It appears advisable to familiarize yourself further with async await functionalities. Refer to these helpful resources for comprehensive understanding:

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Utilizing the this.setState function within a callback

In my current project, I've encountered an issue with setting the state in a Twitter timeline react component. The code snippet causing trouble is as follows: componentWillMount: function() { twitter.get('statuses/user_timeline', ...

Retrieve the element by its Id for precise targeting on a particular webpage

I'm new here and looking for some assistance, so please help if you can :) I'm working on a web page but encountered an issue that I hope you can help me solve. Here are the 3 files involved: home.html --> Main page login.html --> Page ...

Sending a file stream with specific file name and extension in NodeJS: A comprehensive guide

Currently, I am utilizing nodejs to transmit file streams to express response var fileReadStream = fs.createReadStream(filePath); fileReadStream.pipe(res); However, the issue arises on the front-end where the file is solely downloadable with the "download ...

Manipulating information from one format to another

I am currently tackling the task of calculating scores based on departments within groups. For simplicity, I will focus on just one group as an example. Here is the data structure that I have: const data = [{ "id": "cklt7ln1k0922o0sabjkk74m9", ...

Creating a script within a newly generated division element

function addNewChild() { if (childCount <= 2) { childCount++; var newDiv = document.createElement('div'); newDiv.innerHTML = '<br> Prescription '+childCount+':<br><input id="uploadFile" class="disabl ...

MongoDB's implementation of prototypal inheritance within stored objects

Is it possible for MongoDB to save an object with another object as its 'prototype' in the same schema? For example: Assume we have this object in the database: { name : 'foo', lastName : 'bar', email : '<a hre ...

Click on the canvas to fill a specific area with JavaScript

My goal is to implement a JavaScript function that will shade specific areas of canvas drawings when clicked. Below is the code I have that draws a square inside a circle. <!DOCTYPE HTML> <html> <head> </head> <body&g ...

Learn how to dynamically chain where conditions in Firebase without prior knowledge of how many conditions will be added

Currently, I am working on a project using Angular and firebase. My goal is to develop a function that can take two arguments - a string and an object, then return an Observable containing filtered data based on the key-value pairs in the object for a spe ...

Backbone Encompass Nested Elements

Below is the HTML code snippet found in a form... <input type="text" id="test1" name="steps[description]" value="one step"> <input type="text" id="test2" name="steps[description]" value="two step"> I am attempting to generate an array of obje ...

Synchronizing Vuetify Carousel with dynamic route parameters

I'm facing an issue where the v-carousel value does not sync correctly with a route parameter using a computed property. Despite waiting for the mounted hook, the carousel seems to emit its own input and disregard the initial value from the route para ...

Calculating the total value of individual sections using Jquery

I have multiple sections, each containing three input fields: <div class="product_quantity"> <div class="color-quantity"> <input onkeydown="return myFunction(event);" name="custom_small" class="custom_small" type="text"> ...

Issue with slideout menu hyperlinks malfunctioning

Currently developing a site at , everything seemed ready for deployment until testing in 320x480 mode on mobile revealed that the links on the slideout menu were not working on any mobile device I tried, regardless of resolution or page. I've tried u ...

Retrieve Checkbox within a Span using Jquery

Trying to achieve the selection of a checkbox when a user clicks on a div using jQuery. The provided fiddle shows my attempt: http://jsfiddle.net/5PpsJ/ The code snippet I have written so far is as follows, but it doesn't seem to select the checkbox ...

To insert a <div> element within a <tr> element while preserving the exact position of the <tr> tag - here's how you can do it:

I have a challenge with my table where I need to add a green progress bar in the form of a div element within a tr. The width of this progress bar should change dynamically from 0% to 100%, reflecting the current runtime of the video associated with that p ...

Events are failing to appear in their correct positions on the screen

Software Version - "react-big-calendar": "^0.28.2" https://i.sstatic.net/nU4K6.png- Attached is a screenshot for reference. Please see below React code snippet: <Calendar defaultDate={moment().toDate()} defaultVie ...

Error code E401 is being encountered with npm, indicating either an incorrect password has been provided or the

My Node version is 10.15.0 and my NPM version is currently at 6.8.4. After updating npm to 14.16.0 and node to 7.6.2, I encountered the following error - npm ERR! code E401 npm ERR! Incorrect or missing password. npm ERR! If you were trying to log in, ...

What is the best way to switch a Boolean value in React Native?

Struggling with toggling a Boolean state from true to false when the result is undefined. Tried several methods but none seem to work. The boolean state in the constructor is defined like this: class UserInfo extends Component{ constructor(props){ s ...

The preventDefault() method seems ineffective when used with a custom button component that has been imported in a

I'm facing an issue with a form I'm building that includes a custom Button component. Previously, the form worked fine with a normal button, but after switching to the Button component, the handleSubmit function is not triggering when the button ...

Mongoose sparks a confrontation following the preservation of a single document in the database

I'm struggling to understand what minor mistake I'm making in this code. I have simplified the user schema to just one property, which is name. Initially, when I post the first entry to the database, it gets saved without any issues. However, whe ...

Keystroke to activate Ant Design Select and start searching

I'm currently using the 'react-hotkeys-hook' library and have successfully implemented a hotkey that logs in the console when triggered (via onFocus()). My goal now is to use a hotkey that will open a Select component and add the cursor to i ...