Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

Question

Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

I need assistance with handling a basic authentication popup using puppeteer=5.2.1. My specific scenario is outlined below.

1. Start an application
2. Click on a button on the home page
3. This action opens a new window (saml) requesting email input

Within the new window,
4. Input email address
5. Click on Next button
6. A basic authentication popup appears (=> assistance needed here)

The following code reaches up to the 3rd step, where I am able to access the reference of the new page (saml window)

var browser = await puppeteer.launch({
      headless: false, args: ["--start-maximized"]
    });
 var page = await this.browser.newPage();
await waitUntilElementIsVisible(authMethodDropDown);
await wait(5000);
await page.select(authMethodDropDown, "myoption");
const samlPage = await clickAndWaitForTarget(page, authSubmitButton);
wait(5000);
await samlPage.waitForSelector(samlSignInUserTextField);

Below are the different approaches I have tried to handle the basic auth popup in the 6th step.

Option-1: utilizing page.authenticate()

await samlPage.authenticate({username: `${myuser}`, password: `${mypass}`});
await samlPage.type(samlSignInUserTextField, `${myuser}`);
await samlPage.click(samlSignInNextButton);

=> The loading continues and the next page is not displayed.

Option-2: employing setExtraHttpHeaders

const headers = new Map();
headers.set(
   'Autorization',
   `Basic ${new Buffer(`${myuser}:${mypass}`).toString('base64')}`
 );
headers.set('WWW-Authenticate','Negotiate');
headers.set('WWW-Authenticate','NTLM');
await samlPage.setExtraHTTPHeaders(headers);

=> The basic authentication popup persists.

Option-3: Attempted to manage the auth popup using page.keyboard().

await samlPage.keyboard.type("<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="780d0b1d0a381c1715191116561b1715">[email protected]</a>");
await samlPage.keyboard.press("Tab");
await samlPage.keyboard.type("mypassword");
await samlPage.keyboard.press("Enter");

=> Unfortunately, no text input is recognized in the fields.

If anyone can offer guidance on this matter, it would be greatly appreciated.

javascript puppeteer

Answer 1

Answer №1

We encountered a similar issue where we needed to exchange SSO username and password for an Okta Access Token within a federated setup involving OKTA->ADFS->OKTA. Our solution involved a complex flow utilizing an Okta Application, Node.js server, and Node.js client. Below is the Node.js client that achieves the following:

Initiates a puppeteer session
Submits username/password
Follows all redirects
Handles Basic HTTP Authentication dialog when it appears after a redirect

This client works seamlessly in both "headless" and "headed" modes.

'use strict'

const puppeteer = require('puppeteer')
const loginUrl = `http://localhost:8998/login`

async function interceptAuthChallenge(client, username, password) {
    // Registering interception of all Chrome Requests
    await client.send('Network.setRequestInterception', {
        patterns: [{urlPattern: '*'}],
    })

    let authChallengeSubmitted = {}
    await client.on('Network.requestIntercepted', async e => {
        console.log(`EVENT INFO: id=${e.interceptionId}\t type=${e.resourceType}\t isNav=${e.isNavigationRequest}\t isAuthChallenge=${e.authChallenge}`)

        if (e.authChallenge) {
            if (authChallengeSubmitted[e.interceptionId]) {
                delete authChallengeSubmitted[e.interceptionId]
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'CancelAuth'
                    }
                })
            } else {
                authChallengeSubmitted[e.interceptionId] = true

                console.log('Responding to SSO with username/password')
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'ProvideCredentials',
                        'username': username,
                        'password': password
                    }
                })
            }
        } else {
            await client.send('Network.continueInterceptedRequest', {
                interceptionId: e.interceptionId,
            })
        }
    })
}

async function loginWithCDP(username, password) {
    const browser = await puppeteer.launch({
        args: [
            '--disable-setuid-sandbox',
            '--no-sandbox',
            '--ignore-certificate-errors',
        ],
        ignoreHTTPSErrors: true,
        headless: true
    })
    const context = await browser.createIncognitoBrowserContext()
    const page = await context.newPage()
    await page.setViewport({width: 1200, height: 720})

    await page.goto(loginUrl, {waitUntil: 'networkidle0'})
    await page.type('#id-of-the-username-field', username)

    const client = await page.target().createCDPSession()
    await client.send('Network.enable')
    await interceptAuthChallenge(client, username, password)

    await Promise.all([
        page.click('#id-of-the-submit-button'),
        page.waitForNavigation({waitUntil: 'networkidle0'}),
    ])
    
    await page.waitForTimeout(4000)
    await browser.close()
    console.log('Done.')
}


loginWithCDP({YOUR_USERNAME}, {YOUR_PASSWORD})

Answer 2

We encountered a similar issue where we needed to exchange SSO username and password for an Okta Access Token within a federated setup involving OKTA->ADFS->OKTA. Our solution involved a complex flow utilizing an Okta Application, Node.js server, and Node.js client. Below is the Node.js client that achieves the following:

Initiates a puppeteer session
Submits username/password
Follows all redirects
Handles Basic HTTP Authentication dialog when it appears after a redirect

This client works seamlessly in both "headless" and "headed" modes.

'use strict'

const puppeteer = require('puppeteer')
const loginUrl = `http://localhost:8998/login`

async function interceptAuthChallenge(client, username, password) {
    // Registering interception of all Chrome Requests
    await client.send('Network.setRequestInterception', {
        patterns: [{urlPattern: '*'}],
    })

    let authChallengeSubmitted = {}
    await client.on('Network.requestIntercepted', async e => {
        console.log(`EVENT INFO: id=${e.interceptionId}\t type=${e.resourceType}\t isNav=${e.isNavigationRequest}\t isAuthChallenge=${e.authChallenge}`)

        if (e.authChallenge) {
            if (authChallengeSubmitted[e.interceptionId]) {
                delete authChallengeSubmitted[e.interceptionId]
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'CancelAuth'
                    }
                })
            } else {
                authChallengeSubmitted[e.interceptionId] = true

                console.log('Responding to SSO with username/password')
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'ProvideCredentials',
                        'username': username,
                        'password': password
                    }
                })
            }
        } else {
            await client.send('Network.continueInterceptedRequest', {
                interceptionId: e.interceptionId,
            })
        }
    })
}

async function loginWithCDP(username, password) {
    const browser = await puppeteer.launch({
        args: [
            '--disable-setuid-sandbox',
            '--no-sandbox',
            '--ignore-certificate-errors',
        ],
        ignoreHTTPSErrors: true,
        headless: true
    })
    const context = await browser.createIncognitoBrowserContext()
    const page = await context.newPage()
    await page.setViewport({width: 1200, height: 720})

    await page.goto(loginUrl, {waitUntil: 'networkidle0'})
    await page.type('#id-of-the-username-field', username)

    const client = await page.target().createCDPSession()
    await client.send('Network.enable')
    await interceptAuthChallenge(client, username, password)

    await Promise.all([
        page.click('#id-of-the-submit-button'),
        page.waitForNavigation({waitUntil: 'networkidle0'}),
    ])
    
    await page.waitForTimeout(4000)
    await browser.close()
    console.log('Done.')
}


loginWithCDP({YOUR_USERNAME}, {YOUR_PASSWORD})

Answer 3

Answer №2

One issue I encountered was the inability to read the page URL when the authentication popup appeared. To work around this, I discovered a method that allowed me to access and manipulate the URL to send basic credentials in the format of https://user:password@domain.

var user=encodeURIComponent(`${process.env.vault_user}`)
var password=encodeURIComponent(`${process.env.password}`)

// Simulate clicking on a button on the homepage that opens a SAML window and obtain a reference to it.
var samlPage = await pagehelpers.clickAndWaitForTarget(authSubmitButton) 

// Perform actions within the SAML window
await pagehelpers.enterText(samlSignInUserTextField, `${process.env.vault_user}`, samlPage)
await pagehelpers.clickElement(samlSignInNextButton, samlPage) // This will trigger the authentication popup
const ssoRequestLS = await samlPage.waitForRequest(request => request.url().startsWith('https://sso.test.com/adfs/ls/wia'),        )
       modifiedURL = ssoRequestLS.url().replace(
            "sso.test.com/adfs/ls/wia",
            `${user}:${password}@sso.test.com/adfs/ls/wia`,
        )
await pagehelpers.navigateToURL(modifiedURL, samlPage)

async clickAndWaitForTarget(locator, page = this.page) {
        const pageTarget = page.target();
        await page.click(locator);
        const newTarget = await this.browser.waitForTarget(target => target.opener() === pageTarget);
        const newPage = await newTarget.page();
        await newPage.waitForNavigation({ waitUntil: "domcontentloaded" });
        await newPage.waitForSelector("body");
        return newPage;
}

Answer 4

One issue I encountered was the inability to read the page URL when the authentication popup appeared. To work around this, I discovered a method that allowed me to access and manipulate the URL to send basic credentials in the format of https://user:password@domain.

var user=encodeURIComponent(`${process.env.vault_user}`)
var password=encodeURIComponent(`${process.env.password}`)

// Simulate clicking on a button on the homepage that opens a SAML window and obtain a reference to it.
var samlPage = await pagehelpers.clickAndWaitForTarget(authSubmitButton) 

// Perform actions within the SAML window
await pagehelpers.enterText(samlSignInUserTextField, `${process.env.vault_user}`, samlPage)
await pagehelpers.clickElement(samlSignInNextButton, samlPage) // This will trigger the authentication popup
const ssoRequestLS = await samlPage.waitForRequest(request => request.url().startsWith('https://sso.test.com/adfs/ls/wia'),        )
       modifiedURL = ssoRequestLS.url().replace(
            "sso.test.com/adfs/ls/wia",
            `${user}:${password}@sso.test.com/adfs/ls/wia`,
        )
await pagehelpers.navigateToURL(modifiedURL, samlPage)

async clickAndWaitForTarget(locator, page = this.page) {
        const pageTarget = page.target();
        await page.click(locator);
        const newTarget = await this.browser.waitForTarget(target => target.opener() === pageTarget);
        const newPage = await newTarget.page();
        await newPage.waitForNavigation({ waitUntil: "domcontentloaded" });
        await newPage.waitForSelector("body");
        return newPage;
}

Answer 5

Answer №3

utilize the following code snippet to implement basic authentication in Puppeteer: 
await page.authenticate({'username':'YOUR_BASIC_AUTH_USERNAME', 'password': 'YOUR_BASIC_AUTH_PASSWORD'});

source: https://dev.to/sonyarianto/puppeteer-quick-tip-how-to-do-basic-authentication-2pe7

Answer 6

utilize the following code snippet to implement basic authentication in Puppeteer: 
await page.authenticate({'username':'YOUR_BASIC_AUTH_USERNAME', 'password': 'YOUR_BASIC_AUTH_PASSWORD'});

source: https://dev.to/sonyarianto/puppeteer-quick-tip-how-to-do-basic-authentication-2pe7

Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

Answer №1

Answer №2

Answer №3

Similar questions

Is there a way to transfer PHP user information (session) to a Node.js server?

Arrangement of Bootstrap card components

React Higher Order Components (HOCs) are functioning correctly with certain components but not

Validator returns undefined when expressing invalid data

Preventing an event from bubbling up to its parent in a jQuery Ajax call: A step-by-step guide

Displaying the number of tasks completed compared to the total number of tasks within a JavaScript ToDo list

How can I save a PDF file using React?

Add AngularJS to an AJAX call when the user clicks on it

Passing this as a function parameter in Jquery

Unable to access external library using browserify and debowerify

The REST request is preventing the JavaScript on the page from executing

The pagination feature in DataTable does not retain the edited page after making changes

Transmit data from an HTML form to PHP using JSON

What is the best way to create a sleek typewriter effect transition by hovering between tabs?

The useEffect hook in my React app causes the homepage to refresh

Instead of only one menu icon, now there are three menu icons displayed on the screen. (Additionally, two more divs containing

Cross-Origin Resource Sharing Problem - Angular version 8 with NodeJS and ExpressJS

Tips for altering the class of an HTML button dynamically when it's clicked during runtime

Tips on concealing confidential keys within a React.js application on the frontend aspect

The Promise.all function encountered an error: Uncaught TypeError: #<Promise> is not an iterable object