Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

Question

Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

I need assistance with handling a basic authentication popup using puppeteer=5.2.1. My specific scenario is outlined below.

1. Start an application
2. Click on a button on the home page
3. This action opens a new window (saml) requesting email input

Within the new window,
4. Input email address
5. Click on Next button
6. A basic authentication popup appears (=> assistance needed here)

The following code reaches up to the 3rd step, where I am able to access the reference of the new page (saml window)

var browser = await puppeteer.launch({
      headless: false, args: ["--start-maximized"]
    });
 var page = await this.browser.newPage();
await waitUntilElementIsVisible(authMethodDropDown);
await wait(5000);
await page.select(authMethodDropDown, "myoption");
const samlPage = await clickAndWaitForTarget(page, authSubmitButton);
wait(5000);
await samlPage.waitForSelector(samlSignInUserTextField);

Below are the different approaches I have tried to handle the basic auth popup in the 6th step.

Option-1: utilizing page.authenticate()

await samlPage.authenticate({username: `${myuser}`, password: `${mypass}`});
await samlPage.type(samlSignInUserTextField, `${myuser}`);
await samlPage.click(samlSignInNextButton);

=> The loading continues and the next page is not displayed.

Option-2: employing setExtraHttpHeaders

const headers = new Map();
headers.set(
   'Autorization',
   `Basic ${new Buffer(`${myuser}:${mypass}`).toString('base64')}`
 );
headers.set('WWW-Authenticate','Negotiate');
headers.set('WWW-Authenticate','NTLM');
await samlPage.setExtraHTTPHeaders(headers);

=> The basic authentication popup persists.

Option-3: Attempted to manage the auth popup using page.keyboard().

await samlPage.keyboard.type("<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="780d0b1d0a381c1715191116561b1715">[email protected]</a>");
await samlPage.keyboard.press("Tab");
await samlPage.keyboard.type("mypassword");
await samlPage.keyboard.press("Enter");

=> Unfortunately, no text input is recognized in the fields.

If anyone can offer guidance on this matter, it would be greatly appreciated.

javascript puppeteer

Answer 1

Answer №1

We encountered a similar issue where we needed to exchange SSO username and password for an Okta Access Token within a federated setup involving OKTA->ADFS->OKTA. Our solution involved a complex flow utilizing an Okta Application, Node.js server, and Node.js client. Below is the Node.js client that achieves the following:

Initiates a puppeteer session
Submits username/password
Follows all redirects
Handles Basic HTTP Authentication dialog when it appears after a redirect

This client works seamlessly in both "headless" and "headed" modes.

'use strict'

const puppeteer = require('puppeteer')
const loginUrl = `http://localhost:8998/login`

async function interceptAuthChallenge(client, username, password) {
    // Registering interception of all Chrome Requests
    await client.send('Network.setRequestInterception', {
        patterns: [{urlPattern: '*'}],
    })

    let authChallengeSubmitted = {}
    await client.on('Network.requestIntercepted', async e => {
        console.log(`EVENT INFO: id=${e.interceptionId}\t type=${e.resourceType}\t isNav=${e.isNavigationRequest}\t isAuthChallenge=${e.authChallenge}`)

        if (e.authChallenge) {
            if (authChallengeSubmitted[e.interceptionId]) {
                delete authChallengeSubmitted[e.interceptionId]
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'CancelAuth'
                    }
                })
            } else {
                authChallengeSubmitted[e.interceptionId] = true

                console.log('Responding to SSO with username/password')
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'ProvideCredentials',
                        'username': username,
                        'password': password
                    }
                })
            }
        } else {
            await client.send('Network.continueInterceptedRequest', {
                interceptionId: e.interceptionId,
            })
        }
    })
}

async function loginWithCDP(username, password) {
    const browser = await puppeteer.launch({
        args: [
            '--disable-setuid-sandbox',
            '--no-sandbox',
            '--ignore-certificate-errors',
        ],
        ignoreHTTPSErrors: true,
        headless: true
    })
    const context = await browser.createIncognitoBrowserContext()
    const page = await context.newPage()
    await page.setViewport({width: 1200, height: 720})

    await page.goto(loginUrl, {waitUntil: 'networkidle0'})
    await page.type('#id-of-the-username-field', username)

    const client = await page.target().createCDPSession()
    await client.send('Network.enable')
    await interceptAuthChallenge(client, username, password)

    await Promise.all([
        page.click('#id-of-the-submit-button'),
        page.waitForNavigation({waitUntil: 'networkidle0'}),
    ])
    
    await page.waitForTimeout(4000)
    await browser.close()
    console.log('Done.')
}


loginWithCDP({YOUR_USERNAME}, {YOUR_PASSWORD})

Answer 2

We encountered a similar issue where we needed to exchange SSO username and password for an Okta Access Token within a federated setup involving OKTA->ADFS->OKTA. Our solution involved a complex flow utilizing an Okta Application, Node.js server, and Node.js client. Below is the Node.js client that achieves the following:

Initiates a puppeteer session
Submits username/password
Follows all redirects
Handles Basic HTTP Authentication dialog when it appears after a redirect

This client works seamlessly in both "headless" and "headed" modes.

'use strict'

const puppeteer = require('puppeteer')
const loginUrl = `http://localhost:8998/login`

async function interceptAuthChallenge(client, username, password) {
    // Registering interception of all Chrome Requests
    await client.send('Network.setRequestInterception', {
        patterns: [{urlPattern: '*'}],
    })

    let authChallengeSubmitted = {}
    await client.on('Network.requestIntercepted', async e => {
        console.log(`EVENT INFO: id=${e.interceptionId}\t type=${e.resourceType}\t isNav=${e.isNavigationRequest}\t isAuthChallenge=${e.authChallenge}`)

        if (e.authChallenge) {
            if (authChallengeSubmitted[e.interceptionId]) {
                delete authChallengeSubmitted[e.interceptionId]
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'CancelAuth'
                    }
                })
            } else {
                authChallengeSubmitted[e.interceptionId] = true

                console.log('Responding to SSO with username/password')
                await client.send('Network.continueInterceptedRequest', {
                    'interceptionId': e.interceptionId,
                    'authChallengeResponse': {
                        'response': 'ProvideCredentials',
                        'username': username,
                        'password': password
                    }
                })
            }
        } else {
            await client.send('Network.continueInterceptedRequest', {
                interceptionId: e.interceptionId,
            })
        }
    })
}

async function loginWithCDP(username, password) {
    const browser = await puppeteer.launch({
        args: [
            '--disable-setuid-sandbox',
            '--no-sandbox',
            '--ignore-certificate-errors',
        ],
        ignoreHTTPSErrors: true,
        headless: true
    })
    const context = await browser.createIncognitoBrowserContext()
    const page = await context.newPage()
    await page.setViewport({width: 1200, height: 720})

    await page.goto(loginUrl, {waitUntil: 'networkidle0'})
    await page.type('#id-of-the-username-field', username)

    const client = await page.target().createCDPSession()
    await client.send('Network.enable')
    await interceptAuthChallenge(client, username, password)

    await Promise.all([
        page.click('#id-of-the-submit-button'),
        page.waitForNavigation({waitUntil: 'networkidle0'}),
    ])
    
    await page.waitForTimeout(4000)
    await browser.close()
    console.log('Done.')
}


loginWithCDP({YOUR_USERNAME}, {YOUR_PASSWORD})

Answer 3

Answer №2

One issue I encountered was the inability to read the page URL when the authentication popup appeared. To work around this, I discovered a method that allowed me to access and manipulate the URL to send basic credentials in the format of https://user:password@domain.

var user=encodeURIComponent(`${process.env.vault_user}`)
var password=encodeURIComponent(`${process.env.password}`)

// Simulate clicking on a button on the homepage that opens a SAML window and obtain a reference to it.
var samlPage = await pagehelpers.clickAndWaitForTarget(authSubmitButton) 

// Perform actions within the SAML window
await pagehelpers.enterText(samlSignInUserTextField, `${process.env.vault_user}`, samlPage)
await pagehelpers.clickElement(samlSignInNextButton, samlPage) // This will trigger the authentication popup
const ssoRequestLS = await samlPage.waitForRequest(request => request.url().startsWith('https://sso.test.com/adfs/ls/wia'),        )
       modifiedURL = ssoRequestLS.url().replace(
            "sso.test.com/adfs/ls/wia",
            `${user}:${password}@sso.test.com/adfs/ls/wia`,
        )
await pagehelpers.navigateToURL(modifiedURL, samlPage)

async clickAndWaitForTarget(locator, page = this.page) {
        const pageTarget = page.target();
        await page.click(locator);
        const newTarget = await this.browser.waitForTarget(target => target.opener() === pageTarget);
        const newPage = await newTarget.page();
        await newPage.waitForNavigation({ waitUntil: "domcontentloaded" });
        await newPage.waitForSelector("body");
        return newPage;
}

Answer 4

One issue I encountered was the inability to read the page URL when the authentication popup appeared. To work around this, I discovered a method that allowed me to access and manipulate the URL to send basic credentials in the format of https://user:password@domain.

var user=encodeURIComponent(`${process.env.vault_user}`)
var password=encodeURIComponent(`${process.env.password}`)

// Simulate clicking on a button on the homepage that opens a SAML window and obtain a reference to it.
var samlPage = await pagehelpers.clickAndWaitForTarget(authSubmitButton) 

// Perform actions within the SAML window
await pagehelpers.enterText(samlSignInUserTextField, `${process.env.vault_user}`, samlPage)
await pagehelpers.clickElement(samlSignInNextButton, samlPage) // This will trigger the authentication popup
const ssoRequestLS = await samlPage.waitForRequest(request => request.url().startsWith('https://sso.test.com/adfs/ls/wia'),        )
       modifiedURL = ssoRequestLS.url().replace(
            "sso.test.com/adfs/ls/wia",
            `${user}:${password}@sso.test.com/adfs/ls/wia`,
        )
await pagehelpers.navigateToURL(modifiedURL, samlPage)

async clickAndWaitForTarget(locator, page = this.page) {
        const pageTarget = page.target();
        await page.click(locator);
        const newTarget = await this.browser.waitForTarget(target => target.opener() === pageTarget);
        const newPage = await newTarget.page();
        await newPage.waitForNavigation({ waitUntil: "domcontentloaded" });
        await newPage.waitForSelector("body");
        return newPage;
}

Answer 5

Answer №3

utilize the following code snippet to implement basic authentication in Puppeteer: 
await page.authenticate({'username':'YOUR_BASIC_AUTH_USERNAME', 'password': 'YOUR_BASIC_AUTH_PASSWORD'});

source: https://dev.to/sonyarianto/puppeteer-quick-tip-how-to-do-basic-authentication-2pe7

Answer 6

utilize the following code snippet to implement basic authentication in Puppeteer: 
await page.authenticate({'username':'YOUR_BASIC_AUTH_USERNAME', 'password': 'YOUR_BASIC_AUTH_PASSWORD'});

source: https://dev.to/sonyarianto/puppeteer-quick-tip-how-to-do-basic-authentication-2pe7

Puppeteer - How to manage a basic authentication prompt that pops up after clicking a link

Answer №1

Answer №2

Answer №3

Similar questions

Generate a JSON object specifically for the modified input fields upon submitting the form

Arrow icon from Material UI for a smaller device

Invoking a function passed via props that utilizes react-router's features

Ways to have MongoDB present nested JSON data as an array?

Conceal virtual keyboard on mobile when in autocomplete box focus

``Can you provide guidance on excluding matching values from a dictionary object in a Angular project?

What is the best way to accurately establish a new name for the evolving scope

What is the most effective way to send multiple values through the <option> value attribute?

Exploring Node.js and JSON: Retrieving specific object attributes

Issue encountered when calling theme.breakpoints.down('') function from Material UI

Interpolating strings in a graphQL query

Tips for creating a responsive tab indicator in Material UI?

Adding click functionality to dynamically generated list items in jQuery and HTML

Implementing a Response to an AJAX POST Request with Servlets

Is there a standard event triggered upon the closing of a browser tab or window?

Establish and define the global variable "Protractor"

Discovering the method for accessing a variable within jQuery from a regular JavaScript function

Incorporating VueJS with a sleek material design aesthetic

The console is displaying a null value outside of the block, however, the correct value is returned when

An error has occurred in the Next.js App: createContext function is not defined