When the DOM elements have already been rendered, document.querySelector() will return null

I have been working on creating an automated Puppeteer script to download my monthly bank transactions from the bank's website.

However, I have encountered a strange error (refer to the attached Imgur link for images of this issue)

https://i.sstatic.net/suBnQ.jpg

Issue: querySelector Returns Null on Visible DOM Element:

See Screenshot: https://i.sstatic.net/XV2Ka.jpg

(1) The username input box is clearly visible on the website (), (2) However, when I execute 

document.querySelector('#access-code')
, the console returns null.

I am curious about why this behavior is occurring and in what circumstances a browser would return null on a querySelector(#id) query when the DOM element is clearly visible.

# EDIT: Strange Workaround that Resolves the Issue:

While experimenting further with the browser, I used DevTools to inspect the DOM element and copied the JS Path.

Interestingly, after using Chrome Devtools to copy the JS Path, 

document.querySelector('#access-code')
returned the correct element.

Screenshot showing the correct element being returned: https://i.sstatic.net/suBnQ.jpg

In both instances, the same search string is used for document.querySelector.

javascriptseleniumdomscreen-scrapingpuppeteer

Answer №1

It is my belief that attempting to retrieve the proper value using 

document.querySelector('#access-code')
may not be successful when dealing with a website that utilizes framesets.

Within the website, there is a frame with a src attribute that loads content:

<frame src="/internet-banking/Login/Login">

The DOMContentLoading event is triggered when the main document is loaded, without waiting for the frame content to be fully loaded.

Initially, it is crucial to set up a listener for the load event:

window.addEventListener("load",function() {
   ...
 });

Subsequently, you cannot simply use 

document.querySelector('#access-code')
as the input you are trying to access is located within a frame. You will need to find a way to access the frame content and then use a simple querySelector within it.

For instance:

window.addEventListener("load",function() {
   console.log(window.frames[0].document.querySelector('#access-code'));
 });

Additionally, please take a look at the source code of as it appears that the website is primarily rendered on the client side.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

"Surprising notification encountered when trying to shut down the Firefox web-driver using Python

I wrote a simple Python script to log into a website using the Selenium web driver. However, I encountered an unexpected alert from FireFox when trying to close the web driver. Below is my script: from selenium import webdriver from time import * class C ...

pythonfirefoxseleniumselenium-webdriverwebdriver

Jquery Validate doesn't consistently give a positive response

Having a button that triggers the jQuery validation plugin poses an issue where it consistently returns true, except when the fields are left empty. The rules set for validation seem to be disregarded. DEMO http://jsfiddle.net/sw87W/835/ $(document).read ...

javascriptjqueryvalidation

Create a page turning effect in React that simulates scrolling a certain amount at a time

Is there a way to disable default scrolling and have the page automatically scroll to the next item? For example, if I have element2 below element1, how can I set it up so that when I scroll down once, the page scrolls directly to the position of element2 ...

javascriptcssreactjsscrollnodes

Having issues with ToggleClass and RemoveClass functionalities not functioning properly

As a novice in Jquery and CSS/scss, I've managed to create dynamic bootstrap cards for recording players. Each card consists of a music-container with control-play and vinyl elements. I aim to have multiple bootstrap cards generated based on the data ...

javascriptjqueryhtmlcssjquery-ui

Switching up the default font style within TinyMCE

After successfully changing the default font within the editor using the guidelines provided here, I have encountered a new issue. The original default font no longer appears in the font drop-down list. The previous default font was Verdana, and the new d ...

javascriptcssfontstinymcefont-face

Can object methods be called using inline event attributes in an HTML tag?

Exploring the depths of core JavaScript has been my latest obsession. It's fascinating how we can easily trigger functions within the global context using events like this. HTML <span class="name" onclick="clicked()">Name</span> JavaSc ...

javascriptjqueryhtml

Having difficulty rearranging choices within an optgroup

This is a dropdown https://i.sstatic.net/Rk5yL.png <select id="officer-id" placeholder="Choose an officer"> <option selected="selected" >----</option> <optgroup id="pt1" label="To be reviewed"> ...

javascriptjquery

Tips for displaying data by using the append() function when the page is scrolled to the bottom

How can I use the append() function to display data when scrolling to the bottom of the page? Initially, when you load the page index.php, it will display 88888 and more br tags When you scroll to the bottom of the page, I want to show 88888 and more br ...

javascriptphpjqueryhtmlcss

What is the process for producing multiple objects in JSON format and then accessing them from within <g:javascript> in Grails?

My goal is to add two select boxes within the results div using Ajax. I have a function named ajaxFun that retrieves values for the select boxes. However, I am struggling with rendering multiple objects as JSON and then retrieving them in my JavaScript fun ...

javascriptajaxjsongrails

Guide to storing data in the browser's local storage with a Node.js Express backend

Up until this point, I relied on cookies to store preferences for my websites. However, as time has passed, the data stored in these cookies has grown too large and now exceeds the limit of 4096 characters. Therefore, I need to explore an alternative meth ...

javascriptnode.jsexpresscookieslocal-storage

Steps for assigning a date to a hidden element

Currently, I am attempting to extract data from a website using Selenium for Chrome in a VBA Excel macro. The specific information I need pertains to Italian futures, so I navigate to the following page: driver.Get "https://www.eex.com/en/market-data ...

selenium

Using JQuery to switch out images that do not have an ID or class assigned

Currently, I am using a Google Chrome Extension to apply an external theme to a website. Unfortunately, the logo on the site does not have an ID or class that can be used to call it. I am searching for a solution to replace it with a different image. My ...

javascriptjquerycssgoogle-chrome-extension

Reverse the order of jQuery toggle animations

Looking for something specific: How can I create a button that triggers a script and then, when the script is done, reverses the action (toggles)? (I am currently learning javascript/jquery, so I am a beginner in this field) Here's an example: ...

javascriptjqueryhtmlcssjquery-animate

Exploring ways to cycle through a select dropdown in React by utilizing properties sent from the Parent Component

I'm having trouble displaying the props from a parent component in a modal, specifically in a select dropdown. How can I make it so that the dropdown dynamically shows the values from the props instead of the hardcoded 'Agent' value? What am ...

javascriptreactjsreact-hooksrxjsreact-props

Tips for choosing multiple files with the Ctrl key in a list item element:

We have decided to create our own browsing functionality due to the limitations of the existing HTML browse feature. While using the HTML browse, we can only select multiple files by pressing the Ctrl key, and we want to replicate this feature in our custo ...

javascripthtmlangularjs

List of random items:1. Red pen2

By using JavaScript, I want to generate a paragraph that contains a randomly ordered list of items (presented in paragraph format) for a basic website. The input would include: apples concrete a limited dose of contentment North Carolina The final ...

javascripthtmllistrandom

Top solution for maintaining smooth navigation across web pages

As I dive into the world of web development, I find myself intrigued by the idea of reusing navigation and banners across multiple web pages. However, despite my research efforts, I have yet to come across a definitive answer. My objective is simple: The ...

phpjavascripthtmlasp.netcss

Basic use of AJAX for sending the value from jQuery's datepicker

As a novice in JavaScript and AJAX, I'm hoping for your patience as I navigate my way through. In the scope of this project, I am utilizing the CodeIgniter framework. My objective is to implement a datepicker and transmit its value via AJAX. Below is ...

phpjavascriptjqueryajax

Sometimes, the `undefined` TypeError can unexpectedly pop up while making Ajax calls

Here is my issue with AJAX request and response: I have around 85 HTML pages that all use the same AJAX request. However, when working with these files, I sometimes encounter the following error. AJAX $(document).ready(function(){ localStorage.setIte ...

javascriptjqueryhtmlajax

Unable to direct to the main page in index.js of the next.js application

I have been working on a next.js application and encountered an issue with a component I created called <ButtonGroup>. I added a button with the code <Button href="index">Home</Button> to allow users to navigate back to the home ...

javascriptreactjsnext.jsreact-bootstrap