I am currently utilizing Selenium Webdriver to extract content from a webpage. The challenge I'm facing is that the page dynamically loads more content using AJAX as the user scrolls down. While I can programmatically scroll down using JavaScript, I am uncertain about how to determine when to stop scrolling. Since the elements are dynamically attached to the DOM, traditional height properties like document.clientHeight are not applicable in this scenario.
My main question is: How can I identify when I have reached the bottom of the page? Is there a specific condition signaling that the scrollbar is no longer scrollable? Or could this be misleading if additional content is still loading and the scrollbar is temporarily inactive?
Thank you, bsg
Update
The solution that proved effective for me was actually provided in response to another one of my queries (this particular query being a subset of the former). The suggestion involved using jQuery to compare $(document).height() to ($(window).height() + $(window).scrollTop(). If this comparison evaluates to true, it indicates that the bottom of the page has been reached. For further clarification on why this method works, please refer to Meaning of $(window).scrollTop() == $(document).height() - $(window).height() I cannot confirm whether this approach would function without jQuery; therefore, if your test page lacks jQuery, consider utilizing the answer given by user1177636 below.
Here is the actual code snippet I utilized:
JavascriptExecutor js = (JavascriptExecutor)driver;
do{
// Scroll down and perform necessary processing
reachedBottom = Boolean.parseBoolean(js.executeScript("return $(document).height() == ($(window).height() + $(window).scrollTop());").toString());
}while(!reachedBottom);
I trust this information proves useful to someone in a similar situation.