Using Selenium and Java to retrieve population data from en.wikipedia.org for countries

I am attempting to retrieve the population data from a specific element.

I have attempted to use the parent method, but encountered an error. My goal is to extract the population information for China from the table on this page: 

https://en.wikipedia.org/wiki/China

Here is the code I have tried:

WebElement dateBox7 = driver.findElement(By.xpath("//*[contains(text(),'Population')]/parent::tr//td/a[text()]"));

The error I received is:

Exception in thread "main" org.openqa.selenium.NoSuchElementException:

I also attempted to use the following xpath: 

//*[contains(text(),'2016 estimate')]/ancestor::tr/td/text()[1]
and it successfully displayed the population in the DOM. However, when using it in Selenium, I encountered this error:

Exception in thread "main" org.openqa.selenium.InvalidSelectorException: invalid selector: The result of the xpath expression "//*[contains(text(),'2016 estimate')]/ancestor::tr/td/text()[1]" is: [object Text]. It should be an element.

Can anyone explain why this is not working as expected?

javascriptjavaselenium-webdriverxpathwebdriverwait

Answer №1

Check out this xpath solution

"//*[contains(.,'Population')]/parent::tr//th/a[text()]"

It's important to note two changes in this xpath: td has been changed to th, and [contains(text(),'Population')] has been modified to [contains(.,'Population')].

When using [contains(.,'Population')], the "dot" indicates a search within the entire content of the element (including descendants), while "text()" focuses only on the direct textual content.

UPDATE:

Considering your feedback, you can try this revised xpath:

"//*[contains(.,'Population')]/parent::tr/following-sibling::tr[1]/td"

UPDATE #2:

If you want to learn more about Xpath locators, you can visit this resource and for a detailed course on the topic, check out this link.

Answer №2

To retrieve the population of China, specifically the text 1,403,500,365, from the webpage located at 

https://en.wikipedia.org/wiki/China
, you will need to utilize WebDriverWait in conjunction with visibilityOfElementLocated(). Since the desired text is contained within a Text Node, you will also need to employ executeScript() by following this recommended Locator Strategy:

WebElement myElement = new WebDriverWait(driver, 20).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//a[text()='Population']//following::tr[1]//td")));
System.out.println(((JavascriptExecutor)driver).executeScript("return arguments[0].childNodes[2].textContent;", myElement).toString());

For a similar discussion, you can refer to How to retrieve text from an element using Selenium WebDriver and Java

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Improved method for transferring Mongodb query information to Pug

I'm seeking a more efficient method of passing data to my index.js file in a web development application. With only about a month of experience in web development, I acknowledge that this challenge likely stems from my lack of expertise. Here is the w ...

javascriptmongodbexpressmongoosepug

Understanding the time complexity of Object.entries()

Is the complexity of Object.entries() in JavaScript known? According to information from this question, it seems like it could possibly be O(n) if implemented by collecting keys and values as arrays and then combining them together? ...

javascriptarraystime-complexityv8

Using Vue.js, send information from an Ajax request to a Laravel controller for processing

As someone new to development, I have a little confusion with handling data from a multi-step form in an AJAX request on my controller. While I've been able to successfully collect all form inputs at once, I'm struggling to use the data from $req ...

javascriptvue.jsaxios

Creating Angular UI states with parameters in TypeScript

My Understanding In my experience with TypeScript and angular's ui state, I have utilized "type assertion" through the UI-Router definitely typed library. By injecting $state into my code as shown below: function myCtrl($state: ng.ui.IStateService){ ...

javascriptangularjstypescriptangular-ui-routerangular-ui

Extracting the text specifically from the child element in Webdriver using Java

I need to extract the GMT text from a child element. Here's an example: <strong class="EnvMain"> <strong id="currentClock">11:19</strong> GMT </strong> My goal is to retrieve only the GMT text. I attempted to use xpath l ...

javaseleniumxpathwebdriverelement

Running Python code with Selenium on an Azure server

While using Selenium to open web pages and extract content, I encountered an issue on my Azure server (which is command line based) as it does not support Chrome or Firefox browsers. Therefore, I am seeking an alternative method to run Python-based Seleniu ...

javascriptpythonazureselenium

The React application deployed on GitHub Pages is displaying a blank white screen

I developed a simple meme generator app in React.js as an exercise to learn web development, but I am facing issues while trying to host it on Github pages. I followed all the necessary steps such as installing gh-pages with node, updating packages.json wi ...

javascriptreactjsgithub-pages

Issue with showing error messages in view when using ejs templates

I am a beginner with node.js and I'm struggling to show error messages in the view using ejs templates. I want to display This user already exists. Here is my code: node.js router.post('/signup', (req, res) => { var username = req. ...

javascriptmysqlnode.jsejsregistration

Why won't Node.js let me redirect to my error page?

I've been putting together my newsletter project with the Mailchimp API, everything seems to be working fine except for when I try to redirect to a failure page if the status code is not 200. The browser shows an error message saying 'localhost r ...

javascriptnode.jsnewslettermailchimp-api-v3.0

Execute script when the awaited promise is fulfilled

I am looking to retrieve the URL of a cat image using the Pexels API through a script, and then set that image link as the source of an actual image element. I attempted to include some loading text to keep things interesting while waiting for the image l ...

javascripthtmlasynchronousasync-await

Enhancement from the original JavaScript class framework created by John Resig

Greetings everyone, Lately, I've been on the lookout for a straightforward JavaScript class framework that focuses solely on basic inheritance. After some research, I stumbled upon John Resig's sample framework shared on his blog, and it seems t ...

javascriptjavascript-framework

Encrypting and decrypting data using RSA in TypeScript

Currently, I am utilizing Angular 4 to develop the front end of my application. For authentication, I have integrated OAuth2 on the backend (which is created using Spring in Java), ensuring that only authorized individuals can access my app. However, ther ...

javascriptangulartypescriptencryptionrsa

Instructions on keeping a numerical counter at its current value for all site visitors

Recently, I integrated a number counter into my website. However, I am facing an issue where the count resets to zero whenever a new visitor accesses the site. I'd like the count to remain persistent and update based on the previous count. For instanc ...

javascripthtml

When attempting to send a fetch request in the most recent rendition of NextJS, it ends up with an error of 'failed fetch'

I am currently working on a nextjs (v.13.4.19) / strapi (v.4.12.5) application and facing issues when trying to make a request to the strapi endpoint using the fetch function. I have attempted several troubleshooting steps such as: changing localhost:1337 ...

javascriptreactjstypescriptnext.jsstrapi

Making sure to consistently utilize the service API each time the form control is reset within Angular 4

In the component below, an external API service is called within the ngOnInit function to retrieve an array of gifs stored in this.items. The issue arises when the applyGif function is triggered by a user clicking on an image. This function resets the For ...

javascriptangularangular-reactive-formsvaluechangelistener

Guide to embedding an iframe generated with JavaScript into a React component

I'm currently working on developing an app that will utilize an iframe. The goal is to have controllers in place to adjust the style of the iframe and add text to it, essentially creating a preview function. I've been attempting to use Javascript ...

javascripthtmlreactjsiframereact-router

Encountering issues with chromedriver on Heroku platform

I have a discord bot designed to fetch leaderboard data from a specific website. The code functions flawlessly on my MacBook, but when I deploy it on Heroku, an error occurs: /app/chromedriver: 1: Syntax error: word unexpected (expecting ")") app ...

javaseleniumgithubherokuselenium-chromedriver

What is the best approach for managing multiple requests to populate models with RXJava Observables?

Our network stack utilizes ReactiveX and Retrofit to handle API requests asynchronously. The objective is to create a method that can retrieve a fully populated collection of User models. Each User model contains a list of Pet objects. While we can fetch ...

javaandroidretrofitobservablerx-android

What to do when encountering a 404 Error with `meta.json` in a Next.js Application

Hello fellow developers, I'm currently facing a challenge with my Next.js application. Upon running it, the terminal keeps throwing a meta.json 404 error, and I'm stuck on how to resolve it. Here are some key points for context: The Next.js app ...

javascriptreactjsnext.js

An issue occurred while trying to serialize cookies retrieved in getServerSideProps

I am currently working on a project using Reactjs with Next.js. I recently implemented a login module using cookies, but encountered an issue when trying to serialize the .cookies object returned from getServerSideProps. The error message states that undef ...

javascriptreactjsnext.js