What is the process for retrieving data from a website?

Question

What is the process for retrieving data from a website?

Currently, I am in the process of setting up an automated test using the basic page object pattern. The objective is to open Google, input a search query, click on a link from the results, and retrieve the information. So far, I have successfully navigated to the desired page. However, my challenge lies in extracting the contact information located at the bottom of www.sahipro.com and displaying it in the console of my test class. Despite my efforts, I find myself unable to filter out only the emails and numbers as I keep getting additional HTML content. Furthermore, I'm struggling to understand how to reference these page results within my test class.

The task of isolating the contact text and transferring it to a separate test class has proven daunting for me, considering my limited experience in this field.

try {

//Get Document object after parsing the html from given url.
Document document = Jsoup.connect("http://https://sahipro.com/").get();

Element support = document.text("<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="5e2d2b2e2e31...">
print("  Support Email: " + support);

}catch (IOException e) {
e.printStackTrace();
}

print("done");
}

public static void print(String string) {
System.out.println(string);
}

Although I have managed to fetch the information, it appears in HTML format when displayed in the console:

running...
Support Email: <html>
<head>
<meta http-equiv="refresh" 
content="0;url=http://www.dnsrsearch.com/index.php
origURL=http://https%2Fsahipro.com%2F&amp;bc=">
</head>
<body>
<a href="/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="8ffcfaffffe0fd...">[email protected]</a>
</body>
</html>

javascript eclipse selenium jsoup pageobjects

Answer 1

Answer №1

Utilize the Element class method in jsoup to retrieve text from a specified node, rather than extracting text from the document root directly. You can achieve this by using the select method to target a specific node.

 document.select("a[^='mailto:']")[0].text();

Implementing this approach into your code:

try {

 //Retrieve Document object after parsing the HTML content from the provided URL.
      Document document = 
      Jsoup.connect("http://https://sahipro.com/").get();

       String support = document.select("a[^='mailto:']")[0].text();
        String phone = document.select("a[^='tel:']")[0].text();

      print("  Support Email: " + support + " Support Phone: "+ phone ); 

     } catch (IOException e) {
      e.printStackTrace();
     }

     print("done");
 }

public static void print(String string) {
    System.out.println(string);
 }

Answer 2

Utilize the Element class method in jsoup to retrieve text from a specified node, rather than extracting text from the document root directly. You can achieve this by using the select method to target a specific node.

 document.select("a[^='mailto:']")[0].text();

Implementing this approach into your code:

try {

 //Retrieve Document object after parsing the HTML content from the provided URL.
      Document document = 
      Jsoup.connect("http://https://sahipro.com/").get();

       String support = document.select("a[^='mailto:']")[0].text();
        String phone = document.select("a[^='tel:']")[0].text();

      print("  Support Email: " + support + " Support Phone: "+ phone ); 

     } catch (IOException e) {
      e.printStackTrace();
     }

     print("done");
 }

public static void print(String string) {
    System.out.println(string);
 }

What is the process for retrieving data from a website?

Answer №1

Similar questions

Tips for accessing files following the transmission of a post request within the req.body

Is there a way to use Selenium to click on "javascript:void(0)"?

JavaScript tag filtering system: Display only the div elements that contain all the specified classes in an array; otherwise, hide them

Encountering difficulties while attempting to convert JSON to XML resulting in an error being

Encountering timeout issues with Next.JS fetch and Axios requests on Vercel production environment

Utilize vue.js to cache and stream videos

Issue: EMFILE - exceeding maximum number of opened files

Downloading a file utilizing Selenium through programming (versions 2.46 and above)

Switches in a React-Native ListView are not refreshing properly

Troubaling with AngularJS Routing issues

Employing the validateAll() function in conjunction with a personalized v-select component within the scope

Issue with HTTP headers not being effective in $.AJAX request

Having trouble with loading the Select locator in Selenium from a properties file using Java

What is the reason for the PHP condition not functioning properly unless I explicitly echo the variable?

Using a computed property setter in Vue.js/Javascript while focusing on a datepicker can lead to crashing the browser

Refining search outcomes using multiple Multiselect boxes in VueJS

set up the router by defining the behavior for each route

Practicing diligently, JavaScript code snippets

Implementing dynamic URLs using static routing in Express

Using jQuery .animate() leading to erratic input movements