Extract the complete HTML information from the weather website

Currently, I am attempting to retrieve weather data from the following website:

using this code snippet:

 try {
                int i = 0;
                if (googlefirst3.startsWith("http")) {
                    Document document = Jsoup.connect("https://www.ilmeteo.it/meteo/Magenta/previsioni-orarie?refresh_ce").userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11 Firefox/19.0").timeout(0).get();
                    Elements temp = document.select("tr");

                    String verifica;
                    verifica=document.html();
                    for (Element movielist : temp) {
                        i++;
                        html = (i + "|||" + movielist.getElementsByTag("td").first().html());
                        array3b[i] = html;

                    }
                }

            } catch (IOException e) {
                e.printStackTrace();}

I am specifically interested in extracting the table rows containing temperature, wind, and time information:

https://i.sstatic.net/cLbNc.png

However, despite my efforts, I have been unsuccessful in retrieving this data. The document retrieved does not seem to include this information and appears incomplete. Initially, I suspected that the issue might be related to JavaScript-generated HTML, but even after trying a different method as suggested here:

How do I get the web page contents from a WebView?

I still faced difficulties. At this point, I am uncertain whether JavaScript is the root cause of the problem. Could someone provide assistance in determining the nature of the issue? Your help would be greatly appreciated.

Thank you in advance.

Answer №1

The content on the page you're attempting to parse contains data within an iframe.

<iframe name="frmprevi" id="frmprevi" 
src="https://www.ilmeteo.it/portale/meteo/previsioni1.php?citta=Magenta&amp;c=3749&amp;gm=25" 
width="660" height="600" marginheight="0" marginwidth="0" scrolling="no"
frameborder="0" style="margin:0px;padding:0px"></iframe>

This makes it inaccessible to Jsoup. To retrieve the desired data, simply parse the URL directly from the iframe src:

Now that should simplify things, but remember that the parameter gm=25 in the URL could signify the 25th day of the month, so be prepared to adjust it for a different day.

Answer №2

After further investigation, I discovered an iFrame element.

You may want to attempt something along these lines:

Thread(Runnable {

        val document: Document =
            Jsoup.connect("https://www.ilmeteo.it/meteo/Magenta/previsioni-orarie?refresh_ce")
                .userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11 Firefox/19.0")
                .timeout(2000).get()

        val body = document.body()
        val table = body.getElementsByClass("datatable")

        val iframe: Element = body.getElementById("frmprevi")
        val iframeSrc: String = iframe.attr("src")

        if (iframeSrc != null) {
            val iframeContentDoc = Jsoup.connect(iframeSrc).get()
            val temps = iframeContentDoc.body().getElementsByClass("boldval")
            for(temp in temps)
            {
                Log.d("temps",temp.text())
            }
        }

    }).start()

This code is written in Kotlin, but I believe you can easily translate it to Java and extract additional information from the source as well.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the best way to determine if a child div exists within a parent div?

Is there a way to determine if any div exists within the parent div? In my current scenario, I am adding two divs inside the parent div like this. $('#realTimeContents').append("<div style='width:22%; float: left; font-size:18px; line- ...

Using the Vue.js Spread Operator in place of Vue.set or Vue.delete

I'm exploring ways to utilize the spread operator for adding or removing object properties in a manner that preserves reactivity. Within a Vuex mutation, this code snippet is successful: Vue.set(state.sportTypes.sports, sportName, sportProperties) H ...

Updating node content in jsTree

As a seasoned JavaScript developer transitioning to jQuery, I have encountered an issue while working with jsTree. I am attempting to change the name of a node within the tree but have been unsuccessful so far. Despite trying various examples from differen ...

What is the method to create a number using a textbox through javascript?

Whenever a number is entered into the main_textbox, a JavaScript function is called on the onblur event of the textbox using the following code: function generate_bale(){ var bale=document.getElementById("number_of_bale").value; var boxes = "< ...

Draggable shape using Three.js

I have been studying the code for a draggable cube from this link. However, I am struggling to understand the purpose of creating an offset between the plane and the selected object, as seen in this section of the code: function onDocumentMouseDown( even ...

Exploring the connected component feature in React/Redux

While testing the connected component of my React/Redux app, I encountered an error. The test case that caused the error is: App component › shows account info and debits and credits` Invariant Violation: Could not find "store" in either the context or ...

When using VSCode for Next.js projects, automatic imports from "react" are not supported

* While similar, this question is not a duplicate of this other question; the solutions provided there are tailored for TypeScript and do not seem to apply to JavaScript. In my current project using Next.js in Visual Studio Code, I am facing an issue wher ...

How to Track URL Modifications within an Iframe on an Angular Ionic Mobile Application

Issue I'm currently working on identifying modifications in the URL of an embedded <iframe /> within an Ionic (v5) app using Angular. Solution // ts file @ViewChild('myIframe') public iFrame; backButtonSubscription: Subscription; ...

Error message "Undefined is not a function" occurred while using jQuery's .replace and scrollTop functions

I'm having issues with the scroll function in my code. It doesn't seem to be able to locate the ids in my HTML, even though I can't figure out why. I had a previous version that worked perfectly fine (unfortunately, I didn't save it D:) ...

Abnormal scrolling issues observed on handheld devices like mobile phones and tablets

I recently added a parallax effect to the hero section of my website, and while it works perfectly on desktop, I encountered some scrolling issues when viewing it on mobile or tablet devices. Sometimes the scrolling behaves as intended, and other times it ...

What is the best way to remove an item from my online shopping cart using JavaScript?

I am currently developing an online store website. One issue I am facing is deleting items from the cart after a customer completes an order. Below is the array of cart items: const products = [ { id: '0', name: 'Nike Slim Shirt&ap ...

Manage Camera Movement for 360-Degree Image Viewing in A-Frame

I'm struggling to find any guidance on how to control Camera Rotation in A-Frame. I have a 360 image set up as the <a-image> element. I want the image or camera to rotate as I move the mouse over the screen - moving the mouse right should move t ...

Tips for utilizing the nth-child selector to exclude hidden divs

I am facing an issue with displaying random blocks in rows. Each time a block falls into a new row, I want it to have a different style. However, when the user clicks on a button to hide certain blocks using display:none, the problem arises because the nth ...

What is the best way to create a navigation bar that opens on the same page when clicked?

Can someone help me figure out how to create a navbar that, when clicked, opens a small window on the same page like in this example image? ...

Extract the data that was returned from the AJAX post function

I am looking to create a condition that is dependent on the data received from an ajax post outside of the post function function post(){ $.post('page.php',$('#form').serialize(), function(data) { if(data !== 'good'){a ...

Incorporating list items in a React component is not functioning as expected

When I console.log(this.props), here are my props: list:Array(1): {user: "Jack Nicholson", userid: "5b684ed8d3eb1972b6e04d32", socket: "1c0-Jb-kxe6kzPbPAAAD"} However, despite mapping through my list and using the component <UserItem user={user.user} ...

Issue with jQuery Ajax file upload in CodeIgniter

I am attempting to use AJAX to upload a file in the CodeIgniter framework, but I encountered an error message stating 'You did not select a file to upload.' Please review this code: View <form method="POST" action="" enctype="multipart/form- ...

Change the value of the material slide toggle according to the user's response to the JavaScript 'confirm' dialogue

I am currently working on implementing an Angular Material Slide Toggle feature. I want to display a confirmation message if the user tries to switch the toggle from true to false, ensuring they really intend to do this. If the user chooses to cancel, I&ap ...

Linking a pair of checkboxes

I am dealing with two checkboxes on my website. <input class="checkbox1" type="checkbox" name='1' id="example1" value="example1"/> and <input class="checkbox2" type="checkbox" name='2' id="example2" value="example2"/> I ...

Can Angular 2 and Ionic 2 create a Webview build specifically for android devices?

As an Android Developer, I recently discovered the possibility of creating apps using Angular 2 and Ionic 2. Source: https://scotch.io/tutorials/build-a-mobile-app-with-angular-2-and-ionic-2 I am curious to know which option is the better choice. Wi ...