Tips for refining search outcomes from web scraping with Google App Script

Question

Tips for refining search outcomes from web scraping with Google App Script

My objective is to scrape data from the website :

I am specifically interested in extracting the SP500 PE number, which currently stands at 39.57 (at the time of writing). I require this number to be formatted as 39,57, rather than 39.57.

This is my current code:

function webScraper() {

var webURL = "https://www.multpl.com/s-p-500-pe-ratio";

var response = UrlFetchApp.fetch(webURL);

var $ = Cheerio.load(response.getContentText()); 

var itemsOfInterest = $('.info-left').first().text().trim();


Logger.log(itemsOfInterest)

return itemsOfInterest

}

The challenge: While my code successfully retrieves the information, it provides more details than needed. I need to refine the data and convert 39.57 to 39,57.

Here is the full set of information returned by the code:

Current S&P 500 PE Ratio: 39.57

+0.15 (0.39%)

4:00 PM EST, Fri Feb 5

Mean: 15.88

Median: 14.84

Min: 5.31

(Dec 1917)

Max: 123.73 (May 2009)

Price to earnings ratio, based on trailing twelve month “as reported” earnings.
Current PE is estimated from latest reported earnings and current market price.

Source: Robert Shiller and his book Irrational Exuberance for historic S&P 500 PE Ratio.

Answer №1

It is my understanding that your objective is as follows.

You are looking to extract the value of 39.57 from the URL
```
https://www.multpl.com/s-p-500-pe-ratio
```
and convert it to 39,57.
You aim to accomplish this task using Google Apps Script.

Key Points for Modification:

Upon examining the HTML content of the URL, I noticed that the number 39.57 is present in a JSON object. It appears that by utilizing this information, you can easily retrieve the desired value.

In this response, I suggest a workaround which involves retrieving the value from the JSON object embedded within the HTML data. By implementing the following modifications into your script, the process will look like this:

Revised Script:

function webDataExtractor() {
  var websiteURL = "https://www.multpl.com/s-p-500-pe-ratio";
  var results = UrlFetchApp.fetch(websiteURL);
  var content = results.getContentText().match(/var d =([\s\S\w]+?);/);
  if (content && content.length > 0) {
    var objData = JSON.parse(content[1].trim());
    var output = objData.endLabel[0].replace(".", ",");  
    console.log(output) 
    return output;
  }
  return "No value found.";
}

Note:

This revised script specifically caters to the URL
```
https://www.multpl.com/s-p-500-pe-ratio
```
. Using it with other URLs may not yield the same results. Please take note of this distinction.

Resources:

Answer 2

It is my understanding that your objective is as follows.

You are looking to extract the value of 39.57 from the URL
```
https://www.multpl.com/s-p-500-pe-ratio
```
and convert it to 39,57.
You aim to accomplish this task using Google Apps Script.

Key Points for Modification:

Upon examining the HTML content of the URL, I noticed that the number 39.57 is present in a JSON object. It appears that by utilizing this information, you can easily retrieve the desired value.

In this response, I suggest a workaround which involves retrieving the value from the JSON object embedded within the HTML data. By implementing the following modifications into your script, the process will look like this:

Revised Script:

function webDataExtractor() {
  var websiteURL = "https://www.multpl.com/s-p-500-pe-ratio";
  var results = UrlFetchApp.fetch(websiteURL);
  var content = results.getContentText().match(/var d =([\s\S\w]+?);/);
  if (content && content.length > 0) {
    var objData = JSON.parse(content[1].trim());
    var output = objData.endLabel[0].replace(".", ",");  
    console.log(output) 
    return output;
  }
  return "No value found.";
}

Note:

This revised script specifically caters to the URL
```
https://www.multpl.com/s-p-500-pe-ratio
```
. Using it with other URLs may not yield the same results. Please take note of this distinction.

Tips for refining search outcomes from web scraping with Google App Script

Answer №1

Key Points for Modification:

Revised Script:

Note:

Resources:

Similar questions

AJAX: The form submission without refreshing the page only functions for the first attempt

Error message in Material-UI TextField not displaying when state is updated using Object.assign technique

React: The new Date() function displays the date based on the client's machine, not the server

Tips for retrieving the MenuItem name upon click event using Menu and MenuItem components in Material UI React

The <SelectField> component in material-ui is not displaying items properly in ReactJS

I am receiving an undefined value when using document.getElementsByClassName

Can we find a different way to address this issue with a "functional programming" mindset that doesn't involve utilizing the `forEach` method?

Attempting to locate the following div element using a particular class name, however, I consistently retrieve the final one

Oops! The module "rxjs/Subject" seems to be missing the exported member "Subject"

jquery autocomplete utilizing an external json data feed

Divide a collection of objects into groups based on 2-hour time spans

CSS responsive grid - adding a horizontal separator between rows

Troubleshooting base href issues in AngularJS routing

A guide on how to implement the closing functionality of a CSS menu when clicking outside the menu using

How can Typescript be leveraged to enforce a generic constraint on an interface?

Grant the "User" the ability to modify both images and prices

Determining the ultimate height of an element as it transitions in size using CSS

Is it possible for a form to direct submissions to various pages depending on the value of certain fields

What is the best way to generate a dynamically interpolated string in JavaScript?

What are the best practices for utilizing AngularJS's $sanitize service?