Mastering the art of Puppeteer with Javascript

I am currently learning how to make web requests using JavaScript with puppeteer.

After some trial and error, I was able to extract the value of a tag from a random website.

However, I am struggling to figure out how to retrieve 10 consecutive values of tag a using a "for" loop.

This is the code that successfully extracted the value:

const result0 = await page.evaluate(() => {
        return document.querySelectorAll('.ui.divided.list > a')[0].textContent;
      });
      console.log(`The value is ${result0}`);

Here is my attempt at using a "for" loop (which is not working due to the error "ReferenceError: i is not defined"):

for (var i = 0; i < 9; i++) {
    const result = await page.evaluate(() => {
      return document.querySelectorAll('.ui.divided.list > a')[i].textContent;
    });
    console.log(`The value is ${result}`);
  }

I don't want to repeat the same code and use ([1], [2], [3], ...), so how can I achieve this?

Thank you

EDIT: I FORGOT TO MENTION THAT I'M UNABLE TO PRINT THE RESULT USING querySelectorAll AS IT RETURNS UNDEFINED. FOR EXAMPLE:

const result = await page.evaluate(() => {
      return document.querySelectorAll('.ui.divided.list > a').textContent;
    });
    console.log(`The value is ${result[0]}`);

Answer №1

It's important to iterate within page.evaluate (different contexts), and it's recommended to use Array::map for a smoother process:

const result = await page.evaluate(() => {
  // Select all elements with the given class
  const elements = document.querySelectorAll(".ui.divided.list > a");
  // Convert the node list to an array for mapping and extract text content
  return Array.from(elements).map((element) => element.textContent);
});
console.log(`The value is ${result}`);

Answer №2

It seems like you're encountering an issue

The problem arises from using an await request without an async function

For more information, please refer to this documentation:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function

When you console log the variable "resultado", the await operation has not been resolved yet

const resultado = await page.evaluate(() => {
      return document.querySelectorAll('.ui.divided.list > a')[i].textContent;
    });
    console.log(`The value is ${resultado}`);

Creating an async function to handle this call may help resolve your issue

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips for sending an icon as a prop in React components

I'm struggling to implement an icon as a prop while using props for "optionText" and "optionIcon". The optionText is working fine, but I'm facing issues with the OptionIcon. File where I'm creating props import Icon from ...

Why is the jQuery not functioning within the AngularJS function?

I am encountering an issue with writing jQuery inside an AngularJS function. I can't seem to figure out why it's not working properly. HTML <div> <span ng-repeat="image in post.postImages" ng-if="$index <= 3" ng-init="image.showD ...

Issue with Laravel 5.4: AJAX behaving unexpectedly and not returning errors as it should

After going through several tutorials on handling AJAX requests in Laravel, I'm still facing some issues. Each tutorial has its own approach... Finally, one method is not giving me a 500 error, but it's not displaying validation errors as expect ...

Marionette's Take on the Undead: Zombie Perspectives

Error in JS Console: Uncaught ViewDestroyedError: View (cid: "view351") has already been destroyed and cannot be used. backbone.marionette.js?body=1:1715 Code Snippet: initialize: (options) -> HWAs = @model.get('homework_assignments') @ ...

NodeJS: Implementing external URL redirection functionality

I've set up a GET /auth route that is supposed to redirect to an external resource, like https://google.com. However, instead of redirecting me to the correct URL, it redirects me to http:localhost:3000/api/auth/https://google.com. Is there a way to ...

What could be causing the failure in revealing these elements?

Within my Meteor application, I utilize Template.dynamic to seamlessly replace the current Template with the next one. In my "main" html file, the setup looks like this: <div class="container"> {{> postTravelWizard}} </div> </body> ...

React: Implementing localStorage token addition in loginHandler function using State hook is not functioning as expected

I've implemented an AuthContextProvider in my React application to handle user authentication and logout functionality: import React, { useState } from "react"; import axios from "axios"; import { api } from "../api"; co ...

Guide to creating unit tests for document.URL in Angular 5 specifications

Currently attempting to simulate document.URL = 'dashboard'; however, encountering an issue where it states that I can't assign to url because its readonly property. This problem arose while writing jasmine test cases click here for image de ...

What is the best way to display each value from the array arr, containing strings, on separate lines?

Can you complete the function below to display each value in the array 'arr' on separate lines? <!DOCTYPE html> <html> <head> <title> ...

What could be causing the failure to retrieve the salt and hash values from the database in NodeJS?

My current issue involves the retrieval of hash and salt values from the database. Although these values are being stored during sign up, they are not being retrieved when needed by the application. Below, you will find snapshots of the database, console s ...

Error message "Jquery smooth scrolling issue: Unable to retrieve property 'top' as it is not defined"

I'm currently working on a script for a back to top button that will be placed in the footer of my website. However, I'm running into an issue where I'm getting an error message saying "Cannot read property 'top' of undefined". Any ...

Unable to access a nested JSON object that has a repeated name

I'm relatively new to working with JSON, so the issue I'm facing may be simple, but I haven't been able to find a similar problem on stackoverflow. Here's my question: My goal is to access a nested JSON object like: pizza.topping.ratin ...

Getting started with html2canvas: A beginner's guide

So here's a seemingly simple question... I'm diving into new territory and stumbled upon http://html2canvas.hertzen.com with a straightforward tutorial. After successfully running the command npm install -g html2canvas, I hit a roadblock. Where e ...

Bringing in a JavaScript function from a local file into a Node.js

I've been struggling with this issue for a while now and I think it's because of my misunderstanding of how files are linked in node.js. My file structure looks like this: ./main_test.html ./js_test.js ./node_test.js The main_test.html file is ...

Strategies for ensuring a promise is fulfilled before moving on to the next iteration in a never-ending for loop in JavaScript

I've been exploring ways to ensure that a promise is resolved before moving on to the next iteration in a for loop. One suggestion was to use the setInterval() function instead of a for loop, but this isn't ideal since it's hard to predict w ...

Retrieve the parent object within a constructor function

Is there a way to access the parent object when calling a function contained inside that object as a constructor without explicitly referring to it? Take a look at this scenario: var customers = { // Number of customers count: 0, // Naturally ...

Error: JSON parsing failed due to an unexpected character 'W' at the beginning

I am encountering an issue while sending a POST Request through the API in JavaScript. The method I have been using for multiple POST Requests is now displaying the error message: SyntaxError: Unexpected token W in JSON at position 0 Below is the snippet ...

Why does Socket.IO seem to be registering two clients instead of just one when there is only one connection

When using my app, the user first lands on the home screen where they can select their username. They then proceed to another page and from there, navigate to the room entry page. The issue I'm facing is with a specific section of my code that update ...

The Importance of Selenium Events and Patience

Currently, I am using Selenium to automate some testing for our company's website, but encountering issues along the way. TestItemFromSearch: (driver, part, qty) => { Search.SearchItem(driver, part); driver.findElement(By.id('enterQty ...

showcase every value upon submission in the form with options to edit and delete

Is it possible to display all values submitted in a form with edit and delete buttons? Currently, only one value is being displayed at a time. Whenever a new value is displayed, it replaces the old one. How can this be fixed? You can fin ...