Scanning through numerous web pages to tally up the total number of items available

My script is designed to gather the number of reports on each page, then move on to the next page and do the same. The ultimate goal is to calculate the total number of reports across multiple pages.

UPDATED

var casper = require('casper').create({
    clientScripts: ["./lib/jquery-2.1.3.min.js"],
    // verbose: true,
    logLevel: "debug"
});

casper.on('remote.message', function(msg) {
    this.echo('LOG: ' + msg);
});

casper.on('page.error', function (msg, trace) {
    this.echo( 'Error: ' + msg, 'ERROR' );
});

var reportCount, newReportCount, reportPages;

casper.start("reports.html", function() {

    reportPages = this.evaluate(function() {
        return $('#table2 tbody tr td').children('a').length -1;
  });

  //first page of reports
  reportCount = this.evaluate(function() {
      return $('#table1 tbody').first().children('tr').length;
  });

  this.echo('initial count: ' + reportCount);
  this.echo('pages: ' + reportPages);

  //check if more than 1 page and add report count
  if (reportPages > 1) {
    newReportCount = this.thenOpen('reports2.html', function(){
        var newCount = this.evaluate(function(count) {
            add = count + $('#table1 tbody').first().children('tr').length;
            // console.log('new count inside: ' + add);
            return add;
        }, reportCount);
        console.log(newCount); //this shows correct new value 32
    });
    console.log(newReportCount); //this shows [object Casper]

    neoReportCount = this.thenOpen('reports3.html', function(count){
        console.log(newReportCount); //this shows [object Casper]
        //do the same count
    }, newReportCount);
  }

casper.run();

Here is the output in console

Pages: 3
First count: 15
[object Casper], currently at file:///**/reports.html
32
[object Casper], currently at file:///**/reports3.html

Answer №1

Achieving your goal is indeed possible, but it requires using casper.thenOpenAndEvaluate(). The presence of the keyword then indicates that this function operates asynchronously and returns the casper object to facilitate a builder/promise pattern. Consequently, you cannot directly return values from such a function as its execution takes place after the current step has concluded, specifically after console.log(newCount);.

To address this issue, you would have to refactor the function, like so:

// Verify if there are multiple pages and increment report count
if (reportPages > 1) {
  var newCount;
  this.thenOpen('reports2.html', function(count){
    newCount = this.evaluate(function(count){
      add = count + $('#table1 tbody').first().children('tr').length;
      console.log('new count inside: ' + add);
      return add;
    }, reportCount);
    console.log(newCount);
  }).thenOpen('reports3.html', function(count){
    newCount += this.evaluate(function(count){
      add = count + $('#table1 tbody').first().children('tr').length;
      console.log('new count inside: ' + add);
      return add;
    }, reportCount);
    console.log(newCount);
  }).then(function(){
    console.log(newCount);
  });
}

Your scenario implies iterating through multiple pages, typically achieved recursively due to CasperJS' asynchronous nature where the number of required page openings isn't known in advance. For examples illustrating this approach, I recommend exploring this query on Stack Overflow: CasperJS loop or iterate through multiple web pages?

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Using Radio Buttons in a jqGrid Interface

Currently, I am attempting to integrate radio buttons within a jqGrid framework. I am aware that a custom formatter can be utilized for this purpose. Below is my code snippet, however, it fails to provide information on which radio button is selected or if ...

Guide to center align fields in Ionic and Angular

I am working on creating a login screen that resembles the image provided. I have managed to set the background image, input fields, and buttons successfully. However, I am encountering a few issues: The width of my input field is taking up the entire spa ...

How can Vue.js deactivate or activate a dropdown based on a checkbox's state?

My task is to toggle the enabled/disabled state of a dropdown called combo-select when a checkbox is checked. The functionality works as expected when I click on Add Name, but not when I am in Editing Name mode and check the box for No Client. https://i.s ...

Can a gulpfile be written in ES6 syntax?

Question: Is there a way to write my gulp file in ES6 to utilize import instead of require, and use => syntax in place of function()? I have the option to use io.js or any version of node.js. gulpfile.js: import gulp from "./node_modules/gulp/index.j ...

The sign-up button mysteriously vanishes after the page is refreshed

I am encountering an issue with the sign up button during the user registration process. There is a checkbox for Terms & Conditions, and the button should only become enabled after checking this box. Everything seems to be functioning correctly, but when I ...

Locate the submenu item in relation to the main menu item within an external container

I'm working on modifying the behavior of a site's sub-menu. The current sub-menu is displayed as a drop-down, but I want it to show up in a separate horizontal div instead. Here's what I have accomplished so far: jQuery(document).ready(fun ...

Unable to use jQuery to delete class from an element

Here is the markup I am working with: <div class="row"> <img src="image.png" class="download-image regular-image" /> <div class="options"> <p>download</p> </div> <img src="image.png" class="download-image r ...

Utilizing async/await functionality within a React-Native component

I'm currently working on implementing a feature that authenticates users before rendering the main 'App' component. However, I'm facing an issue where my code is executing before the promise is resolved. class App extends Component { ...

Constructing variable names in AngularJS while using ng-repeat

I am working with a series of items that are structured as described below. Unfortunately, I do not have control over the variable names. pubDate0, pubDate1, pubDate2 To access them, I currently use the following format: <div> <i> ...

Deciding the optimal time to start a new JavaScript file and effectively organizing source code

Embarking on a substantial map application project involving approximately 5,000 lines of JavaScript, I am contemplating changing my approach to file organization. Traditionally, I would consolidate all code into a single file named 'main.js', bu ...

Is it possible to adjust the element's position using the code "element.style.top = window.scrollY"?

Recently, I attempted to create a floating input element that would follow the scroll. After much effort, I managed to achieve it and the code is provided below. However, I encountered an issue when trying to change "setText(window.scrollY)" to ...

Building an Angular module that allows for customizable properties: A step-by-step guide

I am in the process of developing an AngularJS module that will simplify interactions with my product's REST API. Since my product is installed on-premise, each user will need to provide their own URL to access the API. Therefore, it is essential that ...

The audio.play() HTML element fails to function in Chrome, preventing the audio from playing

I'm experiencing an issue with playing audio in Chrome when the audio.src is not called before the play call, but Firefox seems to handle it fine. Does anyone have any suggestions? You can check out the fiddle link below - http://jsfiddle.net/vn215r2 ...

Check if the element is located on the active tab using jQuery

I have implemented a tab system to organize a lengthy form with multiple input fields. The form consists of 4 tabs, and beneath them, there is a preview section displaying the user's selections. In this preview area, I added icons that enable users ...

Leverage ajax/js to dynamically refresh a single select tag within a set of multiple

While working on a project, I encountered a challenge while using JavaScript to update a <select> tag within a page. The specific issue arises because the page is designed in a management style with multiple forms generated dynamically through PHP ba ...

Setting a default value for an empty array in Joi

Struggling with the syntax to set a default value for an empty or missing array key. Joi.object({ items: Joi.array().empty([]).items(Joi.number()).default([42]), }) { items: [] } or { } In both scenarios, I want the default value of [42] to fill ...

Should objects passed as props be modified in Vue.js components?

I've observed that in Vue, object fields passed as Props can be changed, and these changes reflect in the parent component. Is this considered a bad practice? Should it be avoided, or is it acceptable to use? What are the potential pitfalls of this a ...

The onEdit() function in Google Apps Script encountered an error indicating that it has exceeded the maximum execution time limit

My script has been running smoothly in Apps Script for the past three weeks, but it suddenly stopped working a few days ago. I noticed that there are multiple executions happening at the same time and many of them are timing out with an error message sayin ...

A guide on inserting text annotations into text notes within a React Next.js 14 (with App Router) application

I am currently in the process of developing an application that allows users to provide commentary on legal rulings. Within my app, I have both judgments and notes. The notes are created based on excerpts from the judgments. Right now, when a user highligh ...

"if the condition is not met, the outcome will not be shown in the while

I have a looping structure that includes conditions for if, else if, and else. However, I have noticed that the else condition at the end of the loop is not being executed as expected. I am currently investigating why this might be happening. Here is the ...