PhantomJS is failing to provide any output

Question

PhantomJS is failing to provide any output

Currently, I am utilizing PhantomJS for web scraping purposes. However, I have encountered an issue where the evaluate method is not returning any data. The script seems to run for a few seconds and then unexpectedly exits.

I have confirmed that PhantomJS is successfully connecting to the webpage. Additionally, PhantomJS is able to extract the page title without any problems. I have also verified multiple times that the class name I am searching for is correct.

var page = require('webpage').create();

page.open('http://www.maccosmetics.com/product/13854/36182/Products/Makeup/Lips/Lipstick/Giambattista-Valli-Lipstick', function(status) {
    page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
        waitFor(function()  {
            return page.evaluate(function() {
                $('.product__price').is(':visible');
            });
        }, function(){
            search = page.evaluate(function() {
                return  $('.product__price').text();
            });
            console.log(search)
        });
    });
    phantom.exit();
});

Despite my efforts, I am still unsure of what could be causing this unexpected behavior.

javascript phantomjs

Answer 1

Answer №1

The reason you're not seeing anything is because you are exiting the process too soon. Remember, all functions (except for evaluate()) that accept a callback function are asynchronous.

When you try to include jQuery in your page using page.includeJs(), make sure you don't exit PhantomJS immediately after. You should only exit once you have finished all necessary operations:

var page = require('webpage').create();

page.open('http://www.maccosmetics.com/product/13854/36182/Products/Makeup/Lips/Lipstick/Giambattista-Valli-Lipstick', function(status) {
    page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
        waitFor(function()  {
            return page.evaluate(function() {
                return $('.product__price').is(':visible');
            });
        }, function(){
            var search = page.evaluate(function() {
                return  $('.product__price').text();
            });
            console.log(search);
            phantom.exit();
        });
    });
});

Answer 2

The reason you're not seeing anything is because you are exiting the process too soon. Remember, all functions (except for evaluate()) that accept a callback function are asynchronous.

When you try to include jQuery in your page using page.includeJs(), make sure you don't exit PhantomJS immediately after. You should only exit once you have finished all necessary operations:

var page = require('webpage').create();

page.open('http://www.maccosmetics.com/product/13854/36182/Products/Makeup/Lips/Lipstick/Giambattista-Valli-Lipstick', function(status) {
    page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {
        waitFor(function()  {
            return page.evaluate(function() {
                return $('.product__price').is(':visible');
            });
        }, function(){
            var search = page.evaluate(function() {
                return  $('.product__price').text();
            });
            console.log(search);
            phantom.exit();
        });
    });
});

PhantomJS is failing to provide any output

Answer №1

Similar questions

Interacting between Jquery and servlets using AJAX

The callback function for ajax completion fails to execute

Create an AngularJS directive that formats the model value for visual display, while retaining the original value in the model

Leveraging JavaScript for pricing calculations within asp.net framework

Enhanced Search and Replace Techniques in HTML using jQuery and JavaScript

Step by step guide on serializing two forms and an entire table

Show or hide a component based on a mouse click in Vue JS

In HTML, adjust column widths for multiple tables according to the widest column present in any of them

Guide on how to "attach" the routes of an Angular 2 module to a specific location within the routing structure

A message appeared in the console warning about Vue using canvas-datagrid

Using Angular to pass an index to a pipe function

Tips for iterating through a collection of arrays with jQuery

Error: The script is not found in the package.json configuration

selecting arrays within arrays according to their date values

How do I retrieve the HSL value for a color selected using an input of type 'color' in JavaScript?

What is the mechanism behind the operation of asynchronous functions within the bcrypt() method in Node.js?

Finding a character that appears either once or thrice

Exploring the differences between selecting an element by its class and making multiple calls to select elements

Obtaining the value of dynamically generated elements using JavaScript within PHP scripting

Executing a script within an ASP.NET MVC Project