Retrieve URLs of HTML elements dynamically with JavaScript, Java, and Selenium

Trying to extract destination URLs from a page filled with links, but the source code is complex and does not include direct web addresses. Here's an example snippet of what it looks like:

<li>
    <a href="javascript:void(0)">
        <span title="Text">Text</span>
    </a>
</li>

The code seems obfuscated, possibly loading URLs dynamically from a micro service. Is there a way to retrieve these URLs programmatically without actually following the link? I'm attempting this using JavaScript / Selenium in Java.

I'm considering triggering the onclick event to capture the URL before any redirection occurs. Any insights on how to achieve this would be appreciated!

javascriptjavaselenium

Answer №1

My solution to the issue involved the following steps:

  1. I implemented a JavaScript code snippet that redefined the open() method of the browser's window object to retrieve the target URL, store it globally, and prevent redirection. For more information, refer to this post on Stack Overflow.
  2. I initiated a click on an HTML link element to trigger the underlying XMLHttpRequest call.
  3. By utilizing JavaScript's clearInterval() and Java's executeAsyncScript() methods, I monitored a global variable until its value was assigned (the value is set by the window.open() method after the click event).

Since I was working with ChromeDriver, the sample code provided below is intended for execution in the Google Chrome browser:

[Java code snippet provided]

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

How can AngularJS utilize variables from an external JavaScript <script> file within an HTML document?

As someone unfamiliar with AngularJS, I have a simple inquiry regarding the subject. The code on my page begins by defining an app and controller: <script> var isisApp = angular.module('isisApp', []); isisApp.controller('Acco ...

javascripthtmlangularjs

Use the inline IF statement to adjust the icon class depending on the Flask variable

Is it feasible to achieve this using the inline if function? Alternatively, I could use JavaScript. While I've come across some similar posts here, the solutions provided were not exactly what I expected or were implemented in PHP. <i class="f ...

javascriptpythonhtmlflaskjinja2

The second div element remains unselected in jQuery

Below is the example of an HTML structure: <span id="17">yes here</span> <div class="PricevariantModification vm-nodisplay"></div> <div class="PricesalesPrice vm-display vm-price-value"> <span class="a"></span> ...

javascriptjqueryhtmlcss

Exploring the application of Checklist-model in Angular

My goal is to have departments as titles and employees as subtitles. When the user checks a department title, all associated employees should also be checked. I am implementing this using fieldset and table in HTML with ng-repeat. It seems similar to this ...

javascriptangularjshtmlchecklist-model

Send two arguments to a personalized validation function using express-validation

I have a requirement to input two values in order to search for a subdocument using the main document ID and then another ID of the subdocument. These IDs are received as parameters, and I am utilizing express-validator with a custom function to ensure th ...

javascriptnode.jsexpressexpress-validator

The value of YOUR_VARIABLE in the Node.js process environment appears to be undefined

Having trouble hiding a discord token for my bot, even though I believe dotenv is set up correctly. However, all I keep getting back is undefined. I've gone as far as using nodemon to refresh the server when making changes, but the issue persists. Co ...

javascriptnode.jsdotenv

Running on Node.js, the Promise is activated, yet there remains an issue with the function

I've encountered a strange issue that I can't seem to diagnose. It's showing a TypeError: My code is returning 'function is undefined', which causes the API call to fail. But oddly enough, when I check the logs and breakpoints, it ...

javascriptnode.jsexpresssequelize.js

What advantages does extending an interface multiple times bring, especially when it has already been extended by its parent?

Consider the following code snippet: interface IAAA { void aaa(); } interface IBBB extends IAAA { void bbb(); } class MyClass implements IAAA, IBBB { public void bbb() { } public void aaa() { } } Question: Is there any advantage to impl ...

javaandroidinterface

Mismatched versions of chromedriver and chrome, unless they are both 96

While working with Selenium python and chromedriver, I encountered the common issue of incompatible chromedriver and chrome versions: session not created from disconnected: unable to connect to renderer (Session info: chrome=96.0.4664.110) Even after do ...

pythonpython-3.xseleniumselenium-webdriverselenium-chromedriver

Saving Android Emoji in a MySQL Database

Users select emojis which are then stored in text body format inside a MySQL database. The column structure is as follows: `body` text CHARACTER SET utf8 COLLATE utf8_bin, Original data : <div id='mobile-question-style' style=\"font-fam ...

javamysqlemoji

Challenges faced when using Selenium with Telerik RadMenu

For the past two days, I've been struggling to find an effective method for writing integration tests for my ASP.NET website using Selenium. The website features a Telerik RadMenu with a hierarchical structure like this: Root Menu - Menu 1 - Sub M ...

cssasp.netseleniumtelerik

JavaScript loop used to create markers on Google Maps

I am currently in the process of developing a basic Google map which includes markers and involves looping through the data source for the markers. When I replace key.lng or key.lat with hardcoded values in the code below, everything functions correctly. ...

javascriptloopsgoogle-maps

Selenium: What causes the CSS selector to fail in IE10 while working fine in Firefox and Chrome?

from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC wait = WebDriverWait(webdriver, 10) elem = wait.until(EC.visibility_of_element_located( (By.CSS_SELECTOR, '.top_laye ...

cssseleniuminternet-explorer-10

How can I select a specific span element using its class name, retrieve the ID associated with it, and then use

I am facing an issue with retrieving the id of a span within a dropdown menu. The classname is fixed, but the id is dynamic. I have attempted using document.querySelector(".tag").id, which works when a span is selected from the dropdown. However, if nothin ...

javascriptjqueryajax

Is it possible to load images top to bottom?

Is there a way to make browsers load images from the bottom to the top? Imagine I have an extremely long image that takes 60 seconds to load. The content is designed to be read from bottom to top. Is there any trick I can use to ensure that the image load ...

javascripthtmlcssbrowserimage-loading

Looking for nested objects within an array in JavaScript

Currently, I am dealing with a REST API that provides data in the following format: students = [{ batch_id: 22 id: 1 image: null name: "a new batch student", attendance: [ { id: 1, student_id: 1, batch_id: 22, absent_on: "2019-09-15", ti ...

javascriptangular

Clear all events from an HTML element and its descendants with TypeScript

Each time the page loads, I have HTML from an API that is constantly changing. Is there a way to strip away all events attached to it? The original HTML looks like this: <div id="content"> <h2 onclick="alert('hi');">Test 1< ...

javascripthtmltypescript

The ng-click functionality in AngularJS is malfunctioning while the ng-change feature is functioning properly when invoking the identical function

UPDATE I have identified the problem - it was due to including 'ngTouch', please refer to my solution provided below. It seems like I must be overlooking a simple error in this scenario, as I cannot seem to locate it despite my efforts. The foll ...

javascriptangularjsangularjs-ng-click

Is there a glitch in the three.js loadOBJMTL loader?

Encountering an issue with the OBJMTL loader in three.js. I'm working with obj/mtl/jpeg files and getting load errors that look like this: "THREE.OBJMTLLoader: Unhandled line 4033/5601/6659" OBJMTLLoader.js:347 Seems like there is a problem with a c ...

javascript3dthree.js

In what way can the values from #result-set-1 be accessed in SimpleJdbcCall?

While using SimpleJdbcCall, I am encountering two parameters: #result-set-1 and #update-count-1. MapSqlParameterSource parameterSource = new MapSqlParameterSource(); parameterSource.addValue("name", "something"); Map<String, Object> resultFromProced ...

javaspring