Running a Selenium driver to execute Javascript's elementFromPoint feature

Question

Running a Selenium driver to execute Javascript's elementFromPoint feature

I am currently working on integrating an "object picker" into my Selenium-based framework, a feature commonly found in many commercial automation tools. I have been using a Javascript command to locate the element at the mouse's current position, but for some reason, I am not getting the expected element.

When using ChromeDriver or InternetExplorerDriver, the script always returns the header object, regardless of the web page being viewed or the mouse position. Even though it seems like the script is fetching coordinates (0, 0) instead of the actual mouse position, I have double-checked and confirmed that Cursor.Position is providing the correct values.

On the other hand, when utilizing FirefoxDriver, I encounter an exception:

"Argument 1 of Document.elementFromPoint is not a finite floating-point value. (UnexpectedJavaScriptError)"

Could someone help me identify what mistake I might be making?

    private void OnHovering()
    {
        if (Control.ModifierKeys == System.Windows.Forms.Keys.Control)
        {
            IWebElement ele = null;
            try
            {
                // Locate the element at the mouse position
                if (driver is IJavaScriptExecutor)
                    ele = (IWebElement)((IJavaScriptExecutor)driver).ExecuteScript(
                        "return document.elementFromPoint(arguments[0], arguments[1])", 
                        new int[] { Cursor.Position.X, Cursor.Position.Y });

                // Select the identified element
                if (ele != null)
                    SelectElement(ele);
            }
            catch (Exception) { }
        }
    }

Thank you in advance!

javascript c#selenium selenium-webdriver

Answer 1

Answer №1

The key here lies in how you pass the coordinates into the script. Remember, when using ExecuteScript(), each script argument must be specified individually. The issue in your situation was that you only specified one argument x, causing it to assume that y should default to a value of 0, which typically corresponds to a header.

Instead of:

ele = (IWebElement)((IJavaScriptExecutor)driver).ExecuteScript(
                        "return document.elementFromPoint(arguments[0], arguments[1])", 
                        new int[] { Cursor.Position.X, Cursor.Position.Y });

You should use:

ele = (IWebElement)((IJavaScriptExecutor)driver).ExecuteScript(
                        "return document.elementFromPoint(arguments[0], arguments[1])", 
                        Cursor.Position.X, Cursor.Position.Y);

Answer 2

The key here lies in how you pass the coordinates into the script. Remember, when using ExecuteScript(), each script argument must be specified individually. The issue in your situation was that you only specified one argument x, causing it to assume that y should default to a value of 0, which typically corresponds to a header.

Instead of:

ele = (IWebElement)((IJavaScriptExecutor)driver).ExecuteScript(
                        "return document.elementFromPoint(arguments[0], arguments[1])", 
                        new int[] { Cursor.Position.X, Cursor.Position.Y });

You should use:

ele = (IWebElement)((IJavaScriptExecutor)driver).ExecuteScript(
                        "return document.elementFromPoint(arguments[0], arguments[1])", 
                        Cursor.Position.X, Cursor.Position.Y);

Answer 3

Answer №2

The provided code by alecxe is effective in most scenarios, however, it may encounter issues if the webpage contains frames or iframes.

To handle frames and iframes gracefully, a more extensive code solution is necessary.

/// <summary>
/// Retrieve the element at the specified viewport coordinates X, Y
/// </summary>
static public RemoteWebElement GetElementFromPoint(RemoteWebDriver driver, int X, int Y)
{
    while (true)
    {
        String script = "return document.elementFromPoint(arguments[0], arguments[1]);";

        RemoteWebElement element = (RemoteWebElement)driver.ExecuteScript(script, X, Y);
        if (element == null)
            return null;

        if (element.TagName != "frame" && element.TagName != "iframe")
            return element;

        Point position = GetElementPosition(element);
        X -= position.X;
        Y -= position.Y;

        driver.SwitchTo().Frame(element);
    }
}

/// <summary>
/// Get the top/left corner position of the Element within the document.
/// NOTE: RemoteWebElement.Location is based on the document's top and does not consider scroll position.
/// </summary>
static public Point GetElementPosition(RemoteWebElement element)
{
    String script = "var X, Y; "
                    + "if (window.pageYOffset) " 
                    + "{ "
                    + "  X = window.pageXOffset; "
                    + "  Y = window.pageYOffset; "
                    + "} "
                    + "else " 
                    + "{ "
                    + "  var elem = document.documentElement; "         
                    + "  if (!elem.clientHeight) elem = document.body; "
                    + "  X = elem.scrollLeft; "
                    + "  Y = elem.scrollTop; "
                    + "} "
                    + "return new Array(X, Y);";

    RemoteWebDriver driver = (RemoteWebDriver)element.WrappedDriver;
    IList<Object>   coordinates  = (IList<Object>)  driver.ExecuteScript(script);

    int scrollX = Convert.ToInt32(coordinates[0]);
    int scrollY = Convert.ToInt32(coordinates[1]);

    return new Point(element.Location.X - scrollX,
                     element.Location.Y - scrollY);
}

This functionality should be incorporated into WebDriver for enhanced performance.

Answer 4

The provided code by alecxe is effective in most scenarios, however, it may encounter issues if the webpage contains frames or iframes.

To handle frames and iframes gracefully, a more extensive code solution is necessary.

/// <summary>
/// Retrieve the element at the specified viewport coordinates X, Y
/// </summary>
static public RemoteWebElement GetElementFromPoint(RemoteWebDriver driver, int X, int Y)
{
    while (true)
    {
        String script = "return document.elementFromPoint(arguments[0], arguments[1]);";

        RemoteWebElement element = (RemoteWebElement)driver.ExecuteScript(script, X, Y);
        if (element == null)
            return null;

        if (element.TagName != "frame" && element.TagName != "iframe")
            return element;

        Point position = GetElementPosition(element);
        X -= position.X;
        Y -= position.Y;

        driver.SwitchTo().Frame(element);
    }
}

/// <summary>
/// Get the top/left corner position of the Element within the document.
/// NOTE: RemoteWebElement.Location is based on the document's top and does not consider scroll position.
/// </summary>
static public Point GetElementPosition(RemoteWebElement element)
{
    String script = "var X, Y; "
                    + "if (window.pageYOffset) " 
                    + "{ "
                    + "  X = window.pageXOffset; "
                    + "  Y = window.pageYOffset; "
                    + "} "
                    + "else " 
                    + "{ "
                    + "  var elem = document.documentElement; "         
                    + "  if (!elem.clientHeight) elem = document.body; "
                    + "  X = elem.scrollLeft; "
                    + "  Y = elem.scrollTop; "
                    + "} "
                    + "return new Array(X, Y);";

    RemoteWebDriver driver = (RemoteWebDriver)element.WrappedDriver;
    IList<Object>   coordinates  = (IList<Object>)  driver.ExecuteScript(script);

    int scrollX = Convert.ToInt32(coordinates[0]);
    int scrollY = Convert.ToInt32(coordinates[1]);

    return new Point(element.Location.X - scrollX,
                     element.Location.Y - scrollY);
}

This functionality should be incorporated into WebDriver for enhanced performance.

Answer 5

Answer №3

I encountered a similar problem recently. Upon investigating, I discovered that the error was occurring due to the element I needed to locate being hidden (having the attribute display:none). Therefore, it did not have a specific location.

Answer 6

I encountered a similar problem recently. Upon investigating, I discovered that the error was occurring due to the element I needed to locate being hidden (having the attribute display:none). Therefore, it did not have a specific location.

Running a Selenium driver to execute Javascript's elementFromPoint feature

Answer №1

Answer №2

Answer №3

Similar questions

The feature of automatically selecting all text input content upon focus is not functioning properly in Microsoft Edge

What are the steps for submitting a form with AJAX?

What is the best way to incorporate auto refresh in a client-side application using vue.js?

Obtain the href attribute using Selenium in Python

How can I trigger a method upon leaving a page in C# (.Net 3.5)?

Automating the WebDriver script in JMeter with advanced tools

What is the best way to monitor and record the height of a div element in a React application?

Unit testing setTimeout in a process.on callback using Jest in NodeJS

Selecting specific elements from an array in JavaScript can be achieved by using various methods and techniques

Abbreviating Column Labels in Google Visualization

Using Vue to handle Promise resolution - incorporating Laravel Gate logic into Vue

What are some effective strategies for resolving the persistent issue of Visual Studio Code warnings, MDN Reference difficulties, and other challenges?

Trouble with selecting inputs within a Div Element

Use two separate AJAX requests to open and close the modal dialog

Discovering distinct colors for this loading dots script

Guide to using Razor syntax in JQuery to effortlessly redirect to a different page

Managing localization in MongoDB

Using Selenium Webdriver to choose a value from a Dropdown menu may require more time than expected

Restricting the time frame in HTML 5

Using *ngIf can lead to a never-ending cycle that never gets resolved