Executing asynchronous JavaScript code within a Golang request: a comprehensive guide

I am in need of a solution to retrieve the content from dynamically generated pages using ajax on a website, specifically with code written in golang. I have found examples for non-ajax pages but struggling to find one that works for ajax pages. Any guidance or package recommendations would be greatly appreciated! Thank you.

package main

import(

    "fmt"
    "time"
    "net/http"
    "github.com/PuerkitoBio/goquery"
    "strings"
    "strconv"

)

func main() {

    var masterURI [1]string
    masterURI[0] = "http://uri1"
    /*masterURI[1] = "http://uri2"
    masterURI[2] = "http://uri3"*/

    for _, uri := range masterURI {

        doc, err := extractHTML(uri)

        check(err)

        search := doc.Find(".pagination li a span").Eq(-1)

        numPages, err := strconv.Atoi(search.Text())

        check(err)

        var i int

        for i = 1; i <= numPages; i++ {

            page := uri + "#/page-" + strconv.Itoa(i)

            fmt.Println("\n========> " + page)

            doc, err := extractHTML(page)

            check(err)

            search := doc.Find(".product-name")

            for i := range search.Nodes {
                product := strings.TrimSpace(search.Eq(i).Text())
                fmt.Println(product)
            }

        }

        fmt.Println(" --- ")

    }

}

func extractHTML(url string) (doc *goquery.Document, err error) {

    userAgent := "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36 OPR/32.0.1948.69";

    trans := &http.Transport{
       DisableKeepAlives: true,
    }

    timeout := time.Duration(600 * time.Second)

    myClient := &http.Client{Transport: trans, Timeout: timeout}

    req, err := http.NewRequest("GET", url, nil)

    check(err)

    req.Header.Add("User-Agent", userAgent)

    resp, err := myClient.Do(req)

    req.Close = true

    check(err)

    if resp.StatusCode == 200 {

        doc, err = goquery.NewDocumentFromResponse(resp) 

    }

    check(err)

    resp.Body.Close()

    return doc, err

}

func check(err error) {

    if err != nil {

        panic(err)

    }

}

Answer №1

Since your answer has received some negative feedback, let me guide you in the correct direction.

Executing JavaScript is not a common practice in Go programming. Typically, this task is performed in web browsers using Development Tools (F12 or similar). You can analyze the Network tab to identify the URL being accessed, and then incorporate that URL into your code for execution.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips for successfully passing an object with a list property in an ajax request

I have encountered a similar problem to This previous question, but I am struggling to implement the solutions provided. I am unsure of where to include the ModelBinder code mentioned in the responses, so I will explain my issue here. I am working with a s ...

Having trouble sending data from AJAX to PHP

My goal is to implement a "load more" feature in my web app that automatically calls a PHP file to load additional products as soon as the page is fully loaded. In order to achieve this, I am using AJAX to call the PHP file: $(document).ready(function() { ...

What methods can be used to transfer data from mapped objects to their parent component in React?

I'm in the midst of creating a cutting-edge shopping cart application. Each item is beautifully displayed as a card component within the app. To achieve this, I've utilized mapping with dummy object data: const Home = () => { const dumm ...

Incorporating User Input: Adding Paragraph Elements with HTML and Javascript to a Div Container

I am attempting to test my ability by creating a basic webpage. My goal is to take the text entered by the user in a textarea and add it to the page using a button. This button should generate a new paragraph element, place the user input inside of it, and ...

Utilize Chrome storage instead of localstorage to generate Parse sessions

I'm currently developing a Chrome Extension that relies on Parse User sessions. Because localstorage is limited to specific domains, I am looking to utilize chrome.storage so the data can be accessed across any site. The existing Parse Javascript SDK ...

Tips for managing state updates in React Redux

Currently, I am utilizing a reducer to manage the state in Redux. The current structure of my state is as follows: { activeConversation: "Jim" conversations: (7) [{…}, {…}, {…}, {…}, {…}, {…}, {…}] user: {id: 8, username: &quo ...

When registering the onHardwareBackButton event in Ionic, the back button continues to successfully navigate to the previous page

I recently started working with Ionic and encountered an issue with the onHardwareBackButton event. The event is functioning properly and directing me to the register function, but even after going to the register function, it still navigates back to the p ...

Any ideas on how to format a date for jQuery Datepicker?

Currently, I have implemented two different datepickers on my website, but I am interested in switching to jQuery's Datepicker for a more unified solution. Here is the current format that I am sending to our backend API: However, I would prefer to i ...

Replacing values in an HTML file with MySql query results

----- Problem solved, solution below ----- In my HTML file, I have a dropdown menu for various courses listed as follows: <ul> <li class="dropbtn" id="1"> <a href="">first</a> <ul class="dropdown-content"> ...

What causes an object to declare itself as undefined only in the event that I attempt to access one of its properties?

Check out this snippet of code: req.Course.find({}).populate('students', 'username firstName lastName registered').populate('teacher', 'image mID firstName lastName').sort({title: 1}).exec(function(err, courses){ ...

The typeof operator is not functioning as anticipated when used with an ajax response

Trying to implement form validation using AJAX and PHP, but encountering issues with the typeof operator. The validation works smoothly except when receiving an empty response, indicating that all fields have passed. In this scenario, the last input retai ...

Is there a way to conditionally redirect to a specific page using NextAuth?

My website has 2 points of user login: one is through my app and the other is via a link on a third-party site. If a user comes from the third-party site, they should be redirected back to it. The only method I can come up with to distinguish if a user is ...

What could be the reason for Laravel 5.8.35 presenting me with this unexpected JavaScript code?

When working with Laravel and attempting to use ajax, the goal was to call a PHP function using jQuery when a user types in a name (at least 3 letters for the first call). The PHP function itself seems to work correctly as confirmed by var dumping the resu ...

What is the best way to make a click handler respond to any click on an "a" tag, regardless of its surrounding elements?

If I have the following HTML code: <a href="/foo">bar</a> <a href="/"> <div></div> </a> And I am looking to create a jQuery handler that will respond when ANY "a" tag is clicked: $(document).click((e) => { ...

Guide to verifying identities and monitoring users through WCF/JSON?

In the process of developing a WCF service, I am in need of creating a customized user authentication system involving usernames and passwords. This particular service is set up as a WebScript(JSON) service. I am curious to find out if it's feasible ...

The jQuery fadeOut function modifies or erases the window hash

While troubleshooting my website, I discovered the following: /* SOME my-web.com/index/#hash HERE... */ me.slides.eq(me.curID).fadeOut(me.options.fade.interval, me.options.fade.easing, function(){ /* HERE HASH IS CLEARED: my-web.com/index/# * ...

Issue with MVC 4 Asynchronous File Upload: Controller Always Receiving Null Value

I'm encountering an issue while attempting to upload a file asynchronously from ajax to my controller. I am passing 3 variables - PictureId, PictureName, and PictureFile. The problem specifically lies with the "PictureFile" variable as it consistently ...

Leverage the AJAX Search Lite Plugin within the top navigation of your WordPress website's header file

I am interested in utilizing the "AJAX Search Lite 3.0.6" plugin within my WordPress Theme. Plugin Link After implementing the shortcode, I noticed that it was not appearing on the navigation bar as expected: <body> <!-- Header --> & ...

Having difficulty loading the controller into my AngularJS module

I've been attempting to organize my angularjs controllers into separate files without success. Folder Structure: --public-- --js/controller/resturant.js --js/master.js --index.php Content of master.js angular.module("rsw",[ ...

The issue of Angular 6 view failing to display accurate data upon page load

In my Angular 6 application, I've implemented a login feature using Firebase's Google login. The interface includes a button labeled Login when the user is logged out, and changes to Logout with the current email address displayed when the user i ...