I am trying to fetch the xpath of a Facebook post using HtmlUnit. To better understand my goal, you can check out these two related questions:
- Supernatural behaviour with a Facebook page
- HtmlUnit commenting out lines of Facebook page
To replicate my process, follow q-1. You can find the HTML code (of the Facebook page) on this pastebin link: http://pastebin.com/MfXsYSJQ.
Alternatively, you can visit . My objective is to retrieve the xpath of the span that contains the post with the text: "Hi! this is the first post of this page."
public class ForStackOverflow {
public static void main(String[] args) throws IOException {
WebClient client=new WebClient(BrowserVersion.FIREFOX_17);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setRedirectEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(true);
client.getOptions().setCssEnabled(true);
client.getOptions().setUseInsecureSSL(true);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
client.setAjaxController(new NicelyResynchronizingAjaxController());
HtmlPage page1=client.getPage("https://www.facebook.com/bhramakarserver");
System.out.println(page1.asXml());
//getting the xpath of span of class="userContent"
HtmlInput input=(HtmlInput)page1.getByXPath("/html/body//input[@type='submit']").get(0);
System.out.println(input.asXml());
//This line gives error as the xpath evaluates to null
HtmlSpan span=(HtmlSpan)page1.getByXPath("/html/body//span[@class='userContent']").get(0);
}
}
The issue seems to be that page1 contains static html. The particular span element:
<span data-ft="{"tn":"K"}" class="userContent">Hi! this is the first post of this page.</span>
is generated dynamically, causing it to appear as commented in the html of page1. However, upon inspection via inspect element, it displays normally. Is there a way to obtain the correct xpath after all dynamic content has been loaded on page1? Can this be achieved using Selenium Webdriver?