When I crawl the page, I come across an element that looks like this.
<a class="a-declarative" href="javascript:void(0)"
data-action="a-expander-toggle"
data-a-expander-toggle='{"allowLinkDefault":true,
"expand_prompt":"See more", "collapse_prompt":"See less"}'>
<i class="a-icon a-icon-extender-expand"></i>
<span class="a-expander-prompt">See more</span>
</a>
In my scrapy spider, I am trying to click on the "See more" text using Selenium and then parse the response.
seemore = response.xpath('//a[contains(@data-action,"a-expander-toggle")]')
seemore.click()
However, I encounter an error when I try to click on seemore. When I log the xpath selector, this is the output I get.
# Log the "see more" link
self.log(response.xpath('//a[contains(@data-action,"a-expander-toggle")]'))
# Log output
[<Selector xpath='//a[contains(@data-action,"a-expander-toggle")]'
data=u'<a href="javascript:void(0)" data-action'>]
It seems like the link is not being read correctly. Can someone confirm if my xpath is correct for selecting this type of link?