Trying to access the page investing.com/rates-bonds/brazil-10 using RCurl and attempting to modify the default time series range through the calendar control on the main table.
Upon inspecting with Firefox Firebug, I captured the post log generated after clicking the -> Apply button on the calendar. A similar method can be executed using Log XMLHTTPRequests in Chrome.
Following the button click, the Firebug Console displays the line:
POST http://www.investing.com/instruments/HistoricalDataAjax
Upon expanding the line, the Headers tab reveals the request header:
Request Headers
Accept text/plain, */*; q=0.01
Accept-Encoding gzip, deflate
Accept-Language en-gb,en;q=0.5
Content-Length 102
Content-Type application/x-www-form-urlencoded; charset=UTF-8
Cookie ... very long string
Host www.investing.com
Referer http://www.investing.com/rates-bonds/brazil-10-year-bond-yield-historical-data
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0
X-Requested-With XMLHttpRequest
The Post tab provides:
Parameters application/x-www-form-urlencoded
action historical_data
curr_id 24029
end_date 05/30/2014
interval_sec Daily
st_date 05/25/2014
Based on these findings, an attempt was made to retrieve the page with:
require(RCurl)
link="http://www.investing.com/instruments/HistoricalDataAjax"
html= postForm(
link,
action="historical_data",
curr_id="24029",
end_date="05/30/2014",
interval_sec="Daily",
st_date="05/25/2014",
.opts=list(
referer="http://www.investing.com/rates-bonds/brazil-10-year-bond-yield-historical-data",
useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0"
))
However, the result obtained was:
html
## [1] ""
## attr(,"Content-Type")
## charset
## "text/html" "utf-8"
An alternative approach was also attempted...
cookie="... long cookie string"
link="http://www.investing.com/instruments/HistoricalDataAjax"
h <- basicTextGatherer()
h$reset()
curlPerform(url = link,
httpheader=c(
'Accept'="text/plain, */*; q=0.01",
'Accept-Encoding'="gzip, deflate",
'Accept-Language'="en-gb,en;q=0.5",
'Content-Length'="102",
'Content-Type'="application/x-www-form-urlencoded; charset=UTF-8",
'Cookie'=cookie,
'Host'="www.investing.com",
'Referer'="http://www.investing.com/rates-bonds/brazil-10-year-bond-yield-historical-data",
'User-Agent'="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0",
'X-Requested-With'="XMLHttpRequest"
),
postfields=c(
action="historical_data",
curr_id="24029",
end_date="05/30/2014",
interval_sec="Daily",
st_date="05/25/2014"
),
writefunction = h$update,
verbose = TRUE)
h$value()
Seeking assistance for this issue.