I am attempting to use JavaScript to scrape the contents of a multi-page website and then export it to an excel or CSV file.
The issue I am facing is that I can only scrape the first page and I am struggling to export it to excel or CSV format.
Below is the code I have written thus far:
const PORT =8000
const axios = require('axios')
const cheerio = require('cheerio')
const express = require('express')
const app = express()
const url = 'https://www.taneps.go.tz/epps/viewAllAwardedContracts.do?d-3998960-p=1&selectedItem=viewAllAwardedContracts.do&T01_ps=100'
axios(url)
.then(response => {
const html = response.data
const $ = cheerio.load(html)
const articles = []
$('#T01',html).each(function(){
const contract = $(this).text()
articles.push({
contract
})
})
console.log(articles)
}).catch(err => console.log(err))
app.listen(PORT,() => console.log(`Server listening on port ${PORT}`))
I am seeking a way to scrape all pages and save the data into a CSV or Excel file.