Currently, I am developing a web scraper that stores results in a JSON format similar to the following:
{"Products" : [
{"Title":"Voice Recorder ISD1932","Results": [
{"Stock":1,"Price":11.4,"Date":"18-8-2014:3:36"},
{"Stock":1,"Price":12.4,"Date":"18-8-2014:3:38"},
{"Stock":1,"Price":12.4,"Date":"19-8-2014:0:40"},
{"Stock":1,"Price":12.4,"Date":"19-8-2014:21:46"},
{"Stock":1,"Price":12.4,"Date":"21-8-2014:22:4"},
{"Stock":1,"Price":12.4,"Date":"22-8-2014:0:40"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:0:48"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"},
{"Stock":1,"Price":12.4,"Date":"23-8-2014:13:56"}],
"id":"4a1e90d7-e578-4bd5-b888-38c7bbfb4af5"}]}
For example, the first element in the results would be:
{"Stock":1,"Price":11.4,"Date":"18-8-2014:3:36"}
The second would be:
{"Stock":1,"Price":12.4,"Date":"18-8-2014:3:38"}
And so on.
Each time I scrape the website, it adds a new element.
I am looking to implement a cleaner function that removes an element if both stock and price are equal to the previous one, excluding the date information.
Using this example, if the third element is identical to the second, I want to remove it. The same goes for subsequent elements.