I utilized s3 select to extract specific data for display on my frontend. I converted an array of bytes to a buffer and then to a string as shown below:
let dataString = Buffer.concat(records).toString('utf8');The resulting string looked like this:
{"id":"1","obj1":"191.25","obj2":"11.81","obj3":"3.44","obj4":"15.62"}
{"id":"2","obj1":"642.00","obj2":"4.33","obj3":"0.00","obj4":"11.33"}
{"id":"3","obj1":"153.76","obj2":"94.77","obj3":"16.80","obj4":"29.79"}
{"id":"4","obj1":"61.71","obj2":"0.43","obj3":"0.00","obj4":"8.14"}
{"id":"5","obj1":"194.33","obj2":"108.89","obj3":"14.13","obj4":"168.60"}
{"id":"6","obj1":"204.31","obj2":"137.41","obj3":"34.76","obj4":"193.16"}
{"id":"7","obj1":"199.53","obj2":"34.53","obj3":"16.29","obj4":"26.56"}
{"id":"8","obj1":"77.33","obj2":"5.00","obj3":"12.50","obj4":"0.00"}
{"id":"9","obj1":"128.54","obj2":"101.60","obj3":"15.76","obj4":"46.23"}
{"id":"10","obj1":"107.00","obj2":"116.67","obj3":"34.42","obj4":"8.75"}
{"id":"12","obj1":"206.05","obj2":"155.03","obj3":"36.96","obj4":"148.99"}
{"id":"13","obj1":"133.93","obj2":"142.79","obj3":"39.91","obj4":"98.30"}
Now, I aim to convert them into a JSON array. The solution provided is as follows:
let dataArray = dataString.split('\n');
//remove white spaces and commas etc
dataArray = dataArray.filter(d=> d.length > 2);
//change string to json
dataArray = dataArray.map(d=> JSON.parse(d));
However, the issue arises when splitting them by new lines; it won't work if the JSON is compressed or contains new lines itself. How should I best handle this situation? I desire the output as presented below:
[{"id":"1","obj1":"191.25","obj2":"11.81","obj3":"3.44","obj4":"15.62"},
{"id":"2","obj1":"642.00","obj2":"4.33","obj3":"0.00","obj4":"11.33"},
{"id":"3","obj1":"153.76","obj2":"94.77","obj3":"16.80","obj4":"29.79"},
{"id":"4","obj1":"61.71","obj2":"0.43","obj3":"0.00","obj4":"8.14"},
{"id":"5","obj1":"194.33","obj2":"108.89","obj3":"14.13","obj4":"168.60"},
{"id":"6","obj1":"204.31","obj2":"137.41","obj3":"34.76","obj4":"193.16"},
{"id":"7","obj1":"199.53","obj2":"34.53","obj3":"16.29","obj4":"26.56"},
{"id":"8","obj1":"77.33","obj2":"5.00","obj3":"12.50","obj4":"0.00"},
{"id":"9","obj1":"128.54","obj2":"101.60","obj3":"15.76","obj4":"46.23"},
{"id":"10","obj1":"107.00","obj2":"116.67","obj3":"34.42","obj4":"8.75"},
{"id":"12","obj1":"206.05","obj2":"155.03","obj3":"36.96","obj4":"148.99"},
{"id":"13","obj1":"133.93","obj2":"142.79","obj3":"39.91","obj4":"98.30"}
]