I've created a Google apps script web application that is able to download the data associated with any Youtube channel's videos and save the result in a .CSV file on the user's Google Drive. The code follows these main steps:
- First, a GAS function (referred to as "X") is invoked when the user clicks a button in their browser (using google.script.run). This function queries the Youtube API in a loop that stops either when the nextPageToken becomes null or after approximately 330 seconds of execution time (due to the maximum allowed runtime for functions being 6 minutes in GAS). For each video, a CSV string is generated and added to an array ("mainArr").
- Once X completes successfully, it returns mainArr and triggers another function called "Y." The purpose of Y is to begin uploading "Utilities.newBlob(mainArr.join('\r\n'))" to the user's Drive in chunks of 256kb each (based on the documentation).
The issue arises when X is executed for shorter durations multiple times, causing Y to be called repeatedly to obtain all the videos. In such cases, the resulting CSV is missing 2 bytes per iteration, leading to discrepancies in the total file size. This discrepancy is problematic as it indicates missing videos within the .CSV file.
To troubleshoot, I have restricted the downloaded fields to only include the videos' IDs to rule out data update issues, such as increasing view counts affecting the file size. Additionally, I have confirmed that the function responsible for creating chunks from mainArr remains intact without corrupting the data.
In terms of the process for calling X and Y multiple times: upon X's completion, Y is triggered to upload mainArr chunk by chunk, excluding the last chunk named "lastChunk." After completing its task, Y clears mainArr (mainArr = []) and returns both mainArr and the base64-encoded lastChunk. Subsequently, X is invoked again, populating mainArr with new data and repeating the cycle of passing mainArr along with the encoded lastChunk for Y to handle.
Here's a snippet from Y outlining part of the described procedure:
var csv = Utilities.newBlob(mainArr.join('\r\n'));
if (lastChunk) {
csv = Utilities.newBlob((Utilities.base64DecodeWebSafe(lastChunk)).concat(csv.getBytes()));
}
var chunkSize = (256*1024); //following drive documentation,
var fileBytes = csv.getBytes();
var fileSize = fileBytes.length;
var chunksCount = Math.ceil(fileSize / chunkSize);
for (var i = 0; i < chunksCount; i++) {
var chunk = csv.setBytes(fileBytes.splice(0, chunkSize));
}