Parse multiple JSON files, manipulate their contents, and store the updated data

I'm currently working on implementing this functionality using Gulp.

  1. Locate and access all files with the extension .json within a designated directory, including any subdirectories.
  2. Perform modifications to the files in some manner, such as adding a new top-level element or similar adjustments.
  3. Save the modified files into a separate directory while preserving the original file structure.

The area where I am encountering difficulties is understanding how to properly utilize the read/write JSON piping syntax in conjunction with the src method.

Here is an outline of my current setup:

gulp.task("migratefiles", function () {
  return gulp.src("files/**/*.json")
      .pipe(/* SEEKING SOLUTION HERE */)
      .pipe(gulp.dest("processed"));
});

Answer №1

Here are several approaches you can take to achieve this task:

(1) Utilize the gulp-json-transform plugin:

var jsonTransform = require('gulp-json-transform');

gulp.task("migratefiles", function () {
  return gulp.src("files/**/*.json")
    .pipe(jsonTransform(function(json, file) {
      var transformedJson = {
        "newRootLevel": json
      };
      return transformedJson;
    }))
    .pipe(gulp.dest("processed"));
 });

Pros:

  • Simplicity in usage
  • Supports asynchronous processing (if a Promise is returned)
  • Provides access to the path of each file

Cons:

  • Basic output formatting capabilities only

(2) Employ the gulp-json-editor plugin:

var jeditor = require('gulp-json-editor');

gulp.task("migratefiles", function () {
   return gulp.src("files/**/*.json")
     .pipe(jeditor(function(json) {
       var transformedJson = {
         "newRootLevel": json
       };
       return transformedJson;
     }))
     .pipe(gulp.dest("processed"));
});

Pros:

  • User-friendly interface
  • Detects and matches the indentation used in input files automatically (e.g., two spaces, four spaces, tabs, etc.) for consistent output file formatting
  • Offers support for various js-beautify options

Cons:

  • Seems not to support asynchronous processing
  • Lacks a direct way to access the path of each file

(3) Manual approach (directly accessing the vinyl file object using map-stream):

var map = require('map-stream');

gulp.task("migratefiles", function () {
   return gulp.src("files/**/*.json")
     .pipe(map(function(file, done) {
       var json = JSON.parse(file.contents.toString());
       var transformedJson = {
         "newRootLevel": json
       };
       file.contents = new Buffer(JSON.stringify(transformedJson));
       done(null, file);
     }))
     .pipe(gulp.dest("processed"));
});

Pros:

  • Complete control and access over all components
  • Supports asynchronous processing with a done callback

Cons:

  • Might be more challenging to work with

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is the best way to remove an item from an array?

I'm currently working on implementing a follow/unfollow feature on my page using React/Redux. However, I am struggling to fully grasp how to achieve this. When the user follows 'something', the reducer does the following: case FOLLOW_CITY: ...

Retrieving information from MongoDB for a specific ObjectID associated with the current authenticated user

Having two collections in my MongoDB structured as follows: Data User: id: ObjectId ("5fb39e3d11eaad3e30cfb1b0") userName: "Tobias" password: "yyy" id: ObjectId ("5fb3c83fb774ff3340482250") userName: "Thor&qu ...

Guide on merging two JArrays using JSON.NET

Is there a way to properly concatenate two JArrays that were obtained through JArray.Parse? It is critical that the order of the arrays is maintained, with the first array appearing first followed by the elements from the second array. ...

Attempting to merge the data from two separate API responses into a single array of objects

I have a current project in which I'm dealing with an object that has three separate arrays of objects. Here's a glimpse of the structure... [ Array 1:[ { key: value} ], Array 2:[ { key: value}, { key: value} ], Array ...

IntelliJ coverage for backend JavaScript

Is it possible to analyze code coverage in IntelliJ without using a browser? http://www.jetbrains.com/webstorm/webhelp/monitoring-code-coverage-for-javascript.html Though there are tutorials by JetBrains on code coverage, they all seem to require a browse ...

Issues arise with Highcharts Sankey chart failing to display all data when the font size for the series is increased

I am currently working with a simple sankey chart in Highcharts. Everything is functioning correctly with the sample data I have implemented, except for one issue - when I increase the font size of the data labels, not all the data is displayed. The info ...

Formik's handleSubmit function seems to be being overlooked and not executed as

I've encountered an issue while trying to validate a form before submission using formik and yup validation. The form is divided into two parts, where the first part needs to be validated before moving on to the second part. I set a state handleShow(t ...

Creating the desired object from Json sub attributes: A step-by-step guide

I have a JSON file example provided below. { "TestOneConfig": { "SvcUrl": "www.abc.com/", "Port": "3455" }, "LiveTestConfig": { "ConnString": "abcd" } } In addition, I have created a class model for the "TestOneConfig" excerpt as s ...

Encountering issues with resolving dependencies in webdriverIO

I'm attempting to execute my WebdriverIo Specs using (npm run test-local) and encountering an error even though I have all the necessary dependencies listed in my package.json as shown below: [0-2] Error: Failed to create a session. Error forwardin ...

Is the unavailability of nodejs's require function in this closure when using the debugger console due to a potential v8 optimization?

I am facing an issue with using the require function in node-inspector. I copied some code from node-inspector and tried to use require in the debugger console to access a module for debugging purposes, but it is showing as not defined. Can someone help me ...

Is Jquery Steps causing interference with the datepicker functionality?

I am currently using the jquery steps plugin for creating a wizard on my website. However, I am experiencing some issues with the functionality of the datepicker and qtip inside the steps. Even after switching the .js references, the problem still persists ...

The dropdown list event does not seem to be triggering when JavaScript is implemented

Having trouble triggering a dropdownlist event. The dropdown in question: asp:dropdownlist id="ddlhello" Runat="server" AutoPostBack="True" onchange="javascript:return ChangeHeader();" An associated selectedindex change event has been added in the code ...

What steps do I need to take to ensure that this Regex pattern only recognizes percentages?

I am attempting to create a specific scenario where I can restrict my string to three digits, followed by a dot and two optional digits after the dot. For example: 100.00 1 10.56 31.5 I've developed a regex pattern that allows me to filter out any ...

Include the clicked link into the text input area using Ajax or Jquery

Hey there, I'm just starting out with jquery and ajax so please be patient with me. Below is a snippet of my script that fetches branch names from the database asynchronously: $(document).ready(function () { $("#pickup").on('keyup' ...

Implementing styles from a constant file in React Native: A simple guide

In my file register.js, I have a UI component. import CustomHeader from '../components/Header'; ... static navigationOptions = ({navigation, navigation: { state } }) => { return { title: '', headerSty ...

Is there a way to display the form values on the table once it has been submitted?

I'm struggling with my form input not showing up under the table headings after submission. I've reviewed the code multiple times, but can't figure out what's causing the issue. If you have any suggestions on how to write the code more ...

Disappearing Cloned Form Fields in jQuery

Hey there! I'm trying to duplicate a section of a form using the code below. But for some reason, the copied fields are only visible for a split-second before they disappear. Can anyone spot any errors that might be causing this strange behavior? jQu ...

What advantages does NextJS offer that set it apart from other frameworks that also provide Server Side Render solutions?

I'm diving into the world of NextJS and as I explore this topic, one burning question arises: "What is the necessity of utilizing NextJS?" From what I understand, NextJS excels in rendering pages from the server and is heavily reliant on ReactJS. Howe ...

An effective method for encoding a tuple within JSON

My aim is to represent the image shape as (4096,2048) in a JSON file, but I keep encountering the following error: I'm unsure whether the problem lies with the code or the way I am inputting the values in the JSON file. ...

Trouble arises when trying to open a new window using the Angular 8 CDK

I am attempting to open my component in a new window, similar to this example: https://stackblitz.com/edit/angular-open-window However, when the window opens, my component is not displayed and I receive the following error in the console: Error: Must pro ...