Generating a JSON document from a collection of text files

Question

Generating a JSON document from a collection of text files

I have a question about JSON files and bash scripting:

My goal is to create a JSON file that contains two keys, "key1" and "key2", each with multiple values stored in separate .txt files. The contents of the .txt files for each key are structured like this:

key1.txt:
val1
val2
val3
..

The structure of key2.txt is similar. Ultimately, I want the JSON file to look like this:

{
    key1:
    [ "val1"
      "val2"
       ...
    ]

    key2:
    [ "val1"
      "val2"
       ...
    ]
}

Does anyone know how to achieve this using bash scripting? Your help would be greatly appreciated!

javascript json bash jq

Answer 1

Answer №1

Give this a try

#!/bin/bash
json="{"
for file in <directory path>/*.txt
do
filename=$(basename "$file")
filename="${filename%.*}"  
json=$json'"'$filename'"'": ["
while IFS= read -r line; do
echo $filename
json=$json'"'$line'",'
done < $file  
json="${json%?}"
json=$json"],"
done
json="${json%?}"
json=$json"}"
echo $json

Answer 2

Give this a try

#!/bin/bash
json="{"
for file in <directory path>/*.txt
do
filename=$(basename "$file")
filename="${filename%.*}"  
json=$json'"'$filename'"'": ["
while IFS= read -r line; do
echo $filename
json=$json'"'$line'",'
done < $file  
json="${json%?}"
json=$json"],"
done
json="${json%?}"
json=$json"}"
echo $json

Answer 3

Answer №2

To read in multiple files using jq, you can utilize the --slurpfile option. However, it is crucial that these files are formatted as actual JSON files. If your files do not meet this criteria, they will need to be converted first.

$ jq -R '.' key1.txt > /tmp/key1.json && \
  jq -R '.' key2.txt > /tmp/key2.json && \
  jq --slurpfile key1 /tmp/key1.json \
     --slurpfile key2 /tmp/key2.json \
     -n '{ key1: $key1, key2: $key2 }'

Alternatively, if you prefer to avoid working with temporary files, you have the option to create the JSON arrays within a subshell and pass them in using --argjson.

$ jq --argjson key1 "$(jq -R '.' key1.txt | jq -s '.')" \
     --argjson key2 "$(jq -R '.' key2.txt | jq -s '.')" \
     -n '{ key1: $key1, key2: $key2 }'

Answer 4

To read in multiple files using jq, you can utilize the --slurpfile option. However, it is crucial that these files are formatted as actual JSON files. If your files do not meet this criteria, they will need to be converted first.

$ jq -R '.' key1.txt > /tmp/key1.json && \
  jq -R '.' key2.txt > /tmp/key2.json && \
  jq --slurpfile key1 /tmp/key1.json \
     --slurpfile key2 /tmp/key2.json \
     -n '{ key1: $key1, key2: $key2 }'

Alternatively, if you prefer to avoid working with temporary files, you have the option to create the JSON arrays within a subshell and pass them in using --argjson.

$ jq --argjson key1 "$(jq -R '.' key1.txt | jq -s '.')" \
     --argjson key2 "$(jq -R '.' key2.txt | jq -s '.')" \
     -n '{ key1: $key1, key2: $key2 }'

Answer 5

Answer №3

Generate an array of entries using a loop and then combine the loop output with a final jq invocation.

for file in *.txt; do
  <"$file" jq -nR --arg key "${file%.txt}" '{$key,value:[inputs]}';
done | jq -s 'from_entries'

Please note that this question and answer are closely related to How to update a json file by the contents read from other files using jq? and Construct JSON through jq tool in shell script loop.

Answer 6

Generate an array of entries using a loop and then combine the loop output with a final jq invocation.

for file in *.txt; do
  <"$file" jq -nR --arg key "${file%.txt}" '{$key,value:[inputs]}';
done | jq -s 'from_entries'

Please note that this question and answer are closely related to How to update a json file by the contents read from other files using jq? and Construct JSON through jq tool in shell script loop.

Generating a JSON document from a collection of text files

Answer №1

Answer №2

Answer №3

Similar questions

What is the reason for requiring this to be passed as an Array instead of a String?

What is the process for parsing a JSON file (array format) with the simple JSON library?

Learn how to extract the first occurrence of repeated data fields by parsing JSON using the org.JSON library in Java

"Enhancing JqGrid functionality with inline editing and custom formatters

Exploring how Java can parse and read JSON using the Jackson library, specifically

tips on locating the next immediate downloadable cell in a table

Retrieve JSON data within an HTML element, where the element is sourced from an AJAX GET response

Tips for avoiding background color interference with raycaster

javascript onload button is selected

Tips on toggling between slides with Bootstrap Carousel

To prevent the need for redundant iterations, arrange an object based on a specific field

Using TypeScript, pass an image as a prop in a Styled Component

Is there a way to hide a paragraph or div using javascript?

I'm having trouble locating the corresponding end tag for "<%". How can I resolve this issue?

Apply express middleware to all routes except for those starting with /api/v1

Show segments of video stream on page using node.js

Unable to extract the 'id' property from 'params' object in Next.js 13.4 due to its undefined value

Is there a way to locate child components without needing to designate the higher-order component encompassing them?

The Runtime Error encountered in NEXTJS: TypeError - Unable to iterate over 'games' as it is not

What steps can be taken to resolve the "npm ERR! code ELIFECYCLE" error in a React application?