How to Work With JSON On the Command Line

JSON data

Linux has no built-in method to properly handle JSON, but many APIs return JSON output. Here’s how to analyze and use JSON data in your shell scripts with a simple utility.

Analyze JSON with jq

jq is a utility that can manage everything you want to do with JSON. It’s super simple to install; Just download the binary and move it to your folder on your PATH, usually / usr / local / bin /. It has no dependencies, so it’s just a binary to download. There are also downloads for macOS and Windows.

To get started, you can redirect JSON to print it. Some APIs will reduce their responses to save space, which will cause this terrible mess when you bend something in the console:

bend in the console

Direct it to jq like this:

curl https://api.example.com/ | jq

And it is now readable, with syntax highlighting to start:

Readable text with syntax highlighting

If you give jq a command, it will use it to select data in the JSON, much like sed works on text. We will use this JSONPlaceholder as an example of API to try commands. For the sake of simplicity, we will leave aside the curl command and the pipe so that everything is readable. You can also loop to a file and direct the chat to jq, if you don’t want to see the curl output every time you test a new command.

A simple period “.” Represents data piped in jq. If you just want to get an object key, you can do:

jq ‘.status’

You can also nest it; for example, ‘.data.geo.host’ would return a value lower in the list. Optionally, you can add a question mark for error checking like “.data.geo?” .Host “, which specifies that the property is optional.

It is by working with tables that jq becomes interesting. You can use square brackets to get a specific array element. For example, you can get the name of the first entry with:

jq ‘.[0].Last name’

You can cut tables using [2:5]and use the same question mark syntax for optional properties. Keep in mind that if an array is empty, it does not mean that it is an invalid property.

However, you will probably want to retrieve a specific entry based on the data it contains, not the order of the table. To do this, pass the entire array to the select utility:

jq ‘.[] | select (.name | contains (“Leanne”)) ‘

This filters the array to include only items that pass the test; in this case, does the name field contain a certain name? It can be any Boolean expression, even arithmetic expressions like select (.count> = 2).

Reformat JSON

jq can also build JSON, which means you can use it to reformat the JSON entry. You can create objects simply by putting them in square brackets and passing jq statements as values. jq will display the new object. For example:

json cat | jq ‘.[0] | {name: .name, company: .enterprise} ‘

Displays well on the command line:

jq releases new object

While this is useful for deleting only the information you want, you can also create new objects using the jq commands.

For a complete reference on all the syntax of jq, you can consult his online manual.

Alternatives to jq

If you don’t want to use another program, you can use a few alternatives. The simplest method is to give up doing it in bash and to use a scripting language like Python or JavaScript (with node) to do the actual processing. You prefer to direct the output of the JSON command to a file:

curl https: //api/example.com/> json.txt

And then load it into a script variable to use it.

You can also use simple text selection utilities; for example, if you just need a specific key in a response, you can use grep next to a regular expression to select the value:

grep “name” | sed / s “name”: ” (. * )” / 1 / ‘

However, this can (and probably will) break very easily with any changes to the JSON. Passing the user API returned the name of each user, but also the user name (which also corresponded to grep), as well as the company name (which had exactly the same key). Unless you plan to account for all of this, you should use something that knows the structure of JSON, rather than treating it as text.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.