Parsing JSON using jq
Parse JSON using jq in the bash command line.
My challenge
- Got a Json from curl get request
- Image base64 is nested in
- I need to get the base64 and convert it to an image
Here is an example Json
{
"id":"001",
"image":{
"full_size":{
"data":"iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII="
},
"half_size":{
"data":"iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAQAAAAnZu5uAAAAEElEQVR42mNk+M8ABYwkMAGbQQUBEvGWBAAAAABJRU5ErkJggg=="
}
}
}
The solution
curl -H "Authorization: Basic myCredentialis" https://togetmyjsonurl.com/getjson | jq -r '.["image"]["full_size"]["data"]' | base64 --decode > randomfilename.jpg
jq - Command-line JSON processor
In the above example, we got the Json from curl, then we parse it using jq, dive 3 levels deep into the Json and arrives at the front door of what we seek.
brew update
brew install jq
which jq
man jq
curl - transfer a URL
which curl
man curl
base64 – Encode and decode using Base64 representation
which base64
man base64
jq Details
Let’s move on with the same example.
1. The pipe operator
The | operator in jq feeds the output of one filter.
2. Get all the keys
$ resulttext | jq 'keys | .[]''
"id"
"image"
3. get all the values
$ resulttext | jq '.[]''
"001"
{
"full_size": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII="
},
"half_size": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAQAAAAnZu5uAAAAEElEQVR42mNk+M8ABYwkMAGbQQUBEvGWBAAAAABJRU5ErkJggg=="
}
}
4. get one from key
$ resulttext | jq '.image''
{
"full_size": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII="
},
"half_size": {
"data": "iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAQAAAAnZu5uAAAAEElEQVR42mNk+M8ABYwkMAGbQQUBEvGWBAAAAABJRU5ErkJggg=="
}
}
5. Get length
- String: length in bytes
- array: number of elements
- Objects: number of key-value pairs
- length of null is 0
$ resulttext | jq '.image | length'
2
6. Go deeper
$ resulttext | jq '.image.full_size.data'
"iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII="
7. Without Quotes
$ resulttext | jq -r '.image.full_size.data'
"iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII="
-r stands for --raw-output.
I got an error with ., so this example could be:
$ resulttext | jq -r '.["image"]["full_size"]["data"]'
iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAQAAAAnOwc2AAAAEUlEQVR42mNk+M+AARiHsiAAcCIKAYwFoQ8AAAAASUVORK5CYII=