JSON pretty printing with a single Python command

JSON is a great way to represent and transport data. It is commonly used in webservices, both for payload and output. It originates from the world of Javascript where it’s a standard notation for objects and other data structures. Hence the name JSON (Javascript Object Notation).

I created a short video to explain the problem and the proposed solution

JSON pretty printing? Why?

Although JSON is human-readable, we often struggle interpreting complex pieces of JSON data. Unfortunately not all services print JSON in a human-readable way.

So we need a way to pretty print the JSON output. This means providing the proper indentation and adding new lines to improve readability.

Imagine the following piece of JSON

{"key":"value","collection":[1,2,3]}

Although it doesn’t seem that complicated, it’s still poorly formatted: no indentation, alle data one a single line.

The following formatting would be a lot easier to interpret

{
 "collection": [
   1,
   2,
   3
 ],
 "key": "value"
}

Looks good, doesn’t it?

JSON pretty printing using Python

A really easy way to pretty print your JSON is by leveraging the Python command-line binary.

No, we won’t be coding in Python. We will only use a oneliner on the command line to perform JSON pretty printing:

cat data.json | python -m json.tool

By piping the data to Python and applying the JSON tool, we get the following output:

{
 "collection": [
   1,
   2,
   3
 ],
 "key": "value"
}

JSON pretty printing of API output

I realize that most people need JSON pretty printing to format the output of an API. That’s also possible. Have a look at the following example:

curl -s http://localhost:3000 | python -m json.tool

cURL is a library/binary to perform HTTP calls on the command line. By adding the -s option, we keep cURL silent and avoid that it outputs metadata. We only care about the JSON output. The JSON output goes to standard out, gets piped to the Python binary and the JSON tool handles the rest.

An example of JSON pretty printing that makes sense

Take the following HTTP call:

curl -s https://en.wikipedia.org/w/api.php\?action\=query\&prop\=revisions\&meta\=siteinfo\&titles\=Main%20Page\&rvprop\=user%7Ccomment\&continue\=\&format\=json

We basically perform an API call on the Wikipedia API and expect JSON output.

This is the output we get:

{"batchcomplete":"","query":{"pages":{"15580374":{"pageid":15580374,"ns":0,"title":"Main Page","revisions":[{"user":"Materialscientist","comment":"Undid revision 696846306 by [[Special:Contributions/Shirt58|Shirt58]] ([[User talk:Shirt58|talk]])"}]}},"general":{"mainpage":"Main Page","base":"https://en.wikipedia.org/wiki/Main_Page","sitename":"Wikipedia","logo":"//en.wikipedia.org/static/images/project-logos/enwiki.png","generator":"MediaWiki 1.27.0-wmf.12","phpversion":"5.6.99-hhvm","phpsapi":"srv","hhvmversion":"3.6.5","dbtype":"mysql","dbversion":"10.0.16-MariaDB-log","imagewhitelistenabled":"","langconversion":"","titleconversion":"","linkprefixcharset":"","linkprefix":"","linktrail":"/^([a-z]+)(.*)$/sD","legaltitlechars":" %!\"$&'()*,\\-.\\/0-9:;=?@A-Z\\\\^_`a-z~\\x80-\\xFF+","invalidusernamechars":"@:","git-hash":"612d297faf85dec6f042bb6e6010279e30be2493","git-branch":"wmf/1.27.0-wmf.12","case":"first-letter","lang":"en","fallback":[],"fallback8bitEncoding":"windows-1252","writeapi":"","timezone":"UTC","timeoffset":0,"articlepath":"/wiki/$1","scriptpath":"/w","script":"/w/index.php","variantarticlepath":false,"server":"//en.wikipedia.org","servername":"en.wikipedia.org","wikiid":"enwiki","time":"2016-02-10T20:35:24Z","misermode":"","uploadsenabled":"","maxuploadsize":1048576000,"minuploadchunksize":1024,"thumblimits":[120,150,180,200,220,250,300,400],"imagelimits":[{"width":320,"height":240},{"width":640,"height":480},{"width":800,"height":600},{"width":1024,"height":768},{"width":1280,"height":1024}],"favicon":"//en.wikipedia.org/static/favicon/wikipedia.ico","centralidlookupprovider":"CentralAuth","allcentralidlookupproviders":["local","CentralAuth"]}}}

Not a pretty sight, right? We could make it look a lot prettier by performing the following command:

curl -s https://en.wikipedia.org/w/api.php\?action\=query\&prop\=revisions\&meta\=siteinfo\&titles\=Main%20Page\&rvprop\=user%7Ccomment\&continue\=\&format\=json | python -m json.tool

Safe to say that the output is a lot more readable:

{
    "batchcomplete": "",
    "query": {
        "general": {
            "allcentralidlookupproviders": [
                "local",
                "CentralAuth"
            ],
            "articlepath": "/wiki/$1",
            "base": "https://en.wikipedia.org/wiki/Main_Page",
            "case": "first-letter",
            "centralidlookupprovider": "CentralAuth",
            "dbtype": "mysql",
            "dbversion": "10.0.16-MariaDB-log",
            "fallback": [],
            "fallback8bitEncoding": "windows-1252",
            "favicon": "//en.wikipedia.org/static/favicon/wikipedia.ico",
            "generator": "MediaWiki 1.27.0-wmf.12",
            "git-branch": "wmf/1.27.0-wmf.12",
            "git-hash": "612d297faf85dec6f042bb6e6010279e30be2493",
            "hhvmversion": "3.6.5",
            "imagelimits": [
                {
                    "height": 240,
                    "width": 320
                },
                {
                    "height": 480,
                    "width": 640
                },
                {
                    "height": 600,
                    "width": 800
                },
                {
                    "height": 768,
                    "width": 1024
                },
                {
                    "height": 1024,
                    "width": 1280
                }
            ],
            "imagewhitelistenabled": "",
            "invalidusernamechars": "@:",
            "lang": "en",
            "langconversion": "",
            "legaltitlechars": " %!\"$&'()*,\\-.\\/0-9:;=?@A-Z\\\\^_`a-z~\\x80-\\xFF+",
            "linkprefix": "",
            "linkprefixcharset": "",
            "linktrail": "/^([a-z]+)(.*)$/sD",
            "logo": "//en.wikipedia.org/static/images/project-logos/enwiki.png",
            "mainpage": "Main Page",
            "maxuploadsize": 1048576000,
            "minuploadchunksize": 1024,
            "misermode": "",
            "phpsapi": "srv",
            "phpversion": "5.6.99-hhvm",
            "script": "/w/index.php",
            "scriptpath": "/w",
            "server": "//en.wikipedia.org",
            "servername": "en.wikipedia.org",
            "sitename": "Wikipedia",
            "thumblimits": [
                120,
                150,
                180,
                200,
                220,
                250,
                300,
                400
            ],
            "time": "2016-02-10T20:37:29Z",
            "timeoffset": 0,
            "timezone": "UTC",
            "titleconversion": "",
            "uploadsenabled": "",
            "variantarticlepath": false,
            "wikiid": "enwiki",
            "writeapi": ""
        },
        "pages": {
            "15580374": {
                "ns": 0,
                "pageid": 15580374,
                "revisions": [
                    {
                        "comment": "Undid revision 696846306 by [[Special:Contributions/Shirt58|Shirt58]] ([[User talk:Shirt58|talk]])",
                        "user": "Materialscientist"
                    }
                ],
                "title": "Main Page"
            }
        }
    }
}