Working With JSON Data

Introducing JSON

JSON stands for JavaScript Object Notation
JSON originated from JavaScript
JSON has become language-agnostic
is now recognized as the standard for data interchange

Examining JSON Syntax

sample JSON file named hello_frieda.json

{
  "name": "Frieda",
  "isDog": true,
  "hobbies": ["eating", "sleeping", "barking"],
  "age": 8,
  "address": {
    "work": null,
    "home": ["Berlin", "Germany"]
  },
  "friends": [
    {
      "name": "Philipp",
      "hobbies": ["eating", "sleeping", "reading"]
    },
    {
      "name": "Mitch",
      "hobbies": ["running", "snacking"]
    }
  ]
}

JSON does not support single quotes

JSON Data Type	Description
object	a collection of key-value pairs inside curly braces ({})
array	a list of values wrapped in square brackets ([])
string	text wrapped in double quotes ("")
number	integers or floating-point numbers
boolean	either true or false without quotes
null	represents a null value, written as null

Writing JSON With Python

the json module is specifically designed for reading and writing strings formatted as JSON
can conveniently convert Python data types into JSON data and the other way around

Convert Python Dictionaries to JSON

after importing the json module, can use .dumps() to convert a Python dictionary to a JSON-formatted string representing a JSON object

>>> import json
>>> food_ratings = {"organic dog food": 2, "human food": 10}
>>> json.dumps(food_ratings)
'{"organic dog food": 2, "human food": 10}'

keys and values don't always directly translate to a JSON data type

>>> numbers_present = {1: True, 2: True, 3: False}
>>> json.dumps(numbers_present)
'{"1": true, "2": true, "3": false}'

when you converting a dictionary to JSON, the dictionary keys will always be strings in JSON

Serialize Other Python Data Types to JSON

the json module allows converting common Python data types to JSON

Python	JSON
dict	object
list	array
tuple	array
str	string
int	number
float	number
True	true
False	false
None	null

different Python data types like lists and tuples serialize to the same JSON array data type
not every Python type can be used as a key

Python Data Type	Allowed as JSON Key
dict	No
list	No
tuple	No
str	Yes
int	Yes
float	Yes
bool	Yes
None	Yes

can't use dictionaries and lists as keys because they're not hashable
tuples are hashable but will cause a TypeError when used as a key

available_nums = {(1, 2): True, 3: False}
# will cause TypeError
json.dumps(available_nums)

the skipkeys argument can prevent a TypeError when creating JSON data with unsupported Python keys

>>> json.dumps(available_nums, skipkeys=True)
'{"3": false}'

when skipkeys in json.dumps() is True, then the keys that are not supported are skipped
avoids raising a TypeError
the result is a JSON-formatted string that only contains a subset of the input dictionary
usually want the JSON data to resemble the input object as close as possible
use skipkeys with caution to not lose information when calling json.dumps()

when using json.dumps() additional arguments can control the look of the resulting JSON-formatted string
can sort the dictionary keys by setting the sort_keys parameter to True

>>> toy_conditions = {"chew bone": 7, "ball": 3, "sock": -1}
>>> json.dumps(toy_conditions, sort_keys=True)
'{"ball": 3, "chew bone": 7, "sock": -1}'

Write a JSON File With Python

to write Python data into an external JSON file use json.dump()

import json

dog_data = {
  "name": "Frieda",
  "is_dog": True,
  "hobbies": ["eating", "sleeping", "barking",],
  "age": 8,
  "address": {
    "work": None,
    "home": ("Berlin", "Germany",),
  },
  "friends": [
    {
      "name": "Philipp",
      "hobbies": ["eating", "sleeping", "reading",],
    },
    {
      "name": "Mitch",
      "hobbies": ["running", "snacking",],
    },
  ],
}

with open("hello_frieda.json", mode="w", encoding="utf-8") as write_file:
    json.dump(dog_data, write_file)

Reading JSON With Python

the json library provides two functions to deserialize JSON data into a Python object

json.loads() - to deserialize a string, bytes, or byte array instances
json.load() - To deserialize a text file or a binary file

conversion from JSON data types and values to Python

JSON	Python
object	dict
array	list
string	str
number	int
number	float
true	True
false	False
null	None

Convert JSON Objects to a Python Dictionary

>>> import json
# create Python dict
>>> dog_registry = {1: {"name": "Frieda"}}
# get JSON string
>>> dog_json = json.dumps(dog_registry)
# JSON converts key to string
>>> dog_json
'{"1": {"name": "Frieda"}}'

# create Python dict using JSON string
>>> new_dog_registry = json.loads(dog_json)

# the two dictionaries are not the same
>>> new_dog_registry == dog_registry
False

# notice difference in keys
>>> new_dog_registry
{'1': {'name': 'Frieda'}}
>>> dog_registry
{1: {'name': 'Frieda'}}

Deserialize JSON Data Types

different data types behave differently in a roundtrip from Python to JSON and back
dict contains Python data types

>>> dog_data = {
...   "name": "Frieda",
...   "is_dog": True,
...   "hobbies": ["eating", "sleeping", "barking",],
...   "age": 8,
...   "address": {
...     "work": None,
...     "home": ("Berlin", "Germany",),
...   },
... }

# convert dict to JSON string
>>> dog_data_json = json.dumps(dog_data)
>>> dog_data_json
'{"name": "Frieda", "is_dog": true, "hobbies": ["eating", "sleeping", "barking"],
"age": 8, "address": {"work": null, "home": ["Berlin", "Germany"]}}'

# convert JSON string to dict
>>> new_dog_data = json.loads(dog_data_json)
>>> new_dog_data
{'name': 'Frieda', 'is_dog': True, 'hobbies': ['eating', 'sleeping', 'barking'],
'age': 8, 'address': {'work': None, 'home': ['Berlin', 'Germany']}}

both dictionaries are the same

one exception is serializing a tuple
its JSON type is an array

>>> type(dog_data["address"]["home"])
<class 'tuple'>

>>> type(new_dog_data["address"]["home"])
<class 'list'>

Open an External JSON File With Python

serializing a Python tuple creates a JSON array
deserializing a JSON array creates a Python list

generally being cautious about data type conversions should be the concern of the Python program which writes the JSON
can always anticipate which Python data types needed as long as the JSON file is valid

if json.load() is passed a file not containing valid JSON syntax a JSONDecodeError will be raised

Interacting With JSON

Prettify JSON With Python

JSON data is human readable and writeable
not easy to use when data is contained in a single line of code
both json.dump() and json.dumps() can take an indent argument

>>> import json
>>> dog_friend = {
...     "name": "Mitch",
...     "age": 6.5,
... }

>>> print(json.dumps(dog_friend))
{"name": "Mitch", "age": 6.5}

>>> print(json.dumps(dog_friend, indent=0))
{
"name": "Mitch",
"age": 6.5
}

>>> print(json.dumps(dog_friend, indent=-2))
{
"name": "Mitch",
"age": 6.5
}

>>> print(json.dumps(dog_friend, indent=""))
{
"name": "Mitch",
"age": 6.5
}

>>> print(json.dumps(dog_friend, indent=" ⮑ "))
{
 ⮑ "name": "Mitch",
 ⮑ "age": 6.5
}

Validate JSON in the Terminal

to check if a JSON file is valid, use Python's json.tool
run the json.tool module as an executable in the terminal using the -m switch
to see json.tool in action, also provide dog_friend.json as the infile positional argument

$ python -m json.tool dog_friend.json
{
    "name": "Mitch",
    "age": 6.5
}

if the file is valid json.tool() will print the file
by default the json.tool prints the JSON data with an indentation of 4
if the file is not valid (in this case using double quotes)

{
    "name": "Mitch"
    "age": 6.5
}

then json.tool outputs

$ python -m json.tool dog_friend.json
Expecting ',' delimiter: line 3 column 5 (char 26)

Pretty Print JSON in the Terminal

set --indent to control which indentation level json.tool uses to display the code

$ python -m json.tool hello_frieda.json --indent 2
{
  "name": "Frieda",
  "is_dog": true,
  "hobbies": [
    "eating",
    "sleeping",
    "barking"
  ],
  "age": 8,
  "address": {
    "work": null,
    "home": [
      "Berlin",
      "Germany"
    ]
  },
  "friends": [
    {
      "name": "Philipp",
      "hobbies": [
        "eating",
        "sleeping",
        "reading"
      ]
    },
    {
      "name": "Mitch",
      "hobbies": [
        "running",
        "snacking"
      ]
    }
  ]
}

by default json.tool writes the output to sys.stdout

Minify JSON With Python

can minify JSON two ways

use json.tool in the terminal

$ python -m json.tool <original file>.json <minified file>.json --compact

use json module in Python code

>>> import json
>>> with open("hello_frieda.json", mode="r", encoding="utf-8") as input_file:
...     original_json = input_file.read()
...

>>> json_data = json.loads(original_json) 
>>> mini_json = json.dumps(json_data, indent=None, separators=(",", ":"))
>>> with open("mini_frieda.json", mode="w", encoding="utf-8") as output_file:
...     output_file.write(mini_json)
...

the separators parameter for json.dumps() defines a tuple with two values

the separator between the key-value pairs or list items
by default the separator is a comma followed by a space (", ")
the separator between the key and the value
by default the separator is a colon followed by a space (": ")