JSON Handling
Parse, generate, and validate JSON with the json module, custom encoders, and error handling.
What you'll learn
- Parse JSON strings with json.loads()
- Generate JSON with json.dumps() and formatting
- Read and write JSON files with load() and dump()
- Handle custom types with default and object_hook
- Deal with JSONDecodeError and encoding issues
Concept
The json Module
JSON (JavaScript Object Notation) is the most common data interchange format on the web. Python's built-in json module converts between JSON strings and Python objects:
| JSON | Python | |------|--------| | object | dict | | array | list | | string | str | | number (int) | int | | number (float) | float | | true/false | True/False | | null | None |
import json
# Parse a JSON string into Python objects
data = json.loads('{"name": "Ada", "age": 25, "active": true}')
print(data["name"]) # Ada
print(type(data)) # <class 'dict'>
# Convert Python objects to a JSON string
text = json.dumps({"name": "Ada", "age": 25})
print(text) # {"name": "Ada", "age": 25}
Parsing JSON: json.loads()
json.loads() (load string) parses a JSON string into Python objects:
import json
json_string = '''
{
"users": [
{"name": "Ada", "age": 25},
{"name": "Bob", "age": 30}
],
"count": 2,
"active": true
}
'''
data = json.loads(json_string)
print(data["users"][0]["name"]) # Ada
print(data["count"]) # 2
print(data["active"]) # True
# Access nested data
for user in data["users"]:
print(f"{user['name']} is {user['age']}")
Generating JSON: json.dumps()
json.dumps() (dump string) converts Python objects to a JSON string:
import json
data = {
"name": "Ada",
"scores": [90, 85, 88],
"metadata": {"class": "CS101", "year": 2024}
}
# Basic — compact, no spaces
compact = json.dumps(data)
# Pretty-printed with indentation
pretty = json.dumps(data, indent=2)
print(pretty)
# {
# "name": "Ada",
# "scores": [90, 85, 88],
# "metadata": {
# "class": "CS101",
# "year": 2024
# }
# }
# Sort keys alphabetically
sorted_json = json.dumps(data, sort_keys=True, indent=2)
Useful dumps() parameters:
indent— number of spaces for indentation (pretty-print)sort_keys— sort dictionary keys alphabeticallyensure_ascii— set to False to keep Unicode characters instead of escaping themseparators— custom item and key separators
Reading and Writing Files
import json
# Write to a file
data = {"name": "Ada", "scores": [90, 85, 88]}
with open("data.json", "w", encoding="utf-8") as f:
json.dump(data, f, indent=2, ensure_ascii=False)
# Read from a file
with open("data.json", "r", encoding="utf-8") as f:
loaded = json.load(f)
print(loaded["name"]) # Ada
Use json.dump() / json.load() (without the 's') for files, and json.dumps() / json.loads() for strings. Always specify encoding="utf-8" when opening the file.
Custom Encoders
By default, json.dumps() only handles built-in types. For custom objects, provide a default function:
import json
from datetime import datetime
class User:
def __init__(self, name, age):
self.name = name
self.age = age
def encode_user(obj):
if isinstance(obj, User):
return {"name": obj.name, "age": obj.age}
if isinstance(obj, datetime):
return obj.isoformat()
raise TypeError(f"Cannot serialize {type(obj)}")
user = User("Ada", 25)
print(json.dumps(user, default=encode_user, indent=2))
The default function is called for any object the encoder does not know how to serialize. It should return a serializable representation or raise TypeError.
Custom Decoders
Use object_hook to customize how JSON objects are decoded:
import json
from datetime import datetime
def decode_dates(obj):
"""Convert ISO date strings back to datetime objects."""
for key, value in obj.items():
if key.endswith("_at") and isinstance(value, str):
obj[key] = datetime.fromisoformat(value)
return obj
json_data = '{"created_at": "2024-01-15T14:30:00", "name": "Ada"}'
data = json.loads(json_data, object_hook=decode_dates)
print(type(data["created_at"])) # <class 'datetime.datetime'>
Error Handling
import json
try:
data = json.loads('{"invalid": json,}')
except json.JSONDecodeError as e:
print(f"JSON parse error at line {e.lineno}, column {e.colno}: {e.msg}")
# Common errors:
# - JSONDecodeError: malformed JSON
# - TypeError: object is not serializable
json.JSONDecodeError provides the line number, column, and error message — useful for debugging malformed JSON.
Practical: API Response Handling
import json
def save_response(response_text, filename):
"""Parse and pretty-save an API JSON response."""
try:
data = json.loads(response_text)
except json.JSONDecodeError as e:
print(f"Invalid JSON: {e}")
return False
with open(filename, "w", encoding="utf-8") as f:
json.dump(data, f, indent=2, ensure_ascii=False, sort_keys=True)
return True
For production JSON handling, consider orjson (much faster) or pydantic (typed JSON with validation).
Example
import json
from datetime import datetime
# Parsing JSON string
json_string = '{"name": "Ada", "age": 25, "scores": [90, 85, 88], "active": true}'
data = json.loads(json_string)
print(f"Name: {data['name']}")
print(f"Scores: {data['scores']}")
print(f"Active: {data['active']}")
# Generating JSON
python_data = {
"project": "CoderNewbie",
"languages": ["Python", "JavaScript", "Rust"],
"version": 1.0,
"open_source": True,
"metadata": {"created": "2024-01-15", "author": None}
}
# Compact
compact = json.dumps(python_data)
print(f"Compact: {compact[:60]}...")
# Pretty-printed
pretty = json.dumps(python_data, indent=2, sort_keys=True)
print("Pretty:")
print(pretty)
# Custom encoder for datetime
def custom_encoder(obj):
if isinstance(obj, datetime):
return obj.isoformat()
raise TypeError(f"Cannot serialize {type(obj)}")
event = {"name": "launch", "timestamp": datetime(2024, 1, 15, 10, 0, 0)}
event_json = json.dumps(event, default=custom_encoder, indent=2)
print(f"Event with datetime: {event_json}")
# Error handling
try:
bad = json.loads('{"broken": json}')
except json.JSONDecodeError as e:
print(f"Parse error: {e.msg} at line {e.lineno}")
# Round-trip: Python → JSON → Python
original = {"a": 1, "b": [2, 3], "c": {"d": True}}
roundtrip = json.loads(json.dumps(original))
print(f"Round-trip equal: {original == roundtrip}")
# Writing to a string with ensure_ascii=False for Unicode
unicode_data = {"greeting": "Hello, 世界!"}
print(json.dumps(unicode_data, ensure_ascii=False))
Shows parsing with loads(), generating with dumps() (compact and pretty), a custom encoder for datetime, error handling for malformed JSON, round-trip verification, and Unicode handling with ensure_ascii=False.
Try it
- JSON Formatter
Practice formatting and validating JSON — the core of json.dumps().
- CSV to JSON
Converting CSV to JSON uses json.dumps() — see the transformation live.
Common mistakes
The mistake
Using json.dump() (for files) when you need a string, or json.dumps() when writing to a file
The fix
json.dumps() returns a string (the 's' stands for string). json.dump() writes to a file object. Use dumps/loads for strings, dump/load for files. Mixing them up is a common error — remember the 's' means string.
The mistake
Forgetting ensure_ascii=False and getting escaped Unicode like \u4e16 instead of actual characters
The fix
By default, json.dumps() escapes non-ASCII characters. Pass ensure_ascii=False to keep Unicode characters as-is in the output. Always pair this with encoding='utf-8' when writing to a file.
Related Resources
Related Cheatsheets
- Python Cheatsheet
json module functions, custom encoders, and error handling.
Practice JSON Handling
2 exercises
Look up unfamiliar terms
A beginner-friendly glossary explains jargon in plain English — variables, functions, DOM, Promise, and more.
Build real projects
Apply what you learned by building guided projects with starter code and solutions.
Decode error messages
Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.