UnicodeDecodeError: 'utf-8' codec can't decode byte
Error message
'utf-8' codec can't decode byte 0x80 in position 0
Quick summary
You tried to decode bytes using an encoding (like UTF-8) that doesn't match the actual byte content.
Why this happens
Reading a file with the wrong encoding (e.g., UTF-8 vs Latin-1)
Decoding binary data as text
Mixing encodings when processing text from different sources
Minimal example
data = b'\x80\x81'
text = data.decode('utf-8') # UnicodeDecodeErrordata = b'\x80\x81'
text = data.decode('utf-8', errors='replace')
print(text) # Two replacement charactersThe bytes aren't valid UTF-8. Using errors='replace' substitutes invalid bytes with the Unicode replacement character.
How to diagnose
Check the file's actual encoding with a hex editor or chardet
Try different encodings: 'utf-8', 'latin-1', 'cp1252'
Use errors='replace' or errors='ignore' to handle bad bytes
How to fix
Specify the correct encoding when opening: open(f, encoding='latin-1')
Use errors='replace' to replace bad bytes with a placeholder
Use errors='ignore' to skip bad bytes
How to prevent
Always specify encoding explicitly when opening files
Use chardet to detect unknown encodings
Standardize on UTF-8 for all text files
Related Resources
Related Lessons
- File Handling
File encoding.