Skip to content

UnicodeDecodeError: 'utf-8' codec can't decode byte

Error message

                'utf-8' codec can't decode byte 0x80 in position 0
              

Quick summary

You tried to decode bytes using an encoding (like UTF-8) that doesn't match the actual byte content.

Why this happens

  • Reading a file with the wrong encoding (e.g., UTF-8 vs Latin-1)

  • Decoding binary data as text

  • Mixing encodings when processing text from different sources

Minimal example

✗ Broken code
data = b'\x80\x81'
text = data.decode('utf-8')  # UnicodeDecodeError
✓ Fixed code
data = b'\x80\x81'
text = data.decode('utf-8', errors='replace')
print(text)  # Two replacement characters

The bytes aren't valid UTF-8. Using errors='replace' substitutes invalid bytes with the Unicode replacement character.

How to diagnose

  • Check the file's actual encoding with a hex editor or chardet

  • Try different encodings: 'utf-8', 'latin-1', 'cp1252'

  • Use errors='replace' or errors='ignore' to handle bad bytes

How to fix

  • Specify the correct encoding when opening: open(f, encoding='latin-1')

  • Use errors='replace' to replace bad bytes with a placeholder

  • Use errors='ignore' to skip bad bytes

How to prevent

  • Always specify encoding explicitly when opening files

  • Use chardet to detect unknown encodings

  • Standardize on UTF-8 for all text files

Related Resources

Related Glossary

Related Lessons

← Back to language