summaryrefslogtreecommitdiff
path: root/bs4/doc/source
diff options
context:
space:
mode:
authorLeonard Richardson <leonard.richardson@canonical.com>2012-02-09 16:15:56 -0500
committerLeonard Richardson <leonard.richardson@canonical.com>2012-02-09 16:15:56 -0500
commit4aff2ee4d6f077e06159c92ab05c0f2ea527c6fa (patch)
tree40951a60046f184794a011a498187053e8ad2a92 /bs4/doc/source
parentcaeb168dc47470607b3cd091e1d35db45c089385 (diff)
As a last-ditch attempt to turn data into Unicode, use errors=replace instead of errors=strict.
Diffstat (limited to 'bs4/doc/source')
-rw-r--r--bs4/doc/source/index.rst9
1 files changed, 9 insertions, 0 deletions
diff --git a/bs4/doc/source/index.rst b/bs4/doc/source/index.rst
index abea5c6..d28787b 100644
--- a/bs4/doc/source/index.rst
+++ b/bs4/doc/source/index.rst
@@ -2076,6 +2076,15 @@ We can fix this by passing in the correct ``from_encoding``::
soup.original_encoding
'iso8859-8'
+In rare cases (usually when a UTF-8 document contains text written in
+a completely different encoding), the only way to get Unicode may be
+to replace some characters with the special Unicode character
+"REPLACEMENT CHARACTER" (U+FFFD, �). If Unicode, Dammit needs to do
+this, it will set the ``.characters_were_replaced`` attribute to
+``True`` on the ``UnicodeDammit`` or ``BeautifulSoup`` object. This
+lets you know that the Unicode representation is not an exact
+representation of the original--some data was lost.
+
Output encoding
---------------