Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
Alexandre Bulté
Validata Core
Commits
4e348784
Commit
4e348784
authored
Jan 29, 2019
by
Pierre Dittgen
Browse files
detect_encoding now test utf-8 first
parent
5fefcdf2
Changes
1
Hide whitespace changes
Inline
Side-by-side
validata_core/source_helpers.py
View file @
4e348784
...
...
@@ -14,13 +14,8 @@ def build_tabulator_params(type, name, source):
# CSV: converts to string
if
format
==
'csv'
:
scheme
=
'text'
# Try UTF-8 first, then fallback to detect_encoding,
# because detection sometimes fails and returns cp1252 instead of utf-8.
try
:
source
=
source
.
decode
(
"utf-8"
)
except
UnicodeDecodeError
:
encoding
=
helpers
.
detect_encoding
(
source
)
source
=
source
.
decode
(
encoding
)
encoding
=
helpers
.
detect_encoding
(
source
)
source
=
source
.
decode
(
encoding
)
# Else use custom BytesLoader
else
:
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment