1
0
mirror of https://github.com/ilri/csv-metadata-quality.git synced 2024-12-22 20:22:18 +01:00
csv-metadata-quality/csv_metadata_quality
Alan Orth 49e3543878
Add Unicode normalization
This will check all strings for un-normalized Unicode characters.
Normalization is done using NFC. This includes tests and updated
sample data (data/test.csv).

See: https://withblue.ink/2019/03/11/why-you-need-to-normalize-unicode-strings.html
2020-01-15 11:37:54 +02:00
..
__init__.py Refactor as package with subpackages 2019-07-26 22:11:10 +03:00
__main__.py Sort imports with isort 2019-08-29 01:15:04 +03:00
app.py Add Unicode normalization 2020-01-15 11:37:54 +02:00
check.py More ISO 639-1 and ISO 639-3 fixes 2019-09-26 07:44:39 +03:00
experimental.py Experimental language detection using langid 2019-09-26 13:46:32 +03:00
fix.py Add Unicode normalization 2020-01-15 11:37:54 +02:00
version.py Version 0.3.1 2019-10-01 17:11:13 +03:00