mirror of
https://github.com/ilri/csv-metadata-quality.git
synced 2025-02-28 12:38:45 +01:00
We actually want to do this after we try to fix mojibake with ftfy. These "unnecessary" Unicode characters could actually help ftfy in some cases because often times they indicate that some character from another encoding was there before (like an accent, dash, or smart quote).