60 Commits

Author	SHA1	Message	Date
Alan Orth	a36454a3ac	Add support for validating languages ... Will validate against ISO 639-2 or ISO 639-3 depending on how long the language field is. Otherwise will return that the language is invalid. Does not currently have any support for generic values like "Other".	2019-07-29 18:59:42 +03:00
Alan Orth	d7888d59a8	csv_metadata_quality/check.py: Return date even if it is invalid ... Otherwise it is missing from the final CSV and then we can't even fix it. :)	2019-07-29 17:40:14 +03:00
Alan Orth	fa4fa3491b	Add check for "suspicious" characters ... These standalone characters often indicate issues with encoding or copy/paste in languages with accents like French and Spanish. For example: foreˆt should be forêt. It is not possible to fix these issues automatically, but this will print a warning so you can notify the owner of the data.	2019-07-29 17:08:49 +03:00
Alan Orth	42920e9c7c	Test Python regular expression matches directly ... Match objects always have a boolean value of True. See: https://docs.python.org/3.7/library/re.html	2019-07-29 16:16:30 +03:00
Alan Orth	87b1997051	Fix whitespace errors found by flake8	2019-07-28 17:47:28 +03:00
Alan Orth	196bb434fa	Add date validation ... I'm only concerned with validating issue dates here. In DSpace they are generally always YYYY, YYY-MM, or YYYY-MM-DD (though in theory they could be any valid ISO8601 format). This also checks for cases where the date is missing and where the metadata has specified multiple dates like "1990||1991", as this is valid, but there is no practical value for it in our system.	2019-07-28 16:11:36 +03:00
Alan Orth	3cf9f9452b	csv_metadata_quality/check.py: Always return field ... We always need to return the field back so apply doesn't set it to null when creating the new data frame.	2019-07-27 01:28:08 +03:00
Alan Orth	aaf3537ba4	Add check for invalid multi-value separators	2019-07-26 23:48:24 +03:00
Alan Orth	02f9d8a736	csv_metadata_quality/check.py: Add check for missing isbn values	2019-07-26 23:45:18 +03:00
Alan Orth	e160b17fb0	Add ISSN and ISBN checks using python-stdnum	2019-07-26 23:14:10 +03:00