csv-metadata-quality/data/test.csv

dc.contributor.author,birthdate,dc.identifier.issn,dc.identifier.isbn
 Alan|| Alan||Alan  Orth||Alan ||Alan Orth  |Alan,1984,0378-5955,978-0-306-40615-6||99921-58-10-7
Stella|| Stella ||Stella Orth||Stella ,1984-11-27,2321-2302,99921-58-10-7
Sophia,2019-06-15,,
Test,2019-06-150,,
"Doe, J.",2019-06-15||2019-01-10,,
Someone,,0378-5955|0378-5955,
Unnecessary Unicode,2019-07-29,,
Suspicious Character||foreˆt,2019-07-29,,
-												Bring test.csv into project

											
										
										
											2019-07-26 22:14:37 +02:00
+								dc.contributor.author,birthdate,dc.identifier.issn,dc.identifier.isbn
-												tests/test.csv: Add invalid multi-value field separator

											
										
										
											2019-07-26 22:48:45 +02:00
+								 Alan|| Alan||Alan  Orth||Alan ||Alan Orth  |Alan,1984,0378-5955,978-0-306-40615-6||99921-58-10-7
-												Bring test.csv into project

											
										
										
											2019-07-26 22:14:37 +02:00
+								Stella|| Stella ||Stella Orth||Stella ,1984-11-27,2321-2302,99921-58-10-7
-												tests/test.csv: Add invalid multi-value field separator

											
										
										
											2019-07-26 22:48:45 +02:00
+								Sophia,2019-06-15,,
-												data/test.csv: Add some new records to test dates

Test invalid, missing, and multiple dates.

											
										
										
											2019-07-28 15:23:55 +02:00
+								Test,2019-06-150,,
 								"Doe, J.",2019-06-15||2019-01-10,,
-												Add "unsafe fixes" runtime option

In this case it fixes occurences of invalid multi-value separators.
DSpace uses "||" to separate multiple values in one field, but our
editors sometimes give us files with mistakes like "|". We can fix
these to be correct multi-value separators if we are sure that the
metadata is not actually using "|" for some legitimate purpose.

											
										
										
											2019-07-28 21:53:39 +02:00
+								Someone,,0378-5955|0378-5955,
-												Add support for fixing "unnecessary" Unicode

These are things like non-breaking spaces, "replacement" characters,
etc that add nothing to the metadata and often cause errors during
parsing or displaying in a UI.

											
										
										
											2019-07-29 15:38:10 +02:00
+								Unnecessary Unicode,2019-07-29,,
-												Add check for "suspicious" characters

These standalone characters often indicate issues with encoding or
copy/paste in languages with accents like French and Spanish. For
example: foreˆt should be forêt.

It is not possible to fix these issues automatically, but this will
print a warning so you can notify the owner of the data.

											
										
										
											2019-07-29 16:08:49 +02:00
+								Suspicious Character||foreˆt,2019-07-29,,