alanorth/csv-metadata-quality
1
0
Fork 0
mirror of https://github.com/ilri/csv-metadata-quality.git synced 2024-12-23 04:32:21 +01:00
Code Releases Activity
c04207fcfc
csv-metadata-quality/data/test.csv

29 lines
1.3 KiB
Plaintext
Raw Normal View History

data/test.csv: Rename contributor column to title This makes more sense as a description of each test and the titles are obviously not authors.
2019-09-26 04:50:40 +02:00
dc.title,birthdate,dc.identifier.issn,dc.identifier.isbn,dc.language.iso,dc.subject,cg.coverage.country,filename
data/test.csv: Add test case for uncommon filename extension
2019-08-10 22:46:56 +02:00
Leading space,2019-07-29,,,,,,
Trailing space ,2019-07-29,,,,,,
Excessive space,2019-07-29,,,,,,
Miscellaenous ||whitespace | issues ,2019-07-29,,,,,,
Duplicate||Duplicate,2019-07-29,,,,,,
Invalid ISSN,2019-07-29,2321-2302,,,,,
Invalid ISBN,2019-07-29,,978-0-306-40615-6,,,,
Multiple valid ISSNs,2019-07-29,0378-5955||0024-9319,,,,,
Multiple valid ISBNs,2019-07-29,,99921-58-10-7||978-0-306-40615-7,,,,
Invalid date,2019-07-260,,,,,,
Multiple dates,2019-07-26||2019-01-10,,,,,,
Invalid multi-value separator,2019-07-29,0378-5955|0024-9319,,,,,
Unnecessary Unicode​,2019-07-29,,,,,,
Suspicious character||foreˆt,2019-07-29,,,,,,
data/test.csv: Update titles of language tests ISO 639-1 is alpha 2 and ISO 639-3 is alpha 3. See: https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes
2019-09-26 06:40:27 +02:00
Invalid ISO 639-1 (alpha 2) language,2019-07-29,,,jp,,,
Invalid ISO 639-3 (alpha 3) language,2019-07-29,,,chi,,,
data/test.csv: Add test case for uncommon filename extension
2019-08-10 22:46:56 +02:00
Invalid language,2019-07-29,,,Span,,,
Invalid AGROVOC subject,2019-07-29,,,,FOREST,,
data/test.csv: Clarify that newline is a line feed
2019-07-31 12:03:43 +02:00
Newline (LF),2019-07-30,,,,"TANZA
data/test.csv: Add test case for uncommon filename extension
2019-08-10 22:46:56 +02:00
NIA",,
Missing date,,,,,,,
Invalid country,2019-08-01,,,,,KENYAA,
Uncommon filename extension,2019-08-10,,,,,,file.pdf.lck
data/test.csv: Add test for replacement of unneccessary Unicode
2019-08-10 23:08:44 +02:00
Unneccesary unicode (U+002D + U+00AD),2019-08-10,,978-­92-­9043-­823-­6,,,,
data/test.csv: Add sample for missing space after comma
2019-08-27 23:08:26 +02:00
"Missing space,after comma",2019-08-27,,,,,,
Experimental language detection using langid Works decenty well assuming the title, abstract, and citation fields are an accurate representation of the language as identified by the language field. Handles ISO 639-1 (alpha 2) and ISO 639-3 (alpha 3) values seamlessly. This includes updated pipenv environment, test data, pytest tests for both correct and incorrect ISO 639-1 and ISO 639-3 languages, and a new command line option "-e".
2019-09-24 17:55:05 +02:00
Incorrect ISO 639-1 language,2019-09-26,,,es,,,
Incorrect ISO 639-3 language,2019-09-26,,,spa,,,
Copy Permalink