mirror of
https://github.com/ilri/csv-metadata-quality.git
synced 2025-05-09 14:46:00 +02:00
Add date validation
I'm only concerned with validating issue dates here. In DSpace they are generally always YYYY, YYY-MM, or YYYY-MM-DD (though in theory they could be any valid ISO8601 format). This also checks for cases where the date is missing and where the metadata has specified multiple dates like "1990||1991", as this is valid, but there is no practical value for it in our system.
This commit is contained in:
@ -1,6 +1,7 @@
|
||||
import csv_metadata_quality.check as check
|
||||
import csv_metadata_quality.fix as fix
|
||||
import pandas as pd
|
||||
import re
|
||||
|
||||
def main():
|
||||
# Read all fields as strings so dates don't get converted from 1998 to 1998.0
|
||||
@ -22,5 +23,10 @@ def main():
|
||||
if column == 'dc.identifier.isbn':
|
||||
df[column] = df[column].apply(check.isbn)
|
||||
|
||||
# check if column is a date column like dc.date.issued
|
||||
match = re.match(r'^.*?date.*$', column)
|
||||
if match is not None:
|
||||
df[column] = df[column].apply(check.date)
|
||||
|
||||
# Write
|
||||
df.to_csv('/tmp/test.fixed.csv', index=False)
|
||||
|
Reference in New Issue
Block a user