From acfdfe4ce62478f26b89dae7f42bfbe10d1ab97c Mon Sep 17 00:00:00 2001 From: Alan Orth Date: Tue, 7 Apr 2020 10:37:48 +0300 Subject: [PATCH] Add notes for 2020-04-07 --- content/posts/2020-04.md | 68 ++++++++++++++++++++++++++++++++++++++++ docs/2020-04/index.html | 65 ++++++++++++++++++++++++++++++++++++-- docs/sitemap.xml | 10 +++--- 3 files changed, 135 insertions(+), 8 deletions(-) diff --git a/content/posts/2020-04.md b/content/posts/2020-04.md index 32bb28eba..c079359d7 100644 --- a/content/posts/2020-04.md +++ b/content/posts/2020-04.md @@ -20,4 +20,72 @@ categories: ["Notes"] - Altmetric responded about [one item](https://hdl.handle.net/10568/101286) that had no donut since at least 2019-12 and said they fixed some problems with their bot's user agent - I decided to [tweet the item](https://twitter.com/mralanorth/status/1245703049445851140), as I can't remember if I ever did it before +## 2020-04-05 + +- Update PostgreSQL JDBC driver to version 42.2.12 + +## 2020-04-07 + +- Yesterday Atmire sent me their [pull request for DSpace 6 modules](https://github.com/ilri/DSpace/pull/445) +- Peter pointed out that some items have his ORCID identifier (`cg.creator.id`) twice + - I think this is because my early `add-orcid-identifiers.py` script was adding identifiers to existing records without properly checking if there was already one present (at first it only checked if there was one with the exact `place` value) + - As a test I dropped all his ORCID identifiers and added them back with the `add-orcid-identifiers.py` script: + +``` +$ psql -h localhost -U postgres dspace -c "DELETE FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=240 AND text_value LIKE '%Ballantyne%';" +DELETE 97 +$ ./add-orcid-identifiers-csv.py -i 2020-04-07-peter-orcids.csv -db dspace -u dspace -p 'fuuu' -d +``` + +- I used this CSV with the script (all records with his name have the name standardized like this): + +``` +dc.contributor.author,cg.creator.id +"Ballantyne, Peter G.","Peter G. Ballantyne: 0000-0001-9346-2893" +``` + +- Then I tried another way, to identify all duplicate ORCID identifiers for a given resource ID and group them so I can see if count is greater than 1: + +``` +dspace=# \COPY (SELECT DISTINCT(resource_id, text_value) as distinct_orcid, COUNT(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 240 GROUP BY distinct_orcid ORDER BY count DESC) TO /tmp/2020-04-07-duplicate-orcids.csv WITH CSV HEADER; +COPY 15209 +``` + +- Of those, about nine authors had duplicate ORCID identifiers over about thirty records, so I created a CSV with all their name variations and ORCID identifiers: + +``` +dc.contributor.author,cg.creator.id +"Ballantyne, Peter G.","Peter G. Ballantyne: 0000-0001-9346-2893" +"Ramirez-Villegas, Julian","Julian Ramirez-Villegas: 0000-0002-8044-583X" +"Villegas-Ramirez, J","Julian Ramirez-Villegas: 0000-0002-8044-583X" +"Ishitani, Manabu","Manabu Ishitani: 0000-0002-6950-4018" +"Manabu, Ishitani","Manabu Ishitani: 0000-0002-6950-4018" +"Ishitani, M.","Manabu Ishitani: 0000-0002-6950-4018" +"Ishitani, M.","Manabu Ishitani: 0000-0002-6950-4018" +"Buruchara, Robin A.","Robin Buruchara: 0000-0003-0934-1218" +"Buruchara, Robin","Robin Buruchara: 0000-0003-0934-1218" +"Jarvis, Andy","Andy Jarvis: 0000-0001-6543-0798" +"Jarvis, Andrew","Andy Jarvis: 0000-0001-6543-0798" +"Jarvis, A.","Andy Jarvis: 0000-0001-6543-0798" +"Tohme, Joseph M.","Joe Tohme: 0000-0003-2765-7101" +"Hansen, James","James Hansen: 0000-0002-8599-7895" +"Hansen, James W.","James Hansen: 0000-0002-8599-7895" +"Asseng, Senthold","Senthold Asseng: 0000-0002-7583-3811" +``` + +- Then I deleted *all* their existing ORCID identifier records: + +``` +dspace=# DELETE FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=240 AND text_value SIMILAR TO '%(0000-0001-6543-0798|0000-0001-9346-2893|0000-0002-6950-4018|0000-0002-7583-3811|0000-0002-8044-583X|0000-0002-8599-7895|0000-0003-0934-1218|0000-0003-2765-7101)%'; +DELETE 994 +``` + +- And then I added them again using the `add-orcid-identifiers` records: + +``` +$ ./add-orcid-identifiers-csv.py -i 2020-04-07-fix-duplicate-orcids.csv -db dspace -u dspace -p 'fuuu' -d +``` + +- I ran the fixes on DSpace Test and CGSpace as well + diff --git a/docs/2020-04/index.html b/docs/2020-04/index.html index 51d817da2..b2d25d406 100644 --- a/docs/2020-04/index.html +++ b/docs/2020-04/index.html @@ -25,7 +25,7 @@ On the same note, the one item Abenet pointed out last week now has a donut with - + @@ -55,9 +55,9 @@ On the same note, the one item Abenet pointed out last week now has a donut with "@type": "BlogPosting", "headline": "April, 2020", "url": "https://alanorth.github.io/cgspace-notes/2020-04/", - "wordCount": "178", + "wordCount": "535", "datePublished": "2020-04-02T10:53:24+03:00", - "dateModified": "2020-04-02T12:33:41+03:00", + "dateModified": "2020-04-02T16:30:44+03:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -154,6 +154,65 @@ On the same note, the one item Abenet pointed out last week now has a donut with +

2020-04-05

+ +

2020-04-07

+ +
$ psql -h localhost -U postgres dspace -c "DELETE FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=240 AND text_value LIKE '%Ballantyne%';"
+DELETE 97
+$ ./add-orcid-identifiers-csv.py -i 2020-04-07-peter-orcids.csv -db dspace -u dspace -p 'fuuu' -d
+
+
dc.contributor.author,cg.creator.id
+"Ballantyne, Peter G.","Peter G. Ballantyne: 0000-0001-9346-2893"
+
+
dspace=# \COPY (SELECT DISTINCT(resource_id, text_value) as distinct_orcid, COUNT(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 240 GROUP BY distinct_orcid ORDER BY count DESC) TO /tmp/2020-04-07-duplicate-orcids.csv WITH CSV HEADER;
+COPY 15209
+
+
dc.contributor.author,cg.creator.id
+"Ballantyne, Peter G.","Peter G. Ballantyne: 0000-0001-9346-2893"
+"Ramirez-Villegas, Julian","Julian Ramirez-Villegas: 0000-0002-8044-583X"
+"Villegas-Ramirez, J","Julian Ramirez-Villegas: 0000-0002-8044-583X"
+"Ishitani, Manabu","Manabu Ishitani: 0000-0002-6950-4018"
+"Manabu, Ishitani","Manabu Ishitani: 0000-0002-6950-4018"
+"Ishitani, M.","Manabu Ishitani: 0000-0002-6950-4018"
+"Ishitani, M.","Manabu Ishitani: 0000-0002-6950-4018"
+"Buruchara, Robin A.","Robin Buruchara: 0000-0003-0934-1218"
+"Buruchara, Robin","Robin Buruchara: 0000-0003-0934-1218"
+"Jarvis, Andy","Andy Jarvis: 0000-0001-6543-0798"
+"Jarvis, Andrew","Andy Jarvis: 0000-0001-6543-0798"
+"Jarvis, A.","Andy Jarvis: 0000-0001-6543-0798"
+"Tohme, Joseph M.","Joe Tohme: 0000-0003-2765-7101"
+"Hansen, James","James Hansen: 0000-0002-8599-7895"
+"Hansen, James W.","James Hansen: 0000-0002-8599-7895"
+"Asseng, Senthold","Senthold Asseng: 0000-0002-7583-3811"
+
+
dspace=# DELETE FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=240 AND text_value SIMILAR TO '%(0000-0001-6543-0798|0000-0001-9346-2893|0000-0002-6950-4018|0000-0002-7583-3811|0000-0002-8044-583X|0000-0002-8599-7895|0000-0003-0934-1218|0000-0003-2765-7101)%';
+DELETE 994
+
+
$ ./add-orcid-identifiers-csv.py -i 2020-04-07-fix-duplicate-orcids.csv -db dspace -u dspace -p 'fuuu' -d
+
diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 7167606bc..fbeaef724 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -4,27 +4,27 @@ https://alanorth.github.io/cgspace-notes/2020-04/ - 2020-04-02T12:33:41+03:00 + 2020-04-02T16:30:44+03:00 https://alanorth.github.io/cgspace-notes/categories/ - 2020-04-02T12:33:41+03:00 + 2020-04-02T16:30:44+03:00 https://alanorth.github.io/cgspace-notes/ - 2020-04-02T12:33:41+03:00 + 2020-04-02T16:30:44+03:00 https://alanorth.github.io/cgspace-notes/categories/notes/ - 2020-04-02T12:33:41+03:00 + 2020-04-02T16:30:44+03:00 https://alanorth.github.io/cgspace-notes/posts/ - 2020-04-02T12:33:41+03:00 + 2020-04-02T16:30:44+03:00