From 56626d12451c99854c06256eb8ac843437a2623c Mon Sep 17 00:00:00 2001 From: Alan Orth Date: Wed, 4 Apr 2018 15:57:34 +0300 Subject: [PATCH] Add notes for 2018-04-04 --- content/posts/2018-03.md | 2 +- content/posts/2018-04.md | 31 ++++++++++++++++++++ docs/2015-11/index.html | 2 +- docs/2015-12/index.html | 2 +- docs/2016-01/index.html | 2 +- docs/2016-02/index.html | 2 +- docs/2016-03/index.html | 2 +- docs/2016-04/index.html | 2 +- docs/2016-05/index.html | 2 +- docs/2016-06/index.html | 2 +- docs/2016-07/index.html | 2 +- docs/2016-08/index.html | 2 +- docs/2016-09/index.html | 2 +- docs/2016-10/index.html | 2 +- docs/2016-11/index.html | 2 +- docs/2016-12/index.html | 2 +- docs/2017-01/index.html | 2 +- docs/2017-02/index.html | 2 +- docs/2017-03/index.html | 2 +- docs/2017-04/index.html | 2 +- docs/2017-05/index.html | 2 +- docs/2017-06/index.html | 2 +- docs/2017-07/index.html | 2 +- docs/2017-08/index.html | 2 +- docs/2017-09/index.html | 2 +- docs/2017-10/index.html | 2 +- docs/2017-11/index.html | 2 +- docs/2017-12/index.html | 2 +- docs/2018-01/index.html | 2 +- docs/2018-02/index.html | 2 +- docs/2018-03/index.html | 15 ++++------ docs/2018-04/index.html | 39 +++++++++++++++++++++++-- docs/404.html | 5 +--- docs/categories/index.html | 2 +- docs/categories/notes/index.html | 2 +- docs/categories/notes/page/2/index.html | 2 +- docs/categories/notes/page/3/index.html | 2 +- docs/categories/notes/page/4/index.html | 2 +- docs/categories/page/2/index.html | 2 +- docs/categories/page/3/index.html | 2 +- docs/categories/page/4/index.html | 2 +- docs/cgiar-library-migration/index.html | 2 +- docs/index.html | 2 +- docs/page/2/index.html | 2 +- docs/page/3/index.html | 2 +- docs/page/4/index.html | 2 +- docs/posts/index.html | 2 +- docs/posts/page/2/index.html | 2 +- docs/posts/page/3/index.html | 2 +- docs/posts/page/4/index.html | 2 +- docs/sitemap.xml | 2 +- docs/tags/index.html | 2 +- docs/tags/notes/index.html | 2 +- docs/tags/notes/page/2/index.html | 2 +- docs/tags/notes/page/3/index.html | 2 +- docs/tags/notes/page/4/index.html | 2 +- docs/tags/page/2/index.html | 2 +- docs/tags/page/3/index.html | 2 +- docs/tags/page/4/index.html | 2 +- 59 files changed, 129 insertions(+), 71 deletions(-) diff --git a/content/posts/2018-03.md b/content/posts/2018-03.md index bac55fd0a..ba6fa89fb 100644 --- a/content/posts/2018-03.md +++ b/content/posts/2018-03.md @@ -127,7 +127,7 @@ if(isBlank(value), "Hernan Ceballos: 0000-0002-8744-7918", value + "||Hernan Ceb - One thing that bothers me is that this won't honor author order - It might be better to do batches of these in PostgreSQL with a script that takes the `place` column of an author into account when setting the `cg.creator.id` -- I wrote a Python script to read the author names and ORCID identifiers from CSV and create matching `cg.creator.id` fieldsa: [add-orcid-identifiers-csv.py ](https://gist.github.com/alanorth/a49d85cd9c5dea89cddbe809813a7050) +- I wrote a Python script to read the author names and ORCID identifiers from CSV and create matching `cg.creator.id` fields: [add-orcid-identifiers-csv.py ](https://gist.github.com/alanorth/a49d85cd9c5dea89cddbe809813a7050) - The CSV should have two columns: author name and ORCID identifier: ``` diff --git a/content/posts/2018-04.md b/content/posts/2018-04.md index 7ade9e440..80c4b6eab 100644 --- a/content/posts/2018-04.md +++ b/content/posts/2018-04.md @@ -25,3 +25,34 @@ Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" jav - For some reason Listings and Reports is not giving any results for any queries now... - I posted a message on Yammer to ask if people are using the Duplicate Check step from the Metadata Quality Module - Help Lili Szilagyi with a question about statistics on some CCAFS items + +## 2018-04-04 + +- Peter noticed that there were still some old CRP names on CGSpace, because I hadn't forced the Discovery index to be updated after I fixed the others last week +- For completeness I re-ran the CRP corrections on CGSpace: + +``` +$ ./fix-metadata-values.py -i /tmp/Correct-21-CRPs-2018-03-16.csv -f cg.contributor.crp -t correct -m 230 -db dspace -u dspace -p 'fuuu' +Fixed 1 occurences of: AGRICULTURE FOR NUTRITION AND HEALTH +``` + +- Then started a full Discovery index: + +``` +$ export JAVA_OPTS='-Dfile.encoding=UTF-8 -Xmx1024m' +$ time schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery -b +``` + +- Elizabeth from CIAT emailed to ask if I could help her by adding ORCID identifiers to all of Joseph Tohme's items +- I used my [add-orcid-identifiers-csv.py](https://gist.githubusercontent.com/alanorth/a49d85cd9c5dea89cddbe809813a7050/raw/f67b6e45a9a940732882ae4bb26897a9b245ef31/add-orcid-identifiers-csv.py) script: + +``` +$ ./add-orcid-identifiers-csv.py -i /tmp/jtohme-2018-04-04.csv -db dspace -u dspace -p 'fuuu' +``` + +- The CSV format of `jtohme-2018-04-04.csv` was: + +```csv +dc.contributor.author,cg.creator.id +"Tohme, Joseph M.",Joe Tohme: 0000-0003-2765-7101 +``` diff --git a/docs/2015-11/index.html b/docs/2015-11/index.html index ad9995f35..1ff9bc292 100644 --- a/docs/2015-11/index.html +++ b/docs/2015-11/index.html @@ -53,7 +53,7 @@ $ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspac "/> - + diff --git a/docs/2015-12/index.html b/docs/2015-12/index.html index 42f9af5ce..23bfc40e9 100644 --- a/docs/2015-12/index.html +++ b/docs/2015-12/index.html @@ -55,7 +55,7 @@ Replace lzop with xz in log compression cron jobs on DSpace Test—it uses less "/> - + diff --git a/docs/2016-01/index.html b/docs/2016-01/index.html index 8445b493e..c0b396048 100644 --- a/docs/2016-01/index.html +++ b/docs/2016-01/index.html @@ -45,7 +45,7 @@ Update GitHub wiki for documentation of maintenance tasks. "/> - + diff --git a/docs/2016-02/index.html b/docs/2016-02/index.html index 4203faacd..86943c703 100644 --- a/docs/2016-02/index.html +++ b/docs/2016-02/index.html @@ -59,7 +59,7 @@ Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE&r "/> - + diff --git a/docs/2016-03/index.html b/docs/2016-03/index.html index 95c9e06a1..954938786 100644 --- a/docs/2016-03/index.html +++ b/docs/2016-03/index.html @@ -45,7 +45,7 @@ Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Ja "/> - + diff --git a/docs/2016-04/index.html b/docs/2016-04/index.html index 91178c56c..aaf7d84af 100644 --- a/docs/2016-04/index.html +++ b/docs/2016-04/index.html @@ -49,7 +49,7 @@ Also, I noticed the checker log has some errors we should pay attention to: "/> - + diff --git a/docs/2016-05/index.html b/docs/2016-05/index.html index deac1004b..6947e8a92 100644 --- a/docs/2016-05/index.html +++ b/docs/2016-05/index.html @@ -53,7 +53,7 @@ There are 3,000 IPs accessing the REST API in a 24-hour period! "/> - + diff --git a/docs/2016-06/index.html b/docs/2016-06/index.html index 6107bfce7..55b196f9e 100644 --- a/docs/2016-06/index.html +++ b/docs/2016-06/index.html @@ -51,7 +51,7 @@ Working on second phase of metadata migration, looks like this will work for mov "/> - + diff --git a/docs/2016-07/index.html b/docs/2016-07/index.html index ef9ce821d..9b784207c 100644 --- a/docs/2016-07/index.html +++ b/docs/2016-07/index.html @@ -67,7 +67,7 @@ In this case the select query was showing 95 results before the update "/> - + diff --git a/docs/2016-08/index.html b/docs/2016-08/index.html index ddee0328b..85d1b8ec5 100644 --- a/docs/2016-08/index.html +++ b/docs/2016-08/index.html @@ -61,7 +61,7 @@ $ git rebase -i dspace-5.5 "/> - + diff --git a/docs/2016-09/index.html b/docs/2016-09/index.html index 63c87c691..4d5ab3d39 100644 --- a/docs/2016-09/index.html +++ b/docs/2016-09/index.html @@ -53,7 +53,7 @@ $ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=or "/> - + diff --git a/docs/2016-10/index.html b/docs/2016-10/index.html index 1a96c71a0..69e073194 100644 --- a/docs/2016-10/index.html +++ b/docs/2016-10/index.html @@ -61,7 +61,7 @@ I exported a random item’s metadata as CSV, deleted all columns except id "/> - + diff --git a/docs/2016-11/index.html b/docs/2016-11/index.html index 06d17e607..ec8b400a2 100644 --- a/docs/2016-11/index.html +++ b/docs/2016-11/index.html @@ -45,7 +45,7 @@ Add dc.type to the output options for Atmire’s Listings and Reports module "/> - + diff --git a/docs/2016-12/index.html b/docs/2016-12/index.html index 39e5d8643..d58cc62b0 100644 --- a/docs/2016-12/index.html +++ b/docs/2016-12/index.html @@ -69,7 +69,7 @@ Another worrying error from dspace.log is: "/> - + diff --git a/docs/2017-01/index.html b/docs/2017-01/index.html index 7aad000c7..063bbd1d0 100644 --- a/docs/2017-01/index.html +++ b/docs/2017-01/index.html @@ -45,7 +45,7 @@ I asked on the dspace-tech mailing list because it seems to be broken, and actua "/> - + diff --git a/docs/2017-02/index.html b/docs/2017-02/index.html index d4310a17d..aedd36501 100644 --- a/docs/2017-02/index.html +++ b/docs/2017-02/index.html @@ -73,7 +73,7 @@ Looks like we’ll be using cg.identifier.ccafsprojectpii as the field name "/> - + diff --git a/docs/2017-03/index.html b/docs/2017-03/index.html index e25437c6b..e4efbcd0c 100644 --- a/docs/2017-03/index.html +++ b/docs/2017-03/index.html @@ -77,7 +77,7 @@ $ identify ~/Desktop/alc_contrastes_desafios.jpg "/> - + diff --git a/docs/2017-04/index.html b/docs/2017-04/index.html index fc0a2a89b..c6adf7618 100644 --- a/docs/2017-04/index.html +++ b/docs/2017-04/index.html @@ -63,7 +63,7 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Th "/> - + diff --git a/docs/2017-05/index.html b/docs/2017-05/index.html index e5925fb6e..2a9619dba 100644 --- a/docs/2017-05/index.html +++ b/docs/2017-05/index.html @@ -29,7 +29,7 @@ - + diff --git a/docs/2017-06/index.html b/docs/2017-06/index.html index 253efbe6d..e3c1ac885 100644 --- a/docs/2017-06/index.html +++ b/docs/2017-06/index.html @@ -29,7 +29,7 @@ - + diff --git a/docs/2017-07/index.html b/docs/2017-07/index.html index dbcebe727..92f1099e3 100644 --- a/docs/2017-07/index.html +++ b/docs/2017-07/index.html @@ -57,7 +57,7 @@ We can use PostgreSQL’s extended output format (-x) plus sed to format the "/> - + diff --git a/docs/2017-08/index.html b/docs/2017-08/index.html index a1b6edd21..ac446b896 100644 --- a/docs/2017-08/index.html +++ b/docs/2017-08/index.html @@ -77,7 +77,7 @@ Then I cleaned up the author authorities and HTML characters in OpenRefine and s "/> - + diff --git a/docs/2017-09/index.html b/docs/2017-09/index.html index 928a6f60c..2478f7ade 100644 --- a/docs/2017-09/index.html +++ b/docs/2017-09/index.html @@ -53,7 +53,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne’s user account "/> - + diff --git a/docs/2017-10/index.html b/docs/2017-10/index.html index 599bd3695..b20fa938b 100644 --- a/docs/2017-10/index.html +++ b/docs/2017-10/index.html @@ -57,7 +57,7 @@ Add Katherine Lutz to the groups for content submission and edit steps of the CG "/> - + diff --git a/docs/2017-11/index.html b/docs/2017-11/index.html index 7a294a1a4..2efe9c295 100644 --- a/docs/2017-11/index.html +++ b/docs/2017-11/index.html @@ -77,7 +77,7 @@ COPY 54701 "/> - + diff --git a/docs/2017-12/index.html b/docs/2017-12/index.html index 15e5ff973..dcc4f367a 100644 --- a/docs/2017-12/index.html +++ b/docs/2017-12/index.html @@ -47,7 +47,7 @@ The list of connections to XMLUI and REST API for today: "/> - + diff --git a/docs/2018-01/index.html b/docs/2018-01/index.html index c5b79e9ef..83b4d1a1b 100644 --- a/docs/2018-01/index.html +++ b/docs/2018-01/index.html @@ -185,7 +185,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv "/> - + diff --git a/docs/2018-02/index.html b/docs/2018-02/index.html index 2a50033b0..7aae83cd6 100644 --- a/docs/2018-02/index.html +++ b/docs/2018-02/index.html @@ -47,7 +47,7 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl "/> - + diff --git a/docs/2018-03/index.html b/docs/2018-03/index.html index 6d0a59890..f40378bf3 100644 --- a/docs/2018-03/index.html +++ b/docs/2018-03/index.html @@ -20,7 +20,7 @@ Export a CSV of the IITA community metadata for Martin Mueller - + @@ -41,7 +41,7 @@ Export a CSV of the IITA community metadata for Martin Mueller "/> - + @@ -51,9 +51,9 @@ Export a CSV of the IITA community metadata for Martin Mueller "@type": "BlogPosting", "headline": "March, 2018", "url": "https://alanorth.github.io/cgspace-notes/2018-03/", - "wordCount": "2961", + "wordCount": "2960", "datePublished": "2018-03-02T16:07:54+02:00", - "dateModified": "2018-03-28T22:01:13+03:00", + "dateModified": "2018-04-04T15:14:21+03:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -255,7 +255,7 @@ UPDATE 2309 @@ -536,11 +536,6 @@ COPY 56156 - -

2018-03-20

- - diff --git a/docs/2018-04/index.html b/docs/2018-04/index.html index 091728949..7def8b1b2 100644 --- a/docs/2018-04/index.html +++ b/docs/2018-04/index.html @@ -43,7 +43,7 @@ Catalina logs at least show some memory errors yesterday: "/> - + @@ -53,7 +53,7 @@ Catalina logs at least show some memory errors yesterday: "@type": "BlogPosting", "headline": "April, 2018", "url": "https://alanorth.github.io/cgspace-notes/2018-04/", - "wordCount": "118", + "wordCount": "253", "datePublished": "2018-04-01T16:13:54+02:00", "dateModified": "2018-04-01T17:21:41+03:00", "author": { @@ -145,6 +145,41 @@ Exception in thread "ContainerBackgroundProcessor[StandardEngine[Catalina]]
  • Help Lili Szilagyi with a question about statistics on some CCAFS items
    • +

    2018-04-04

    + + + +
    $ ./fix-metadata-values.py -i /tmp/Correct-21-CRPs-2018-03-16.csv -f cg.contributor.crp -t correct -m 230 -db dspace -u dspace -p 'fuuu'
+Fixed 1 occurences of: AGRICULTURE FOR NUTRITION AND HEALTH
+
    + + + +
    $ export JAVA_OPTS='-Dfile.encoding=UTF-8 -Xmx1024m'
+$ time schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery -b
+
    + + + +
    $ ./add-orcid-identifiers-csv.py -i /tmp/jtohme-2018-04-04.csv -db dspace -u dspace -p 'fuuu'
+
    + + + +
    dc.contributor.author,cg.creator.id
+"Tohme, Joseph M.",Joe Tohme: 0000-0003-2765-7101
+
    + diff --git a/docs/404.html b/docs/404.html index 76faf9586..c5ceba8cd 100644 --- a/docs/404.html +++ b/docs/404.html @@ -26,7 +26,7 @@ - + @@ -41,9 +41,6 @@ - - - diff --git a/docs/categories/index.html b/docs/categories/index.html index 5d7674d69..bc43e5959 100644 --- a/docs/categories/index.html +++ b/docs/categories/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/notes/index.html b/docs/categories/notes/index.html index da26dd865..bcdde0f1d 100644 --- a/docs/categories/notes/index.html +++ b/docs/categories/notes/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/notes/page/2/index.html b/docs/categories/notes/page/2/index.html index 45a35a031..33f67a914 100644 --- a/docs/categories/notes/page/2/index.html +++ b/docs/categories/notes/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/notes/page/3/index.html b/docs/categories/notes/page/3/index.html index 26fe91717..005ff7452 100644 --- a/docs/categories/notes/page/3/index.html +++ b/docs/categories/notes/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/notes/page/4/index.html b/docs/categories/notes/page/4/index.html index 4abeb00d4..bd382a091 100644 --- a/docs/categories/notes/page/4/index.html +++ b/docs/categories/notes/page/4/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/page/2/index.html b/docs/categories/page/2/index.html index ab0d441a8..02c92e94f 100644 --- a/docs/categories/page/2/index.html +++ b/docs/categories/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/page/3/index.html b/docs/categories/page/3/index.html index 54cf84562..9091180f1 100644 --- a/docs/categories/page/3/index.html +++ b/docs/categories/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/categories/page/4/index.html b/docs/categories/page/4/index.html index 23a8aa536..6de1ad70b 100644 --- a/docs/categories/page/4/index.html +++ b/docs/categories/page/4/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/cgiar-library-migration/index.html b/docs/cgiar-library-migration/index.html index c969bf0b4..e58f9b9b3 100644 --- a/docs/cgiar-library-migration/index.html +++ b/docs/cgiar-library-migration/index.html @@ -29,7 +29,7 @@ - + diff --git a/docs/index.html b/docs/index.html index fb38d7470..5b1a32c7b 100644 --- a/docs/index.html +++ b/docs/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/page/2/index.html b/docs/page/2/index.html index b4960c58a..5756e25a1 100644 --- a/docs/page/2/index.html +++ b/docs/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/page/3/index.html b/docs/page/3/index.html index c77e52b52..1b3e779c7 100644 --- a/docs/page/3/index.html +++ b/docs/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/page/4/index.html b/docs/page/4/index.html index be05cf557..2411236c4 100644 --- a/docs/page/4/index.html +++ b/docs/page/4/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/posts/index.html b/docs/posts/index.html index 62b8da683..8e5d27ba6 100644 --- a/docs/posts/index.html +++ b/docs/posts/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/posts/page/2/index.html b/docs/posts/page/2/index.html index 846551f9b..bad683ae1 100644 --- a/docs/posts/page/2/index.html +++ b/docs/posts/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/posts/page/3/index.html b/docs/posts/page/3/index.html index a6989dde0..715587427 100644 --- a/docs/posts/page/3/index.html +++ b/docs/posts/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/posts/page/4/index.html b/docs/posts/page/4/index.html index 3161cb4bf..d75192745 100644 --- a/docs/posts/page/4/index.html +++ b/docs/posts/page/4/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 987657c03..bdb06e5b2 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -9,7 +9,7 @@ https://alanorth.github.io/cgspace-notes/2018-03/ - 2018-03-28T22:01:13+03:00 + 2018-04-04T15:14:21+03:00 diff --git a/docs/tags/index.html b/docs/tags/index.html index fd0e7be7c..eb0a9919d 100644 --- a/docs/tags/index.html +++ b/docs/tags/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/notes/index.html b/docs/tags/notes/index.html index 7503998d6..f4f016301 100644 --- a/docs/tags/notes/index.html +++ b/docs/tags/notes/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/notes/page/2/index.html b/docs/tags/notes/page/2/index.html index a0764a84e..9df6ade1d 100644 --- a/docs/tags/notes/page/2/index.html +++ b/docs/tags/notes/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/notes/page/3/index.html b/docs/tags/notes/page/3/index.html index cc28585c3..d49aaa68f 100644 --- a/docs/tags/notes/page/3/index.html +++ b/docs/tags/notes/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/notes/page/4/index.html b/docs/tags/notes/page/4/index.html index 06702dbe0..4b0863b6d 100644 --- a/docs/tags/notes/page/4/index.html +++ b/docs/tags/notes/page/4/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/page/2/index.html b/docs/tags/page/2/index.html index 50be4a9b8..fefdbe87e 100644 --- a/docs/tags/page/2/index.html +++ b/docs/tags/page/2/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/page/3/index.html b/docs/tags/page/3/index.html index ebb3ab357..23f2ea6eb 100644 --- a/docs/tags/page/3/index.html +++ b/docs/tags/page/3/index.html @@ -26,7 +26,7 @@ - + diff --git a/docs/tags/page/4/index.html b/docs/tags/page/4/index.html index 816f1149f..62d597d52 100644 --- a/docs/tags/page/4/index.html +++ b/docs/tags/page/4/index.html @@ -26,7 +26,7 @@ - +