CGSpace Notes

Documenting day-to-day work on the CGSpace repository.

June, 2018

by Alan Orth in  

2018-06-04

  • Test the DSpace 5.8 module upgrades from Atmire (#378)
    • There seems to be a problem with the CUA and L&R versions in pom.xml because they are using SNAPSHOT and it doesn’t build
  • I added the new CCAFS Phase II Project Tag PII-FP1_PACCA2 and merged it into the 5_x-prod branch (#379)
  • I proofed and tested the ILRI author corrections that Peter sent back to me this week:
$ ./fix-metadata-values.py -i /tmp/2018-05-30-Correct-660-authors.csv -db dspace -u dspace -p 'fuuu' -f dc.contributor.author -t correct -m 3 -n
  • I think a sane proofing workflow in OpenRefine is to apply the custom text facets for check/delete/remove and illegal characters that I developed in March, 2018
  • Time to index ~70,000 items on CGSpace:
$ time schedtool -D -e ionice -c2 -n7 nice -n19 [dspace]/bin/dspace index-discovery -b                                  

real    74m42.646s
user    8m5.056s
sys     2m7.289s

2018-06-06

  • It turns out that I needed to add a server block for atmire.com-snapshots to my Maven settings, so now the Atmire code builds
  • Now Maven and Ant run properly, but I’m getting SQL migration errors in dspace.log after starting Tomcat
  • I’ve updated my ticket on Atmire’s bug tracker: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560

2018-06-07

  • Proofing 200 IITA records on DSpace Test for Sisay: IITA_Junel_06 (10568⁄95391)
    • Mispelled authorship type: CGAIR single center should be: CGIAR single centre
    • I see some encoding errors in author affiliations, for example:
    • Universidade de SÆo Paulo
    • Institut National des Recherches Agricoles du B nin
    • Centre de Coop ration Internationale en Recherche Agronomique pour le D veloppement
    • Institut des Recherches Agricoles du B nin
    • Institut des Savannes, C te d’ Ivoire
    • Institut f r Pflanzenpathologie und Pflanzenschutz der Universit t, Germany
    • Projet de Gestion des Ressources Naturelles, B nin
    • Universit t Hannover
    • Universit F lix Houphouet-Boigny
  • I uploaded fixes for all those now, but I will continue with the rest of the data later
  • Regarding the SQL migration errors, Atmire told me I need to run some migrations manually in PostgreSQL:
delete from schema_version where version = '5.6.2015.12.03.2';
update schema_version set version = '5.6.2015.12.03.2' where version = '5.5.2015.12.03.2';
update schema_version set version = '5.8.2015.12.03.3' where version = '5.5.2015.12.03.3';
  • And then I need to ignore the ignored ones:
$ ~/dspace/bin/dspace database migrate ignored
  • Now DSpace starts up properly!
  • Gabriela from CIP got back to me about the author names we were correcting on CGSpace
  • I did a quick sanity check on them and then did a test import with my fix-metadata-value.py script:
$ ./fix-metadata-values.py -i /tmp/2018-06-08-CIP-Authors.csv -db dspace -u dspace -p 'fuuu' -f dc.contributor.author -t correct -m 3
  • I will apply them on CGSpace tomorrow I think…

2018-06-09

  • It’s pretty annoying, but the JVM monitoring for Munin was never set up when I migrated DSpace Test to its new server a few months ago
  • I ran the tomcat and munin-node tags in Ansible again and now the stuff is all wired up and recording stats properly
  • I applied the CIP author corrections on CGSpace and DSpace Test and re-ran the Discovery indexing

2018-06-10

  • I spent some time removing the Atmire Metadata Quality Module (MQM) from the proposed DSpace 5.8 changes
  • After removing all code mentioning MQM, mqm, metadata-quality, batchedit, duplicatechecker, etc, I think I got most of it removed, but there is a Spring error during Tomcat startup:
 INFO [org.dspace.servicemanager.DSpaceServiceManager] Shutdown DSpace core service manager
Failed to startup the DSpace Service Manager: failure starting up spring service manager: Error creating bean with name 'org.dspace.servicemanager.spring.DSpaceBeanPostProcessor#0' defined in class path resource [spring/spring-dspace-applicationContext.xml]: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.dspace.servicemanager.config.DSpaceConfigurationService]: : Cannot find class [com.atmire.dspace.discovery.ItemCollectionPlugin] for bean with name 'itemCollectionPlugin' defined in file [/home/aorth/dspace/config/spring/api/discovery.xml];
  • I can fix this by commenting out the ItemCollectionPlugin line of discovery.xml, but from looking at the git log I’m not actually sure if that is related to MQM or not
  • I will have to ask Atmire