CGSpace Notes

August, 2016

Posted on
#notes

2016-08-01

  • Add updated distribution license from Sisay (#259)
  • Play with upgrading Mirage 2 dependencies in bower.json because most are several versions of out date
  • Bootstrap is at 3.3.0 but upstream is at 3.3.7, and upgrading to anything beyond 3.3.1 breaks glyphicons and probably more
  • bower stuff is a dead end, waste of time, too many issues
  • Anything after Bootstrap 3.3.1 makes glyphicons disappear (HTTP 404 trying to access from incorrect path of fonts)
  • Start working on DSpace 5.1 → 5.5 port:
$ git checkout -b 55new 5_x-prod
$ git reset --hard ilri/5_x-prod
$ git rebase -i dspace-5.5
  • Lots of conflicts that don’t make sense (ie, shouldn’t conflict!)
  • This file in particular conflicts almost 10 times: dspace/modules/xmlui-mirage2/src/main/webapp/themes/CGIAR/styles/_style.scss
  • Checking out a clean branch at 5.5 and cherry-picking our commits works where that file would normally have a conflict
  • Seems to be related to merge commits
  • git rebase --preserve-merges doesn’t seem to help
  • Eventually I just turned on git rerere and solved the conflicts and completed the 403 commit rebase
  • The 5.5 code now builds but doesn’t run (white page in Tomcat)

2016-08-02

  • Ask Atmire for help with DSpace 5.5 issue
  • Vanilla DSpace 5.5 deploys and runs fine
  • Playing with DSpace in Ubuntu 16.04 and Tomcat 7
  • Everything is still fucked up, even vanilla DSpace 5.5

2016-08-04

  • Ask on DSpace mailing list about duplicate authors, Discovery and author text values
  • Atmire responded with some new DSpace 5.5 ready versions to try for their modules

2016-08-05

  • Fix item display incorrectly displaying Species when Breeds were present (#260)
  • Experiment with fixing more authors, like Delia Grace:
dspacetest=# update metadatavalue set authority='0b4fcbc1-d930-4319-9b4d-ea1553cca70b', confidence=600 where metadata_field_id=3 and text_value='Grace, D.';

2016-08-06

  • Finally figured out how to remove “View/Open” and “Bitstreams” from the item view

2016-08-07

  • Start working on Ubuntu 16.04 Ansible playbook for Tomcat 8, PostgreSQL 9.5, Oracle 8, etc

2016-08-08

  • Still troubleshooting Atmire modules on DSpace 5.5
  • Vanilla DSpace 5.5 works on Tomcat 7…
  • Ooh, and vanilla DSpace 5.5 works on Tomcat 8 with Java 8!
  • Some notes about setting up Tomcat 8, since it’s new on this machine…
  • Install latest Oracle Java 8 JDK
  • Create setenv.sh in Tomcat 8 libexec/bin directory: 
CATALINA_OPTS="-Djava.awt.headless=true -Xms3072m -Xmx3072m -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -Dfile.encoding=UTF-8"
CATALINA_OPTS="$CATALINA_OPTS -Djava.library.path=/opt/brew/Cellar/tomcat-native/1.2.8/lib"

JRE_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home
  • Edit Tomcat 8 server.xml to add regular HTTP listener for solr
  • Symlink webapps:
$ rm -rf /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/ROOT
$ ln -sv ~/dspace/webapps/xmlui /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/ROOT
$ ln -sv ~/dspace/webapps/oai /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/oai
$ ln -sv ~/dspace/webapps/jspui /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/jspui
$ ln -sv ~/dspace/webapps/rest /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/rest
$ ln -sv ~/dspace/webapps/solr /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/solr

2016-08-09

2016-08-10

  • Turns out DSpace 5.x isn’t ready for Tomcat 8: https://jira.duraspace.org/browse/DS-3092
  • So we’ll need to use Tomcat 7 + Java 8 on Ubuntu 16.04
  • More work on the Ansible stuff for this, allowing Tomcat 7 to use Java 8
  • Merge pull request for fixing the type Discovery index to use dc.type (#262)
  • Merge pull request for removing “Bitstream” text from item display, as it confuses users and isn’t necessary (#263)

2016-08-11

  • Finally got DSpace (5.5) running on Ubuntu 16.04, Tomcat 7, Java 8, PostgreSQL 9.5 via the updated Ansible stuff

DSpace 5.5 on Ubuntu 16.04, Tomcat 7, Java 8, PostgreSQL 9.5

2016-08-14

2016-08-15

ExpressJS running behind nginx

2016-08-16

  • Troubleshoot Paramiko connection issues with Ansible on ILRI servers: #37
  • Turns out we need to add some MACs to our sshd_config: hmac-sha2-512,hmac-sha2-256
  • Update DSpace Test’s Java to version 8 to start testing this configuration (seeing as Solr recommends it)

2016-08-17

  • More work on Let’s Encrypt stuff for Ansible roles
  • Yesterday Atmire responded about DSpace 5.5 issues and asked me to try the dspace database repair command to fix Flyway issues
  • The dspace database command doesn’t even run: https://gist.github.com/alanorth/c43c8d89e8df346d32c0ee938be90cd5
  • Oops, it looks like the missing classes causing dspace database to fail were coming from the old ~/dspace/config/spring folder
  • After removing the spring folder and running ant install again, dspace database works
  • I see there are missing and pending Flyway migrations, but running dspace database repair and dspace database migrate does nothing: https://gist.github.com/alanorth/41ed5abf2ff32d8ac9eedd1c3d015d70

2016-08-18

  • Fix “CONGO,DR” country name in input-forms.xml (#264)
  • Also need to fix existing records using the incorrect form in the database:
dspace=# update metadatavalue set text_value='CONGO, DR' where resource_type_id=2 and metadata_field_id=228 and text_value='CONGO,DR';
  • I asked a question on the DSpace mailing list about updating “preferred” forms of author names from ORCID

2016-08-21

  • A few days ago someone on the DSpace mailing list suggested I try dspace dsrun org.dspace.authority.UpdateAuthorities to update preferred author names from ORCID
  • If you set auto-update-items=true in dspace/config/modules/solrauthority.cfg it is supposed to update records it finds automatically
  • I updated my name format on ORCID and I’ve been running that script a few times per day since then but nothing has changed
  • Still troubleshooting Atmire modules on DSpace 5.5
  • I sent them some new verbose logs: https://gist.github.com/alanorth/700748995649688148ceba89d760253e