CGSpace Notes

2021-01-03

• Peter notified me that some filters on AReS were broken again
  • It’s the same issue with the field names getting .keyword appended to the end that I already filed an issue on OpenRXV about last month
  • I fixed the broken filters (careful to not edit any others, lest they break too!)

2020-12-01

• Atmire responded about the issue with duplicate data in our Solr statistics
  • They noticed that some records in the statistics-2015 core haven’t been migrated with the AtomicStatisticsUpdateCLI tool yet and assumed that I haven’t migrated any of the records yet
  • That’s strange, as I checked all ten cores and 2015 is the only one with some unmigrated documents, as according to the cua_version field
  • I started processing those (about 411,000 records):

Notes about the DSpace 6 upgrade on CGSpace in 2020-11.

2020-11-01

• Continue with processing the statistics-2019 Solr core with the AtomicStatisticsUpdateCLI tool on DSpace Test
  • So far we’ve spent at least fifty hours to process the statistics and statistics-2019 core… wow.

2020-10-06

• Add tests for the new /items POST handlers to the DSpace 6.x branch of my dspace-statistics-api
  • It took a bit of extra work because I had to learn how to mock the responses for when Solr is not available
  • Tag and release version 1.3.0 on GitHub: https://github.com/ilri/dspace-statistics-api/releases/tag/v1.3.0
• Trying to test the changes Atmire sent last week but I had to re-create my local database from a recent CGSpace dump
  • During the FlywayDB migration I got an error:

2020-09-02

• Replace Marissa van Epp for Rhys Bucknall in the CCAFS groups on CGSpace because Marissa no longer works at CCAFS
• The AReS Explorer hasn’t updated its index since 2020-08-22 when I last forced it
  • I restarted it again now and told Moayad that the automatic indexing isn’t working
• Add Alliance of Bioversity International and CIAT to affiliations on CGSpace
• Abenet told me that the general search text on AReS doesn’t get reset when you use the “Reset Filters” button
  • I filed a bug on OpenRXV: https://github.com/ilri/OpenRXV/issues/39
• I filed an issue on OpenRXV to make some minor edits to the admin UI: https://github.com/ilri/OpenRXV/issues/40

2020-08-02

• I spent a few days working on a Java-based curation task to tag items with ISO 3166-1 Alpha2 country codes based on their cg.coverage.country text values
  • It looks up the names in ISO 3166-1 first, and then in our CGSpace countries mapping (which has five or so of Peter’s preferred “display” country names)
  • It implements a “force” mode too that will clear existing country codes and re-tag everything
  • It is class based so I can easily add support for other vocabularies, and the technique could even be used for organizations with mappings to ROR and Clarisa…

2020-07-01

• A few users noticed that CGSpace wasn’t loading items today, item pages seem blank
  • I looked at the PostgreSQL locks but they don’t seem unusual
  • I guess this is the same “blank item page” issue that we had a few times in 2019 that we never solved
  • I restarted Tomcat and PostgreSQL and the issue was gone
• Since I was restarting Tomcat anyways I decided to redeploy the latest changes from the 5_x-prod branch and I added a note about COVID-19 items to the CGSpace frontpage at Peter’s request

2020-06-01

• I tried to run the AtomicStatisticsUpdateCLI CUA migration script on DSpace Test (linode26) again and it is still going very slowly and has tons of errors like I noticed yesterday
  • I sent Atmire the dspace.log from today and told them to log into the server to debug the process
• In other news, I checked the statistics API on DSpace 6 and it’s working
• I tried to build the OAI registry on the freshly migrated DSpace 6 on DSpace Test and I get an error:

2020-05-02

• Peter said that CTA is having problems submitting an item to CGSpace
  • Looking at the PostgreSQL stats it seems to be the same issue that Tezira was having last week, as I see the number of connections in ‘idle in transaction’ and ‘waiting for lock’ state are increasing again
  • I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2.11, and there were some bugs related to transactions fixed in 42.2.12 (which I had updated in the Ansible playbooks, but not deployed yet)