cgspace-notes/content/posts/2018-07.md

---
title: "July, 2018"
date: 2018-07-01T12:56:54+03:00
author: "Alan Orth"
tags: ["Notes"]
---

## 2018-07-01

- I want to upgrade DSpace Test to DSpace 5.8 so I took a backup of its current database just in case:

```
$ pg_dump -b -v -o --format=custom -U dspace -f dspace-2018-07-01.backup dspace
```

- During the `mvn package` stage on the 5.8 branch I kept getting issues with java running out of memory:

```
There is insufficient memory for the Java Runtime Environment to continue.
```

<!--more-->

- As the machine only has 8GB of RAM, I reduced the Tomcat memory heap from 5120m to 4096m so I could try to allocate more to the build process:

```
$ export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx1024m"
$ mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false -Denv=dspacetest.cgiar.org -P \!dspace-lni,\!dspace-rdf,\!dspace-sword,\!dspace-swordv2 clean package
```

- Then I stopped the Tomcat 7 service, ran the ant update, and manually ran the old and ignored SQL migrations:

```
$ sudo su - postgres
$ psql dspace
...
dspace=# begin;
BEGIN
dspace=# \i Atmire-DSpace-5.8-Schema-Migration.sql
DELETE 0
UPDATE 1
DELETE 1
dspace=# commit
dspace=# \q
$ exit
$ dspace database migrate ignored
```

- After that I started Tomcat 7 and DSpace seems to be working, now I need to tell our colleagues to try stuff and report issues they have

## 2018-07-02

- Discuss AgriKnowledge including our Handle identifier on their harvested items from CGSpace
- They seem to be only interested in Gates-funded outputs, for example: https://www.agriknowledge.org/files/tm70mv21t

## 2018-07-03

- Finally finish with the CIFOR Archive records (a total of 2448):
  - I mapped the 50 items that were duplicates from elsewhere in CGSpace into [CIFOR Archive](https://cgspace.cgiar.org/handle/10568/16702)
  - I did one last check of the remaining 2398 items and found eight who have a `cg.identifier.doi` that links to some URL other than a DOI so I moved those to `cg.identifier.url` and `cg.identifier.googleurl` as appropriate
  - Also, thirteen items had a DOI in their citation, but did not have a `cg.identifier.doi` field, so I added those
  - Then I imported those 2398 items in two batches (to deal with memory issues):

```
$ export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx1024m"
$ dspace metadata-import -e aorth@mjanja.ch -f /tmp/2018-06-27-New-CIFOR-Archive.csv
$ dspace metadata-import -e aorth@mjanja.ch -f /tmp/2018-06-27-New-CIFOR-Archive2.csv
```

- I noticed there are many items that use HTTP instead of HTTPS for their Google Books URL, and some missing HTTP entirely:

```
dspace=# select count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=222 and text_value like 'http://books.google.%';
 count
-------
   785
dspace=# select count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=222 and text_value ~ '^books\.google\..*';
 count
-------
     4
```

- I think I should fix that as well as some other garbage values like "test" and "dspace.ilri.org" etc:

```
dspace=# begin;
dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'http://books.google', 'https://books.google') where resource_type_id=2 and metadata_field_id=222 and text_value like 'http://books.google.%';
UPDATE 785
dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'books.google', 'https://books.google') where resource_type_id=2 and metadata_field_id=222 and text_value ~ '^books\.google\..*';
UPDATE 4
dspace=# update metadatavalue set text_value='https://books.google.com/books?id=meF1CLdPSF4C' where resource_type_id=2 and metadata_field_id=222 and text_value='meF1CLdPSF4C';
UPDATE 1
dspace=# delete from metadatavalue where resource_type_id=2 and metadata_field_id=222 and metadata_value_id in (2299312, 10684, 10700, 996403);
DELETE 4
dspace=# commit;
```

- Testing DSpace 5.8 with PostgreSQL 9.6 and Tomcat 8.5.32 (instead of my usual 7.0.88) and for some reason I get autowire errors on Catalina startup with 8.5.32:

```
03-Jul-2018 19:51:37.272 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [org.dspace.servicemanager.servlet.DSpaceKernelServletContextListener]
 java.lang.RuntimeException: Failure during filter init: Failed to startup the DSpace Service Manager: failure starting up spring service manager: Error creating bean with name 'conversionService' defined in file [/home/aorth/dspace/config/spring/xmlui/spring-dspace-addon-cua-services.xml]: Cannot create inner bean 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a' of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter] while setting bean property 'converters' with key [1]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter.filterConverter; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No matching bean of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
	at org.dspace.servicemanager.servlet.DSpaceKernelServletContextListener.contextInitialized(DSpaceKernelServletContextListener.java:92)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
	at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:629)
	at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1839)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Failed to startup the DSpace Service Manager: failure starting up spring service manager: Error creating bean with name 'conversionService' defined in file [/home/aorth/dspace/config/spring/xmlui/spring-dspace-addon-cua-services.xml]: Cannot create inner bean 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a' of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter] while setting bean property 'converters' with key [1]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter.filterConverter; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No matching bean of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
```

- Gotta check that out later...

## 2018-07-04

- I verified that the autowire error indeed only occurs on Tomcat 8.5, but the application works fine on Tomcat 7
- I have raised this in the [DSpace 5.8 compatibility ticket on Atmire's tracker](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560)
- Abenet wants me to add "United Kingdom government" to the sponsors on CGSpace so I created a ticket to track it ([#381](https://github.com/ilri/DSpace/issues/381))
- Also, Udana wants me to add "Enhancing Sustainability Across Agricultural Systems" to the WLE Phase II research themes so I created a ticket to track that ([#382](https://github.com/ilri/DSpace/issues/382)
- I need to try to finish this DSpace 5.8 business first because I have too many branches with cherry-picks going on right now!

## 2018-07-06

- CCAFS want me to add "PII-FP2_MSCCCAFS" to their Phase II project tags on CGSpace ([#383](https://github.com/ilri/DSpace/issues/383)
- I'll do it in a batch with all the other metadata updates next week

## 2018-07-08

- I was tempted to do the Linode instance upgrade on CGSpace (linode18), but after looking closely at the system backups I noticed that Solr isn't being backed up to S3
- I apparently noticed this—and fixed it!—in [2016-07]({{< relref "2016-07.md" >}}), but it doesn't look like the backup has been updated since then!
- It looks like I added Solr to the `backup_to_s3.sh` script, but that script is not even being used (`s3cmd` is run directly from root's crontab)
- For now I have just initiated a manual S3 backup of the Solr data:

```
# s3cmd sync --delete-removed /home/backup/solr/ s3://cgspace.cgiar.org/solr/
```

- But I need to add this to cron!
- I wonder if I should convert some of the cron jobs to systemd services / timers...
- I sent a note to all our users on Yammer to ask them about possible maintenance on Sunday, July 14th

<!-- vim: set sw=2 ts=2: -->
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00			`---`
			`title: "July, 2018"`
			`date: 2018-07-01T12:56:54+03:00`
			`author: "Alan Orth"`
			`tags: ["Notes"]`
			`---`

			`## 2018-07-01`

Update notes for 2018-07-01 2018-07-01 17:05:01 +02:00			`- I want to upgrade DSpace Test to DSpace 5.8 so I took a backup of its current database just in case:`
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00
			```
			`$ pg_dump -b -v -o --format=custom -U dspace -f dspace-2018-07-01.backup dspace`
			```

Update notes for 2018-07-01 2018-07-01 17:05:01 +02:00			- During the `mvn package` stage on the 5.8 branch I kept getting issues with java running out of memory:
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00
			```
			`There is insufficient memory for the Java Runtime Environment to continue.`
			```

Update notes for 2018-07-01 2018-07-01 17:05:01 +02:00			`<!--more-->`

			`- As the machine only has 8GB of RAM, I reduced the Tomcat memory heap from 5120m to 4096m so I could try to allocate more to the build process:`
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00
			```
			`$ export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx1024m"`
Update notes for 2018-07-01 2018-07-01 17:05:01 +02:00			`$ mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false -Denv=dspacetest.cgiar.org -P \!dspace-lni,\!dspace-rdf,\!dspace-sword,\!dspace-swordv2 clean package`
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00			```

Update notes for 2018-07-01 2018-07-01 17:05:01 +02:00			`- Then I stopped the Tomcat 7 service, ran the ant update, and manually ran the old and ignored SQL migrations:`

			```
			`$ sudo su - postgres`
			`$ psql dspace`
			`...`
			`dspace=# begin;`
			`BEGIN`
			`dspace=# \i Atmire-DSpace-5.8-Schema-Migration.sql`
			`DELETE 0`
			`UPDATE 1`
			`DELETE 1`
			`dspace=# commit`
			`dspace=# \q`
			`$ exit`
			`$ dspace database migrate ignored`
			```

			`- After that I started Tomcat 7 and DSpace seems to be working, now I need to tell our colleagues to try stuff and report issues they have`
Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00
Update notes for 2018-07-02 2018-07-02 16:33:38 +02:00			`## 2018-07-02`

			`- Discuss AgriKnowledge including our Handle identifier on their harvested items from CGSpace`
			`- They seem to be only interested in Gates-funded outputs, for example: https://www.agriknowledge.org/files/tm70mv21t`

Add notes for 2018-07-03 2018-07-03 13:37:30 +02:00			`## 2018-07-03`

			`- Finally finish with the CIFOR Archive records (a total of 2448):`
			`- I mapped the 50 items that were duplicates from elsewhere in CGSpace into [CIFOR Archive](https://cgspace.cgiar.org/handle/10568/16702)`
			- I did one last check of the remaining 2398 items and found eight who have a `cg.identifier.doi` that links to some URL other than a DOI so I moved those to `cg.identifier.url` and `cg.identifier.googleurl` as appropriate
			- Also, thirteen items had a DOI in their citation, but did not have a `cg.identifier.doi` field, so I added those
			`- Then I imported those 2398 items in two batches (to deal with memory issues):`

			```
			`$ export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx1024m"`
			`$ dspace metadata-import -e aorth@mjanja.ch -f /tmp/2018-06-27-New-CIFOR-Archive.csv`
			`$ dspace metadata-import -e aorth@mjanja.ch -f /tmp/2018-06-27-New-CIFOR-Archive2.csv`
			```

			`- I noticed there are many items that use HTTP instead of HTTPS for their Google Books URL, and some missing HTTP entirely:`

			```
			`dspace=# select count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=222 and text_value like 'http://books.google.%';`
			`count`
			`-------`
			`785`
			`dspace=# select count() from metadatavalue where resource_type_id=2 and metadata_field_id=222 and text_value ~ '^books\.google\..';`
			`count`
			`-------`
			`4`
			```

			`- I think I should fix that as well as some other garbage values like "test" and "dspace.ilri.org" etc:`

			```
			`dspace=# begin;`
			`dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'http://books.google', 'https://books.google') where resource_type_id=2 and metadata_field_id=222 and text_value like 'http://books.google.%';`
			`UPDATE 785`
			`dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'books.google', 'https://books.google') where resource_type_id=2 and metadata_field_id=222 and text_value ~ '^books\.google\..*';`
			`UPDATE 4`
			`dspace=# update metadatavalue set text_value='https://books.google.com/books?id=meF1CLdPSF4C' where resource_type_id=2 and metadata_field_id=222 and text_value='meF1CLdPSF4C';`
			`UPDATE 1`
			`dspace=# delete from metadatavalue where resource_type_id=2 and metadata_field_id=222 and metadata_value_id in (2299312, 10684, 10700, 996403);`
			`DELETE 4`
			`dspace=# commit;`
			```

Update notes for 2018-07-03 2018-07-03 19:01:51 +02:00			`- Testing DSpace 5.8 with PostgreSQL 9.6 and Tomcat 8.5.32 (instead of my usual 7.0.88) and for some reason I get autowire errors on Catalina startup with 8.5.32:`

			```
			`03-Jul-2018 19:51:37.272 SEVERE [localhost-startStop-1] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [org.dspace.servicemanager.servlet.DSpaceKernelServletContextListener]`
			java.lang.RuntimeException: Failure during filter init: Failed to startup the DSpace Service Manager: failure starting up spring service manager: Error creating bean with name 'conversionService' defined in file [/home/aorth/dspace/config/spring/xmlui/spring-dspace-addon-cua-services.xml]: Cannot create inner bean 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a' of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter] while setting bean property 'converters' with key [1]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter.filterConverter; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No matching bean of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
			`at org.dspace.servicemanager.servlet.DSpaceKernelServletContextListener.contextInitialized(DSpaceKernelServletContextListener.java:92)`
			`at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)`
			`at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)`
			`at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)`
			`at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)`
			`at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)`
			`at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)`
			`at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:629)`
			`at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1839)`
			`at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)`
			`at java.util.concurrent.FutureTask.run(FutureTask.java:266)`
			`at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)`
			`at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)`
			`at java.lang.Thread.run(Thread.java:748)`
			Caused by: java.lang.RuntimeException: Failed to startup the DSpace Service Manager: failure starting up spring service manager: Error creating bean with name 'conversionService' defined in file [/home/aorth/dspace/config/spring/xmlui/spring-dspace-addon-cua-services.xml]: Cannot create inner bean 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a' of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter] while setting bean property 'converters' with key [1]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter#3f6c3e6a': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$ColumnsConverter.filterConverter; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No matching bean of type [com.atmire.app.xmlui.aspect.statistics.mostpopular.MostPopularConfig$FilterConverter] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
			```

			`- Gotta check that out later...`

Add notes for 2018-07-04 2018-07-04 11:46:02 +02:00			`## 2018-07-04`

			`- I verified that the autowire error indeed only occurs on Tomcat 8.5, but the application works fine on Tomcat 7`
			`- I have raised this in the [DSpace 5.8 compatibility ticket on Atmire's tracker](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560)`
			`- Abenet wants me to add "United Kingdom government" to the sponsors on CGSpace so I created a ticket to track it ([#381](https://github.com/ilri/DSpace/issues/381))`
			`- Also, Udana wants me to add "Enhancing Sustainability Across Agricultural Systems" to the WLE Phase II research themes so I created a ticket to track that ([#382](https://github.com/ilri/DSpace/issues/382)`
			`- I need to try to finish this DSpace 5.8 business first because I have too many branches with cherry-picks going on right now!`

Add notes for 2018-07-08 2018-07-08 14:05:34 +02:00			`## 2018-07-06`

			`- CCAFS want me to add "PII-FP2_MSCCCAFS" to their Phase II project tags on CGSpace ([#383](https://github.com/ilri/DSpace/issues/383)`
			`- I'll do it in a batch with all the other metadata updates next week`

			`## 2018-07-08`

			`- I was tempted to do the Linode instance upgrade on CGSpace (linode18), but after looking closely at the system backups I noticed that Solr isn't being backed up to S3`
			`- I apparently noticed this—and fixed it!—in [2016-07]({{< relref "2016-07.md" >}}), but it doesn't look like the backup has been updated since then!`
			- It looks like I added Solr to the `backup_to_s3.sh` script, but that script is not even being used (`s3cmd` is run directly from root's crontab)
			`- For now I have just initiated a manual S3 backup of the Solr data:`

			```
			`# s3cmd sync --delete-removed /home/backup/solr/ s3://cgspace.cgiar.org/solr/`
			```

			`- But I need to add this to cron!`
			`- I wonder if I should convert some of the cron jobs to systemd services / timers...`
			`- I sent a note to all our users on Yammer to ask them about possible maintenance on Sunday, July 14th`

Add notes for 2018-07-01 2018-07-01 13:34:56 +02:00			`<!-- vim: set sw=2 ts=2: -->`