2020-06-01
- I tried to run the
AtomicStatisticsUpdateCLI
CUA migration script on DSpace Test (linode26) again and it is still going very slowly and has tons of errors like I noticed yesterday
- I sent Atmire the dspace.log from today and told them to log into the server to debug the process
- In other news, I checked the statistics API on DSpace 6 and it’s working
- I tried to build the OAI registry on the freshly migrated DSpace 6 on DSpace Test and I get an error:
$ dspace oai import -c
OAI 2.0 manager action started
Loading @mire database changes for module MQM
Changes have been processed
Clearing index
Index cleared
Using full import.
Full import
java.lang.NullPointerException
at org.dspace.xoai.app.XOAI.willChangeStatus(XOAI.java:438)
at org.dspace.xoai.app.XOAI.index(XOAI.java:368)
at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227)
at org.dspace.xoai.app.XOAI.index(XOAI.java:134)
at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
2020-06-02
- I noticed that I was able to do a partial OAI import (ie, without
-c
)
- Then I tried to clear the OAI Solr core and import, but I get the same error:
$ curl http://localhost:8080/solr/oai/update -H "Content-type: text/xml" --data-binary '<delete><query>*:*</query></delete>'
$ curl http://localhost:8080/solr/oai/update -H "Content-type: text/xml" --data-binary '<commit />'
$ ~/dspace63/bin/dspace oai import
OAI 2.0 manager action started
...
There are no indexed documents, using full import.
Full import
java.lang.NullPointerException
at org.dspace.xoai.app.XOAI.willChangeStatus(XOAI.java:438)
at org.dspace.xoai.app.XOAI.index(XOAI.java:368)
at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227)
at org.dspace.xoai.app.XOAI.index(XOAI.java:143)
at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
- I found a bug report on DSpace Jira describing this issue affecting someone else running DSpace 6.3
- They suspect it has to do with the item having some missing group names in its authorization policies
- I added some debugging to
dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java
to print the Handle of the item that causes the crash and then I looked at its authorization policies
- Indeed there are some blank group names:
- The same item on CGSpace (DSpace 5.8) also has groups with no name:
- I added some debugging and found exactly where this happens
- As it turns out we can just check if the group policy is null there and it allows the OAI import to proceed
- Aaaaand as it turns out, this was fixed in
dspace-6_x
in 2018 after DSpace 6.3 was released (see DS-4019), so that was a waste of three hours.
- I cherry picked 150e83558103ed7f50e8f323b6407b9cbdf33717 into our current
6_x-dev-atmire-modules
branch