CGSpace Notes

Documenting day-to-day work on the CGSpace repository.

June, 2020

2020-06-01

  • I tried to run the AtomicStatisticsUpdateCLI CUA migration script on DSpace Test (linode26) again and it is still going very slowly and has tons of errors like I noticed yesterday
    • I sent Atmire the dspace.log from today and told them to log into the server to debug the process
  • In other news, I checked the statistics API on DSpace 6 and it’s working
  • I tried to build the OAI registry on the freshly migrated DSpace 6 on DSpace Test and I get an error:
$ dspace oai import -c
OAI 2.0 manager action started
Loading @mire database changes for module MQM
Changes have been processed
Clearing index
Index cleared
Using full import.
Full import
java.lang.NullPointerException
        at org.dspace.xoai.app.XOAI.willChangeStatus(XOAI.java:438)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:368)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
        at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:134)
        at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)

2020-06-02

  • I noticed that I was able to do a partial OAI import (ie, without -c)
    • Then I tried to clear the OAI Solr core and import, but I get the same error:
$ curl http://localhost:8080/solr/oai/update -H "Content-type: text/xml" --data-binary '<delete><query>*:*</query></delete>'
$ curl http://localhost:8080/solr/oai/update -H "Content-type: text/xml" --data-binary '<commit />'
$ ~/dspace63/bin/dspace oai import
OAI 2.0 manager action started
...
There are no indexed documents, using full import.
Full import
java.lang.NullPointerException
        at org.dspace.xoai.app.XOAI.willChangeStatus(XOAI.java:438)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:368)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:280)
        at org.dspace.xoai.app.XOAI.indexAll(XOAI.java:227)
        at org.dspace.xoai.app.XOAI.index(XOAI.java:143)
        at org.dspace.xoai.app.XOAI.main(XOAI.java:560)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
  • I found a bug report on DSpace Jira describing this issue affecting someone else running DSpace 6.3
    • They suspect it has to do with the item having some missing group names in its authorization policies
    • I added some debugging to dspace-oai/src/main/java/org/dspace/xoai/app/XOAI.java to print the Handle of the item that causes the crash and then I looked at its authorization policies
    • Indeed there are some blank group names:

Missing group names in DSpace 6.3 item authorization policy

  • The same item on CGSpace (DSpace 5.8) also has groups with no name:

Missing group names in DSpace 5.8 item authorization policy

  • I added some debugging and found exactly where this happens
    • As it turns out we can just check if the group policy is null there and it allows the OAI import to proceed
    • Aaaaand as it turns out, this was fixed in dspace-6_x in 2018 after DSpace 6.3 was released (see DS-4019), so that was a waste of three hours.
    • I cherry picked 150e83558103ed7f50e8f323b6407b9cbdf33717 into our current 6_x-dev-atmire-modules branch