CGSpace Notes

2021-11-02

• I experimented with manually sharding the Solr statistics on DSpace Test
• First I exported all the 2019 stats from CGSpace:

$ ./run.sh -s http://localhost:8081/solr/statistics -f 'time:2019-*' -a export -o statistics-2019.json -k uid
$ zstd statistics-2019.json

• Then on DSpace Test I created a statistics-2019 core with the same instance dir as the main statistics core (as illustrated in the DSpace docs)

$ mkdir -p /home/dspacetest.cgiar.org/solr/statistics-2019/data
# create core in Solr admin
$ curl -s "http://localhost:8081/solr/statistics/update?softCommit=true" -H "Content-Type: text/xml" --data-binary "<delete><query>time:2019-*</query></delete>"
$ ./run.sh -s http://localhost:8081/solr/statistics-2019 -a import -o statistics-2019.json -k uid

• The key thing above is that you create the core in the Solr admin UI, but the data directory must already exist so you have to do that first in the file system
• I restarted the server after the import was done to see if the cores would come back up OK
  • I remember last time I tried this the manually created statistics cores didn’t come back up after I rebooted, but this time they did

2021-11-03

• While inspecting the stats for the new statistics-2019 shard on DSpace Test I noticed that I can’t find any stats via the DSpace Statistics API for an item that should have some
  • I checked on CGSpace’s and I can’t find them there either, but I see them in Solr when I query in the admin UI
  • I need to debug that, but it doesn’t seem to be related to the sharding…