cgspace-notes/content/posts/2023-09.md

title	date	author	categories
September, 2023	2023-09-02T17:29:36+03:00	Alan Orth	Notes

2023-09-02

• Export CGSpace to check for missing Initiative collection mappings
• Start a harvest on AReS

2023-09-03

• I figured out how to use Altmetric and Dimensions badges in the DSpace Angular frontend
  • It still feels hacky, but using AfterViewInit, and importing the Altmetric embed.js in the component works
  • The style on mobile also needs work...

2023-09-06

• Discussion with Marie about finalizing the output types list on GitHub
  • I did some review and cleanup in preparation for publishing the new list

2023-09-07

• Export CGSpace to start doing a review of the metadata
• First I will start by extracting all items with DOIs, along with some fields I can compare against Crossref:

$ csvgrep -c 'cg.identifier.doi[en_US]' -r 'doi.org' ~/Downloads/2023-09-07-cgspace.csv \
    | csvcut -c 'id,dc.title[en_US],dcterms.issued[en_US],dcterms.available[en_US],cg.issn[en_US],cg.isbn[en_US],cg.volume[en_US],cg.issue[en_US],cg.number[en_US],dcterms.extent[en_US],cg.identifier.doi[en_US],cg.reviewStatus[en_US],cg.isijournal[en_US],dcterms.license[en_US],dcterms.accessRights[en_US],dcterms.type[en_US],dc.identifier.uri[en_US]' \
    > /tmp/2023-09-07-cgspace-dois.csv
$ csvgrep -c 'cg.identifier.doi[en_US]' -r 'doi.org' ~/Downloads/2023-09-07-cgspace.csv | csvcut -c 'cg.identifier.doi[en_US]' | sed 1d > /tmp/2023-09-07-cgspace-dois.txt

• Then I resolved the DOIs from Crossref:

$ ./ilri/crossref_doi_lookup.py -i /tmp/2023-09-07-cgspace-dois.txt -o /tmp/2023-09-07-cgspace-dois-results.csv -e a.orth@cgiar.org

• A user emailed to ask about uploading a 180MB PDF to CGSpace
  • I used GhostScript to try reducing it using the screen, ebook and prepress presets:

$ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/screen -dNOPAUSE -dQUIET -dBATCH -sOutputFile=primer-screen.pdf Primer\ \(digital\)_Climate-\ smart\ and\ regenerative\ agriculture\ in\ climate\ change\ adaptation.pdf 
$ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook -dNOPAUSE -dQUIET -dBATCH -sOutputFile=primer-ebook.pdf Primer\ \(digital\)_Climate-\ smart\ and\ regenerative\ agriculture\ in\ climate\ change\ adaptation.pdf
$ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/prepress -dNOPAUSE -dQUIET -dBATCH -sOutputFile=primer-prepress.pdf Primer\ \(digital\)_Climate-\ smart\ and\ regenerative\ agriculture\ in\ climate\ change\ adaptation.pdf

• The prepress one is 300DPI and looks visually identical to the original, so I proposed that we use that one

2023-09-08

• I did a review of the metadata for our items with DOIs, comparing with data from Crossref
  • I spot checked a handful of issue / online dates and licenses, and saw that Crossref's dates are always more accurate than ours when they differ
  • I also filled in some missing volumes, issues, ISSNs, and extents
  • This results in 14,000 changes to existing items, which will take several days to import unfortunately
  • After eight hours the first file is only about 2/3 finished... sigh
• Meet with Peter to discuss changes to the DSpace 7 test
  • Minor updates to submission forms and some new ideas for the home page and item page
• I figured out how to use a themed home page component and add a cards UI to our CGSpace theme

2023-09-09

• I can't believe that almost 18 hours later the first CSV import with 5,000 changes is not done...
• Run all system updates on CGSpace and reboot it, as it had been two months since the last time

2023-09-10

• Minor work on the DSpace 7 home page

2023-09-11

• Export CGSpace to check for missing Initiative collection mappings
• Start a harvest on AReS

2023-09-12

• Minor work on DSpace 7 home page
• Minor work on CG Core types
  • I published a new HTML version of the updated IPtypes and archived the current version as v2.0.0 so we can still reference it

2023-09-13

• Stefano reminded me about the updated OAI MODS mappings on CGSpace so I re-applied them on DSpace Test and updated the OAI index so he could confirm
  • Now I'm ready to put it on CGSpace if he confirms
• I created a basic theme for CIP on DSpace 7
  • While doing that I noticed that a bunch of CIP bitstreams didn't have the latest 500px thumbnails so I re-ran filter-media on a handful of their collections
• I had two occurrences of an OOM kill of the Tomcat 9 java process on DSpace 7 test tonight
  • Once while doing a Discovery index, the other while doing filter media

2023-09-15

• Discuss issues with the Altmetric API with the Altmetric support team
  • Apparently we can use a different API, the Explorer API, since we already have access to the Explorer dashboard
• I reduced the Solr heap size on DSpace 7 from 3GB to 2GB
  • Apparentlty I already did this from 4GB to 3GB a few months ago
  • The Solr admin interface was showing Solr taking ~1GB of RAM so I think this should be safe
• Mark on DSpace Slack said he uses PM2's --max-memory-restart so the processes restart when they hit the limit
  • Also, he said he had to reduce cache:serverSide:botCache:max from 1000 to 500 to cache less SSR pages in memory
• I decided to try deploying DSpace 7 Test on a Hetzner server with 64GB RAM, 6 CPUs, and 2x512GB NVMe SSD

2023-09-16

• Export CGSpace to check for missing Initiative collection mappings
• Start a harvest on AReS
• Configure the privacy policy page on DSpace 7 using a themed component with the text from our DSpace 6 site
• I realized that for all my custom Angular components I should be using routerLink instead of href when I am constructing links
  • The routerLink routes within the single page application and saves state, while the href reloads the page
  • Using the routerLink way is faster and results in less flashing and jumping in the page when navigating
  • See: https://stackoverflow.com/a/61588147