- Megan says there are still some mapped items are not appearing since last week, so I forced a full `index-discovery -b`
- Need to remember to check if the collection has more items (currently 39 on CGSpace, but 118 on the freshly reindexed DSPace Test) tomorrow: https://cgspace.cgiar.org/handle/10568/80731
- Create ticket on Atmire tracker to ask about commissioning them to develop the feature to expose ORCID via REST/OAI: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=510
- According to the [DSpace curation docs](https://wiki.duraspace.org/display/DSDOC5x/Curation+System) the fact that the `requiredmetadata` curation task stops when it finds a missing metadata field is by design
- When ingesting some collections I was getting `java.lang.OutOfMemoryError: GC overhead limit exceeded`, which can be solved by disabling the GC timeout with `-XX:-UseGCOverheadLimit`
- Other times I was getting an error about heap space, so I kept bumping the RAM allocation by 512MB each time (up to 4096m!) it crashed
- This leads to tens of thousands of abandoned files in the assetstore, which need to be cleaned up using `dspace cleanup -v`, or else you'll run out of disk space
- In the end I realized it's better to use submission mode (`-s`) to ingest the community object as a single AIP without its children, followed by each of the collections:
$ for collection in /home/aorth/10947-1/COLLECTION@10947-*; do [dspace]/bin/dspace packager -s -o ignoreHandle=false -t AIP -e aorth@mjanja.ch -p 10947/1 $collection; done
$ for item in /home/aorth/10947-1/ITEM@10947-*; do [dspace]/bin/dspace packager -r -f -u -t AIP -e aorth@mjanja.ch $item; done
```
- Note that in submission mode DSpace ignores the handle specified in `mets.xml` in the zip file, so you need to turn that off with `-o ignoreHandle=false`
- Give feedback to CIFOR about their data quality:
- Suggestion: uppercase dc.subject, cg.coverage.region, and cg.coverage.subregion in your crosswalk so they match CGSpace and therefore can be faceted / reported on easier
- Suggestion: use CGSpace's CRP names (cg.contributor.crp), see: dspace/config/input-forms.xml
- Suggestion: clean up duplicates and errors in funders, perhaps use a controlled vocabulary like ours, see: dspace/config/controlled-vocabularies/dc-description-sponsorship.xml
- Suggestion: use dc.type "Blog Post" instead of "Blog" for your blog post items (we are also adding a "Blog Post" type to CGSpace soon)
- Question: many of your items use dc.document.uri AND cg.identifier.url with the same text value?
- Help Marianne from WLE with an Open Search query to show the latest WLE CRP outputs: https://cgspace.cgiar.org/open-search/discover?query=crpsubject:WATER%2C+LAND+AND+ECOSYSTEMS&sort_by=2&order=DESC
- This uses the webui's item list sort options, see `webui.itemlist.sort-option` in `dspace.cfg`
- The equivalent Discovery search would be: https://cgspace.cgiar.org/discover?filtertype_1=crpsubject&filter_relational_operator_1=equals&filter_1=WATER%2C+LAND+AND+ECOSYSTEMS&submit_apply_filter=&query=&rpp=10&sort_by=dc.date.issued_dt&order=desc
- I ended up running into issues during data cleaning and decided to wipe out the entire community and re-sync DSpace Test assetstore and database from CGSpace rather than waiting for the cleanup task to clean up
- Hours into the re-ingestion I ran into more errors, and had to erase everything and start over _again_!
- Now, no matter what I do I keep getting foreign key errors...
