I exported it, but a quick run through the csv-metadata-quality tool shows that there are some low-hanging fruits we can fix before I send him the data I will limit the scope to the titles, regions, subregions, and river basins for now to manually fix some non-breaking spaces (U+00A0) there that would otherwise be removed by the csv-metadata-quality script’s “unneccesary Unicode” fix:" />
I exported it, but a quick run through the csv-metadata-quality tool shows that there are some low-hanging fruits we can fix before I send him the data I will limit the scope to the titles, regions, subregions, and river basins for now to manually fix some non-breaking spaces (U+00A0) there that would otherwise be removed by the csv-metadata-quality script’s “unneccesary Unicode” fix:"/>
<li><p>Udana from IWMI asked me for a CSV export of their community on CGSpace</p>
<ul>
<li>I exported it, but a quick run through the <code>csv-metadata-quality</code> tool shows that there are some low-hanging fruits we can fix before I send him the data</li>
<li><p>I will limit the scope to the titles, regions, subregions, and river basins for now to manually fix some non-breaking spaces (U+00A0) there that would otherwise be removed by the csv-metadata-quality script’s “unneccesary Unicode” fix:</p>
<li><p>Then I replace them in vim with <code>:% s/\%u00a0/ /g</code> because I can’t figure out the correct sed syntax to do it directly from the pipe above</p></li>
<li><p>I uploaded those to CGSpace and then re-exported the metadata</p></li>
<li><p>Now that I think about it, I shouldn’t be removing non-breaking spaces (U+00A0), I should be replacing them with normal spaces!</p></li>
<li><p>I modified the script so it replaces the non-breaking spaces instead of removing them</p></li>
<li><p>Then I ran the csv-metadata-quality script to do some general cleanups (though I temporarily commented out the whitespace fixes because it was too many thousands of rows):</p>
<li><p>Release <ahref="https://github.com/ilri/csv-metadata-quality/releases/tag/v0.3.1">version 0.3.1 of the csv-metadata-quality script</a> with the non-breaking spaces change</p></li>
<li><p>Create an account for Bioversity’s ICT consultant Francesco on DSpace Test:</p>
<pre><code>$ dspace user -a -m blah@mail.it -g Francesco -s Vernocchi -p 'fffff'
</code></pre></li>
<li><p>Email Francesca and Carol to ask for follow up about the test upload I did on 2019-09-21</p>
<ul>
<li>I suggested that if they still want to do value addition of those records (like adding countries, regions, etc) that they could maybe do it after we migrate the records to CGSpace</li>
<li>Carol responded to tell me where to map the items with type Brochure, Journal Item, and Thesis, so I applied them to the <ahref="https://dspacetest.cgiar.org/handle/10568/103688">collection on DSpace Test</a></li>
<li>Hector from CCAFS responded about my feedback of their CLARISA API
<ul>
<li>He made some fixes to the metadata values they are using based on my feedback and said they are happy if we would use it</li>
</ul></li>
<li>Gabriela from CIP asked me if it was possible to generate an RSS feed of items that have the CIP subject “POTATO AGRI-FOOD SYSTEMS”
<ul>
<li>I notice that there is a similar term “SWEETPOTATO AGRI-FOOD SYSTEMS” so I had to come up with a way to exclude that using the boolean “AND NOT” in the <ahref="https://cgspace.cgiar.org/open-search/discover?query=cipsubject:POTATO%20AGRI%E2%80%90FOOD%20SYSTEMS%20AND%20NOT%20cipsubject:SWEETPOTATO%20AGRI%E2%80%90FOOD%20SYSTEMS&scope=10568/51671&sort_by=3&order=DESC">OpenSearch query</a></li>
<li>Again, the <code>sort_by=3</code> parameter is the accession date, as configured in <code>dspace.cfg</code></li>
<li>Continue working on identifying duplicates in the Bioversity migration
<ul>
<li>I have been recording the originals and duplicates in a spreadsheet so I can map them later</li>
<li>For now I am just reconciling any incorrect or missing metadata in the original items on CGSpace, deleting the duplicate from DSpace Test, and mapping the original to the correct place on CGSpace</li>
<li>So far I have deleted thirty duplicates and mapped fourteen</li>
</ul></li>
<li>Run all system updates on DSpace Test (linode19) and reboot the server</li>
