Update notes

content/posts/2018-05.md

@@ -356,3 +356,12 @@ dspace=# select email, netid from eperson where email not like '%cgiar.org%' and
 - It might be possible to adjust some things, but eventually we'll need a larger VPS instance
 - For some reason there are no JVM stats in Munin, ugh
 - Run all system updates on DSpace Test and reboot it
+- I generated a list of CIFOR duplicates from the `CIFOR_May_9` collection using the Atmire MQM module and then dumped the HTML source so I could process it for sending to Vika
+- I used grep to filter all relevant handle lines from the HTML source then used sed to insert a newline before each "Item1" line (as the duplicates are grouped like Item1, Item2, Item3 for each set of duplicates):
+
+```
+$ grep -E 'aspect.duplicatechecker.DuplicateResults.field.del_handle_[0-9]{1,3}_Item' ~/Desktop/https\ _dspacetest.cgiar.org_atmire_metadata-quality_duplicate-checker.html > ~/cifor-duplicates.txt
+$ sed 's/.*Item1.*/\n&/g' ~/cifor-duplicates.txt > ~/cifor-duplicates-cleaned.txt
+```
+
+- I told Vika to look through the list manually and indicate which ones are indeed duplicates that we should delete, and which ones to map to CIFOR's collection