Add notes for 2020-07-23

content/posts/2020-07.md

@@ -648,5 +648,27 @@ OpenRXV harvesting bot; https://github.com/ilri/OpenRXV
   - [COUNTER_Robots_list.json: Remove anchors from okhttp](https://github.com/atmire/COUNTER-Robots/pull/33)
 - I will update the agent patterns on the CGSpace `5_x-prod` and 6.x branches
 - Make some changes to the Bootstrap CSS and HTML configuration to improve readability and style on the CG Core v2 metadata reference guide and send a pull request to Marie ([#29](https://github.com/AgriculturalSemantics/cg-core/pull/29))
+- The `solr-upgrade-statistics-6x` tool keeps crashing due to memory issues when processing 2018 stats
+  - I reduced the number of records per batch from 10,000 to 5,000 and increased the memory to 3072 and it still crashes...
+  - I reduced the number of records per batch to 1,000 and it works, but still took like twenty minutes before it even started!
+  - Eventually after processing a few million records it crashed with this error:
+
+```
+Exception: Error while creating field 'p_group_id{type=uuid,properties=indexed,stored,multiValued}' from value '10'
+org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error while creating field 'p_group_id{type=uuid,properties=indexed,stored,multiValued}' from value '10'
+```
+
+- There were four records so I deleted them:
+
+```
+$ curl -s "http://localhost:8081/solr/statistics-2018/update?softCommit=true" -H "Content-Type: text/xml" --data-binary '<delete><query>id:10</query></delete>'
+```
+
+- Meeting with Moayad and Peter and Abenet to discuss the latest AReS changes
+
+## 2020-07-23
+
+- I closed all issues in the [OpenRXV](https://github.com/ilri/OpenRXV/issues) and [AReS](https://github.com/ilri/AReS/issues) GitHub repositories with screenshots so that Moayad can use them for his invoice
+- The statistics-2018 core always crashes with the same error even after I deleted the "id:10" records...
 
 <!-- vim: set sw=2 ts=2: -->