Add notes for 2020-07-22

content/posts/2020-07.md

@@ -593,5 +593,60 @@ Removing unnecessary Unicode (U+200B): Agencia de Servicios a la Comercializaci
 - I had previously thought these were fixed by setting the `font-family` on the elements, but it doesn't appear to be working now
   - I filed a ticket with Atmire to ask them to use the HTML elements instead, as their code already uses those elsewhere
   - I don't want to go back to using the large webfonts with CSS because the SVG + JS method saves us ~140KiB and causes at least three fewer network requests
+- I started processing the 2019 stats in a batch of 1 million on DSpace Test:
+
+```
+$ export JAVA_OPTS='-Dfile.encoding=UTF-8 -Xmx2048m'
+$ chrt -b 0 dspace solr-upgrade-statistics-6x -n 1000000 -i statistics-2019
+...
+        *** Statistics Records with Legacy Id ***
+
+           6,359,966    Bistream View
+           2,204,775    Item View
+             139,266    Community View
+             131,234    Collection View
+             948,529    Community Search
+             593,974    Collection Search
+           1,682,818    Unexpected Type & Full Site
+        --------------------------------------
+          12,060,562    TOTAL
+```
+
+- The statistics-2019 finished processing after about 9 hours so I started the 2018 ones:
+
+```
+$ export JAVA_OPTS='-Dfile.encoding=UTF-8 -Xmx2048m'
+$ chrt -b 0 dspace solr-upgrade-statistics-6x -n 1000000 -i statistics-2018
+        *** Statistics Records with Legacy Id ***
+
+           3,684,394    Bistream View
+           2,183,032    Item View
+             131,222    Community View
+              79,348    Collection View
+             345,529    Collection Search
+             322,223    Community Search
+             874,107    Unexpected Type & Full Site
+        --------------------------------------
+           7,619,855    TOTAL
+```
+
+- Moayad finally made OpenRXV use a unique user agent:
+
+```
+OpenRXV harvesting bot; https://github.com/ilri/OpenRXV
+```
+
+- I see nearly 200,000 hits in Solr from the IP address, though, so I need to make sure those are old ones from before today
+  - I purged the hits for 178.62.93.141 as well as any from the old `axios/0.19.2` user agent
+  - I made some requests with and without the new user agent and only the ones without showed up in Solr
+
+## 2020-07-22
+
+- Atmire merged my latest bot suggestions to the COUNTER-Robots project:
+  - [Add new bots](https://github.com/atmire/COUNTER-Robots/pull/36)
+  - [COUNTER_Robots_list.json: Escape literal dots](https://github.com/atmire/COUNTER-Robots/pull/35)
+  - [COUNTER_Robots_list.json: Remove anchors from okhttp](https://github.com/atmire/COUNTER-Robots/pull/33)
+- I will update the agent patterns on the CGSpace `5_x-prod` and 6.x branches
+- Make some changes to the Bootstrap CSS and HTML configuration to improve readability and style on the CG Core v2 metadata reference guide and send a pull request to Marie ([#29](https://github.com/AgriculturalSemantics/cg-core/pull/29))
 
 <!-- vim: set sw=2 ts=2: -->