Add notes for 2021-11-21 and regenerate public

content/posts/2021-11.md

@@ -118,4 +118,69 @@ $ rsync -av --partial --progress --delete provisioning@ares:/tmp/data/ backend/d
   - He found conflicting numbers when using the community and collection modes in Content and Usage Analysis
   - I sent him more numbers directly from the DSpace Statistics API
 
+## 2021-11-09
+
+- I migrated the 2013, 2012, and 2011 statistics to yearly shards on DSpace Test's Solr to continute my testing of memory / latency impact
+- I found out why the CI jobs for the DSpace Statistics API had been failing the past few weeks
+  - When I reverted to using the original falcon-swagger-ui project after they apparently merged my Falcon 3 changes, it seems that they actually only merged the Swagger UI changes, not the Falcon 3 fix!
+  - I switched back to using my own fork and now it's working
+  - Unfortunately now I'm getting an error installing my dependencies with Poetry:
+
+```console
+RuntimeError
+
+Unable to find installation candidates for regex (2021.11.9)
+
+at /usr/lib/python3.9/site-packages/poetry/installation/chooser.py:72 in choose_for
+     68│
+     69│             links.append(link)
+     70│
+     71│         if not links:
+  →  72│             raise RuntimeError(
+     73│                 "Unable to find installation candidates for {}".format(package)
+     74│             )
+     75│
+     76│         # Get the best link
+```
+
+- So that's super annoying... I'm going to try using Pipenv again...
+
+## 2021-11-10
+
+- 93.158.91.62 is scraping us again
+  - That's an IP in Sweden that is clearly a bot, but pretending to use a normal user agent
+  - I added them to the "bot" list in nginx so the requests will share a common DSpace session with other bots and not create Solr hits, but still they are causing high outbound traffic
+  - I modified the nginx configuration to send them an HTTP 403 and tell them to use a bot user agent
+
+## 2021-11-14
+
+- I decided to update AReS to the latest OpenRXV version with Elasticsearch 7.13
+  - First I took backups of the Elasticsearch volume and OpenRXV backend data:
+
+```console
+$ docker-compose down
+$ sudo tar czf openrxv_esData_7-2021-11-14.tar.xz /var/lib/docker/volumes/openrxv_esData_7
+$ cp -a backend/data backend/data.2021-11-14
+```
+
+- Then I checked out the latest git commit, updated all images, rebuilt the project:
+
+```console
+$ docker images | grep -v ^REPO | sed 's/ \+/:/g' | cut -d: -f1,2 | xargs -L1 docker pull
+$ docker-compose build
+$ docker-compose up -d
+```
+
+- Then I updated the repository configurations and started a fresh harvest
+- Help Francesca from the Alliance with a question about embargos on CGSpace items
+  - I logged in as a normal user and a CGIAR user, and I was unable to access the PDF or full text of the item
+  - I was only able to access the PDF when I was logged in as an admin
+
+## 2021-11-21
+
+- Update all Docker images on AReS (linode20) and re-build OpenRXV
+  - Run all system updates and reboot the server
+  - Start a full harvest, but I notice that the number of items being harvested is not complete, so I stopped it
+- Run all system updates on CGSpace (linode18) and DSpace Test (linode26) and reboot them
+
 <!-- vim: set sw=2 ts=2: -->