Add notes for 2021-08-05

content/posts/2021-08.md

@@ -60,4 +60,33 @@ categories: ["Notes"]
   - Now it seems to be verified (all green): https://www.openarchives.org/Register/ValidateSite?log=R23ZWX85
   - We are listed in the OpenArchives list of databases conforming to OAI 2.0
 
+## 2021-08-03
+
+- Run fresh re-harvest on AReS
+
+## 2021-08-05
+
+- Have a quick call with Mishell Portilla from CIP about a journal article that was flagged as being in a predatory journal (Beall's List)
+  - We agreed to unmap it from RTB's collection for now, and I asked for advice from Peter and Abenet for what to do in the future
+- A developer from the Alliance asked for access to the CGSpace database so they can make some integration with PowerBI
+  - I told them we don't allow direct database access, and that it would be tricky anyways (that's what APIs are for!)
+- I'm curious if there are still any requests coming in to CGSpace from the abusive Russian networks
+  - I extracted all the unique IPs that nginx processed in the last week:
+
+```console
+# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/access.log.2 /var/log/nginx/access.log.3 /var/log/nginx/access.log.4 /var/log/nginx/access.log.5 /var/log/nginx/access.log.6 /var/log/nginx/access.log.7 /var/log/nginx/access.log.8 | grep -E " (200|499) " | grep -v -E "(mahider|Googlebot|Turnitin|Grammarly|Unpaywall|UptimeRobot|bot)" | awk '{print $1}' | sort | uniq > /tmp/2021-08-05-all-ips.txt
+# wc -l /tmp/2021-08-05-all-ips.txt
+43428 /tmp/2021-08-05-all-ips.txt
+```
+
+- Already I can see that the total is much less than during the attack on one weekend last month (over 50,000!)
+  - Indeed, now I see that there are no IPs from those networks coming in now:
+
+```console
+$ ./ilri/resolve-addresses-geoip2.py -i /tmp/2021-08-05-all-ips.txt -o /tmp/2021-08-05-all-ips.csv
+$ csvgrep -c asn -r '^(49453|46844|206485|62282|36352|35913|35624|8100)$' /tmp/2021-08-05-all-ips.csv | csvcut -c ip | sed 1d | sort | uniq > /tmp/2021-08-05-all-ips-to-purge.csv
+$ wc -l /tmp/2021-08-05-all-ips-to-purge.csv
+0 /tmp/2021-08-05-all-ips-to-purge.csv
+```
+
 <!-- vim: set sw=2 ts=2: -->