Add notes for 2017-11-04

content/post/2017-11.md

@@ -47,3 +47,46 @@ COPY 54701
 
 - Atmire got back to us to say that they estimate it will take two days of labor to implement the change to Listings and Reports
 - I said I'd ask Abenet if she wants that feature
+
+## 2017-11-04
+
+- I finished looking through Sisay's CIAT records for the "Alianzas de Aprendizaje" data
+- I corrected about half of the authors to standardize them
+- Linode emailed this morning to say that the CPU usage was high again, this time at 6:14AM
+- It's the first time in a few days that this has happened
+- I had a look to see what was going on, but it isn't the CORE bot:
+
+```
+# awk '{print $1}' /var/log/nginx/access.log | sort -n | uniq -c | sort -h | tail
+    306 68.180.229.31
+    323 61.148.244.116
+    414 66.249.66.91
+    507 40.77.167.16
+    618 157.55.39.161
+    652 207.46.13.103
+    666 157.55.39.254
+   1173 104.196.152.243
+   1737 66.249.66.90
+  23101 138.201.52.218
+```
+
+- 138.201.52.218 is from some Hetzner server, and I see it making 40,000 requests yesterday too, but none before that:
+
+```
+# zgrep -c 138.201.52.218 /var/log/nginx/access.log*
+/var/log/nginx/access.log:24403
+/var/log/nginx/access.log.1:45958
+/var/log/nginx/access.log.2.gz:0
+/var/log/nginx/access.log.3.gz:0
+/var/log/nginx/access.log.4.gz:0
+/var/log/nginx/access.log.5.gz:0
+/var/log/nginx/access.log.6.gz:0
+```
+
+- It's clearly a bot as it's making tens of thousands of requests, but it's using a "normal" user agent:
+
+```
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
+```
+
+- For now I don't know what this user is!