Add notes for 2018-12-06

content/posts/2018-12.md

@@ -308,4 +308,42 @@ $ grep -o -E 'session_id=[A-Z0-9]{32}:ip_addr=78.46.79.71' dspace.log.2018-12-03
 
 - Discuss RSS issues with IWMI and WLE people
 
+## 2018-12-06
+
+- Linode sent a message that the CPU usage of CGSpace (linode18) is too high last night
+- I looked in the logs and there's nothing particular going on:
+
+```
+# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "05/Dec/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
+   1225 157.55.39.177
+   1240 207.46.13.12
+   1261 207.46.13.101
+   1411 207.46.13.157
+   1529 34.218.226.147
+   2085 50.116.102.77
+   3334 2a01:7e00::f03c:91ff:fe0a:d645
+   3733 66.249.70.27
+   3815 35.237.175.180
+   7669 54.70.40.11
+```
+
+- `54.70.40.11` is some new bot with the following user agent:
+
+```
+Mozilla/5.0 (compatible) SemanticScholarBot (+https://www.semanticscholar.org/crawler)
+```
+
+- But Tomcat is forcing them to re-use their Tomcat sessions with the Crawler Session Manager valve:
+
+```
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=54.70.40.11' dspace.log.2018-12-05
+6980
+$ grep -o -E 'session_id=[A-Z0-9]{32}:ip_addr=54.70.40.11' dspace.log.2018-12-05 | sort | uniq | wc -l
+1156
+```
+
+- `2a01:7e00::f03c:91ff:fe0a:d645` appears to be the CKM dev server where Danny is testing harvesting via Drupal
+- It seems they are hitting the XMLUI's OpenSearch a bit, but mostly on the REST API so no issues here yet
+- `Drupal` is already in the Tomcat Crawler Session Manager Valve's regex so that's good!
+
 <!-- vim: set sw=2 ts=2: -->