Add notes for 2022-06-06

content/posts/2022-06.md

@@ -0,0 +1,50 @@
+---
+title: "June, 2022"
+date: 2022-06-06T09:01:36+03:00
+author: "Alan Orth"
+categories: ["Notes"]
+---
+
+## 2022-06-06
+
+- Look at the Solr statistics on CGSpace
+  - I see 167,000 hits from a bunch of Microsoft IPs with reverse DNS "msnbot-" using the Solr query `dns:*msnbot* AND dns:*.msn.com`
+  - I purged these first so I could see the other "real" IPs in the Solr facets
+- I see 47,500 hits from 80.248.237.167 on a data center ISP in Sweden, using a normal user agent
+- I see 13,000 hits from 163.237.216.11 on a data center ISP in Australia, using a normal user agent
+- I see 7,300 hits from 208.185.238.57 from Britanica, using a normal user agent
+  - There seem to be many more of these:
+
+<!--more-->
+
+```console
+# zcat --force /var/log/nginx/access.log* | grep 208.185.238. | awk '{print $1}' | sort | uniq -c | sort -h
+      2 208.185.238.1
+    166 208.185.238.54
+   1293 208.185.238.51
+   2587 208.185.238.59
+   4692 208.185.238.56
+   5480 208.185.238.53
+   6277 208.185.238.52
+   6400 208.185.238.58
+   8261 208.185.238.55
+  17549 208.185.238.57
+```
+
+- I see 3,000 hits from 178.208.75.33 by a Russian-owned IP in the Netherlands that is making a GET to / every one minute, using a normal user agent
+- I see 3,000 hits from 134.122.124.196 on Digital Ocean to the REST API with a normal user agent
+- I purged all these hits from IPs for a total of about 265,000
+- Then I faceted by user agent and found
+  - 1,000 hits by `insomnia/2022.2.1`, which I also saw last month and submitted to COUNTER-Robots
+  - 265 hits by `omgili/0.5 +http://omgili.com`
+  - 150 hits by `Vizzit`
+  - 132 hits by `MetaInspector/5.7.0 (+https://github.com/jaimeiniesta/metainspector)`
+  - 73 hits by `Scoop.it`
+  - 62 hits by `bitdiscovery`
+  - 59 hits by `Asana/1.4.0 WebsiteMetadataRetriever`
+  - 32 hits by `Sprout Social (Link Attachment)`
+  - 29 hits by `CyotekWebCopy/1.9 CyotekHTTP/6.2`
+  - 20 hits by `Hootsuite-Authoring/1.0`
+- I purged about 4,100 hits from these user agents
+
+<!-- vim: set sw=2 ts=2: -->