--- title: "November, 2018" date: 2018-11-01T16:41:30+02:00 author: "Alan Orth" tags: ["Notes"] --- ## 2018-11-01 - Finalize AReS Phase I and Phase II ToRs - Send a note about my [dspace-statistics-api](https://github.com/ilri/dspace-statistics-api) to the dspace-tech mailing list ## 2018-11-03 - Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage - Today these are the top 10 IPs: ``` # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "03/Nov/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10 1300 66.249.64.63 1384 35.237.175.180 1430 138.201.52.218 1455 207.46.13.156 1500 40.77.167.175 1979 50.116.102.77 2790 66.249.64.61 3367 84.38.130.177 4537 70.32.83.92 22508 66.249.64.59 ``` - The `66.249.64.x` are definitely Google - `70.32.83.92` is well known, probably CCAFS or something, as it's only a few thousand requests and always to REST API - `84.38.130.177` is some new IP in Latvia that is only hitting the XMLUI, using the following user agent: ``` Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.792.0 Safari/535.1 ``` - They at least seem to be re-using their Tomcat sessions: ``` $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=84.38.130.177' dspace.log.2018-11-03 | sort | uniq 342 ``` - `50.116.102.77` is also a regular REST API user - `40.77.167.175` and `207.46.13.156` seem to be Bing - `138.201.52.218` seems to be on Hetzner in Germany, but is using this user agent: ``` Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:62.0) Gecko/20100101 Firefox/62.0 ``` - And it doesn't seem they are re-using their Tomcat sessions: ``` $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=138.201.52.218' dspace.log.2018-11-03 | sort | uniq 1243 ``` - Ah, we've apparently seen this server exactly a year ago in 2017-11, making 40,000 requests in one day... - I wonder if it's worth adding them to the list of bots in the nginx config? - Linode sent a mail that CGSpace (linode18) is using high outgoing bandwidth - Looking at the nginx logs again I see the following top ten IPs: ``` # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "03/Nov/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10 1979 50.116.102.77 1980 35.237.175.180 2186 207.46.13.156 2208 40.77.167.175 2843 66.249.64.63 4220 84.38.130.177 4537 70.32.83.92 5593 66.249.64.61 12557 78.46.89.18 32152 66.249.64.59 ``` - `78.46.89.18` is new since I last checked a few hours ago, and it's from Hetzner with the following user agent: ``` Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:62.0) Gecko/20100101 Firefox/62.0 ``` - It's making lots of requests and using quite a number of Tomcat sessions: ``` $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=78.46.89.18' /home/cgspace.cgiar.org/log/dspace.log.2018-11-03 | sort | uniq 8449 ``` - I could add this IP to the list of bot IPs in nginx, but it seems like a futile effort when some new IP could come along and do the same thing - Perhaps I should think about adding rate limits to dynamic pages like `/discover` and `/browse` - I think it's reasonable for a human to click one of those links five or ten times a minute... - To contrast, `78.46.89.18` made about 300 requests per minute for a few hours today: ``` # grep 78.46.89.18 /var/log/nginx/access.log | grep -o -E '03/Nov/2018:[0-9][0-9]:[0-9][0-9]' | sort | uniq -c | sort -n | tail -n 20 286 03/Nov/2018:18:02 287 03/Nov/2018:18:21 289 03/Nov/2018:18:23 291 03/Nov/2018:18:27 293 03/Nov/2018:18:34 300 03/Nov/2018:17:58 300 03/Nov/2018:18:22 300 03/Nov/2018:18:32 304 03/Nov/2018:18:12 305 03/Nov/2018:18:13 305 03/Nov/2018:18:24 312 03/Nov/2018:18:39 322 03/Nov/2018:18:17 326 03/Nov/2018:18:38 327 03/Nov/2018:18:16 330 03/Nov/2018:17:57 332 03/Nov/2018:18:19 336 03/Nov/2018:17:56 340 03/Nov/2018:18:14 341 03/Nov/2018:18:18 ``` - If they want to download all our metadata and PDFs they should use an API rather than scraping the XMLUI - I will add them to the list of bot IPs in nginx for now and think about enforcing rate limits in XMLUI later - Also, this is the third (?) time a mysterious IP in Hetzner has done this... who is this?