diff --git a/content/post/2017-12.md b/content/post/2017-12.md index 7a0877dc3..bed6f7b9c 100644 --- a/content/post/2017-12.md +++ b/content/post/2017-12.md @@ -325,3 +325,48 @@ Elapsed time: 2 secs (2559 msecs) - I need to keep an eye on this issue because it has nice fixes for reducing the number of database connections in DSpace 5.7: https://jira.duraspace.org/browse/DS-3551 - Update text on CGSpace about page to give some tips to developers about using the resources more wisely ([#352](https://github.com/ilri/DSpace/pull/352)) +- Linode alerted that CGSpace was using 396.3% CPU from 12 to 2 PM +- The REST and OAI API logs look pretty much the same as earlier this morning, but there's a new IP harvesting XMLUI: + +``` +# cat /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/library-access.log /var/log/nginx/library-access.log.1 | grep -E "18/Dec/2017" | awk '{print $1}' | sort -n | uniq -c | sort -h | tail + 360 95.108.181.88 + 477 66.249.66.90 + 526 86.101.203.216 + 691 207.46.13.13 + 698 197.210.168.174 + 819 207.46.13.146 + 878 68.180.229.254 + 1965 104.196.152.243 + 17701 2.86.72.181 + 52532 137.108.70.7 +``` + +- 2.86.72.181 appears to be from Greece, and has the following user agent: + +``` +Mozilla/3.0 (compatible; Indy Library) +``` + +- Surprisingly it seems they are re-using their Tomcat session for all those 17,000 requests: + +``` +$ grep 2.86.72.181 dspace.log.2017-12-18 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l +1 +``` + +- I guess there's nothing I can do to them for now +- In other news, I am curious how many PostgreSQL connection pool errors we've had in the last month: + +``` +$ grep -c "Cannot get a connection, pool error Timeout waiting for idle object" dspace.log.2017-1* | grep -v :0 +dspace.log.2017-11-07:15695 +dspace.log.2017-11-08:135 +dspace.log.2017-11-17:1298 +dspace.log.2017-11-26:4160 +dspace.log.2017-11-28:107 +dspace.log.2017-11-29:3972 +dspace.log.2017-12-01:1601 +dspace.log.2017-12-02:1274 +dspace.log.2017-12-07:2769 +``` diff --git a/public/2017-12/index.html b/public/2017-12/index.html index 7b64c92aa..e350b7027 100644 --- a/public/2017-12/index.html +++ b/public/2017-12/index.html @@ -23,7 +23,7 @@ The list of connections to XMLUI and REST API for today: - + @@ -56,9 +56,9 @@ The list of connections to XMLUI and REST API for today: "@type": "BlogPosting", "headline": "December, 2017", "url": "https://alanorth.github.io/cgspace-notes/2017-12/", - "wordCount": "1787", + "wordCount": "1972", "datePublished": "2017-12-01T13:53:54+03:00", - "dateModified": "2017-12-18T14:02:24+02:00", + "dateModified": "2017-12-18T15:20:37+02:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -483,8 +483,55 @@ Elapsed time: 2 secs (2559 msecs) +
# cat /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/library-access.log /var/log/nginx/library-access.log.1 | grep -E "18/Dec/2017" | awk '{print $1}' | sort -n | uniq -c | sort -h | tail            
+    360 95.108.181.88
+    477 66.249.66.90
+    526 86.101.203.216
+    691 207.46.13.13
+    698 197.210.168.174
+    819 207.46.13.146
+    878 68.180.229.254
+   1965 104.196.152.243
+  17701 2.86.72.181
+  52532 137.108.70.7
+
+ + + +
Mozilla/3.0 (compatible; Indy Library)
+
+ + + +
$ grep 2.86.72.181 dspace.log.2017-12-18 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l                                                                                          
+1
+
+ + + +
$ grep -c "Cannot get a connection, pool error Timeout waiting for idle object" dspace.log.2017-1* | grep -v :0
+dspace.log.2017-11-07:15695
+dspace.log.2017-11-08:135
+dspace.log.2017-11-17:1298
+dspace.log.2017-11-26:4160
+dspace.log.2017-11-28:107
+dspace.log.2017-11-29:3972
+dspace.log.2017-12-01:1601
+dspace.log.2017-12-02:1274
+dspace.log.2017-12-07:2769
+
+ diff --git a/public/sitemap.xml b/public/sitemap.xml index 214651147..ae58e79e7 100644 --- a/public/sitemap.xml +++ b/public/sitemap.xml @@ -4,7 +4,7 @@ https://alanorth.github.io/cgspace-notes/2017-12/ - 2017-12-18T14:02:24+02:00 + 2017-12-18T15:20:37+02:00 @@ -139,7 +139,7 @@ https://alanorth.github.io/cgspace-notes/ - 2017-12-18T14:02:24+02:00 + 2017-12-18T15:20:37+02:00 0 @@ -150,7 +150,7 @@ https://alanorth.github.io/cgspace-notes/tags/notes/ - 2017-12-18T14:02:24+02:00 + 2017-12-18T15:20:37+02:00 0 @@ -162,13 +162,13 @@ https://alanorth.github.io/cgspace-notes/post/ - 2017-12-18T14:02:24+02:00 + 2017-12-18T15:20:37+02:00 0 https://alanorth.github.io/cgspace-notes/tags/ - 2017-12-18T14:02:24+02:00 + 2017-12-18T15:20:37+02:00 0