diff --git a/content/posts/2017-11.md b/content/posts/2017-11.md index a1e3bcb98..3b3ec349c 100644 --- a/content/posts/2017-11.md +++ b/content/posts/2017-11.md @@ -540,9 +540,9 @@ $ grep 104.196.152.243 dspace.log.2017-11-07 | grep -o -E 'session_id=[A-Z0-9]{3 - What's amazing is that it seems to reuse its Java session across all requests: ``` -$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=5.9.6.51' /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=5.9.6.51' dspace.log.2017-11-12 1558 -$ grep 5.9.6.51 /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l +$ grep 5.9.6.51 dspace.log.2017-11-12 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l 1 ``` @@ -552,7 +552,7 @@ $ grep 5.9.6.51 /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 | grep -o -E ' ``` # grep 95.108.181.88 /var/log/nginx/access.log | tail -n 1 95.108.181.88 - - [12/Nov/2017:08:33:17 +0000] "GET /bitstream/handle/10568/57004/GenebankColombia_23Feb2015.pdf HTTP/1.1" 200 972019 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" -$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' dspace.log.2017-11-12 991 ``` diff --git a/content/posts/2018-04.md b/content/posts/2018-04.md index 202210d6c..1a5b39394 100644 --- a/content/posts/2018-04.md +++ b/content/posts/2018-04.md @@ -78,3 +78,145 @@ $ git rebase -i dspace-5.8 - DS-3583 Usage of correct Collection Array (#1731) (upstream commit on dspace-5_x: c8f62e6f496fa86846bfa6bcf2d16811087d9761) - ... but somehow git knew, and didn't include them in my interactive rebase! - I need to send this branch to Atmire and also arrange payment (see [ticket #560](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560) in their tracker) +- Fix Sisay's SSH access to the new DSpace Test server (linode19) + +## 2018-04-05 + +- Fix Sisay's sudo access on the new DSpace Test server (linode19) +- The reindexing process on DSpace Test took _forever_ yesterday: + +``` +$ time schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery -b + +real 599m32.961s +user 9m3.947s +sys 2m52.585s +``` + +- So we really should not use this Linode block storage for Solr +- Assetstore might be fine but would complicate things with configuration and deployment (ughhh) +- Better to use Linode block storage only for backup +- Help Peter with the GDPR compliance / reporting form for CGSpace +- DSpace Test crashed due to memory issues again: + +``` +# grep -c 'java.lang.OutOfMemoryError: Java heap space' /var/log/tomcat7/catalina.out +16 +``` + +- I ran all system updates on DSpace Test and rebooted it +- Proof some records on DSpace Test for Udana from IWMI +- He has done better with the small syntax and consistency issues but then there are larger concerns with not linking to DOIs, copying titles incorrectly, etc + +## 2018-04-10 + +- I got a notice that CGSpace CPU usage was very high this morning +- Looking at the nginx logs, here are the top users today so far: + +``` +# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "10/Apr/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10 + 282 207.46.13.112 + 286 54.175.208.220 + 287 207.46.13.113 + 298 66.249.66.153 + 322 207.46.13.114 + 780 104.196.152.243 + 3994 178.154.200.38 + 4295 70.32.83.92 + 4388 95.108.181.88 + 7653 45.5.186.2 +``` + +- 45.5.186.2 is of course CIAT +- 95.108.181.88 appears to be Yandex: + +``` +95.108.181.88 - - [09/Apr/2018:06:34:16 +0000] "GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1" 200 2638 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" +``` + +- And for some reason Yandex created a lot of Tomcat sessions today: + +``` +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' dspace.log.2018-04-10 +4363 +``` + +- 70.32.83.92 appears to be some harvester we've seen before, but on a new IP +- They are not creating new Tomcat sessions so there is no problem there +- 178.154.200.38 also appears to be Yandex, and is also creating many Tomcat sessions: + +``` +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=178.154.200.38' dspace.log.2018-04-10 +3982 +``` + +- I'm not sure why Yandex creates so many Tomcat sessions, as its user agent should match the Crawler Session Manager valve +- Let's try a manual request with and without their user agent: + +``` +$ http --print Hh https://cgspace.cgiar.org/bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg 'User-Agent:Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)' +GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1 +Accept: */* +Accept-Encoding: gzip, deflate +Connection: keep-alive +Host: cgspace.cgiar.org +User-Agent: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) + +HTTP/1.1 200 OK +Connection: keep-alive +Content-Language: en-US +Content-Length: 2638 +Content-Type: image/jpeg;charset=ISO-8859-1 +Date: Tue, 10 Apr 2018 05:18:37 GMT +Expires: Tue, 10 Apr 2018 06:18:37 GMT +Last-Modified: Tue, 25 Apr 2017 07:05:54 GMT +Server: nginx +Strict-Transport-Security: max-age=15768000 +Vary: User-Agent +X-Cocoon-Version: 2.2.0 +X-Content-Type-Options: nosniff +X-Frame-Options: SAMEORIGIN +X-XSS-Protection: 1; mode=block + +$ http --print Hh https://cgspace.cgiar.org/bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg +GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1 +Accept: */* +Accept-Encoding: gzip, deflate +Connection: keep-alive +Host: cgspace.cgiar.org +User-Agent: HTTPie/0.9.9 + +HTTP/1.1 200 OK +Connection: keep-alive +Content-Language: en-US +Content-Length: 2638 +Content-Type: image/jpeg;charset=ISO-8859-1 +Date: Tue, 10 Apr 2018 05:20:08 GMT +Expires: Tue, 10 Apr 2018 06:20:08 GMT +Last-Modified: Tue, 25 Apr 2017 07:05:54 GMT +Server: nginx +Set-Cookie: JSESSIONID=31635DB42B66D6A4208CFCC96DD96875; Path=/; Secure; HttpOnly +Strict-Transport-Security: max-age=15768000 +Vary: User-Agent +X-Cocoon-Version: 2.2.0 +X-Content-Type-Options: nosniff +X-Frame-Options: SAMEORIGIN +X-XSS-Protection: 1; mode=block +``` + +- So it definitely looks like Yandex requests are getting assigned a session from the Crawler Session Manager valve +- And if I look at the DSpace log I see its IP sharing a session with other crawlers like Google (66.249.66.153) +- Indeed the number of Tomcat sessions appears to be normal: + +![Tomcat sessions week](/cgspace-notes/2018/04/jmx_dspace_sessions-week.png) + +- Looks like the number of total requests processed by nginx in March went down from the previous months: + +``` +# time zcat --force /var/log/nginx/* | grep -cE "[0-9]{1,2}/Mar/2018" +2266594 + +real 0m13.658s +user 0m16.533s +sys 0m1.087s +``` diff --git a/docs/2015-11/index.html b/docs/2015-11/index.html index 1ff9bc292..eb737955c 100644 --- a/docs/2015-11/index.html +++ b/docs/2015-11/index.html @@ -53,7 +53,7 @@ $ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspac "/> - + diff --git a/docs/2015-12/index.html b/docs/2015-12/index.html index 23bfc40e9..c3c14ca2e 100644 --- a/docs/2015-12/index.html +++ b/docs/2015-12/index.html @@ -55,7 +55,7 @@ Replace lzop with xz in log compression cron jobs on DSpace Test—it uses less "/> - + diff --git a/docs/2016-01/index.html b/docs/2016-01/index.html index c0b396048..2be4103c9 100644 --- a/docs/2016-01/index.html +++ b/docs/2016-01/index.html @@ -45,7 +45,7 @@ Update GitHub wiki for documentation of maintenance tasks. "/> - + diff --git a/docs/2016-02/index.html b/docs/2016-02/index.html index 86943c703..f9416e75c 100644 --- a/docs/2016-02/index.html +++ b/docs/2016-02/index.html @@ -59,7 +59,7 @@ Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE&r "/> - + diff --git a/docs/2016-03/index.html b/docs/2016-03/index.html index 954938786..7dbb9700d 100644 --- a/docs/2016-03/index.html +++ b/docs/2016-03/index.html @@ -45,7 +45,7 @@ Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Ja "/> - + diff --git a/docs/2016-04/index.html b/docs/2016-04/index.html index aaf7d84af..3d29e21e6 100644 --- a/docs/2016-04/index.html +++ b/docs/2016-04/index.html @@ -49,7 +49,7 @@ Also, I noticed the checker log has some errors we should pay attention to: "/> - + diff --git a/docs/2016-05/index.html b/docs/2016-05/index.html index 6947e8a92..c26187fb8 100644 --- a/docs/2016-05/index.html +++ b/docs/2016-05/index.html @@ -53,7 +53,7 @@ There are 3,000 IPs accessing the REST API in a 24-hour period! "/> - + diff --git a/docs/2016-06/index.html b/docs/2016-06/index.html index 55b196f9e..d387d738a 100644 --- a/docs/2016-06/index.html +++ b/docs/2016-06/index.html @@ -51,7 +51,7 @@ Working on second phase of metadata migration, looks like this will work for mov "/> - + diff --git a/docs/2016-07/index.html b/docs/2016-07/index.html index 9b784207c..3b2dc1fed 100644 --- a/docs/2016-07/index.html +++ b/docs/2016-07/index.html @@ -67,7 +67,7 @@ In this case the select query was showing 95 results before the update "/> - + diff --git a/docs/2016-08/index.html b/docs/2016-08/index.html index 85d1b8ec5..25f11725b 100644 --- a/docs/2016-08/index.html +++ b/docs/2016-08/index.html @@ -61,7 +61,7 @@ $ git rebase -i dspace-5.5 "/> - + diff --git a/docs/2016-09/index.html b/docs/2016-09/index.html index 4d5ab3d39..8167ea366 100644 --- a/docs/2016-09/index.html +++ b/docs/2016-09/index.html @@ -53,7 +53,7 @@ $ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=or "/> - + diff --git a/docs/2016-10/index.html b/docs/2016-10/index.html index c52765814..1e4c2e447 100644 --- a/docs/2016-10/index.html +++ b/docs/2016-10/index.html @@ -61,7 +61,7 @@ I exported a random item’s metadata as CSV, deleted all columns except id "/> - + diff --git a/docs/2016-11/index.html b/docs/2016-11/index.html index ec8b400a2..6ba10768d 100644 --- a/docs/2016-11/index.html +++ b/docs/2016-11/index.html @@ -45,7 +45,7 @@ Add dc.type to the output options for Atmire’s Listings and Reports module "/> - + diff --git a/docs/2016-12/index.html b/docs/2016-12/index.html index d58cc62b0..57d7edf83 100644 --- a/docs/2016-12/index.html +++ b/docs/2016-12/index.html @@ -69,7 +69,7 @@ Another worrying error from dspace.log is: "/> - + diff --git a/docs/2017-01/index.html b/docs/2017-01/index.html index 063bbd1d0..d00f14343 100644 --- a/docs/2017-01/index.html +++ b/docs/2017-01/index.html @@ -45,7 +45,7 @@ I asked on the dspace-tech mailing list because it seems to be broken, and actua "/> - + diff --git a/docs/2017-02/index.html b/docs/2017-02/index.html index aedd36501..9b0911d32 100644 --- a/docs/2017-02/index.html +++ b/docs/2017-02/index.html @@ -73,7 +73,7 @@ Looks like we’ll be using cg.identifier.ccafsprojectpii as the field name "/> - + diff --git a/docs/2017-03/index.html b/docs/2017-03/index.html index e4efbcd0c..e4830f7f6 100644 --- a/docs/2017-03/index.html +++ b/docs/2017-03/index.html @@ -77,7 +77,7 @@ $ identify ~/Desktop/alc_contrastes_desafios.jpg "/> - + diff --git a/docs/2017-04/index.html b/docs/2017-04/index.html index c6adf7618..3c1ce3dbf 100644 --- a/docs/2017-04/index.html +++ b/docs/2017-04/index.html @@ -63,7 +63,7 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Th "/> - + diff --git a/docs/2017-05/index.html b/docs/2017-05/index.html index 2a9619dba..58fb4d78a 100644 --- a/docs/2017-05/index.html +++ b/docs/2017-05/index.html @@ -29,7 +29,7 @@ - + diff --git a/docs/2017-06/index.html b/docs/2017-06/index.html index e3c1ac885..2dc13a8a3 100644 --- a/docs/2017-06/index.html +++ b/docs/2017-06/index.html @@ -29,7 +29,7 @@ - + diff --git a/docs/2017-07/index.html b/docs/2017-07/index.html index 92f1099e3..39ac8e9de 100644 --- a/docs/2017-07/index.html +++ b/docs/2017-07/index.html @@ -57,7 +57,7 @@ We can use PostgreSQL’s extended output format (-x) plus sed to format the "/> - + diff --git a/docs/2017-08/index.html b/docs/2017-08/index.html index ac446b896..947c0379d 100644 --- a/docs/2017-08/index.html +++ b/docs/2017-08/index.html @@ -77,7 +77,7 @@ Then I cleaned up the author authorities and HTML characters in OpenRefine and s "/> - + diff --git a/docs/2017-09/index.html b/docs/2017-09/index.html index 2478f7ade..00d1ac386 100644 --- a/docs/2017-09/index.html +++ b/docs/2017-09/index.html @@ -53,7 +53,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne’s user account "/> - + diff --git a/docs/2017-10/index.html b/docs/2017-10/index.html index b20fa938b..c9fb785d7 100644 --- a/docs/2017-10/index.html +++ b/docs/2017-10/index.html @@ -57,7 +57,7 @@ Add Katherine Lutz to the groups for content submission and edit steps of the CG "/> - + diff --git a/docs/2017-11/index.html b/docs/2017-11/index.html index 2efe9c295..9ef5651d0 100644 --- a/docs/2017-11/index.html +++ b/docs/2017-11/index.html @@ -77,7 +77,7 @@ COPY 54701 "/> - + @@ -754,9 +754,9 @@ $ grep 104.196.152.243 dspace.log.2017-11-07 | grep -o -E 'session_id=[A-Z0-9]{3
$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=5.9.6.51' /home/cgspace.cgiar.org/log/dspace.log.2017-11-12
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=5.9.6.51' dspace.log.2017-11-12
1558
-$ grep 5.9.6.51 /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
+$ grep 5.9.6.51 dspace.log.2017-11-12 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
1
@@ -767,7 +767,7 @@ $ grep 5.9.6.51 /home/cgspace.cgiar.org/log/dspace.log.2017-11-12 | grep -o -E '
# grep 95.108.181.88 /var/log/nginx/access.log | tail -n 1
95.108.181.88 - - [12/Nov/2017:08:33:17 +0000] "GET /bitstream/handle/10568/57004/GenebankColombia_23Feb2015.pdf HTTP/1.1" 200 972019 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
-$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' /home/cgspace.cgiar.org/log/dspace.log.2017-11-12
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' dspace.log.2017-11-12
991
diff --git a/docs/2017-12/index.html b/docs/2017-12/index.html
index dcc4f367a..b345a78b7 100644
--- a/docs/2017-12/index.html
+++ b/docs/2017-12/index.html
@@ -47,7 +47,7 @@ The list of connections to XMLUI and REST API for today:
"/>
-
+
diff --git a/docs/2018-01/index.html b/docs/2018-01/index.html
index 83b4d1a1b..91d4385b1 100644
--- a/docs/2018-01/index.html
+++ b/docs/2018-01/index.html
@@ -185,7 +185,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv
"/>
-
+
diff --git a/docs/2018-02/index.html b/docs/2018-02/index.html
index 7aae83cd6..c0f68a3c0 100644
--- a/docs/2018-02/index.html
+++ b/docs/2018-02/index.html
@@ -47,7 +47,7 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl
"/>
-
+
diff --git a/docs/2018-03/index.html b/docs/2018-03/index.html
index 2cfb72e6f..c8d221c5c 100644
--- a/docs/2018-03/index.html
+++ b/docs/2018-03/index.html
@@ -41,7 +41,7 @@ Export a CSV of the IITA community metadata for Martin Mueller
"/>
-
+
diff --git a/docs/2018-04/index.html b/docs/2018-04/index.html
index 2143c1014..698fdbfad 100644
--- a/docs/2018-04/index.html
+++ b/docs/2018-04/index.html
@@ -21,7 +21,7 @@ Catalina logs at least show some memory errors yesterday:
-
+
@@ -43,7 +43,7 @@ Catalina logs at least show some memory errors yesterday:
"/>
-
+
@@ -53,9 +53,9 @@ Catalina logs at least show some memory errors yesterday:
"@type": "BlogPosting",
"headline": "April, 2018",
"url": "https://alanorth.github.io/cgspace-notes/2018-04/",
- "wordCount": "423",
+ "wordCount": "1005",
"datePublished": "2018-04-01T16:13:54+02:00",
- "dateModified": "2018-04-04T15:57:34+03:00",
+ "dateModified": "2018-04-04T17:01:08+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@@ -206,8 +206,162 @@ $ git rebase -i dspace-5.8
… but somehow git knew, and didn’t include them in my interactive rebase!
I need to send this branch to Atmire and also arrange payment (see ticket #560 in their tracker)
+Fix Sisay’s SSH access to the new DSpace Test server (linode19)
+2018-04-05
+
+
+- Fix Sisay’s sudo access on the new DSpace Test server (linode19)
+- The reindexing process on DSpace Test took forever yesterday:
+
+
+$ time schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery -b
+
+real 599m32.961s
+user 9m3.947s
+sys 2m52.585s
+
+
+
+- So we really should not use this Linode block storage for Solr
+- Assetstore might be fine but would complicate things with configuration and deployment (ughhh)
+- Better to use Linode block storage only for backup
+- Help Peter with the GDPR compliance / reporting form for CGSpace
+- DSpace Test crashed due to memory issues again:
+
+
+# grep -c 'java.lang.OutOfMemoryError: Java heap space' /var/log/tomcat7/catalina.out
+16
+
+
+
+- I ran all system updates on DSpace Test and rebooted it
+- Proof some records on DSpace Test for Udana from IWMI
+- He has done better with the small syntax and consistency issues but then there are larger concerns with not linking to DOIs, copying titles incorrectly, etc
+
+
+2018-04-10
+
+
+- I got a notice that CGSpace CPU usage was very high this morning
+- Looking at the nginx logs, here are the top users today so far:
+
+
+# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "10/Apr/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
+ 282 207.46.13.112
+ 286 54.175.208.220
+ 287 207.46.13.113
+ 298 66.249.66.153
+ 322 207.46.13.114
+ 780 104.196.152.243
+ 3994 178.154.200.38
+ 4295 70.32.83.92
+ 4388 95.108.181.88
+ 7653 45.5.186.2
+
+
+
+- 45.5.186.2 is of course CIAT
+- 95.108.181.88 appears to be Yandex:
+
+
+95.108.181.88 - - [09/Apr/2018:06:34:16 +0000] "GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1" 200 2638 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
+
+
+
+- And for some reason Yandex created a lot of Tomcat sessions today:
+
+
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=95.108.181.88' dspace.log.2018-04-10
+4363
+
+
+
+- 70.32.83.92 appears to be some harvester we’ve seen before, but on a new IP
+- They are not creating new Tomcat sessions so there is no problem there
+- 178.154.200.38 also appears to be Yandex, and is also creating many Tomcat sessions:
+
+
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=178.154.200.38' dspace.log.2018-04-10
+3982
+
+
+
+- I’m not sure why Yandex creates so many Tomcat sessions, as its user agent should match the Crawler Session Manager valve
+- Let’s try a manual request with and without their user agent:
+
+
+$ http --print Hh https://cgspace.cgiar.org/bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg 'User-Agent:Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)'
+GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1
+Accept: */*
+Accept-Encoding: gzip, deflate
+Connection: keep-alive
+Host: cgspace.cgiar.org
+User-Agent: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
+
+HTTP/1.1 200 OK
+Connection: keep-alive
+Content-Language: en-US
+Content-Length: 2638
+Content-Type: image/jpeg;charset=ISO-8859-1
+Date: Tue, 10 Apr 2018 05:18:37 GMT
+Expires: Tue, 10 Apr 2018 06:18:37 GMT
+Last-Modified: Tue, 25 Apr 2017 07:05:54 GMT
+Server: nginx
+Strict-Transport-Security: max-age=15768000
+Vary: User-Agent
+X-Cocoon-Version: 2.2.0
+X-Content-Type-Options: nosniff
+X-Frame-Options: SAMEORIGIN
+X-XSS-Protection: 1; mode=block
+
+$ http --print Hh https://cgspace.cgiar.org/bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg
+GET /bitstream/handle/10568/21794/ILRI_logo_usage.jpg.jpg HTTP/1.1
+Accept: */*
+Accept-Encoding: gzip, deflate
+Connection: keep-alive
+Host: cgspace.cgiar.org
+User-Agent: HTTPie/0.9.9
+
+HTTP/1.1 200 OK
+Connection: keep-alive
+Content-Language: en-US
+Content-Length: 2638
+Content-Type: image/jpeg;charset=ISO-8859-1
+Date: Tue, 10 Apr 2018 05:20:08 GMT
+Expires: Tue, 10 Apr 2018 06:20:08 GMT
+Last-Modified: Tue, 25 Apr 2017 07:05:54 GMT
+Server: nginx
+Set-Cookie: JSESSIONID=31635DB42B66D6A4208CFCC96DD96875; Path=/; Secure; HttpOnly
+Strict-Transport-Security: max-age=15768000
+Vary: User-Agent
+X-Cocoon-Version: 2.2.0
+X-Content-Type-Options: nosniff
+X-Frame-Options: SAMEORIGIN
+X-XSS-Protection: 1; mode=block
+
+
+
+- So it definitely looks like Yandex requests are getting assigned a session from the Crawler Session Manager valve
+- And if I look at the DSpace log I see its IP sharing a session with other crawlers like Google (66.249.66.153)
+- Indeed the number of Tomcat sessions appears to be normal:
+
+
+
+
+
+- Looks like the number of total requests processed by nginx in March went down from the previous months:
+
+
+# time zcat --force /var/log/nginx/* | grep -cE "[0-9]{1,2}/Mar/2018"
+2266594
+
+real 0m13.658s
+user 0m16.533s
+sys 0m1.087s
+
+
diff --git a/docs/2018/04/jmx_dspace_sessions-week.png b/docs/2018/04/jmx_dspace_sessions-week.png
new file mode 100644
index 000000000..3d91f0b72
Binary files /dev/null and b/docs/2018/04/jmx_dspace_sessions-week.png differ
diff --git a/docs/404.html b/docs/404.html
index c5ceba8cd..4e2225b6a 100644
--- a/docs/404.html
+++ b/docs/404.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/index.html b/docs/categories/index.html
index bc43e5959..5aeb33660 100644
--- a/docs/categories/index.html
+++ b/docs/categories/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/notes/index.html b/docs/categories/notes/index.html
index bcdde0f1d..27a1a0347 100644
--- a/docs/categories/notes/index.html
+++ b/docs/categories/notes/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/notes/page/2/index.html b/docs/categories/notes/page/2/index.html
index 33f67a914..27969f47a 100644
--- a/docs/categories/notes/page/2/index.html
+++ b/docs/categories/notes/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/notes/page/3/index.html b/docs/categories/notes/page/3/index.html
index 005ff7452..3899c7dbf 100644
--- a/docs/categories/notes/page/3/index.html
+++ b/docs/categories/notes/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/notes/page/4/index.html b/docs/categories/notes/page/4/index.html
index bd382a091..3c89c270a 100644
--- a/docs/categories/notes/page/4/index.html
+++ b/docs/categories/notes/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/page/2/index.html b/docs/categories/page/2/index.html
index 02c92e94f..39e2b3b45 100644
--- a/docs/categories/page/2/index.html
+++ b/docs/categories/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/page/3/index.html b/docs/categories/page/3/index.html
index 9091180f1..9c10d24c0 100644
--- a/docs/categories/page/3/index.html
+++ b/docs/categories/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/categories/page/4/index.html b/docs/categories/page/4/index.html
index 6de1ad70b..e81ca1015 100644
--- a/docs/categories/page/4/index.html
+++ b/docs/categories/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/cgiar-library-migration/index.html b/docs/cgiar-library-migration/index.html
index e58f9b9b3..778a3382d 100644
--- a/docs/cgiar-library-migration/index.html
+++ b/docs/cgiar-library-migration/index.html
@@ -29,7 +29,7 @@
-
+
diff --git a/docs/index.html b/docs/index.html
index 5b1a32c7b..71f2353c9 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/page/2/index.html b/docs/page/2/index.html
index 5756e25a1..7fd9322cc 100644
--- a/docs/page/2/index.html
+++ b/docs/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/page/3/index.html b/docs/page/3/index.html
index 1b3e779c7..540e37b9f 100644
--- a/docs/page/3/index.html
+++ b/docs/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/page/4/index.html b/docs/page/4/index.html
index 2411236c4..b0e3b018b 100644
--- a/docs/page/4/index.html
+++ b/docs/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/posts/index.html b/docs/posts/index.html
index 8e5d27ba6..dad2e113d 100644
--- a/docs/posts/index.html
+++ b/docs/posts/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/posts/page/2/index.html b/docs/posts/page/2/index.html
index bad683ae1..f2cffc251 100644
--- a/docs/posts/page/2/index.html
+++ b/docs/posts/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/posts/page/3/index.html b/docs/posts/page/3/index.html
index 715587427..8d5607749 100644
--- a/docs/posts/page/3/index.html
+++ b/docs/posts/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/posts/page/4/index.html b/docs/posts/page/4/index.html
index d75192745..85cef1a46 100644
--- a/docs/posts/page/4/index.html
+++ b/docs/posts/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/sitemap.xml b/docs/sitemap.xml
index 06601e0d2..7d8e06ab1 100644
--- a/docs/sitemap.xml
+++ b/docs/sitemap.xml
@@ -4,7 +4,7 @@
https://alanorth.github.io/cgspace-notes/2018-04/
- 2018-04-04T15:57:34+03:00
+ 2018-04-04T17:01:08+03:00
@@ -159,7 +159,7 @@
https://alanorth.github.io/cgspace-notes/
- 2018-04-04T15:57:34+03:00
+ 2018-04-04T17:01:08+03:00
0
@@ -170,7 +170,7 @@
https://alanorth.github.io/cgspace-notes/tags/notes/
- 2018-04-04T15:57:34+03:00
+ 2018-04-04T17:01:08+03:00
0
@@ -182,13 +182,13 @@
https://alanorth.github.io/cgspace-notes/posts/
- 2018-04-04T15:57:34+03:00
+ 2018-04-04T17:01:08+03:00
0
https://alanorth.github.io/cgspace-notes/tags/
- 2018-04-04T15:57:34+03:00
+ 2018-04-04T17:01:08+03:00
0
diff --git a/docs/tags/index.html b/docs/tags/index.html
index eb0a9919d..eca4adf9d 100644
--- a/docs/tags/index.html
+++ b/docs/tags/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/notes/index.html b/docs/tags/notes/index.html
index f4f016301..036659c15 100644
--- a/docs/tags/notes/index.html
+++ b/docs/tags/notes/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/notes/page/2/index.html b/docs/tags/notes/page/2/index.html
index 9df6ade1d..a5a339e9f 100644
--- a/docs/tags/notes/page/2/index.html
+++ b/docs/tags/notes/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/notes/page/3/index.html b/docs/tags/notes/page/3/index.html
index d49aaa68f..ee89ade19 100644
--- a/docs/tags/notes/page/3/index.html
+++ b/docs/tags/notes/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/notes/page/4/index.html b/docs/tags/notes/page/4/index.html
index 4b0863b6d..f20c1f1b8 100644
--- a/docs/tags/notes/page/4/index.html
+++ b/docs/tags/notes/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/page/2/index.html b/docs/tags/page/2/index.html
index fefdbe87e..5341b51e2 100644
--- a/docs/tags/page/2/index.html
+++ b/docs/tags/page/2/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/page/3/index.html b/docs/tags/page/3/index.html
index 23f2ea6eb..5d8780725 100644
--- a/docs/tags/page/3/index.html
+++ b/docs/tags/page/3/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/docs/tags/page/4/index.html b/docs/tags/page/4/index.html
index 62d597d52..2dbc0376d 100644
--- a/docs/tags/page/4/index.html
+++ b/docs/tags/page/4/index.html
@@ -26,7 +26,7 @@
-
+
diff --git a/static/2018/04/jmx_dspace_sessions-week.png b/static/2018/04/jmx_dspace_sessions-week.png
new file mode 100644
index 000000000..3d91f0b72
Binary files /dev/null and b/static/2018/04/jmx_dspace_sessions-week.png differ