diff --git a/content/post/2018-01.md b/content/post/2018-01.md index 98fef3ae0..5617ebeef 100644 --- a/content/post/2018-01.md +++ b/content/post/2018-01.md @@ -1004,3 +1004,46 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled - I see I can monitor the number of Tomcat threads and some detailed JVM memory stuff if I install `munin-plugins-java` - I'd still like to get arbitrary mbeans like activeSessions etc, though - I can't remember if I had to configure the jmx settings in `/etc/munin/plugin-conf.d/munin-node` or not—I think all I did was re-run the `munin-node-configure` script and of course enable JMX in Tomcat's JVM options + +## 2018-01-23 + +- Thinking about generating a jmeter test plan for DSpace, along the lines of [Georgetown's dspace-performance-test](https://github.com/Georgetown-University-Libraries/dspace-performance-test) +- I got a list of all the GET requests on CGSpace for January 21st (the last time Linode complained the load was high), excluding admin calls: + +``` +# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -c -v "/admin" +56405 +``` + +- Apparently about 28% of these requests were for bitstreams, 30% for the REST API, and 30% for handles: + +``` +# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo "^/(handle|bitstream|rest|oai)/" | sort | uniq -c | sort -n + 38 /oai/ + 14406 /bitstream/ + 15179 /rest/ + 15191 /handle/ +``` + +- And 3% were to the homepage or search: + +``` +# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo '^/($|open-search|discover)' | sort | uniq -c + 1050 / + 413 /discover + 170 /open-search +``` + +- The last 10% or so seem to be for static assets that would be served by nginx anyways: + +``` +# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -v bitstream | grep -Eo '\.(js|css|png|jpg|jpeg|php|svg|gif|txt|map)$' | sort | uniq -c | sort -n + 2 .gif + 7 .css + 84 .js + 433 .php + 882 .txt + 2551 .png +``` + +- I can definitely design a test plan on this! diff --git a/public/2018-01/index.html b/public/2018-01/index.html index ef7222a70..169934994 100644 --- a/public/2018-01/index.html +++ b/public/2018-01/index.html @@ -92,7 +92,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv - + @@ -194,9 +194,9 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv "@type": "BlogPosting", "headline": "January, 2018", "url": "https://alanorth.github.io/cgspace-notes/2018-01/", - "wordCount": "5709", + "wordCount": "5977", "datePublished": "2018-01-02T08:35:54-08:00", - "dateModified": "2018-01-22T23:24:39+02:00", + "dateModified": "2018-01-23T00:08:32+02:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -1369,6 +1369,55 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled
  • I can’t remember if I had to configure the jmx settings in /etc/munin/plugin-conf.d/munin-node or not—I think all I did was re-run the munin-node-configure script and of course enable JMX in Tomcat’s JVM options
  • +

    2018-01-23

    + + + +
    # zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -c -v "/admin"
    +56405
    +
    + + + +
    # zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo "^/(handle|bitstream|rest|oai)/" | sort | uniq -c | sort -n
    +     38 /oai/
    +  14406 /bitstream/
    +  15179 /rest/
    +  15191 /handle/
    +
    + + + +
    # zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo '^/($|open-search|discover)' | sort | uniq -c
    +   1050 /
    +    413 /discover
    +    170 /open-search
    +
    + + + +
    # zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -v bitstream | grep -Eo '\.(js|css|png|jpg|jpeg|php|svg|gif|txt|map)$' | sort | uniq -c | sort -n
    +      2 .gif
    +      7 .css
    +     84 .js
    +    433 .php
    +    882 .txt
    +   2551 .png
    +
    + + + diff --git a/public/sitemap.xml b/public/sitemap.xml index 36cf8e889..d037301a4 100644 --- a/public/sitemap.xml +++ b/public/sitemap.xml @@ -4,7 +4,7 @@ https://alanorth.github.io/cgspace-notes/2018-01/ - 2018-01-22T23:24:39+02:00 + 2018-01-23T00:08:32+02:00 @@ -144,7 +144,7 @@ https://alanorth.github.io/cgspace-notes/ - 2018-01-22T23:24:39+02:00 + 2018-01-23T00:08:32+02:00 0 @@ -155,7 +155,7 @@ https://alanorth.github.io/cgspace-notes/tags/notes/ - 2018-01-22T23:24:39+02:00 + 2018-01-23T00:08:32+02:00 0 @@ -167,13 +167,13 @@ https://alanorth.github.io/cgspace-notes/post/ - 2018-01-22T23:24:39+02:00 + 2018-01-23T00:08:32+02:00 0 https://alanorth.github.io/cgspace-notes/tags/ - 2018-01-22T23:24:39+02:00 + 2018-01-23T00:08:32+02:00 0