Update notes

content/post/2018-01.md

@@ -1004,3 +1004,46 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled
 - I see I can monitor the number of Tomcat threads and some detailed JVM memory stuff if I install `munin-plugins-java`
 - I'd still like to get arbitrary mbeans like activeSessions etc, though
 - I can't remember if I had to configure the jmx settings in `/etc/munin/plugin-conf.d/munin-node` or not—I think all I did was re-run the `munin-node-configure` script and of course enable JMX in Tomcat's JVM options
+
+## 2018-01-23
+
+- Thinking about generating a jmeter test plan for DSpace, along the lines of [Georgetown's dspace-performance-test](https://github.com/Georgetown-University-Libraries/dspace-performance-test)
+- I got a list of all the GET requests on CGSpace for January 21st (the last time Linode complained the load was high), excluding admin calls:
+
+```
+# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -c -v "/admin"
+56405
+```
+
+- Apparently about 28% of these requests were for bitstreams, 30% for the REST API, and 30% for handles:
+
+```
+# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo "^/(handle|bitstream|rest|oai)/" | sort | uniq -c | sort -n
+     38 /oai/
+  14406 /bitstream/
+  15179 /rest/
+  15191 /handle/
+```
+
+- And 3% were to the homepage or search:
+
+```
+# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -Eo '^/($|open-search|discover)' | sort | uniq -c
+   1050 /
+    413 /discover
+    170 /open-search
+```
+
+- The last 10% or so seem to be for static assets that would be served by nginx anyways:
+
+```
+# zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -v bitstream | grep -Eo '\.(js|css|png|jpg|jpeg|php|svg|gif|txt|map)$' | sort | uniq -c | sort -n
+      2 .gif
+      7 .css
+     84 .js
+    433 .php
+    882 .txt
+   2551 .png
+```
+
+- I can definitely design a test plan on this!

public/2018-01/index.html

@@ -92,7 +92,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv
 
 <meta property="article:published_time" content="2018-01-02T08:35:54-08:00"/>
 
-<meta property="article:modified_time" content="2018-01-22T23:24:39&#43;02:00"/>
+<meta property="article:modified_time" content="2018-01-23T00:08:32&#43;02:00"/>
 
 
 
@@ -194,9 +194,9 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv
   "@type": "BlogPosting",
   "headline": "January, 2018",
   "url": "https://alanorth.github.io/cgspace-notes/2018-01/",
-  "wordCount": "5709",
+  "wordCount": "5977",
   "datePublished": "2018-01-02T08:35:54-08:00",
-  "dateModified": "2018-01-22T23:24:39&#43;02:00",
+  "dateModified": "2018-01-23T00:08:32&#43;02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -1369,6 +1369,55 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled
 <li>I can&rsquo;t remember if I had to configure the jmx settings in <code>/etc/munin/plugin-conf.d/munin-node</code> or not—I think all I did was re-run the <code>munin-node-configure</code> script and of course enable JMX in Tomcat&rsquo;s JVM options</li>
 </ul>
 
+<h2 id="2018-01-23">2018-01-23</h2>
+
+<ul>
+<li>Thinking about generating a jmeter test plan for DSpace, along the lines of <a href="https://github.com/Georgetown-University-Libraries/dspace-performance-test">Georgetown&rsquo;s dspace-performance-test</a></li>
+<li>I got a list of all the GET requests on CGSpace for January 21st (the last time Linode complained the load was high), excluding admin calls:</li>
+</ul>
+
+<pre><code># zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep &quot;21/Jan/2018&quot; | grep &quot;GET &quot; | grep -c -v &quot;/admin&quot;
+56405
+</code></pre>
+
+<ul>
+<li>Apparently about 28% of these requests were for bitstreams, 30% for the REST API, and 30% for handles:</li>
+</ul>
+
+<pre><code># zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep &quot;21/Jan/2018&quot; | grep &quot;GET &quot; | grep -v &quot;/admin&quot; | awk '{print $7}' | grep -Eo &quot;^/(handle|bitstream|rest|oai)/&quot; | sort | uniq -c | sort -n
+     38 /oai/
+  14406 /bitstream/
+  15179 /rest/
+  15191 /handle/
+</code></pre>
+
+<ul>
+<li>And 3% were to the homepage or search:</li>
+</ul>
+
+<pre><code># zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep &quot;21/Jan/2018&quot; | grep &quot;GET &quot; | grep -v &quot;/admin&quot; | awk '{print $7}' | grep -Eo '^/($|open-search|discover)' | sort | uniq -c
+   1050 /
+    413 /discover
+    170 /open-search
+</code></pre>
+
+<ul>
+<li>The last 10% or so seem to be for static assets that would be served by nginx anyways:</li>
+</ul>
+
+<pre><code># zcat --force /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz /var/log/nginx/library-access.log.2.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/rest.log.2.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/oai.log.2.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/error.log.2.gz /var/log/nginx/error.log.3.gz | grep &quot;21/Jan/2018&quot; | grep &quot;GET &quot; | grep -v &quot;/admin&quot; | awk '{print $7}' | grep -v bitstream | grep -Eo '\.(js|css|png|jpg|jpeg|php|svg|gif|txt|map)$' | sort | uniq -c | sort -n
+      2 .gif
+      7 .css
+     84 .js
+    433 .php
+    882 .txt
+   2551 .png
+</code></pre>
+
+<ul>
+<li>I can definitely design a test plan on this!</li>
+</ul>
+
   
 
   

public/sitemap.xml

@@ -4,7 +4,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2018-01/</loc>
-    <lastmod>2018-01-22T23:24:39+02:00</lastmod>
+    <lastmod>2018-01-23T00:08:32+02:00</lastmod>
   </url>
   
   <url>
@@ -144,7 +144,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2018-01-22T23:24:39+02:00</lastmod>
+    <lastmod>2018-01-23T00:08:32+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -155,7 +155,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2018-01-22T23:24:39+02:00</lastmod>
+    <lastmod>2018-01-23T00:08:32+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -167,13 +167,13 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/post/</loc>
-    <lastmod>2018-01-22T23:24:39+02:00</lastmod>
+    <lastmod>2018-01-23T00:08:32+02:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2018-01-22T23:24:39+02:00</lastmod>
+    <lastmod>2018-01-23T00:08:32+02:00</lastmod>
     <priority>0</priority>
   </url>