Update notes for 2017-12-20

content/post/2017-12.md

@@ -423,18 +423,18 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery
 - Then I create a global `Resource` in the main Tomcat _server.xml_ (inside `GlobalNamingResources`):
 
 ```
-	<Resource name="jdbc/dspace" auth="Container" type="javax.sql.DataSource"
+<Resource name="jdbc/dspace" auth="Container" type="javax.sql.DataSource"
-		  driverClassName="org.postgresql.Driver"
+	  driverClassName="org.postgresql.Driver"
-		  url="jdbc:postgresql://localhost:5432/dspace"
+	  url="jdbc:postgresql://localhost:5432/dspace"
-		  username="dspace"
+	  username="dspace"
-		  password="dspace"
+	  password="dspace"
-          initialSize='5'
+      initialSize='5'
-          maxActive='50'
+      maxActive='50'
-          maxIdle='15'
+      maxIdle='15'
-          minIdle='5'
+      minIdle='5'
-          maxWait='5000'
+      maxWait='5000'
-          validationQuery='SELECT 1'
+      validationQuery='SELECT 1'
-          testOnBorrow='true' />
+      testOnBorrow='true' />
 ```
 
 - Most of the parameters are from comments by Mark Wood about his JNDI setup: https://jira.duraspace.org/browse/DS-3564
@@ -623,3 +623,30 @@ javax.naming.NoInitialContextException: Need to specify class name in environmen
 - Oh cool! `select * from pg_stat_activity` shows "PostgreSQL JDBC Driver" for the application name! That's how you know it's working!
 - If you monitor the `pg_stat_activity` while you run `dspace database info` you can see that it doesn't use the JNDI and creates ~9 extra PostgreSQL connections!
 - And in the middle of all of this Linode sends an alert that CGSpace has high CPU usage from 2 to 4 PM
+
+## 2017-12-20
+
+- The database connection pooling is definitely better!
+
+![PostgreSQL connection pooling on DSpace Test](/cgspace-notes/2017/12/postgres-connections-week-dspacetest.png)
+
+- Now there is only one set of idle connections shared among all the web applications, instead of 10+ per application
+- There are short bursts of connections up to 10, but it generally stays around 5
+- Test and import 13 records to CGSpace for Abenet:
+
+```
+$ export JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx512m -XX:+TieredCompilation -XX:TieredStopAtLevel=1"
+$ dspace import -a -e aorth@mjanja.ch -s /home/aorth/cg_system_20Dec/SimpleArchiveFormat -m systemoffice.map &> systemoffice.log
+```
+
+- The fucking database went from 47 to 72 to 121 connections while I was importing so it stalled.
+- Since I had to restart Tomcat anyways, I decided to just deploy the new JNDI connection pooling stuff on CGSpace
+- There was an initial connection storm of 50 PostgreSQL connections, but then it settled down to 7
+- After that CGSpace came up fine and I was able to import the 13 items just fine:
+
+```
+$ dspace import -a -e aorth@mjanja.ch -s /home/aorth/cg_system_20Dec/SimpleArchiveFormat -m systemoffice.map &> systemoffice.log
+$ schedtool -D -e ionice -c2 -n7 nice -n19 dspace filter-media -i 10568/89287
+```
+
+- The final code for the JNDI work in the Ansible infrastructure scripts is here: https://github.com/ilri/rmg-ansible-public/commit/1959d9cb7a0e7a7318c77f769253e5e029bdfa3b

public/2017-12/index.html

@@ -23,7 +23,7 @@ The list of connections to XMLUI and REST API for today:
 
 <meta property="article:published_time" content="2017-12-01T13:53:54&#43;03:00"/>
 
-<meta property="article:modified_time" content="2017-12-19T18:33:19&#43;02:00"/>
+<meta property="article:modified_time" content="2017-12-19T19:16:47&#43;02:00"/>
 
 
 
@@ -56,9 +56,9 @@ The list of connections to XMLUI and REST API for today:
   "@type": "BlogPosting",
   "headline": "December, 2017",
   "url": "https://alanorth.github.io/cgspace-notes/2017-12/",
-  "wordCount": "3411",
+  "wordCount": "3593",
   "datePublished": "2017-12-01T13:53:54&#43;03:00",
-  "dateModified": "2017-12-19T18:33:19&#43;02:00",
+  "dateModified": "2017-12-19T19:16:47&#43;02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -591,18 +591,18 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery
 <li>Then I create a global <code>Resource</code> in the main Tomcat <em>server.xml</em> (inside <code>GlobalNamingResources</code>):</li>
 </ul>
 
-<pre><code>	&lt;Resource name=&quot;jdbc/dspace&quot; auth=&quot;Container&quot; type=&quot;javax.sql.DataSource&quot;
+<pre><code>&lt;Resource name=&quot;jdbc/dspace&quot; auth=&quot;Container&quot; type=&quot;javax.sql.DataSource&quot;
-		  driverClassName=&quot;org.postgresql.Driver&quot;
+	  driverClassName=&quot;org.postgresql.Driver&quot;
-		  url=&quot;jdbc:postgresql://localhost:5432/dspace&quot;
+	  url=&quot;jdbc:postgresql://localhost:5432/dspace&quot;
-		  username=&quot;dspace&quot;
+	  username=&quot;dspace&quot;
-		  password=&quot;dspace&quot;
+	  password=&quot;dspace&quot;
-          initialSize='5'
+      initialSize='5'
-          maxActive='50'
+      maxActive='50'
-          maxIdle='15'
+      maxIdle='15'
-          minIdle='5'
+      minIdle='5'
-          maxWait='5000'
+      maxWait='5000'
-          validationQuery='SELECT 1'
+      validationQuery='SELECT 1'
-          testOnBorrow='true' /&gt;
+      testOnBorrow='true' /&gt;
 </code></pre>
 
 <ul>
@@ -811,6 +811,39 @@ javax.naming.NoInitialContextException: Need to specify class name in environmen
 <li>And in the middle of all of this Linode sends an alert that CGSpace has high CPU usage from 2 to 4 PM</li>
 </ul>
 
+<h2 id="2017-12-20">2017-12-20</h2>
+
+<ul>
+<li>The database connection pooling is definitely better!</li>
+</ul>
+
+<p><img src="/cgspace-notes/2017/12/postgres-connections-week-dspacetest.png" alt="PostgreSQL connection pooling on DSpace Test" /></p>
+
+<ul>
+<li>Now there is only one set of idle connections shared among all the web applications, instead of 10+ per application</li>
+<li>There are short bursts of connections up to 10, but it generally stays around 5</li>
+<li>Test and import 13 records to CGSpace for Abenet:</li>
+</ul>
+
+<pre><code>$ export JAVA_OPTS=&quot;-Dfile.encoding=UTF-8 -Xmx512m -XX:+TieredCompilation -XX:TieredStopAtLevel=1&quot;
+$ dspace import -a -e aorth@mjanja.ch -s /home/aorth/cg_system_20Dec/SimpleArchiveFormat -m systemoffice.map &amp;&gt; systemoffice.log
+</code></pre>
+
+<ul>
+<li>The fucking database went from 47 to 72 to 121 connections while I was importing so it stalled.</li>
+<li>Since I had to restart Tomcat anyways, I decided to just deploy the new JNDI connection pooling stuff on CGSpace</li>
+<li>There was an initial connection storm of 50 PostgreSQL connections, but then it settled down to 7</li>
+<li>After that CGSpace came up fine and I was able to import the 13 items just fine:</li>
+</ul>
+
+<pre><code>$ dspace import -a -e aorth@mjanja.ch -s /home/aorth/cg_system_20Dec/SimpleArchiveFormat -m systemoffice.map &amp;&gt; systemoffice.log
+$ schedtool -D -e ionice -c2 -n7 nice -n19 dspace filter-media -i 10568/89287
+</code></pre>
+
+<ul>
+<li>The final code for the JNDI work in the Ansible infrastructure scripts is here: <a href="https://github.com/ilri/rmg-ansible-public/commit/1959d9cb7a0e7a7318c77f769253e5e029bdfa3b">https://github.com/ilri/rmg-ansible-public/commit/1959d9cb7a0e7a7318c77f769253e5e029bdfa3b</a></li>
+</ul>
+
   
 
   

public/sitemap.xml

@@ -4,7 +4,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2017-12/</loc>
-    <lastmod>2017-12-19T18:33:19+02:00</lastmod>
+    <lastmod>2017-12-19T19:16:47+02:00</lastmod>
   </url>
   
   <url>
@@ -139,7 +139,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2017-12-19T18:33:19+02:00</lastmod>
+    <lastmod>2017-12-19T19:16:47+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -150,7 +150,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2017-12-19T18:33:19+02:00</lastmod>
+    <lastmod>2017-12-19T19:16:47+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -162,13 +162,13 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/post/</loc>
-    <lastmod>2017-12-19T18:33:19+02:00</lastmod>
+    <lastmod>2017-12-19T19:16:47+02:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2017-12-19T18:33:19+02:00</lastmod>
+    <lastmod>2017-12-19T19:16:47+02:00</lastmod>
     <priority>0</priority>
   </url>