Update notes for 2022-05-05

content/posts/2022-05.md

@@ -43,4 +43,42 @@ categories: ["Notes"]
 - Run all system updates on AReS server, update all Docker containers, and restart the server
   - Start a harvest on AReS
 
+## 2022-05-05
+
+- Update PostgreSQL JDBC driver to 42.3.5 in the Ansible infrastructure playbooks and deploy on DSpace Test
+- Peter asked me how many items we add to CGSpace every year
+  - I wrote a SQL query to check the number of items grouped by their accession dates since 2009:
+
+```console
+localhost/dspacetest= ☘ SELECT EXTRACT(year from text_value::date) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
+ yyyy │ count 
+──────┼───────
+ 2022 │  2073
+ 2021 │  6471
+ 2020 │  4074
+ 2019 │  7330
+ 2018 │  8899
+ 2017 │  6860
+ 2016 │  8451
+ 2015 │ 15692
+ 2014 │ 16479
+ 2013 │  4388
+ 2012 │  6472
+ 2011 │  2694
+ 2010 │  2457
+ 2009 │   293
+```
+
+- Note that I had an issue with casting `text_value` to date because one item had an accession date of `2016` instead of `2016-09-29T20:14:47Z`
+  - Once I fixed that PostgreSQL was able to [extract() the year](https://www.postgresql.org/docs/12/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT)
+  - There were some other methods I tried that worked also, for example `TO_DATE()`:
+
+```console
+localhost/dspacetest= ☘ SELECT EXTRACT(year from TO_DATE(text_value, 'YYYY-MM-DD"T"HH24:MI:SS"Z"')) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
+```
+
+- But it seems PostgreSQL is smart enough to recognize date formatting in strings automatically when we cast so we don't need to convert to date first
+- Another thing I noticed is that a few hundred items have accession dates from decades ago, perhaps this is due to importing items from the CGIAR Library?
+- I spent some time merging a few pull requests for DSpace 6.4 and porting one to `main` for DSpace 7.x
+
 <!-- vim: set sw=2 ts=2: -->

docs/2022-05/index.html

@@ -35,7 +35,7 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-05/" />
 <meta property="article:published_time" content="2022-05-04T09:13:39+03:00" />
-<meta property="article:modified_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="article:modified_time" content="2022-05-05T12:46:13+03:00" />
 
 
 
@@ -76,9 +76,9 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
   "@type": "BlogPosting",
   "headline": "May, 2022",
   "url": "https://alanorth.github.io/cgspace-notes/2022-05/",
-  "wordCount": "300",
+  "wordCount": "558",
   "datePublished": "2022-05-04T09:13:39+03:00",
-  "dateModified": "2022-05-04T11:09:45+03:00",
+  "dateModified": "2022-05-05T12:46:13+03:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -204,6 +204,46 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
 </ul>
 </li>
 </ul>
+<h2 id="2022-05-05">2022-05-05</h2>
+<ul>
+<li>Update PostgreSQL JDBC driver to 42.3.5 in the Ansible infrastructure playbooks and deploy on DSpace Test</li>
+<li>Peter asked me how many items we add to CGSpace every year
+<ul>
+<li>I wrote a SQL query to check the number of items grouped by their accession dates since 2009:</li>
+</ul>
+</li>
+</ul>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>localhost/dspacetest= ☘ SELECT EXTRACT(year from text_value::date) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
+</span></span><span style="display:flex;"><span> yyyy │ count 
+</span></span><span style="display:flex;"><span>──────┼───────
+</span></span><span style="display:flex;"><span> 2022 │  2073
+</span></span><span style="display:flex;"><span> 2021 │  6471
+</span></span><span style="display:flex;"><span> 2020 │  4074
+</span></span><span style="display:flex;"><span> 2019 │  7330
+</span></span><span style="display:flex;"><span> 2018 │  8899
+</span></span><span style="display:flex;"><span> 2017 │  6860
+</span></span><span style="display:flex;"><span> 2016 │  8451
+</span></span><span style="display:flex;"><span> 2015 │ 15692
+</span></span><span style="display:flex;"><span> 2014 │ 16479
+</span></span><span style="display:flex;"><span> 2013 │  4388
+</span></span><span style="display:flex;"><span> 2012 │  6472
+</span></span><span style="display:flex;"><span> 2011 │  2694
+</span></span><span style="display:flex;"><span> 2010 │  2457
+</span></span><span style="display:flex;"><span> 2009 │   293
+</span></span></code></pre></div><ul>
+<li>Note that I had an issue with casting <code>text_value</code> to date because one item had an accession date of <code>2016</code> instead of <code>2016-09-29T20:14:47Z</code>
+<ul>
+<li>Once I fixed that PostgreSQL was able to <a href="https://www.postgresql.org/docs/12/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT">extract() the year</a></li>
+<li>There were some other methods I tried that worked also, for example <code>TO_DATE()</code>:</li>
+</ul>
+</li>
+</ul>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>localhost/dspacetest= ☘ SELECT EXTRACT(year from TO_DATE(text_value, &#39;YYYY-MM-DD&#34;T&#34;HH24:MI:SS&#34;Z&#34;&#39;)) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
+</span></span></code></pre></div><ul>
+<li>But it seems PostgreSQL is smart enough to recognize date formatting in strings automatically when we cast so we don&rsquo;t need to convert to date first</li>
+<li>Another thing I noticed is that a few hundred items have accession dates from decades ago, perhaps this is due to importing items from the CGIAR Library?</li>
+<li>I spent some time merging a few pull requests for DSpace 6.4 and porting one to <code>main</code> for DSpace 7.x</li>
+</ul>
 <!-- raw HTML omitted -->
 
   

docs/categories/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/categories/notes/page/6/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/6/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/7/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/8/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/page/9/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/6/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/7/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/8/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/posts/page/9/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
+<meta property="og:updated_time" content="2022-05-05T12:46:13+03:00" />
 
 
 

docs/sitemap.xml

@@ -3,19 +3,19 @@
   xmlns:xhtml="http://www.w3.org/1999/xhtml">
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
-    <lastmod>2022-05-04T11:09:45+03:00</lastmod>
+    <lastmod>2022-05-05T12:46:13+03:00</lastmod>
   </url><url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2022-05-04T11:09:45+03:00</lastmod>
+    <lastmod>2022-05-05T12:46:13+03:00</lastmod>
   </url><url>
     <loc>https://alanorth.github.io/cgspace-notes/2022-05/</loc>
-    <lastmod>2022-05-04T11:09:45+03:00</lastmod>
+    <lastmod>2022-05-05T12:46:13+03:00</lastmod>
   </url><url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
-    <lastmod>2022-05-04T11:09:45+03:00</lastmod>
+    <lastmod>2022-05-05T12:46:13+03:00</lastmod>
   </url><url>
     <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2022-05-04T11:09:45+03:00</lastmod>
+    <lastmod>2022-05-05T12:46:13+03:00</lastmod>
   </url><url>
     <loc>https://alanorth.github.io/cgspace-notes/2022-04/</loc>
     <lastmod>2022-05-04T11:09:45+03:00</lastmod>