alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-22 05:02:19 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2022-05-05

Browse Source
This commit is contained in:
Alan Orth 2022-05-05 12:46:13 +03:00
parent ca294919f2
commit 3890c1fd7d
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
28 changed files with 109 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
content/posts/2022-05.md
View File

@ -43,4 +43,41 @@ categories: ["Notes"]
- Run all system updates on AReS server, update all Docker containers, and restart the server
- Start a harvest on AReS
## 2022-05-05
- Update PostgreSQL JDBC driver to 42.3.5 in the Ansible infrastructure playbooks and deploy on DSpace Test
- Peter asked me how many items we add to CGSpace every year
- I wrote a SQL query to check the number of items grouped by their accession dates since 2009:
```console
localhost/dspacetest= ☘ SELECT EXTRACT(year from text_value::date) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
yyyy │ count
──────┼───────
2022 │ 2073
2021 │ 6471
2020 │ 4074
2019 │ 7330
2018 │ 8899
2017 │ 6860
2016 │ 8451
2015 │ 15692
2014 │ 16479
2013 │ 4388
2012 │ 6472
2011 │ 2694
2010 │ 2457
2009 │ 293
```
- Note that I had an issue with casting `text_value` to date because one item had an accession date of `2016` instead of `2016-09-29T20:14:47Z`
- Once I fixed that PostgreSQL was able to [extract() the year](https://www.postgresql.org/docs/12/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT)
- There were some other methods I tried that worked also, for example `TO_DATE()`:
```console
localhost/dspacetest= ☘ SELECT EXTRACT(year from TO_DATE(text_value, 'YYYY-MM-DD"T"HH24:MI:SS"Z"')) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
```
- But it seems PostgreSQL is smart enough to recognize date formatting in strings automatically when we cast so we don't need to convert to date first
- Another thing I noticed is that a few hundred items have accession dates from decades ago, perhaps this is due to importing items from the CGIAR Library?
<!-- vim: set sw=2 ts=2: -->

45
docs/2022-05/index.html
View File

@ -35,7 +35,7 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-05/" />
<meta property="article:published_time" content="2022-05-04T09:13:39+03:00" />
<meta property="article:modified_time" content="2022-05-04T11:09:45+03:00" />
<meta property="article:modified_time" content="2022-05-04T16:48:24+03:00" />
@ -76,9 +76,9 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
"@type": "BlogPosting",
"headline": "May, 2022",
"url": "https://alanorth.github.io/cgspace-notes/2022-05/",
"wordCount": "300",
"wordCount": "538",
"datePublished": "2022-05-04T09:13:39+03:00",
"dateModified": "2022-05-04T11:09:45+03:00",
"dateModified": "2022-05-04T16:48:24+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -204,6 +204,45 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
</ul>
</li>
</ul>
<h2 id="2022-05-05">2022-05-05</h2>
<ul>
<li>Update PostgreSQL JDBC driver to 42.3.5 in the Ansible infrastructure playbooks and deploy on DSpace Test</li>
<li>Peter asked me how many items we add to CGSpace every year
<ul>
<li>I wrote a SQL query to check the number of items grouped by their accession dates since 2009:</li>
</ul>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>localhost/dspacetest= ☘ SELECT EXTRACT(year from text_value::date) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
</span></span><span style="display:flex;"><span> yyyy │ count
</span></span><span style="display:flex;"><span>──────┼───────
</span></span><span style="display:flex;"><span> 2022 │ 2073
</span></span><span style="display:flex;"><span> 2021 │ 6471
</span></span><span style="display:flex;"><span> 2020 │ 4074
</span></span><span style="display:flex;"><span> 2019 │ 7330
</span></span><span style="display:flex;"><span> 2018 │ 8899
</span></span><span style="display:flex;"><span> 2017 │ 6860
</span></span><span style="display:flex;"><span> 2016 │ 8451
</span></span><span style="display:flex;"><span> 2015 │ 15692
</span></span><span style="display:flex;"><span> 2014 │ 16479
</span></span><span style="display:flex;"><span> 2013 │ 4388
</span></span><span style="display:flex;"><span> 2012 │ 6472
</span></span><span style="display:flex;"><span> 2011 │ 2694
</span></span><span style="display:flex;"><span> 2010 │ 2457
</span></span><span style="display:flex;"><span> 2009 │ 293
</span></span></code></pre></div><ul>
<li>Note that I had an issue with casting <code>text_value</code> to date because one item had an accession date of <code>2016</code> instead of <code>2016-09-29T20:14:47Z</code>
<ul>
<li>Once I fixed that PostgreSQL was able to <a href="https://www.postgresql.org/docs/12/functions-datetime.html#FUNCTIONS-DATETIME-EXTRACT">extract() the year</a></li>
<li>There were some other methods I tried that worked also, for example <code>TO_DATE()</code>:</li>
</ul>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>localhost/dspacetest= ☘ SELECT EXTRACT(year from TO_DATE(text_value, &#39;YYYY-MM-DD&#34;T&#34;HH24:MI:SS&#34;Z&#34;&#39;)) AS YYYY, COUNT(*) FROM metadatavalue WHERE metadata_field_id=11 GROUP BY YYYY ORDER BY YYYY DESC LIMIT 14;
</span></span></code></pre></div><ul>
<li>But it seems PostgreSQL is smart enough to recognize date formatting in strings automatically when we cast so we don&rsquo;t need to convert to date first</li>
<li>Another thing I noticed is that a few hundred items have accession dates from decades ago, perhaps this is due to importing items from the CGIAR Library?</li>
</ul>
<!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/categories/notes/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

2
docs/posts/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-04T11:09:45+03:00" />
<meta property="og:updated_time" content="2022-05-04T16:48:24+03:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>
<lastmod>2022-05-04T16:48:24+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>
<lastmod>2022-05-04T16:48:24+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-05/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>
<lastmod>2022-05-04T16:48:24+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>
<lastmod>2022-05-04T16:48:24+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>
<lastmod>2022-05-04T16:48:24+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-04/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>