alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-26 23:14:29 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2020-12-10

Browse Source
This commit is contained in:
Alan Orth 2020-12-10 23:43:09 +02:00
parent a61fd6b1ad
commit 7a83648463
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
29 changed files with 72 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
content/posts/2020-12.md
View File

@ -184,4 +184,23 @@ Caused by: org.apache.http.TruncatedChunkException: Truncated chunk ( expected s
...
```
## 2020-12-10
- The statistics-2019 core finished processing the duplicate removal so I started the statistics-2017 core
- Peter asked me to add ONE HEALTH to ILRI subjects on CGSpace
- A few items that got "lost" after approval during the database issues earlier this week seem to have gone back into their workflows
- Abenet approved them again and they got new handles, phew
- Abenet was having an issue with the date filter on AReS and it turns out that it's the same `.keyword` issue I had noticed before that causes the filter to stop working
- I fixed the filter to use the correct field name and filed a bug on OpenRXV: https://github.com/ilri/OpenRXV/issues/63
- I checked the Solr statistics on DSpace Test to see if the Atmire duplicates remover was working, but now I see a comical amount of duplicates...
![Solr stats with dozens of duplicates](/cgspace-notes/2020/12/solr-stats-duplicates.png)
- I sent feedback about this to Atmire
- I will re-sync the Solr stats from CGSpace so we can try again...
- In other news, it has been a few days since we deployed the fix for the database locking issue and things seem much better now:
![PostgreSQL connections all week](/cgspace-notes/2020/12/postgres_connections_ALL-week.png)
![PostgreSQL locks all week](/cgspace-notes/2020/12/postgres_locks_ALL-week.png)
<!-- vim: set sw=2 ts=2: -->

32
docs/2020-12/index.html
View File

@ -20,7 +20,7 @@ I started processing those (about 411,000 records):
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-12/" />
<meta property="article:published_time" content="2020-12-01T11:32:54+02:00" />
<meta property="article:modified_time" content="2020-12-08T11:14:33+02:00" />
<meta property="article:modified_time" content="2020-12-09T22:48:19+02:00" />
@ -46,9 +46,9 @@ I started processing those (about 411,000 records):
"@type": "BlogPosting",
"headline": "December, 2020",
"url": "https://alanorth.github.io/cgspace-notes/2020-12/",
"wordCount": "1051",
"wordCount": "1231",
"datePublished": "2020-12-01T11:32:54+02:00",
"dateModified": "2020-12-08T11:14:33+02:00",
"dateModified": "2020-12-09T22:48:19+02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -309,7 +309,31 @@ Caused by: org.apache.http.TruncatedChunkException: Truncated chunk ( expected s
at org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:125)
at org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:152)
...
</code></pre><!-- raw HTML omitted -->
</code></pre><h2 id="2020-12-10">2020-12-10</h2>
<ul>
<li>The statistics-2019 core finished processing the duplicate removal so I started the statistics-2017 core</li>
<li>Peter asked me to add ONE HEALTH to ILRI subjects on CGSpace</li>
<li>A few items that got &ldquo;lost&rdquo; after approval during the database issues earlier this week seem to have gone back into their workflows
<ul>
<li>Abenet approved them again and they got new handles, phew</li>
</ul>
</li>
<li>Abenet was having an issue with the date filter on AReS and it turns out that it&rsquo;s the same <code>.keyword</code> issue I had noticed before that causes the filter to stop working
<ul>
<li>I fixed the filter to use the correct field name and filed a bug on OpenRXV: <a href="https://github.com/ilri/OpenRXV/issues/63">https://github.com/ilri/OpenRXV/issues/63</a></li>
</ul>
</li>
<li>I checked the Solr statistics on DSpace Test to see if the Atmire duplicates remover was working, but now I see a comical amount of duplicates&hellip;</li>
</ul>
<p><img src="/cgspace-notes/2020/12/solr-stats-duplicates.png" alt="Solr stats with dozens of duplicates"></p>
<ul>
<li>I sent feedback about this to Atmire</li>
<li>I will re-sync the Solr stats from CGSpace so we can try again&hellip;</li>
<li>In other news, it has been a few days since we deployed the fix for the database locking issue and things seem much better now:</li>
</ul>
<p><img src="/cgspace-notes/2020/12/postgres_connections_ALL-week.png" alt="PostgreSQL connections all week">
<img src="/cgspace-notes/2020/12/postgres_locks_ALL-week.png" alt="PostgreSQL locks all week"></p>
<!-- raw HTML omitted -->

BIN
docs/2020/12/postgres_connections_ALL-week.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
docs/2020/12/postgres_locks_ALL-week.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
docs/2020/12/solr-stats-duplicates.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 139 KiB

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2020-12-08T11:14:33+02:00" />
<meta property="og:updated_time" content="2020-12-09T22:48:19+02:00" />

10
docs/sitemap.xml
View File

@ -4,27 +4,27 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2020-12-08T11:14:33+02:00</lastmod>
<lastmod>2020-12-09T22:48:19+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2020-12-08T11:14:33+02:00</lastmod>
<lastmod>2020-12-09T22:48:19+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/2020-12/</loc>
<lastmod>2020-12-08T11:14:33+02:00</lastmod>
<lastmod>2020-12-09T22:48:19+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2020-12-08T11:14:33+02:00</lastmod>
<lastmod>2020-12-09T22:48:19+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2020-12-08T11:14:33+02:00</lastmod>
<lastmod>2020-12-09T22:48:19+02:00</lastmod>
</url>
<url>

BIN
static/2020/12/postgres_connections_ALL-week.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
static/2020/12/postgres_locks_ALL-week.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
static/2020/12/solr-stats-duplicates.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 139 KiB