Add notes for 2020-12-21

2024-12-22 13:12:19 +01:00 · 2020-12-21 08:47:34 +02:00 · 2020-12-21 08:47:34 +02:00 · 4717a187c3
commit 4717a187c3
parent 461428c926
23 changed files with 96 additions and 31 deletions
--- a/content/posts/2020-12.md
+++ b/content/posts/2020-12.md
@ -592,6 +592,42 @@ java.lang.OutOfMemoryError: Java heap space
 - I looked in Solr and found dozens of duplicates of each field again...
  - I sent [feedback to Atmire](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=839)
 - I finished the technical work on adding community and collection support to the DSpace Statistics API
-  - I still need to update the tests as well as the documentation
+  - I still need to update ~~the tests~~ as well as the documentation
+- I started a harvesting of AReS
+
+## 2020-12-21
+
+- The AReS harvest finished this morning and I moved the Elasticsearch index manually
+- First, check the number of records in the temp index to make sure it seems complete and not with double data:
+
+```console
+$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&pretty'
+{
+  "count" : 100135,
+  "_shards" : {
+    "total" : 1,
+    "successful" : 1,
+    "skipped" : 0,
+    "failed" : 0
+  }
+}
+```
+
+- Then delete the old backup and clone the current items index as a backup:
+
+```console
+$ curl -XDELETE 'http://localhost:9200/openrxv-items-2020-12-14?pretty'
+$ curl -X PUT "localhost:9200/openrxv-items/_settings?pretty" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
+$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2020-12-21
+```
+
+- Then delete the current items index and clone it from temp:
+
+```console
+$ curl -XDELETE 'http://localhost:9200/openrxv-items?pretty'
+$ curl -X PUT "localhost:9200/openrxv-items-temp/_settings?pretty" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
+$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
+$ curl -X PUT "localhost:9200/openrxv-items-temp/_settings?pretty" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": false}}'
+```

 <!-- vim: set sw=2 ts=2: -->
--- a/docs/2020-12/index.html
+++ b/docs/2020-12/index.html
@ -20,7 +20,7 @@ I started processing those (about 411,000 records):
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-12/" />
 <meta property="article:published_time" content="2020-12-01T11:32:54+02:00" />
-<meta property="article:modified_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="article:modified_time" content="2020-12-20T16:47:45+02:00" />



@ -46,9 +46,9 @@ I started processing those (about 411,000 records):
  "@type": "BlogPosting",
  "headline": "December, 2020",
  "url": "https://alanorth.github.io/cgspace-notes/2020-12/",
-  "wordCount": "3242",
+  "wordCount": "3386",
  "datePublished": "2020-12-01T11:32:54+02:00",
-  "dateModified": "2020-12-17T16:50:56+02:00",
+  "dateModified": "2020-12-20T16:47:45+02:00",
  "author": {
    "@type": "Person",
    "name": "Alan Orth"
@ -727,11 +727,40 @@ java.lang.OutOfMemoryError: Java heap space
 </li>
 <li>I finished the technical work on adding community and collection support to the DSpace Statistics API
 <ul>
-<li>I still need to update the tests as well as the documentation</li>
+<li>I still need to update <del>the tests</del> as well as the documentation</li>
 </ul>
 </li>
+<li>I started a harvesting of AReS</li>
 </ul>
-<!-- raw HTML omitted -->
+<h2 id="2020-12-21">2020-12-21</h2>
+<ul>
+<li>The AReS harvest finished this morning and I moved the Elasticsearch index manually</li>
+<li>First, check the number of records in the temp index to make sure it seems complete and not with double data:</li>
+</ul>
+<pre><code class="language-console" data-lang="console">$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&amp;pretty'
+{
+  &quot;count&quot; : 100135,
+  &quot;_shards&quot; : {
+    &quot;total&quot; : 1,
+    &quot;successful&quot; : 1,
+    &quot;skipped&quot; : 0,
+    &quot;failed&quot; : 0
+  }
+}
+</code></pre><ul>
+<li>Then delete the old backup and clone the current items index as a backup:</li>
+</ul>
+<pre><code class="language-console" data-lang="console">$ curl -XDELETE 'http://localhost:9200/openrxv-items-2020-12-14?pretty'
+$ curl -X PUT &quot;localhost:9200/openrxv-items/_settings?pretty&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: true}}'
+$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2020-12-21
+</code></pre><ul>
+<li>Then delete the current items index and clone it from temp:</li>
+</ul>
+<pre><code class="language-console" data-lang="console">$ curl -XDELETE 'http://localhost:9200/openrxv-items?pretty'
+$ curl -X PUT &quot;localhost:9200/openrxv-items-temp/_settings?pretty&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: true}}'
+$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
+$ curl -X PUT &quot;localhost:9200/openrxv-items-temp/_settings?pretty&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: false}}'
+</code></pre><!-- raw HTML omitted -->

  

--- a/docs/categories/index.html
+++ b/docs/categories/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/categories/notes/index.html
+++ b/docs/categories/notes/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/categories/notes/page/2/index.html
+++ b/docs/categories/notes/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/categories/notes/page/3/index.html
+++ b/docs/categories/notes/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/categories/notes/page/4/index.html
+++ b/docs/categories/notes/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/categories/notes/page/5/index.html
+++ b/docs/categories/notes/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/index.html
+++ b/docs/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/2/index.html
+++ b/docs/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/3/index.html
+++ b/docs/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/4/index.html
+++ b/docs/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/5/index.html
+++ b/docs/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/6/index.html
+++ b/docs/page/6/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/page/7/index.html
+++ b/docs/page/7/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/index.html
+++ b/docs/posts/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/2/index.html
+++ b/docs/posts/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/3/index.html
+++ b/docs/posts/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/4/index.html
+++ b/docs/posts/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/5/index.html
+++ b/docs/posts/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/6/index.html
+++ b/docs/posts/page/6/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/posts/page/7/index.html
+++ b/docs/posts/page/7/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2020-12-17T16:50:56+02:00" />
+<meta property="og:updated_time" content="2020-12-20T16:47:45+02:00" />



--- a/docs/sitemap.xml
+++ b/docs/sitemap.xml
@ -4,27 +4,27 @@
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
-    <lastmod>2020-12-17T16:50:56+02:00</lastmod>
+    <lastmod>2020-12-20T16:47:45+02:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2020-12-17T16:50:56+02:00</lastmod>
+    <lastmod>2020-12-20T16:47:45+02:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/2020-12/</loc>
-    <lastmod>2020-12-17T16:50:56+02:00</lastmod>
+    <lastmod>2020-12-20T16:47:45+02:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
-    <lastmod>2020-12-17T16:50:56+02:00</lastmod>
+    <lastmod>2020-12-20T16:47:45+02:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2020-12-17T16:50:56+02:00</lastmod>
+    <lastmod>2020-12-20T16:47:45+02:00</lastmod>
  </url>
  
  <url>