content/posts/2021-02.md

@@ -234,96 +234,4 @@ $ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
 # start indexing in AReS
 ```
 
-## 2021-02-08
-
-- Finish rotating the AReS indexes after the harvesting last night:
-
-```console
-$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&pretty'
-{
-  "count" : 100983,
-  "_shards" : {
-    "total" : 1,
-    "successful" : 1,
-    "skipped" : 0,
-    "failed" : 0
-  }
-}
-$ curl -X PUT "localhost:9200/openrxv-items/_settings" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write":true}}'
-$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2021-02-08
-$ curl -XDELETE 'http://localhost:9200/openrxv-items'
-$ curl -X PUT "localhost:9200/openrxv-items-temp/_settings" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
-$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
-$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
-$ curl -XDELETE 'http://localhost:9200/openrxv-items-2021-02-08'
-```
-
-## 2021-02-10
-
-- Talk to Abdullah from CodeObia about a few of the issues we filed on OpenRXV
-  - Verify a fix he made for the issue with spaces in template file names
-  - He says that the [Angular expressions support should be enabled](https://github.com/ilri/OpenRXV/issues/49), but I tried it and couldn't get a few simple examples working
-- Atmire responded to a few issues today:
-  - First, the one about a crash while exporting a community CSV, which appears to be a [vanilla DSpace issue with a patch in DSpace 6.4](https://jira.lyrasis.org/browse/DS-4211)
-  - Second, the MQM batch consumer issue, which appears to be harmless log spam in *most* cases and they have sent a patch that adjusts the logging as such
-  - Third, a version bump for CUA to fix the `java.lang.UnsupportedOperationException: Multiple update components target the same field:solr_update_time_stamp` error
-- I cherry-picked the patches for DS-4111 and was able to export the ILRI community finally, but the results are almost twice as many items as in the community!
-  - Investigating with csvcut I see there are some ids that appear up to five, six, or seven times!
-
-```console
-$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | wc -l
-30354
-$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | sort -u | wc -l         
-18555
-$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | sort | uniq -c | sort -h | tail     
-      5 c21a79e5-e24e-4861-aa07-e06703d1deb7
-      5 c2460aa1-ae28-4003-9a99-2d7c5cd7fd38
-      5 d73fb3ae-9fac-4f7e-990f-e394f344246c
-      5 dc0e24fa-b7f5-437e-ac09-e15c0704be00
-      5 dc50bcca-0abf-473f-8770-69d5ab95cc33
-      5 e714bdf9-cc0f-4d9a-a808-d572e25c9238
-      6 7dfd1c61-9e8c-4677-8d41-e1c4b11d867d
-      6 fb76888c-03ae-4d53-b27d-87d7ca91371a
-      6 ff42d1e6-c489-492c-a40a-803cabd901ed
-      7 094e9e1d-09ff-40ca-a6b9-eca580936147
-```
-
-- I added a comment to that bug to ask if this is a side effect of the patch
-- I started working on tagging pre-2010 ILRI items with license information, like we talked about with Peter and Abenet last week
-  - Due to the export bug I had to sort and remove duplicates first, then use csvgrep to filter out books and journal articles:
-
-```console
-$ csvcut -c 'id,dc.date.issued,dc.date.issued[],dc.date.issued[en_US],dc.rights,dc.rights[],dc.rights[en],dc.rights[en_US],dc.publisher,dc.publisher[],dc.publisher[en_US],dc.type[en_US]' /tmp/2021-02-10-ILRI.csv | csvgrep -c 'dc.type[en_US]' -r '^.+[^(Journal Item|Journal Article|Book|Book Chapter)]'
-```
-
-- I imported the CSV into OpenRefine and converted the date text values to date types so I could facet by dates before 2010:
-
-```console
-if(diff(value,"01/01/2010".toDate(),"days")<0, true, false)
-```
-
-- Then I filtered by publisher to make sure they were only ours:
-
-```console
-or(
-  value.contains("International Livestock Research Institute"),
-  value.contains("ILRI"),
-  value.contains("International Livestock Centre for Africa"),
-  value.contains("ILCA"),
-  value.contains("ILRAD"),
-  value.contains("International Laboratory for Research on Animal Diseases")
-)
-```
-
-- I tagged these pre-2010 items with "Other" if they didn't already have a license
-- I checked 2010 to 2015, and 2016 to date, but they were all tagged already!
-- In the end I added the "Other" license to 1,523 items from before 2010
-
-## 2021-02-11
-
-- CodeObia keeps working on a few more small issues on OpenRXV
-  - Abdullah sent fixes for two issues but I couldn't verify them myself so I asked him to check again
-  - Call with Abdullah and Yousef to discuss some issues
-  - We got the Angular expressions parser working...
-
 <!-- vim: set sw=2 ts=2: -->

docs/2021-02/index.html

@@ -32,7 +32,7 @@ $ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&pretty&#3
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2021-02/" />
 <meta property="article:published_time" content="2021-02-01T10:13:54+02:00" />
-<meta property="article:modified_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="article:modified_time" content="2021-02-06T14:00:36+02:00" />
 
 
 
@@ -70,9 +70,9 @@ $ curl -s &#39;http://localhost:9200/openrxv-items-temp/_count?q=*&amp;pretty&#3
   "@type": "BlogPosting",
   "headline": "February, 2021",
   "url": "https://alanorth.github.io/cgspace-notes/2021-02/",
-  "wordCount": "2064",
+  "wordCount": "1517",
   "datePublished": "2021-02-01T10:13:54+02:00",
-  "dateModified": "2021-02-11T09:36:09+02:00",
+  "dateModified": "2021-02-06T14:00:36+02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -389,103 +389,7 @@ sys     2m26.050s
 </ul>
 <pre><code class="language-console" data-lang="console">$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
 # start indexing in AReS
-</code></pre><h2 id="2021-02-08">2021-02-08</h2>
+</code></pre><!-- raw HTML omitted -->
-<ul>
-<li>Finish rotating the AReS indexes after the harvesting last night:</li>
-</ul>
-<pre><code class="language-console" data-lang="console">$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&amp;pretty'
-{
-  &quot;count&quot; : 100983,
-  &quot;_shards&quot; : {
-    &quot;total&quot; : 1,
-    &quot;successful&quot; : 1,
-    &quot;skipped&quot; : 0,
-    &quot;failed&quot; : 0
-  }
-}
-$ curl -X PUT &quot;localhost:9200/openrxv-items/_settings&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;:true}}'
-$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2021-02-08
-$ curl -XDELETE 'http://localhost:9200/openrxv-items'
-$ curl -X PUT &quot;localhost:9200/openrxv-items-temp/_settings&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: true}}'
-$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
-$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
-$ curl -XDELETE 'http://localhost:9200/openrxv-items-2021-02-08'
-</code></pre><h2 id="2021-02-10">2021-02-10</h2>
-<ul>
-<li>Talk to Abdullah from CodeObia about a few of the issues we filed on OpenRXV
-<ul>
-<li>Verify a fix he made for the issue with spaces in template file names</li>
-<li>He says that the <a href="https://github.com/ilri/OpenRXV/issues/49">Angular expressions support should be enabled</a>, but I tried it and couldn&rsquo;t get a few simple examples working</li>
-</ul>
-</li>
-<li>Atmire responded to a few issues today:
-<ul>
-<li>First, the one about a crash while exporting a community CSV, which appears to be a <a href="https://jira.lyrasis.org/browse/DS-4211">vanilla DSpace issue with a patch in DSpace 6.4</a></li>
-<li>Second, the MQM batch consumer issue, which appears to be harmless log spam in <em>most</em> cases and they have sent a patch that adjusts the logging as such</li>
-<li>Third, a version bump for CUA to fix the <code>java.lang.UnsupportedOperationException: Multiple update components target the same field:solr_update_time_stamp</code> error</li>
-</ul>
-</li>
-<li>I cherry-picked the patches for DS-4111 and was able to export the ILRI community finally, but the results are almost twice as many items as in the community!
-<ul>
-<li>Investigating with csvcut I see there are some ids that appear up to five, six, or seven times!</li>
-</ul>
-</li>
-</ul>
-<pre><code class="language-console" data-lang="console">$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | wc -l
-30354
-$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | sort -u | wc -l         
-18555
-$ csvcut -c id /tmp/2021-02-10-ILRI.csv | sed '1d' | sort | uniq -c | sort -h | tail     
-      5 c21a79e5-e24e-4861-aa07-e06703d1deb7
-      5 c2460aa1-ae28-4003-9a99-2d7c5cd7fd38
-      5 d73fb3ae-9fac-4f7e-990f-e394f344246c
-      5 dc0e24fa-b7f5-437e-ac09-e15c0704be00
-      5 dc50bcca-0abf-473f-8770-69d5ab95cc33
-      5 e714bdf9-cc0f-4d9a-a808-d572e25c9238
-      6 7dfd1c61-9e8c-4677-8d41-e1c4b11d867d
-      6 fb76888c-03ae-4d53-b27d-87d7ca91371a
-      6 ff42d1e6-c489-492c-a40a-803cabd901ed
-      7 094e9e1d-09ff-40ca-a6b9-eca580936147
-</code></pre><ul>
-<li>I added a comment to that bug to ask if this is a side effect of the patch</li>
-<li>I started working on tagging pre-2010 ILRI items with license information, like we talked about with Peter and Abenet last week
-<ul>
-<li>Due to the export bug I had to sort and remove duplicates first, then use csvgrep to filter out books and journal articles:</li>
-</ul>
-</li>
-</ul>
-<pre><code class="language-console" data-lang="console">$ csvcut -c 'id,dc.date.issued,dc.date.issued[],dc.date.issued[en_US],dc.rights,dc.rights[],dc.rights[en],dc.rights[en_US],dc.publisher,dc.publisher[],dc.publisher[en_US],dc.type[en_US]' /tmp/2021-02-10-ILRI.csv | csvgrep -c 'dc.type[en_US]' -r '^.+[^(Journal Item|Journal Article|Book|Book Chapter)]'
-</code></pre><ul>
-<li>I imported the CSV into OpenRefine and converted the date text values to date types so I could facet by dates before 2010:</li>
-</ul>
-<pre><code class="language-console" data-lang="console">if(diff(value,&quot;01/01/2010&quot;.toDate(),&quot;days&quot;)&lt;0, true, false)
-</code></pre><ul>
-<li>Then I filtered by publisher to make sure they were only ours:</li>
-</ul>
-<pre><code class="language-console" data-lang="console">or(
-  value.contains(&quot;International Livestock Research Institute&quot;),
-  value.contains(&quot;ILRI&quot;),
-  value.contains(&quot;International Livestock Centre for Africa&quot;),
-  value.contains(&quot;ILCA&quot;),
-  value.contains(&quot;ILRAD&quot;),
-  value.contains(&quot;International Laboratory for Research on Animal Diseases&quot;)
-)
-</code></pre><ul>
-<li>I tagged these pre-2010 items with &ldquo;Other&rdquo; if they didn&rsquo;t already have a license</li>
-<li>I checked 2010 to 2015, and 2016 to date, but they were all tagged already!</li>
-<li>In the end I added the &ldquo;Other&rdquo; license to 1,523 items from before 2010</li>
-</ul>
-<h2 id="2021-02-11">2021-02-11</h2>
-<ul>
-<li>CodeObia keeps working on a few more small issues on OpenRXV
-<ul>
-<li>Abdullah sent fixes for two issues but I couldn&rsquo;t verify them myself so I asked him to check again</li>
-<li>Call with Abdullah and Yousef to discuss some issues</li>
-<li>We got the Angular expressions parser working&hellip;</li>
-</ul>
-</li>
-</ul>
-<!-- raw HTML omitted -->
 
   
 

docs/categories/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/categories/notes/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/categories/notes/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/categories/notes/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/categories/notes/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/categories/notes/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/6/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/page/7/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/2/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/3/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/4/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/5/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/6/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/posts/page/7/index.html

@@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2021-02-11T09:36:09+02:00" />
+<meta property="og:updated_time" content="2021-02-06T14:00:36+02:00" />
 
 
 

docs/sitemap.xml

@@ -4,27 +4,27 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
-    <lastmod>2021-02-11T09:36:09+02:00</lastmod>
+    <lastmod>2021-02-06T14:00:36+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2021-02-11T09:36:09+02:00</lastmod>
+    <lastmod>2021-02-06T14:00:36+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2021-02/</loc>
-    <lastmod>2021-02-11T09:36:09+02:00</lastmod>
+    <lastmod>2021-02-06T14:00:36+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
-    <lastmod>2021-02-11T09:36:09+02:00</lastmod>
+    <lastmod>2021-02-06T14:00:36+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2021-02-11T09:36:09+02:00</lastmod>
+    <lastmod>2021-02-06T14:00:36+02:00</lastmod>
   </url>
   
   <url>