Add notes for 2022-08-18

2022-08-18 13:45:48 -07:00 · 2022-08-18 13:45:48 -07:00 · e203ee6dcc
parent 6c61d1c102
commit e203ee6dcc
29 changed files with 85 additions and 34 deletions
--- a/content/posts/2022-08.md
+++ b/content/posts/2022-08.md
@ -96,4 +96,26 @@ $ dspace import --add --eperson=aorth@mjanja.ch --source /tmp/SimpleArchiveForma
 - Add CONSERVATION to ILRI subjects on CGSpace
  - I see that AGROVOC has `conservation agriculture` and I suggested that we use that instead

+## 2022-08-17
+
+- Peter and Jose sent more feedback about the CRP Innovation records from MARLO
+  - We expanded the CRP names in the citation and removed the `cg.identifier.url` URLs because they are ugly and will stop working eventually
+  - The mappings of MARLO links will be done internally with the `cg.number` IDs like "IN-1119" and the Handle URIs
+
+## 2022-08-18
+
+- I talked to Jose about the CCAFS MARLO records
+  - He still hasn't finished re-processing the PDFs to update the internal MARLO links
+  - I started looking at the other records (MELIAs, OICRs, Policies) and found some minor issues in the MELIAs so I sent feedback to Jose
+  - On second thought, I opened the MELIAs file in OpenRefine and it looks OK, so this must have been a parsing issue in LibreOffice when I was checking the file (or perhaps I didn't use the correct quoting when importing)
+- Import the original MELIA v2 CSV file into OpenRefine to fix encoding before processing with csvcut/csvjoin
+  - Then extract the IDs and filenames from the original V2 file and join with the UTF-8 file:
+
+```console
+$ csvcut -c 'cg.number (series/report No.)',File ~/Downloads/MELIA-Metadata-v2-csv.csv > MELIA-v2-IDs-Files.csv
+$ csvjoin -c 'cg.number (series/report No.)' MELIAs\ metadata\ utf8\ 20220816_JM.csv MELIA-v2-IDs-Files.csv > MELIAs-UTF-8-with-files.csv
+```
+
+- Then I imported them into OpenRefine to start metadata cleaning and enrichment
+
 <!-- vim: set sw=2 ts=2: -->
--- a/docs/2022-08/index.html
+++ b/docs/2022-08/index.html
@ -14,7 +14,7 @@ Our request to add CC-BY-3.0-IGO to SPDX was approved a few weeks ago
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-08/" />
 <meta property="article:published_time" content="2022-08-01T10:22:36+03:00" />
-<meta property="article:modified_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="article:modified_time" content="2022-08-15T18:46:57-07:00" />



@ -34,9 +34,9 @@ Our request to add CC-BY-3.0-IGO to SPDX was approved a few weeks ago
  "@type": "BlogPosting",
  "headline": "August, 2022",
  "url": "https://alanorth.github.io/cgspace-notes/2022-08/",
-  "wordCount": "871",
+  "wordCount": "1080",
  "datePublished": "2022-08-01T10:22:36+03:00",
-  "dateModified": "2022-08-13T21:51:49-07:00",
+  "dateModified": "2022-08-15T18:46:57-07:00",
  "author": {
    "@type": "Person",
    "name": "Alan Orth"
@ -221,6 +221,35 @@ Our request to add CC-BY-3.0-IGO to SPDX was approved a few weeks ago
 </ul>
 </li>
 </ul>
+<h2 id="2022-08-17">2022-08-17</h2>
+<ul>
+<li>Peter and Jose sent more feedback about the CRP Innovation records from MARLO
+<ul>
+<li>We expanded the CRP names in the citation and removed the <code>cg.identifier.url</code> URLs because they are ugly and will stop working eventually</li>
+<li>The mappings of MARLO links will be done internally with the <code>cg.number</code> IDs like &ldquo;IN-1119&rdquo; and the Handle URIs</li>
+</ul>
+</li>
+</ul>
+<h2 id="2022-08-18">2022-08-18</h2>
+<ul>
+<li>I talked to Jose about the CCAFS MARLO records
+<ul>
+<li>He still hasn&rsquo;t finished re-processing the PDFs to update the internal MARLO links</li>
+<li>I started looking at the other records (MELIAs, OICRs, Policies) and found some minor issues in the MELIAs so I sent feedback to Jose</li>
+<li>On second thought, I opened the MELIAs file in OpenRefine and it looks OK, so this must have been a parsing issue in LibreOffice when I was checking the file (or perhaps I didn&rsquo;t use the correct quoting when importing)</li>
+</ul>
+</li>
+<li>Import the original MELIA v2 CSV file into OpenRefine to fix encoding before processing with csvcut/csvjoin
+<ul>
+<li>Then extract the IDs and filenames from the original V2 file and join with the UTF-8 file:</li>
+</ul>
+</li>
+</ul>
+<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ csvcut -c <span style="color:#e6db74">&#39;cg.number (series/report No.)&#39;</span>,File ~/Downloads/MELIA-Metadata-v2-csv.csv &gt; MELIA-v2-IDs-Files.csv
+</span></span><span style="display:flex;"><span>$ csvjoin -c <span style="color:#e6db74">&#39;cg.number (series/report No.)&#39;</span> MELIAs<span style="color:#ae81ff">\ </span>metadata<span style="color:#ae81ff">\ </span>utf8<span style="color:#ae81ff">\ </span>20220816_JM.csv MELIA-v2-IDs-Files.csv &gt; MELIAs-UTF-8-with-files.csv
+</span></span></code></pre></div><ul>
+<li>Then I imported them into OpenRefine to start metadata cleaning and enrichment</li>
+</ul>
 <!-- raw HTML omitted -->

  
--- a/docs/categories/index.html
+++ b/docs/categories/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/index.html
+++ b/docs/categories/notes/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/2/index.html
+++ b/docs/categories/notes/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/3/index.html
+++ b/docs/categories/notes/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/4/index.html
+++ b/docs/categories/notes/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/5/index.html
+++ b/docs/categories/notes/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/6/index.html
+++ b/docs/categories/notes/page/6/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/categories/notes/page/7/index.html
+++ b/docs/categories/notes/page/7/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/index.html
+++ b/docs/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/2/index.html
+++ b/docs/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/3/index.html
+++ b/docs/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/4/index.html
+++ b/docs/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/5/index.html
+++ b/docs/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/6/index.html
+++ b/docs/page/6/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/7/index.html
+++ b/docs/page/7/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/8/index.html
+++ b/docs/page/8/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/page/9/index.html
+++ b/docs/page/9/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/index.html
+++ b/docs/posts/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/2/index.html
+++ b/docs/posts/page/2/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/3/index.html
+++ b/docs/posts/page/3/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/4/index.html
+++ b/docs/posts/page/4/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/5/index.html
+++ b/docs/posts/page/5/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/6/index.html
+++ b/docs/posts/page/6/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/7/index.html
+++ b/docs/posts/page/7/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/8/index.html
+++ b/docs/posts/page/8/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/posts/page/9/index.html
+++ b/docs/posts/page/9/index.html
@ -10,7 +10,7 @@
 <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
 <meta property="og:type" content="website" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
-<meta property="og:updated_time" content="2022-08-13T21:51:49-07:00" />
+<meta property="og:updated_time" content="2022-08-15T18:46:57-07:00" />



--- a/docs/sitemap.xml
+++ b/docs/sitemap.xml
@ -3,19 +3,19 @@
  xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/2022-08/</loc>
-    <lastmod>2022-08-13T21:51:49-07:00</lastmod>
+    <lastmod>2022-08-15T18:46:57-07:00</lastmod>
  </url><url>
    <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
-    <lastmod>2022-08-13T21:51:49-07:00</lastmod>
+    <lastmod>2022-08-15T18:46:57-07:00</lastmod>
  </url><url>
    <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2022-08-13T21:51:49-07:00</lastmod>
+    <lastmod>2022-08-15T18:46:57-07:00</lastmod>
  </url><url>
    <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
-    <lastmod>2022-08-13T21:51:49-07:00</lastmod>
+    <lastmod>2022-08-15T18:46:57-07:00</lastmod>
  </url><url>
    <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2022-08-13T21:51:49-07:00</lastmod>
+    <lastmod>2022-08-15T18:46:57-07:00</lastmod>
  </url><url>
    <loc>https://alanorth.github.io/cgspace-notes/2022-07/</loc>
    <lastmod>2022-07-31T15:49:35+03:00</lastmod>