Update notes for 2019-09-27

2024-11-22 06:35:03 +01:00 · 2019-09-27 01:20:09 +03:00 · 2019-09-27 01:20:09 +03:00 · a8f833a6c6
commit a8f833a6c6
parent 71ed97ce53
5 changed files with 36 additions and 11 deletions
--- a/content/posts/2019-08.md
+++ b/content/posts/2019-08.md
@ -358,5 +358,12 @@ sys     2m27.496s
  - After reading the code I see that XSLT is reading the community titles from the DIM representation (stored in the `$dim` variable) created from METS
  - I modified the patterns in my sed script so that those lines are not replaced and then the community list works again
  - This is actually not a problem at all because this metadata is only used in the HTML meta tags in XMLUI community lists and has nothing to do with item metadata
+- Get a list of institutions from CCAFS's Clarisa API and try to parse it with `jq` and pass it through `csvcut` to add line numbers:
+
+```
+$ cat ~/Downloads/institutions.json| jq '.[] | {name: .name}' | grep name | awk -F: '{print $2}' | sed 's/"//g' | csvcut -l > /tmp/investors.csv
+```
+
+- I could potentially use this with reconcile-csv and OpenRefine as a source to validate our institutional authors against...

 <!-- vim: set sw=2 ts=2: -->
--- a/content/posts/2019-09.md
+++ b/content/posts/2019-09.md
@ -316,5 +316,8 @@ $ dspace import -a me@cgiar.org -m 2019-09-20-bioversity2.map -s /home/aorth/Bio
 - Release [version 0.3.0 of the csv-metadata-quality](https://github.com/ilri/csv-metadata-quality/releases/tag/v0.3.0) tool
  - This version includes the experimental validation of languages using the Python `langid` library
  - I also included updated pytest tests and test files that specifically test this functionality
+- Give more feedback to Bosede about the [IITA Sept 6 (20196th.xls) records on DSpace Test](https://dspacetest.cgiar.org/handle/10568/105116)
+  - I told her to delete one item that appears to be a duplicate, or to fix its citation to be correct if she thinks it is not a duplicate
+  - I deleted another item that I had previously identified as a duplicate that she had fixed by incorrectly deleting the original (ugh)

 <!-- vim: set sw=2 ts=2: -->
--- a/docs/2019-08/index.html
+++ b/docs/2019-08/index.html
@ -59,7 +59,7 @@ Run system updates on DSpace Test (linode19) and reboot it
  "@type": "BlogPosting",
  "headline": "August, 2019",
  "url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-08\/",
-  "wordCount": "2703",
+  "wordCount": "2770",
  "datePublished": "2019-08-03T12:39:51\x2b03:00",
  "dateModified": "2019-09-01T01:54:55\x2b03:00",
  "author": {
@ -603,6 +603,13 @@ sys     2m27.496s
 <li>I modified the patterns in my sed script so that those lines are not replaced and then the community list works again</li>
 <li>This is actually not a problem at all because this metadata is only used in the HTML meta tags in XMLUI community lists and has nothing to do with item metadata</li>
 </ul></li>
+
+<li><p>Get a list of institutions from CCAFS&rsquo;s Clarisa API and try to parse it with <code>jq</code> and pass it through <code>csvcut</code> to add line numbers:</p>
+
+<pre><code>$ cat ~/Downloads/institutions.json| jq '.[] | {name: .name}' | grep name | awk -F: '{print $2}' | sed 's/&quot;//g' | csvcut -l &gt; /tmp/investors.csv
+</code></pre></li>
+
+<li><p>I could potentially use this with reconcile-csv and OpenRefine as a source to validate our institutional authors against&hellip;</p></li>
 </ul>

 <!-- vim: set sw=2 ts=2: -->
--- a/docs/2019-09/index.html
+++ b/docs/2019-09/index.html
@ -40,7 +40,7 @@ Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-09/" />
 <meta property="article:published_time" content="2019-09-01T10:17:51+03:00" />
-<meta property="article:modified_time" content="2019-09-24T20:13:20+03:00" />
+<meta property="article:modified_time" content="2019-09-26T14:21:41+03:00" />

 <meta name="twitter:card" content="summary"/>
 <meta name="twitter:title" content="September, 2019"/>
@ -85,9 +85,9 @@ Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:
  "@type": "BlogPosting",
  "headline": "September, 2019",
  "url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-09\/",
-  "wordCount": "2417",
+  "wordCount": "2497",
  "datePublished": "2019-09-01T10:17:51\x2b03:00",
-  "dateModified": "2019-09-24T20:13:20\x2b03:00",
+  "dateModified": "2019-09-26T14:21:41\x2b03:00",
  "author": {
    "@type": "Person",
    "name": "Alan Orth"
@ -552,11 +552,19 @@ $ dspace import -a me@cgiar.org -m 2019-09-20-bioversity2.map -s /home/aorth/Bio
 <li>Release <a href="https://github.com/ilri/csv-metadata-quality/releases/tag/v0.3.0">version 0.3.0 of the csv-metadata-quality</a> tool

 <ul>
-<li>This version includes the experimental validation of languages using the Python <code>langid</code> library
-<!-- vim: set sw=2 ts=2: --></li>
+<li>This version includes the experimental validation of languages using the Python <code>langid</code> library</li>
+<li>I also included updated pytest tests and test files that specifically test this functionality</li>
+</ul></li>
+<li>Give more feedback to Bosede about the <a href="https://dspacetest.cgiar.org/handle/10568/105116">IITA Sept 6 (20196th.xls) records on DSpace Test</a>
+
+<ul>
+<li>I told her to delete one item that appears to be a duplicate, or to fix its citation to be correct if she thinks it is not a duplicate</li>
+<li>I deleted another item that I had previously identified as a duplicate that she had fixed by incorrectly deleting the original (ugh)</li>
 </ul></li>
 </ul>

+<!-- vim: set sw=2 ts=2: -->
+
  

  
--- a/docs/sitemap.xml
+++ b/docs/sitemap.xml
@ -4,27 +4,27 @@
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2019-09-24T20:13:20+03:00</lastmod>
+    <lastmod>2019-09-26T14:21:41+03:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2019-09-24T20:13:20+03:00</lastmod>
+    <lastmod>2019-09-26T14:21:41+03:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2019-09-24T20:13:20+03:00</lastmod>
+    <lastmod>2019-09-26T14:21:41+03:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/2019-09/</loc>
-    <lastmod>2019-09-24T20:13:20+03:00</lastmod>
+    <lastmod>2019-09-26T14:21:41+03:00</lastmod>
  </url>
  
  <url>
    <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2019-09-24T20:13:20+03:00</lastmod>
+    <lastmod>2019-09-26T14:21:41+03:00</lastmod>
  </url>
  
  <url>