alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-26 00:18:21 +01:00
Code Issues Projects Releases Wiki Activity

Update notes for 2019-04-08

Browse Source
This commit is contained in:
Alan Orth 2019-04-08 11:26:20 +03:00
parent a9596bdfed
commit 6f44a3bcdd
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 73 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
content/posts/2019-04.md
View File

@ -401,4 +401,31 @@ $ psql -c 'select * from pg_stat_activity' | grep -o -E '(dspaceWeb|dspaceApi|ds
- It seems that the issue with CGSpace being "down" is actually because of CPU steal again!!!
- I opened a ticket with support and asked them to migrate the VM to a less busy host
## 2019-04-08
- Start checking IITA's last round of batch uploads from [March on DSpace Test](https://dspacetest.cgiar.org/handle/10568/100333) (20193rd.xls)
- Lots of problems with affiliations, I had to correct about sixty of them
- I used lein to host the latest CSV of our affiliations for OpenRefine to reconcile against:
```
$ lein run ~/src/git/DSpace/2019-02-22-affiliations.csv name id
```
- After matching the values and creating some new matches I had trouble remembering how to copy the reconciled values to a new column
- The matched values can be accessed with `cell.recon.match.name`, but some of the new values don't appear, perhaps because I edited the original cell values?
- I ended up using this GREL expression to copy all values to a new column:
```
if(cell.recon.matched, cell.recon.match.name, value)
```
- See the [OpenRefine variables documentation](https://github.com/OpenRefine/OpenRefine/wiki/Variables#recon) for more notes about the `recon` object
- I also noticed a handful of errors in our current list of affiliations so I corrected them:
```
$ ./fix-metadata-values.py -i 2019-04-08-fix-13-affiliations.csv -db dspace -u dspace -p 'fuuu' -f cg.contributor.affiliation -m 211 -t correct -d
```
- We should create a new list of affiliations to update our controlled vocabulary again
<!-- vim: set sw=2 ts=2: -->

44
docs/2019-04/index.html
View File

@ -38,7 +38,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-02-21-delete-1-region.csv -db dspace
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-04/" />
<meta property="article:published_time" content="2019-04-01T09:00:43&#43;03:00"/>
<meta property="article:modified_time" content="2019-04-07T21:15:03&#43;03:00"/>
<meta property="article:modified_time" content="2019-04-07T21:17:16&#43;03:00"/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="April, 2019"/>
@ -81,9 +81,9 @@ $ ./delete-metadata-values.py -i /tmp/2019-02-21-delete-1-region.csv -db dspace
"@type": "BlogPosting",
"headline": "April, 2019",
"url": "https://alanorth.github.io/cgspace-notes/2019-04/",
"wordCount": "2222",
"wordCount": "2397",
"datePublished": "2019-04-01T09:00:43&#43;03:00",
"dateModified": "2019-04-07T21:15:03&#43;03:00",
"dateModified": "2019-04-07T21:17:16&#43;03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -623,6 +623,44 @@ X-XSS-Protection: 1; mode=block
<li>I opened a ticket with support and asked them to migrate the VM to a less busy host</li>
</ul>
<h2 id="2019-04-08">2019-04-08</h2>
<ul>
<li>Start checking IITA&rsquo;s last round of batch uploads from <a href="https://dspacetest.cgiar.org/handle/10568/100333">March on DSpace Test</a> (20193rd.xls)
<ul>
<li>Lots of problems with affiliations, I had to correct about sixty of them</li>
<li>I used lein to host the latest CSV of our affiliations for OpenRefine to reconcile against:</li>
</ul></li>
</ul>
<pre><code>$ lein run ~/src/git/DSpace/2019-02-22-affiliations.csv name id
</code></pre>
<ul>
<li>After matching the values and creating some new matches I had trouble remembering how to copy the reconciled values to a new column
<ul>
<li>The matched values can be accessed with <code>cell.recon.match.name</code>, but some of the new values don&rsquo;t appear, perhaps because I edited the original cell values?</li>
<li>I ended up using this GREL expression to copy all values to a new column:</li>
</ul></li>
</ul>
<pre><code>if(cell.recon.matched, cell.recon.match.name, value)
</code></pre>
<ul>
<li>See the <a href="https://github.com/OpenRefine/OpenRefine/wiki/Variables#recon">OpenRefine variables documentation</a> for more notes about the <code>recon</code> object</li>
<li>I also noticed a handful of errors in our current list of affiliations so I corrected them:</li>
</ul>
<pre><code>$ ./fix-metadata-values.py -i 2019-04-08-fix-13-affiliations.csv -db dspace -u dspace -p 'fuuu' -f cg.contributor.affiliation -m 211 -t correct -d
</code></pre>
<ul>
<li>We should create a new list of affiliations to update our controlled vocabulary again</li>
</ul>
<!-- vim: set sw=2 ts=2: -->

10
docs/sitemap.xml
View File

@ -4,7 +4,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/2019-04/</loc>
<lastmod>2019-04-07T21:15:03+03:00</lastmod>
<lastmod>2019-04-07T21:17:16+03:00</lastmod>
</url>
<url>
@ -219,7 +219,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2019-04-07T21:15:03+03:00</lastmod>
<lastmod>2019-04-07T21:17:16+03:00</lastmod>
<priority>0</priority>
</url>
@ -230,7 +230,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2019-04-07T21:15:03+03:00</lastmod>
<lastmod>2019-04-07T21:17:16+03:00</lastmod>
<priority>0</priority>
</url>
@ -242,13 +242,13 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2019-04-07T21:15:03+03:00</lastmod>
<lastmod>2019-04-07T21:17:16+03:00</lastmod>
<priority>0</priority>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2019-04-07T21:15:03+03:00</lastmod>
<lastmod>2019-04-07T21:17:16+03:00</lastmod>
<priority>0</priority>
</url>