alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-22 21:22:19 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2020-01-15

Browse Source
This commit is contained in:
Alan Orth 2020-01-15 13:51:35 +02:00
parent ba5755d441
commit 3736370c6e
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 41 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
content/posts/2020-01.md
View File

@ -93,4 +93,23 @@ In [8]: unicodedata.is_normalized('NFC', 'é')
Out[8]: True
```
## 2020-01-15
- I added support for Unicode normalization to my [csv-metadata-quality](https://github.com/ilri/csv-metadata-quality) tool in [v0.4.0](https://github.com/ilri/csv-metadata-quality/releases/tag/v0.4.0)
- Generate ILRI and Bioversity subject lists for Elizabeth Arnaud from Bioversity:
```
dspace=# \COPY (SELECT DISTINCT text_value as "cg.subject.ilri", count(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 203 GROUP BY text_value ORDER BY count DESC) to /tmp/2020-01-15-ilri-subjects.csv WITH CSV HEADER;
COPY 144
dspace=# \COPY (SELECT DISTINCT text_value as "cg.subject.bioversity", count(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 120 GROUP BY text_value ORDER BY count DESC) to /tmp/2020-01-15-bioversity-subjects.csv WITH CSV HEADER;
COPY 1325
```
- She will be meeting with FAO and will look over the terms to see if they can add some to AGROVOC
- I noticed a few errors in the ILRI subjects so I fixed them locally and on CGSpace (linode18) using my `fix-metadata.py` script:
```
$ ./fix-metadata-values.py -i 2020-01-15-fix-8-ilri-subjects.csv -db dspace -u dspace -p 'fuuu' -f cg.subject.ilri -m 203 -t correct -d
```
<!-- vim: set sw=2 ts=2: -->

20
docs/2020-01/index.html
View File

@ -29,7 +29,7 @@ I tweeted the CGSpace repository link
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-01/" />
<meta property="article:published_time" content="2020-01-06T10:48:30+02:00" />
<meta property="article:modified_time" content="2020-01-08T15:33:56+02:00" />
<meta property="article:modified_time" content="2020-01-14T20:40:41+02:00" />
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="January, 2020"/>
@ -63,9 +63,9 @@ I tweeted the CGSpace repository link
"@type": "BlogPosting",
"headline": "January, 2020",
"url": "https:\/\/alanorth.github.io\/cgspace-notes\/2020-01\/",
"wordCount": "617",
"wordCount": "765",
"datePublished": "2020-01-06T10:48:30+02:00",
"dateModified": "2020-01-08T15:33:56+02:00",
"dateModified": "2020-01-14T20:40:41+02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -222,6 +222,20 @@ Out[7]: False
In [8]: unicodedata.is_normalized('NFC', 'é')
Out[8]: True
</code></pre><h2 id="2020-01-15">2020-01-15</h2>
<ul>
<li>I added support for Unicode normalization to my <a href="https://github.com/ilri/csv-metadata-quality">csv-metadata-quality</a> tool in <a href="https://github.com/ilri/csv-metadata-quality/releases/tag/v0.4.0">v0.4.0</a></li>
<li>Generate ILRI and Bioversity subject lists for Elizabeth Arnaud from Bioversity:</li>
</ul>
<pre><code>dspace=# \COPY (SELECT DISTINCT text_value as &quot;cg.subject.ilri&quot;, count(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 203 GROUP BY text_value ORDER BY count DESC) to /tmp/2020-01-15-ilri-subjects.csv WITH CSV HEADER;
COPY 144
dspace=# \COPY (SELECT DISTINCT text_value as &quot;cg.subject.bioversity&quot;, count(*) FROM metadatavalue WHERE resource_type_id = 2 AND metadata_field_id = 120 GROUP BY text_value ORDER BY count DESC) to /tmp/2020-01-15-bioversity-subjects.csv WITH CSV HEADER;
COPY 1325
</code></pre><ul>
<li>She will be meeting with FAO and will look over the terms to see if they can add some to AGROVOC</li>
<li>I noticed a few errors in the ILRI subjects so I fixed them locally and on CGSpace (linode18) using my <code>fix-metadata.py</code> script:</li>
</ul>
<pre><code>$ ./fix-metadata-values.py -i 2020-01-15-fix-8-ilri-subjects.csv -db dspace -u dspace -p 'fuuu' -f cg.subject.ilri -m 203 -t correct -d
</code></pre><!-- raw HTML omitted -->

10
docs/sitemap.xml
View File

@ -4,27 +4,27 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2020-01-08T15:33:56+02:00</lastmod>
<lastmod>2020-01-14T20:40:41+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2020-01-08T15:33:56+02:00</lastmod>
<lastmod>2020-01-14T20:40:41+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/2020-01/</loc>
<lastmod>2020-01-08T15:33:56+02:00</lastmod>
<lastmod>2020-01-14T20:40:41+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2020-01-08T15:33:56+02:00</lastmod>
<lastmod>2020-01-14T20:40:41+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2020-01-08T15:33:56+02:00</lastmod>
<lastmod>2020-01-14T20:40:41+02:00</lastmod>
</url>
<url>