alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-22 06:35:03 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2022-05-13

Browse Source
This commit is contained in:
Alan Orth 2022-05-13 10:28:23 +03:00
parent f07c04bd7e
commit 7916af5417
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
28 changed files with 72 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
content/posts/2022-05.md
View File

@ -102,7 +102,7 @@ localhost/dspacetest= ☘ SELECT EXTRACT(year from TO_DATE(text_value, 'YYYY-MM-
- We discussed the future of CGSpace and DSpace in general in the new One CGIAR
- We discussed how to prepare for bringing in content from the Initiatives, whether we need new metadata fields to support people from IFPRI etc
- We discussed the need for good quality Drupal and WordPress modules so sites can harvest content from the repository
- Peter asked me to send him a list of investors/funders/donors so he can clean it up, but also to try to align it with RoR and evntually do something like we do with country codes, adding the RoR IDs and potentially showing the badge on item views
- Peter asked me to send him a list of investors/funders/donors so he can clean it up, but also to try to align it with ROR and evntually do something like we do with country codes, adding the ROR IDs and potentially showing the badge on item views
- We also discussed removing some Mirage 2 themes for old programs and CRPs that don't have custom branding, ie only Google Analytics
- Export a list of donors for Peter to clean up:
@ -135,4 +135,24 @@ $ grep -c "Adding ORCID" /tmp/add-orcids.log
# zcat --force /var/log/nginx/access.log.* | grep 'GET /explorer' | goaccess --log-format=COMBINED - -o /tmp/ares_report.html
```
## 2022-05-13
- Peter finalized the corrections on donors from yesterday so I extracted them into fix/delete CSVs and ran them on CGSpace:
```console
$ ./ilri/fix-metadata-values.py -i 2022-05-13-fix-CGSpace-Donors.csv -db dspace -u dspace -p 'fuuu' -f cg.contributor.donor -m 248 -t correct -d
$ ./ilri/delete-metadata-values.py -i 2022-05-13-delete-CGSpace-Donors.csv -db dspace -u dspace -p 'fuuu' -f cg.contributor.donor -m 248 -d
```
- I cleaned up a few records manually (like some that had \r\n) then re-exported the donors and checked against the latest ROR dump:
```console
$ ./ilri/ror-lookup.py -i /tmp/2022-05-13-donors.csv -r v1.0-2022-03-17-ror-data.json -o /tmp/2022-05-13-ror.csv
$ csvgrep -c matched -m true /tmp/2022-05-13-ror.csv | wc -l
230
$ csvgrep -c matched -m false /tmp/2022-05-13-ror.csv | csvcut -c organization > /tmp/2022-05-13-ror-unmatched.csv
```
- Then I sent Peter a list so he can try to update some from ROR
<!-- vim: set sw=2 ts=2: -->

26
docs/2022-05/index.html
View File

@ -35,7 +35,7 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-05/" />
<meta property="article:published_time" content="2022-05-04T09:13:39+03:00" />
<meta property="article:modified_time" content="2022-05-10T16:35:50+03:00" />
<meta property="article:modified_time" content="2022-05-13T08:39:15+03:00" />
@ -76,9 +76,9 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
"@type": "BlogPosting",
"headline": "May, 2022",
"url": "https://alanorth.github.io/cgspace-notes/2022-05/",
"wordCount": "947",
"wordCount": "1070",
"datePublished": "2022-05-04T09:13:39+03:00",
"dateModified": "2022-05-10T16:35:50+03:00",
"dateModified": "2022-05-13T08:39:15+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -269,7 +269,7 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
<li>We discussed the future of CGSpace and DSpace in general in the new One CGIAR</li>
<li>We discussed how to prepare for bringing in content from the Initiatives, whether we need new metadata fields to support people from IFPRI etc</li>
<li>We discussed the need for good quality Drupal and WordPress modules so sites can harvest content from the repository</li>
<li>Peter asked me to send him a list of investors/funders/donors so he can clean it up, but also to try to align it with RoR and evntually do something like we do with country codes, adding the RoR IDs and potentially showing the badge on item views</li>
<li>Peter asked me to send him a list of investors/funders/donors so he can clean it up, but also to try to align it with ROR and evntually do something like we do with country codes, adding the ROR IDs and potentially showing the badge on item views</li>
<li>We also discussed removing some Mirage 2 themes for old programs and CRPs that don&rsquo;t have custom branding, ie only Google Analytics</li>
</ul>
</li>
@ -300,7 +300,23 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script
<li>I generated a report from the AReS nginx logs on linode18:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span># zcat --force /var/log/nginx/access.log.* | grep <span style="color:#e6db74">&#39;GET /explorer&#39;</span> | goaccess --log-format<span style="color:#f92672">=</span>COMBINED - -o /tmp/ares_report.html
</span></span></code></pre></div><!-- raw HTML omitted -->
</span></span></code></pre></div><h2 id="2022-05-13">2022-05-13</h2>
<ul>
<li>Peter finalized the corrections on donors from yesterday so I extracted them into fix/delete CSVs and ran them on CGSpace:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ ./ilri/fix-metadata-values.py -i 2022-05-13-fix-CGSpace-Donors.csv -db dspace -u dspace -p <span style="color:#e6db74">&#39;fuuu&#39;</span> -f cg.contributor.donor -m <span style="color:#ae81ff">248</span> -t correct -d
</span></span><span style="display:flex;"><span>$ ./ilri/delete-metadata-values.py -i 2022-05-13-delete-CGSpace-Donors.csv -db dspace -u dspace -p <span style="color:#e6db74">&#39;fuuu&#39;</span> -f cg.contributor.donor -m <span style="color:#ae81ff">248</span> -d
</span></span></code></pre></div><ul>
<li>I cleaned up a few records manually (like some that had \r\n) then re-exported the donors and checked against the latest ROR dump:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ ./ilri/ror-lookup.py -i /tmp/2022-05-13-donors.csv -r v1.0-2022-03-17-ror-data.json -o /tmp/2022-05-13-ror.csv
</span></span><span style="display:flex;"><span>$ csvgrep -c matched -m true /tmp/2022-05-13-ror.csv | wc -l
</span></span><span style="display:flex;"><span>230
</span></span><span style="display:flex;"><span>$ csvgrep -c matched -m false /tmp/2022-05-13-ror.csv | csvcut -c organization &gt; /tmp/2022-05-13-ror-unmatched.csv
</span></span></code></pre></div><ul>
<li>Then I sent Peter a list so he can try to update some from ROR</li>
</ul>
<!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/categories/notes/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

2
docs/posts/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-05-12T12:51:45+03:00" />
<meta property="og:updated_time" content="2022-05-13T08:39:15+03:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2022-05-12T12:51:45+03:00</lastmod>
<lastmod>2022-05-13T08:39:15+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2022-05-12T12:51:45+03:00</lastmod>
<lastmod>2022-05-13T08:39:15+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-05/</loc>
<lastmod>2022-05-10T16:35:50+03:00</lastmod>
<lastmod>2022-05-13T08:39:15+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2022-05-12T12:51:45+03:00</lastmod>
<lastmod>2022-05-13T08:39:15+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2022-05-12T12:51:45+03:00</lastmod>
<lastmod>2022-05-13T08:39:15+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-04/</loc>
<lastmod>2022-05-04T11:09:45+03:00</lastmod>