alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-22 21:22:19 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2021-01-18

Browse Source
This commit is contained in:
Alan Orth 2021-01-18 16:21:24 +02:00
parent 87fd47e7d2
commit 5c7d01cbc4
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
26 changed files with 173 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
content/posts/2021-01.md
View File

@ -205,4 +205,64 @@ $ dspace community-filiator --remove --parent=10568/66598 --child=10568/106605
- Publish [v1.4.1 of the DSpace Statistics API](https://github.com/ilri/dspace-statistics-api/releases/tag/v1.4.1) based on feedback from the community
- This includes the fix for limiting the Solr query to UUIDs
## 2021-01-17
- Start a re-index on AReS
- First delete the old Elasticsearch temp index:
```console
$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
# start indexing in AReS
```
- Then, the next morning when it's done, check the results of the harvesting, backup the current `openrxv-items` index, and clone the `openrxv-items-temp` index to `openrxv-items`:
```console
$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&pretty'
{
"count" : 100540,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
}
}
$ curl -X PUT "localhost:9200/openrxv-items/_settings" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2021-01-18
$ curl -XDELETE 'http://localhost:9200/openrxv-items'
$ curl -X PUT "localhost:9200/openrxv-items-temp/_settings" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
$ curl -XDELETE 'http://localhost:9200/openrxv-items-2021-01-18'
```
## 2021-01-18
- Finish the indexing on AReS that I started yesterday
- Udana from IWMI emailed me to ask why the iwmi.csv doesn't include items he approved to CGSpace this morning
- I told him it is generated every Sunday night
- I regenerated the file manually for him
- I adjusted the script to run on Monday and Friday
- Meeting with Peter and Abenet about CG Core v2
- We also need to remove CTA and CPWF subjects from the input form since they are both closed now and no longer submitting items
- Peter also wants to create new fields on CGSpace for the SDGs and CGIAR Impact Areas
- I suggested `cg.subject.sdg` and `cg.subject.impactArea`
- We also agreed to remove the following fields:
- cg.livestock.agegroup
- cg.livestock.function
- cg.message.sms
- cg.message.voice
- I removed them from the input form, metadata registry, and deleted all the values in the database:
```
localhost/dspace63= > BEGIN;
localhost/dspace63= > DELETE FROM metadatavalue WHERE metadata_field_id IN (115, 116, 117, 118);
DELETE 27
localhost/dspace63= > COMMIT;
```
- I submitted [an issue](https://github.com/AgriculturalSemantics/cg-core/issues/31) to CG Core v2 to propose standardizing the camel case convention for a few more fields of ours
- I submitted [an issue](https://github.com/AgriculturalSemantics/cg-core/issues/32) to CG Core v2 to propose removing `cg.series` and `cg.pages` in favor of `dcterms.isPartOf` and `dcterms.extent`, respectively
<!-- vim: set sw=2 ts=2: -->

74
docs/2021-01/index.html
View File

@ -27,7 +27,7 @@ For example, this item has 51 views on CGSpace, but 0 on AReS
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2021-01/" />
<meta property="article:published_time" content="2021-01-03T10:13:54+02:00" />
<meta property="article:modified_time" content="2021-01-13T15:44:47+02:00" />
<meta property="article:modified_time" content="2021-01-14T16:27:57+02:00" />
@ -60,9 +60,9 @@ For example, this item has 51 views on CGSpace, but 0 on AReS
"@type": "BlogPosting",
"headline": "January, 2021",
"url": "https://alanorth.github.io/cgspace-notes/2021-01/",
"wordCount": "1515",
"wordCount": "1834",
"datePublished": "2021-01-03T10:13:54+02:00",
"dateModified": "2021-01-13T15:44:47+02:00",
"dateModified": "2021-01-14T16:27:57+02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -385,6 +385,74 @@ $ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
</ul>
</li>
</ul>
<h2 id="2021-01-17">2021-01-17</h2>
<ul>
<li>Start a re-index on AReS
<ul>
<li>First delete the old Elasticsearch temp index:</li>
</ul>
</li>
</ul>
<pre><code class="language-console" data-lang="console">$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
# start indexing in AReS
</code></pre><ul>
<li>Then, the next morning when it&rsquo;s done, check the results of the harvesting, backup the current <code>openrxv-items</code> index, and clone the <code>openrxv-items-temp</code> index to <code>openrxv-items</code>:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&amp;pretty'
{
&quot;count&quot; : 100540,
&quot;_shards&quot; : {
&quot;total&quot; : 1,
&quot;successful&quot; : 1,
&quot;skipped&quot; : 0,
&quot;failed&quot; : 0
}
}
$ curl -X PUT &quot;localhost:9200/openrxv-items/_settings&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: true}}'
$ curl -s -X POST http://localhost:9200/openrxv-items/_clone/openrxv-items-2021-01-18
$ curl -XDELETE 'http://localhost:9200/openrxv-items'
$ curl -X PUT &quot;localhost:9200/openrxv-items-temp/_settings&quot; -H 'Content-Type: application/json' -d'{&quot;settings&quot;: {&quot;index.blocks.write&quot;: true}}'
$ curl -s -X POST http://localhost:9200/openrxv-items-temp/_clone/openrxv-items
$ curl -XDELETE 'http://localhost:9200/openrxv-items-temp'
$ curl -XDELETE 'http://localhost:9200/openrxv-items-2021-01-18'
</code></pre><h2 id="2021-01-18">2021-01-18</h2>
<ul>
<li>Finish the indexing on AReS that I started yesterday</li>
<li>Udana from IWMI emailed me to ask why the iwmi.csv doesn&rsquo;t include items he approved to CGSpace this morning
<ul>
<li>I told him it is generated every Sunday night</li>
<li>I regenerated the file manually for him</li>
<li>I adjusted the script to run on Monday and Friday</li>
</ul>
</li>
<li>Meeting with Peter and Abenet about CG Core v2
<ul>
<li>We also need to remove CTA and CPWF subjects from the input form since they are both closed now and no longer submitting items</li>
<li>Peter also wants to create new fields on CGSpace for the SDGs and CGIAR Impact Areas
<ul>
<li>I suggested <code>cg.subject.sdg</code> and <code>cg.subject.impactArea</code></li>
</ul>
</li>
<li>We also agreed to remove the following fields:
<ul>
<li>cg.livestock.agegroup</li>
<li>cg.livestock.function</li>
<li>cg.message.sms</li>
<li>cg.message.voice</li>
</ul>
</li>
<li>I removed them from the input form, metadata registry, and deleted all the values in the database:</li>
</ul>
</li>
</ul>
<pre><code>localhost/dspace63= &gt; BEGIN;
localhost/dspace63= &gt; DELETE FROM metadatavalue WHERE metadata_field_id IN (115, 116, 117, 118);
DELETE 27
localhost/dspace63= &gt; COMMIT;
</code></pre><ul>
<li>I submitted <a href="https://github.com/AgriculturalSemantics/cg-core/issues/31">an issue</a> to CG Core v2 to propose standardizing the camel case convention for a few more fields of ours</li>
<li>I submitted <a href="https://github.com/AgriculturalSemantics/cg-core/issues/32">an issue</a> to CG Core v2 to propose removing <code>cg.series</code> and <code>cg.pages</code> in favor of <code>dcterms.isPartOf</code> and <code>dcterms.extent</code>, respectively</li>
</ul>
<!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

24
docs/cgspace-cgcorev2-migration/index.html
View File

@ -11,7 +11,7 @@
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/cgspace-cgcorev2-migration/" />
<meta property="article:published_time" content="2019-10-28T13:27:35+02:00" />
<meta property="article:modified_time" content="2020-04-13T15:30:24+03:00" />
<meta property="article:modified_time" content="2021-01-18T16:20:57+02:00" />
@ -30,7 +30,7 @@
"url": "https://alanorth.github.io/cgspace-notes/cgspace-cgcorev2-migration/",
"wordCount": "555",
"datePublished": "2019-10-28T13:27:35+02:00",
"dateModified": "2020-04-13T15:30:24+03:00",
"dateModified": "2021-01-18T16:20:57+02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -116,7 +116,7 @@
<li><a href="#implementation-progress">Implementation Progress</a></li>
</ul>
<h2 id="proposed-changes">Proposed Changes</h2>
<p>As of 2019-12-22 the scope of the changes includes the following fields:</p>
<p>As of 2021-01-18 the scope of the changes includes the following fields:</p>
<ul>
<li>cg.creator.id→cg.creator.identifier
<ul>
@ -132,8 +132,8 @@
<li>values from CrossRef or Grid.ac if possible</li>
</ul>
</li>
<li>dc.description.version→cg.review-status</li>
<li>cg.fulltextstatus→cg.howpublished
<li>dc.description.version→cg.reviewStatus</li>
<li>cg.fulltextstatus→cg.howPublished
<ul>
<li>CGSpace uses values like &ldquo;Formally Published&rdquo; or &ldquo;Grey Literature&rdquo;</li>
</ul>
@ -179,13 +179,13 @@
<h2 id="fields-to-create">Fields to Create</h2>
<p>Make sure the following fields exist:</p>
<ul>
<li><input checked="" disabled="" type="checkbox"> cg.creator.identifier (242)</li>
<li><input checked="" disabled="" type="checkbox"> cg.contributor.donor (243)</li>
<li><input checked="" disabled="" type="checkbox"> cg.review-status (244)</li>
<li><input checked="" disabled="" type="checkbox"> cg.howpublished (245)</li>
<li><input checked="" disabled="" type="checkbox"> cg.journal (246)</li>
<li><input checked="" disabled="" type="checkbox"> cg.isbn (247)</li>
<li><input checked="" disabled="" type="checkbox"> cg.issn (248)</li>
<li>[] cg.creator.identifier</li>
<li>[] cg.contributor.donor</li>
<li>[] cg.reviewStatus</li>
<li>[] cg.howPublished</li>
<li>[] cg.journal</li>
<li>[] cg.isbn</li>
<li>[] cg.issn</li>
</ul>
<h2 id="fields-to-delete">Fields to delete</h2>
<p>Fields to delete after migration:</p>

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-01-13T15:44:47+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

16
docs/sitemap.xml
View File

@ -4,27 +4,27 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2021-01-13T15:44:47+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2021-01-13T15:44:47+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/2021-01/</loc>
<lastmod>2021-01-13T15:44:47+02:00</lastmod>
<lastmod>2021-01-14T16:27:57+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2021-01-13T15:44:47+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2021-01-13T15:44:47+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
@ -39,12 +39,12 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/migration/</loc>
<lastmod>2020-12-01T19:15:48+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2020-12-01T19:15:48+02:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>
@ -114,7 +114,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/cgspace-cgcorev2-migration/</loc>
<lastmod>2020-04-13T15:30:24+03:00</lastmod>
<lastmod>2021-01-18T16:20:57+02:00</lastmod>
</url>
<url>

2
docs/tags/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
<meta property="og:updated_time" content="2020-12-01T19:15:48+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />

2
docs/tags/migration/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/migration/" />
<meta property="og:updated_time" content="2020-12-01T19:15:48+02:00" />
<meta property="og:updated_time" content="2021-01-18T16:20:57+02:00" />