alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-25 16:08:19 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2021-03-09

Browse Source
This commit is contained in:
Alan Orth 2021-03-09 15:48:47 +02:00
parent 551adc7053
commit 4f60b2aff3
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
23 changed files with 63 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
content/posts/2021-03.md
View File

@ -276,4 +276,18 @@ $ curl -s -X POST http://localhost:9200/openrxv-items-final/_clone/openrxv-items
- As I saw on my local test instance, even when you cancel a harvesting, it replaces the `openrxv-items-final` index with whatever is in `openrxv-items-temp` automatically, so I assume it will do the same now
## 2021-03-09
- The harvesting on AReS finished last night and everything worked as expected, with no manual intervention
- This means that [the issue](https://github.com/ilri/OpenRXV/issues/64) we were facing for a few months was due to the `openrxv-items` index being deleted and re-created as a standalone index instead of an alias of `openrxv-items-final`
- Talk to Moayad about OpenRXV development
- We realized that the missing/duplicate items issue is probably due to the long harvesting time on the REST API, as the time between starting the harvesting on page 0 and finishing the harvesting on page 900 (in the CGSpace example), some items will have been added to the repository, which causes the pages to shift
- I proposed a solution in the [GitHub issue](https://github.com/ilri/OpenRXV/issues/67), where we consult the site's XML sitemap after harvesting to see if we missed any items, and then we harvest them individually
- Peter sent me a list of 356 DOIs from Altmetric that don't have our Handles, so we need to Tweet them
- I used my `doi-to-handle.py` script to generate a list of handles and titles for him:
```console
$ ./ilri/doi-to-handle.py -i /tmp/dois.txt -o /tmp/handles.txt -db dspace -u dspace -p 'fuuu'
```
<!-- vim: set sw=2 ts=2: -->

28
docs/2021-03/index.html
View File

@ -19,7 +19,7 @@ Also, we found some issues building and running OpenRXV currently due to ecosyst
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2021-03/" />
<meta property="article:published_time" content="2021-03-01T10:13:54+02:00" />
<meta property="article:modified_time" content="2021-03-07T15:51:12+02:00" />
<meta property="article:modified_time" content="2021-03-08T20:13:40+02:00" />
@ -44,9 +44,9 @@ Also, we found some issues building and running OpenRXV currently due to ecosyst
"@type": "BlogPosting",
"headline": "March, 2021",
"url": "https://alanorth.github.io/cgspace-notes/2021-03/",
"wordCount": "1673",
"wordCount": "1861",
"datePublished": "2021-03-01T10:13:54+02:00",
"dateModified": "2021-03-07T15:51:12+02:00",
"dateModified": "2021-03-08T20:13:40+02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -415,7 +415,27 @@ $ curl -s -X POST http://localhost:9200/openrxv-items-final/_clone/openrxv-items
</code></pre><ul>
<li>As I saw on my local test instance, even when you cancel a harvesting, it replaces the <code>openrxv-items-final</code> index with whatever is in <code>openrxv-items-temp</code> automatically, so I assume it will do the same now</li>
</ul>
<!-- raw HTML omitted -->
<h2 id="2021-03-09">2021-03-09</h2>
<ul>
<li>The harvesting on AReS finished last night and everything worked as expected, with no manual intervention
<ul>
<li>This means that <a href="https://github.com/ilri/OpenRXV/issues/64">the issue</a> we were facing for a few months was due to the <code>openrxv-items</code> index being deleted and re-created as a standalone index instead of an alias of <code>openrxv-items-final</code></li>
</ul>
</li>
<li>Talk to Moayad about OpenRXV development
<ul>
<li>We realized that the missing/duplicate items issue is probably due to the long harvesting time on the REST API, as the time between starting the harvesting on page 0 and finishing the harvesting on page 900 (in the CGSpace example), some items will have been added to the repository, which causes the pages to shift</li>
<li>I proposed a solution in the <a href="https://github.com/ilri/OpenRXV/issues/67">GitHub issue</a>, where we consult the site&rsquo;s XML sitemap after harvesting to see if we missed any items, and then we harvest them individually</li>
</ul>
</li>
<li>Peter sent me a list of 356 DOIs from Altmetric that don&rsquo;t have our Handles, so we need to Tweet them
<ul>
<li>I used my <code>doi-to-handle.py</code> script to generate a list of handles and titles for him:</li>
</ul>
</li>
</ul>
<pre><code class="language-console" data-lang="console">$ ./ilri/doi-to-handle.py -i /tmp/dois.txt -o /tmp/handles.txt -db dspace -u dspace -p 'fuuu'
</code></pre><!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-03-07T15:51:12+02:00" />
<meta property="og:updated_time" content="2021-03-08T20:13:40+02:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2021-03-07T15:51:12+02:00</lastmod>
<lastmod>2021-03-08T20:13:40+02:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2021-03-07T15:51:12+02:00</lastmod>
<lastmod>2021-03-08T20:13:40+02:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2021-03/</loc>
<lastmod>2021-03-07T15:51:12+02:00</lastmod>
<lastmod>2021-03-08T20:13:40+02:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2021-03-07T15:51:12+02:00</lastmod>
<lastmod>2021-03-08T20:13:40+02:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2021-03-07T15:51:12+02:00</lastmod>
<lastmod>2021-03-08T20:13:40+02:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2021-02/</loc>
<lastmod>2021-03-04T22:46:05+02:00</lastmod>