Add notes for 2019-08-20

This commit is contained in:
Alan Orth 2019-08-21 01:26:32 +03:00
parent 35125e6fd1
commit d765b637c6
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 43 additions and 8 deletions

View File

@ -210,4 +210,19 @@ statistics-2015: org.apache.solr.common.SolrException:org.apache.solr.common.Sol
- After reboot the statistics-2018 core failed to load so I restarted `tomcat7` again - After reboot the statistics-2018 core failed to load so I restarted `tomcat7` again
- After this last restart all Solr cores seem to be up and running - After this last restart all Solr cores seem to be up and running
## 2019-08-20
- Francesco sent me a new CSV with the raw filenames and paths for the Bioversity migration
- All file paths are relative to the Typo3 upload path of `/fileadmin` on the Bioversity website
- I create a new column with the derived URL that I can use to download the PDFs with my `generate-thumbnails.py` script
- Unfortunately now the filename column has paths too, so I have to use a simple Python/Jython script in OpenRefine to get the basename of the files in the filename column:
```
import os
return os.path.basename(value)
```
- Then I can try to download all the files again with the script
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -27,7 +27,7 @@ Run system updates on DSpace Test (linode19) and reboot it
<meta property="og:type" content="article" /> <meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-08/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-08/" />
<meta property="article:published_time" content="2019-08-03T12:39:51+03:00" /> <meta property="article:published_time" content="2019-08-03T12:39:51+03:00" />
<meta property="article:modified_time" content="2019-08-16T17:10:11+03:00" /> <meta property="article:modified_time" content="2019-08-18T23:07:48+03:00" />
<meta name="twitter:card" content="summary"/> <meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="August, 2019"/> <meta name="twitter:title" content="August, 2019"/>
@ -59,9 +59,9 @@ Run system updates on DSpace Test (linode19) and reboot it
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "August, 2019", "headline": "August, 2019",
"url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-08\/", "url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-08\/",
"wordCount": "1536", "wordCount": "1637",
"datePublished": "2019-08-03T12:39:51\x2b03:00", "datePublished": "2019-08-03T12:39:51\x2b03:00",
"dateModified": "2019-08-16T17:10:11\x2b03:00", "dateModified": "2019-08-18T23:07:48\x2b03:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -400,6 +400,26 @@ $ dspace metadata-import -f /tmp/bioversity2.csv -e blah@blah.com
<li><p>After this last restart all Solr cores seem to be up and running</p></li> <li><p>After this last restart all Solr cores seem to be up and running</p></li>
</ul> </ul>
<h2 id="2019-08-20">2019-08-20</h2>
<ul>
<li><p>Francesco sent me a new CSV with the raw filenames and paths for the Bioversity migration</p>
<ul>
<li>All file paths are relative to the Typo3 upload path of <code>/fileadmin</code> on the Bioversity website</li>
<li>I create a new column with the derived URL that I can use to download the PDFs with my <code>generate-thumbnails.py</code> script</li>
<li><p>Unfortunately now the filename column has paths too, so I have to use a simple Python/Jython script in OpenRefine to get the basename of the files in the filename column:</p>
<pre><code>import os
return os.path.basename(value)
</code></pre></li>
</ul></li>
<li><p>Then I can try to download all the files again with the script</p></li>
</ul>
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -4,27 +4,27 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/2019-08/</loc> <loc>https://alanorth.github.io/cgspace-notes/2019-08/</loc>
<lastmod>2019-08-16T17:10:11+03:00</lastmod> <lastmod>2019-08-18T23:07:48+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2019-08-16T17:10:11+03:00</lastmod> <lastmod>2019-08-18T23:07:48+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2019-08-16T17:10:11+03:00</lastmod> <lastmod>2019-08-18T23:07:48+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc> <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2019-08-16T17:10:11+03:00</lastmod> <lastmod>2019-08-18T23:07:48+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2019-08-16T17:10:11+03:00</lastmod> <lastmod>2019-08-18T23:07:48+03:00</lastmod>
</url> </url>
<url> <url>