alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2019-12-17

Browse Source
This commit is contained in:
Alan Orth
2019-12-17 14:49:24 +02:00
parent d83c951532
commit d54e5b69f1
90 changed files with 1420 additions and 1377 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
docs/2018-10/index.html
View File

@ -23,7 +23,7 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items
I created a GitHub issue to track this #389, because I'm super busy in Nairobi right now
"/>
<meta name="generator" content="Hugo 0.60.1" />
<meta name="generator" content="Hugo 0.61.0" />
@ -104,12 +104,12 @@ I created a GitHub issue to track this #389, because I&#39;m super busy in Nairo
</p>
</header>
<h2 id="20181001">2018-10-01</h2>
<h2 id="2018-10-01">2018-10-01</h2>
<ul>
<li>Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items</li>
<li>I created a GitHub issue to track this <a href="https://github.com/ilri/DSpace/issues/389">#389</a>, because I'm super busy in Nairobi right now</li>
</ul>
<h2 id="20181003">2018-10-03</h2>
<h2 id="2018-10-03">2018-10-03</h2>
<ul>
<li>I see Moayad was busy collecting item views and downloads from CGSpace yesterday:</li>
</ul>
@ -193,7 +193,7 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
&quot;Thornton, Philip K.&quot;,Philip Thornton: 0000-0002-1854-0182
&quot;Thornton, Phillip&quot;,Philip Thornton: 0000-0002-1854-0182
&quot;Thornton, Phillip K.&quot;,Philip Thornton: 0000-0002-1854-0182
</code></pre><h2 id="20181004">2018-10-04</h2>
</code></pre><h2 id="2018-10-04">2018-10-04</h2>
<ul>
<li>Salem raised an issue that the dspace-statistics-api reports downloads for some items that have no bitstreams (like many limited access items)</li>
<li>Every item has at least a <code>LICENSE</code> bundle, and some have a <code>THUMBNAIL</code> bundle, but the indexing code is specifically checking for downloads from the <code>ORIGINAL</code> bundle
@ -213,24 +213,24 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
<li>I found a logic error in the dspace-statistics-api <code>indexer.py</code> script that was causing item views to be inserted into downloads</li>
<li>I tagged version 0.4.2 of the tool and redeployed it on CGSpace</li>
</ul>
<h2 id="20181005">2018-10-05</h2>
<h2 id="2018-10-05">2018-10-05</h2>
<ul>
<li>Meet with Peter, Abenet, and Sisay to discuss CGSpace meeting in Nairobi and Sisay's work plan</li>
<li>We agreed that he would do monthly updates of the controlled vocabularies and generate a new one for the top 1,000 AGROVOC terms</li>
<li>Add a link to <a href="https://cgspace.cgiar.org/explorer/">AReS explorer</a> to the CGSpace homepage introduction text</li>
</ul>
<h2 id="20181006">2018-10-06</h2>
<h2 id="2018-10-06">2018-10-06</h2>
<ul>
<li>Follow up with AgriKnowledge about including Handle links (<code>dc.identifier.uri</code>) on their item pages</li>
<li>In July, 2018 they had said their programmers would include the field in the next update of their website software</li>
<li><a href="https://repository.cimmyt.org/">CIMMYT's DSpace repository</a> is now running DSpace 5.x!</li>
<li>It's running OAI, but not REST, so I need to talk to Richard about that!</li>
</ul>
<h2 id="20181008">2018-10-08</h2>
<h2 id="2018-10-08">2018-10-08</h2>
<ul>
<li>AgriKnowledge says they're going to add the <code>dc.identifier.uri</code> to their item view in November when they update their website software</li>
</ul>
<h2 id="20181010">2018-10-10</h2>
<h2 id="2018-10-10">2018-10-10</h2>
<ul>
<li>Peter noticed that some recently added PDFs don't have thumbnails</li>
<li>When I tried to force them to be generated I got an error that I've never seen before:</li>
@ -249,7 +249,7 @@ org.im4java.core.InfoException: org.im4java.core.CommandException: org.im4java.c
<li>This works, but I'm not sure what ImageMagick's long-term plan is if they are going to disable ALL image formats&hellip;</li>
<li>I suppose I need to enable a workaround for this in Ansible?</li>
</ul>
<h2 id="20181011">2018-10-11</h2>
<h2 id="2018-10-11">2018-10-11</h2>
<ul>
<li>I emailed DuraSpace to update <a href="https://duraspace.org/registry/entry/4188/?gvid=178">our entry in their DSpace registry</a> (the data was still on DSpace 3, JSPUI, etc)</li>
<li>Generate a list of the top 1500 values for <code>dc.subject</code> so Sisay can start making a controlled vocabulary for it:</li>
@ -288,7 +288,7 @@ COPY 10000
<li>CTA uploaded some infographics that are very tall and their thumbnails disrupt the item lists on the front page and in their communities and collections</li>
<li>I decided to constrain the max height of these to 200px using CSS (<a href="https://github.com/ilri/DSpace/pull/392">#392</a>)</li>
</ul>
<h2 id="20181013">2018-10-13</h2>
<h2 id="2018-10-13">2018-10-13</h2>
<ul>
<li>Run all system updates on DSpace Test (linode19) and reboot it</li>
<li>Look through Peter's list of 746 author corrections in OpenRefine</li>
@ -308,7 +308,7 @@ COPY 10000
</code></pre><ul>
<li>I will apply these on CGSpace when I do the other updates tomorrow, as well as double check the high scoring ones to see if they are correct in Sisay's author controlled vocabulary</li>
</ul>
<h2 id="20181014">2018-10-14</h2>
<h2 id="2018-10-14">2018-10-14</h2>
<ul>
<li>Merge the authors controlled vocabulary (<a href="https://github.com/ilri/DSpace/pull/393">#393</a>), usage rights (<a href="https://github.com/ilri/DSpace/pull/394">#394</a>), and the upstream DSpace 5.x cherry-picks (<a href="https://github.com/ilri/DSpace/pull/395">#394</a>) into our <code>5_x-prod</code> branch</li>
<li>Switch to new CGIAR LDAP server on CGSpace, as it's been running (at least for authentication) on DSpace Test for the last few weeks, and I think they old one will be deprecated soon (today?)</li>
@ -330,7 +330,7 @@ COPY 10000
</li>
<li>I limited the tall thumbnails even further to 170px because Peter said CTA's were still too tall at 200px (<a href="https://github.com/ilri/DSpace/pull/396">#396</a>)</li>
</ul>
<h2 id="20181015">2018-10-15</h2>
<h2 id="2018-10-15">2018-10-15</h2>
<ul>
<li>Tomcat on DSpace Test (linode19) has somehow stopped running all the DSpace applications</li>
<li>I don't see anything in the Catalina logs or <code>dmesg</code>, and the Tomcat manager shows XMLUI, REST, OAI, etc all &ldquo;Running: false&rdquo;</li>
@ -353,7 +353,7 @@ $ psql -h localhost -U postgres dspacetest -c 'alter user dspacetest superuser;'
$ pg_restore -h localhost -U postgres -d dspacetest -O --role=dspacetest -h localhost ~/Downloads/cgspace_2018-10-11.backup
$ psql -h localhost -U postgres -f ~/src/git/DSpace/dspace/etc/postgres/update-sequences.sql dspacetest
$ psql -h localhost -U postgres dspacetest -c 'alter user dspacetest nosuperuser;'
</code></pre><h2 id="20181016">2018-10-16</h2>
</code></pre><h2 id="2018-10-16">2018-10-16</h2>
<ul>
<li>Generate a list of the schema on CGSpace so CodeObia can compare with MELSpace:</li>
</ul>
@ -401,7 +401,7 @@ $ time http --print h 'https://dspacetest.cgiar.org/rest/items?expand=metadata,b
</code></pre><ul>
<li>I sent a mail to dspace-tech to ask how to profile this&hellip;</li>
</ul>
<h2 id="20181017">2018-10-17</h2>
<h2 id="2018-10-17">2018-10-17</h2>
<ul>
<li>I decided to update most of the existing metadata values that we have in <code>dc.rights</code> on CGSpace to be machine readable in SPDX format (with Creative Commons version if it was included)</li>
<li>Most of the are from Bioversity, and I asked Maria for permission before updating them</li>
@ -444,7 +444,7 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
<li>I made a pull request and merged the ORCID updates into the <code>5_x-prod</code> branch (<a href="https://github.com/ilri/DSpace/pull/397">#397</a>)</li>
<li>Improve the logic of name checking in my <a href="https://gist.github.com/alanorth/57a88379126d844563c1410bd7b8d12b">resolve-orcids.py</a> script</li>
</ul>
<h2 id="20181018">2018-10-18</h2>
<h2 id="2018-10-18">2018-10-18</h2>
<ul>
<li>I granted MEL's deposit user admin access to IITA, CIP, Bioversity, and RTB communities on DSpace Test so they can start testing real depositing</li>
<li>After they do some tests and we check the values Enrico will send a formal email to Peter et al to ask that they start depositing officially</li>
@ -455,7 +455,7 @@ $ /usr/lib/postgresql/9.6/bin/pg_upgrade -b /usr/lib/postgresql/9.5/bin -B /usr/
$ exit
# systemctl start postgresql
# dpkg -r postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
</code></pre><h2 id="20181019">2018-10-19</h2>
</code></pre><h2 id="2018-10-19">2018-10-19</h2>
<ul>
<li>Help Francesca from Bioversity generate a report about items they uploaded in 2015 through 2018</li>
<li>Linode emailed me to say that CGSpace (linode18) had high CPU usage for a few hours this afternoon</li>
@ -475,7 +475,7 @@ $ exit
</code></pre><ul>
<li>5.9.6.51 is MegaIndex, which I've seen before&hellip;</li>
</ul>
<h2 id="20181020">2018-10-20</h2>
<h2 id="2018-10-20">2018-10-20</h2>
<ul>
<li>I was going to try to run Solr in Docker because I learned I can run Docker on Travis-CI (for testing my dspace-statistics-api), but the oldest official Solr images are for 5.5, and DSpace's Solr configuration is for 4.9</li>
<li>This means our existing Solr configuration doesn't run in Solr 5.5:</li>
@ -522,11 +522,11 @@ ERROR: Error CREATEing SolrCore 'statistics': Unable to create core [statistics]
</code></pre><ul>
<li>So I'm not sure why this bot uses so many sessions — is it because it requests very slowly?</li>
</ul>
<h2 id="20181021">2018-10-21</h2>
<h2 id="2018-10-21">2018-10-21</h2>
<ul>
<li>Discuss AfricaRice joining CGSpace</li>
</ul>
<h2 id="20181022">2018-10-22</h2>
<h2 id="2018-10-22">2018-10-22</h2>
<ul>
<li>Post message to Yammer about usage rights (dc.rights)</li>
<li>Change <code>build.properties</code> to use HTTPS for Handles in our <a href="https://github.com/ilri/rmg-ansible-public">Ansible infrastructure playbooks</a></li>
@ -546,7 +546,7 @@ UPDATE 76608
<li>Skype with Peter about ToRs for the AReS open source work and future plans to develop tools around the DSpace ecosystem</li>
<li>Help CGSpace users with some issues related to usage rights</li>
</ul>
<h2 id="20181023">2018-10-23</h2>
<h2 id="2018-10-23">2018-10-23</h2>
<ul>
<li>Improve the usage rights (dc.rights) on CGSpace again by adding the long names in the submission form, as well as adding versio 3.0 and Creative Commons Zero (CC0) public domain license (<a href="https://github.com/ilri/DSpace/pull/399">#399</a>)</li>
<li>Add &ldquo;usage rights&rdquo; to the XMLUI item display (<a href="https://github.com/ilri/DSpace/pull/400">#400</a>)</li>
@ -571,14 +571,14 @@ $ curl -X GET -H &quot;Content-Type: application/json&quot; -H &quot;Accept: app
<li>Improve the documentatin of my <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a></li>
<li>Email Modi and Jayashree from ICRISAT to ask if they want to join CGSpace as partners</li>
</ul>
<h2 id="20181024">2018-10-24</h2>
<h2 id="2018-10-24">2018-10-24</h2>
<ul>
<li>I deployed the new Creative Commons choices to the usage rights on the CGSpace submission form</li>
<li>Also, I deployed the changes to show usage rights on the item view</li>
<li>Re-work the <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a> to use Python's native json instead of ujson to make it easier to deploy in places where we don't have — or don't want to have — Python headers and a compiler (like containers)</li>
<li>Re-work the deployment of the API to use systemd's <code>EnvironmentFile</code> to read the environment variables instead of <code>Environment</code> in the <a href="https://github.com/ilri/rmg-ansible-public">RMG Ansible infrastructure scripts</a></li>
</ul>
<h2 id="20181025">2018-10-25</h2>
<h2 id="2018-10-25">2018-10-25</h2>
<ul>
<li>Send Peter and Jane a list of technical ToRs for AReS open source work:</li>
<li>Basic version of AReS that works with metadata fields present in default DSpace 5.x/6.x (for example author, date issued, type, subjects)
@ -595,7 +595,7 @@ $ curl -X GET -H &quot;Content-Type: application/json&quot; -H &quot;Accept: app
</li>
<li>Maria asked if we can add publisher (<code>dc.publisher</code>) to the advanced search filters, so I created a <a href="https://github.com/ilri/DSpace/issues/401">GitHub issue</a> to track it</li>
</ul>
<h2 id="20181028">2018-10-28</h2>
<h2 id="2018-10-28">2018-10-28</h2>
<ul>
<li>I forked the <a href="https://github.com/alanorth/SolrClient/tree/kazoo-2.5.0">SolrClient library and updated its kazoo dependency to be version 2.5.0</a> so we stop getting errors about &ldquo;async&rdquo; being a reserved keyword in Python 3.7</li>
<li>Then I re-generated the <code>requirements.txt</code> in the <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-library</a> and released version 0.5.2</li>
@ -606,12 +606,12 @@ $ curl -X GET -H &quot;Content-Type: application/json&quot; -H &quot;Accept: app
<li>I merged the changes for adding versionless Creative Commons licenses to the submission form to the <code>5_x-prod</code> branch (<a href="https://github.com/ilri/DSpace/pull/403">#403</a>)</li>
<li>I will deploy them later this week</li>
</ul>
<h2 id="20181029">2018-10-29</h2>
<h2 id="2018-10-29">2018-10-29</h2>
<ul>
<li>I deployed the publisher and Creative Commons changes to CGSpace, ran all system updates, and rebooted the server</li>
<li>I sent the email to Jane Poole and ILRI ICT and Finance to start the admin process of getting a new Linode server for AReS</li>
</ul>
<h2 id="20181030">2018-10-30</h2>
<h2 id="2018-10-30">2018-10-30</h2>
<ul>
<li>Meet with the COPO guys to walk them through the CGSpace submission workflow and discuss CG core, REST API, etc
<ul>
@ -621,7 +621,7 @@ $ curl -X GET -H &quot;Content-Type: application/json&quot; -H &quot;Accept: app
</ul>
</li>
</ul>
<h2 id="20181031">2018-10-31</h2>
<h2 id="2018-10-31">2018-10-31</h2>
<ul>
<li>More discussion and planning for AReS open sourcing and Amman meeting in 2019-10</li>
<li>I did some work to clean up and improve the dspace-statistics-api README.md and project structure and <a href="https://github.com/ilri/dspace-statistics-api">moved it to the ILRI organization on GitHub</a></li>