mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2025-01-27 05:49:12 +01:00
Add notes for 2020-01-27
This commit is contained in:
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="2018-10-01
|
||||
|
||||
Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items
|
||||
I created a GitHub issue to track this #389, because I'm super busy in Nairobi right now
|
||||
I created a GitHub issue to track this #389, because I’m super busy in Nairobi right now
|
||||
" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2018-10/" />
|
||||
@ -21,9 +21,9 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
|
||||
<meta name="twitter:description" content="2018-10-01
|
||||
|
||||
Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items
|
||||
I created a GitHub issue to track this #389, because I'm super busy in Nairobi right now
|
||||
I created a GitHub issue to track this #389, because I’m super busy in Nairobi right now
|
||||
"/>
|
||||
<meta name="generator" content="Hugo 0.62.2" />
|
||||
<meta name="generator" content="Hugo 0.63.1" />
|
||||
|
||||
|
||||
|
||||
@ -53,7 +53,7 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
|
||||
|
||||
<!-- combined, minified CSS -->
|
||||
|
||||
<link href="https://alanorth.github.io/cgspace-notes/css/style.a20c1a4367639632cdb341d23c27ca44fedcc75b0f8b3cbea6203010da153d3c.css" rel="stylesheet" integrity="sha256-ogwaQ2djljLNs0HSPCfKRP7cx1sPizy+piAwENoVPTw=" crossorigin="anonymous">
|
||||
<link href="https://alanorth.github.io/cgspace-notes/css/style.23e2c3298bcc8c1136c19aba330c211ec94c36f7c4454ea15cf4d3548370042a.css" rel="stylesheet" integrity="sha256-I+LDKYvMjBE2wZq6MwwhHslMNvfERU6hXPTTVINwBCo=" crossorigin="anonymous">
|
||||
|
||||
|
||||
<!-- RSS 2.0 feed -->
|
||||
@ -100,7 +100,7 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
|
||||
<header>
|
||||
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-10-01T22:31:54+03:00">Mon Oct 01, 2018</time> by Alan Orth in
|
||||
<i class="fa fa-folder" aria-hidden="true"></i> <a href="/cgspace-notes/categories/notes" rel="category tag">Notes</a>
|
||||
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/cgspace-notes/categories/notes" rel="category tag">Notes</a>
|
||||
|
||||
|
||||
</p>
|
||||
@ -108,7 +108,7 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
|
||||
<h2 id="2018-10-01">2018-10-01</h2>
|
||||
<ul>
|
||||
<li>Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items</li>
|
||||
<li>I created a GitHub issue to track this <a href="https://github.com/ilri/DSpace/issues/389">#389</a>, because I'm super busy in Nairobi right now</li>
|
||||
<li>I created a GitHub issue to track this <a href="https://github.com/ilri/DSpace/issues/389">#389</a>, because I’m super busy in Nairobi right now</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-03">2018-10-03</h2>
|
||||
<ul>
|
||||
@ -133,7 +133,7 @@ I created a GitHub issue to track this #389, because I'm super busy in Nairo
|
||||
118927 200
|
||||
31435 500
|
||||
</code></pre><ul>
|
||||
<li>I added Phil Thornton and Sonal Henson's ORCID identifiers to the controlled vocabulary for <code>cg.creator.orcid</code> and then re-generated the names using my <a href="https://gist.github.com/alanorth/57a88379126d844563c1410bd7b8d12b">resolve-orcids.py</a> script:</li>
|
||||
<li>I added Phil Thornton and Sonal Henson’s ORCID identifiers to the controlled vocabulary for <code>cg.creator.orcid</code> and then re-generated the names using my <a href="https://gist.github.com/alanorth/57a88379126d844563c1410bd7b8d12b">resolve-orcids.py</a> script:</li>
|
||||
</ul>
|
||||
<pre><code>$ grep -oE '[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{4}-[A-Z0-9]{4}' ~/src/git/DSpace/dspace/config/controlled-vocabularies/cg-creator-id.xml | sort | uniq > 2018-10-03-orcids.txt
|
||||
$ ./resolve-orcids.py -i 2018-10-03-orcids.txt -o 2018-10-03-names.txt -d
|
||||
@ -160,14 +160,14 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
87646 34.218.226.147
|
||||
111729 213.139.53.62
|
||||
</code></pre><ul>
|
||||
<li>But in super positive news, he says they are using my new <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a> and it's MUCH faster than using Atmire CUA's internal “restlet” API</li>
|
||||
<li>I don't recognize the <code>138.201.49.199</code> IP, but it is in Germany (Hetzner) and appears to be paginating over some browse pages and downloading bitstreams:</li>
|
||||
<li>But in super positive news, he says they are using my new <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a> and it’s MUCH faster than using Atmire CUA’s internal “restlet” API</li>
|
||||
<li>I don’t recognize the <code>138.201.49.199</code> IP, but it is in Germany (Hetzner) and appears to be paginating over some browse pages and downloading bitstreams:</li>
|
||||
</ul>
|
||||
<pre><code># grep 138.201.49.199 /var/log/nginx/access.log | grep -o -E 'GET /[a-z]+' | sort | uniq -c
|
||||
8324 GET /bitstream
|
||||
4193 GET /handle
|
||||
</code></pre><ul>
|
||||
<li>Suspiciously, it's only grabbing the CGIAR System Office community (handle prefix 10947):</li>
|
||||
<li>Suspiciously, it’s only grabbing the CGIAR System Office community (handle prefix 10947):</li>
|
||||
</ul>
|
||||
<pre><code># grep 138.201.49.199 /var/log/nginx/access.log | grep -o -E 'GET /handle/[0-9]{5}' | sort | uniq -c
|
||||
7 GET /handle/10568
|
||||
@ -177,9 +177,9 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
</ul>
|
||||
<pre><code>Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
|
||||
</code></pre><ul>
|
||||
<li>It's clearly a bot and it's not re-using its Tomcat session, so I will add its IP to the nginx bad bot list</li>
|
||||
<li>I looked in Solr's statistics core and these hits were actually all counted as <code>isBot:false</code> (of course)… hmmm</li>
|
||||
<li>I tagged all of Sonal and Phil's items with their ORCID identifiers on CGSpace using my <a href="https://gist.github.com/alanorth/a49d85cd9c5dea89cddbe809813a7050">add-orcid-identifiers.py</a> script:</li>
|
||||
<li>It’s clearly a bot and it’s not re-using its Tomcat session, so I will add its IP to the nginx bad bot list</li>
|
||||
<li>I looked in Solr’s statistics core and these hits were actually all counted as <code>isBot:false</code> (of course)… hmmm</li>
|
||||
<li>I tagged all of Sonal and Phil’s items with their ORCID identifiers on CGSpace using my <a href="https://gist.github.com/alanorth/a49d85cd9c5dea89cddbe809813a7050">add-orcid-identifiers.py</a> script:</li>
|
||||
</ul>
|
||||
<pre><code>$ ./add-orcid-identifiers-csv.py -i 2018-10-03-add-orcids.csv -db dspace -u dspace -p 'fuuu'
|
||||
</code></pre><ul>
|
||||
@ -205,7 +205,7 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
</li>
|
||||
<li>I see there are other bundles we might need to pay attention to: <code>TEXT</code>, <code>@_LOGO-COLLECTION_@</code>, <code>@_LOGO-COMMUNITY_@</code>, etc…</li>
|
||||
<li>On a hunch I dropped the statistics table and re-indexed and now those two items above have no downloads</li>
|
||||
<li>So it's fixed, but I'm not sure why!</li>
|
||||
<li>So it’s fixed, but I’m not sure why!</li>
|
||||
<li>Peter wants to know the number of API requests per month, which was about 250,000 in September (exluding statlet requests):</li>
|
||||
</ul>
|
||||
<pre><code># zcat --force /var/log/nginx/{oai,rest}.log* | grep -E 'Sep/2018' | grep -c -v 'statlets'
|
||||
@ -216,7 +216,7 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
</ul>
|
||||
<h2 id="2018-10-05">2018-10-05</h2>
|
||||
<ul>
|
||||
<li>Meet with Peter, Abenet, and Sisay to discuss CGSpace meeting in Nairobi and Sisay's work plan</li>
|
||||
<li>Meet with Peter, Abenet, and Sisay to discuss CGSpace meeting in Nairobi and Sisay’s work plan</li>
|
||||
<li>We agreed that he would do monthly updates of the controlled vocabularies and generate a new one for the top 1,000 AGROVOC terms</li>
|
||||
<li>Add a link to <a href="https://cgspace.cgiar.org/explorer/">AReS explorer</a> to the CGSpace homepage introduction text</li>
|
||||
</ul>
|
||||
@ -224,30 +224,30 @@ Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
<ul>
|
||||
<li>Follow up with AgriKnowledge about including Handle links (<code>dc.identifier.uri</code>) on their item pages</li>
|
||||
<li>In July, 2018 they had said their programmers would include the field in the next update of their website software</li>
|
||||
<li><a href="https://repository.cimmyt.org/">CIMMYT's DSpace repository</a> is now running DSpace 5.x!</li>
|
||||
<li>It's running OAI, but not REST, so I need to talk to Richard about that!</li>
|
||||
<li><a href="https://repository.cimmyt.org/">CIMMYT’s DSpace repository</a> is now running DSpace 5.x!</li>
|
||||
<li>It’s running OAI, but not REST, so I need to talk to Richard about that!</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-08">2018-10-08</h2>
|
||||
<ul>
|
||||
<li>AgriKnowledge says they're going to add the <code>dc.identifier.uri</code> to their item view in November when they update their website software</li>
|
||||
<li>AgriKnowledge says they’re going to add the <code>dc.identifier.uri</code> to their item view in November when they update their website software</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-10">2018-10-10</h2>
|
||||
<ul>
|
||||
<li>Peter noticed that some recently added PDFs don't have thumbnails</li>
|
||||
<li>When I tried to force them to be generated I got an error that I've never seen before:</li>
|
||||
<li>Peter noticed that some recently added PDFs don’t have thumbnails</li>
|
||||
<li>When I tried to force them to be generated I got an error that I’ve never seen before:</li>
|
||||
</ul>
|
||||
<pre><code>$ dspace filter-media -v -f -i 10568/97613
|
||||
org.im4java.core.InfoException: org.im4java.core.CommandException: org.im4java.core.CommandException: identify: not authorized `/tmp/impdfthumb5039464037201498062.pdf' @ error/constitute.c/ReadImage/412.
|
||||
</code></pre><ul>
|
||||
<li>I see there was an update to Ubuntu's ImageMagick on 2018-10-05, so maybe something changed or broke?</li>
|
||||
<li>I get the same error when forcing <code>filter-media</code> to run on DSpace Test too, so it's gotta be an ImageMagic bug</li>
|
||||
<li>I see there was an update to Ubuntu’s ImageMagick on 2018-10-05, so maybe something changed or broke?</li>
|
||||
<li>I get the same error when forcing <code>filter-media</code> to run on DSpace Test too, so it’s gotta be an ImageMagic bug</li>
|
||||
<li>The ImageMagick version is currently 8:6.8.9.9-7ubuntu5.13, and there is an <a href="https://usn.ubuntu.com/3785-1/">Ubuntu Security Notice from 2018-10-04</a></li>
|
||||
<li>Wow, someone on <a href="https://twitter.com/rosscampbell/status/1048268966819319808">Twitter posted about this breaking his web application</a> (and it was retweeted by the ImageMagick acount!)</li>
|
||||
<li>I commented out the line that disables PDF thumbnails in <code>/etc/ImageMagick-6/policy.xml</code>:</li>
|
||||
</ul>
|
||||
<pre><code> <!--<policy domain="coder" rights="none" pattern="PDF" />-->
|
||||
</code></pre><ul>
|
||||
<li>This works, but I'm not sure what ImageMagick's long-term plan is if they are going to disable ALL image formats…</li>
|
||||
<li>This works, but I’m not sure what ImageMagick’s long-term plan is if they are going to disable ALL image formats…</li>
|
||||
<li>I suppose I need to enable a workaround for this in Ansible?</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-11">2018-10-11</h2>
|
||||
@ -292,7 +292,7 @@ COPY 10000
|
||||
<h2 id="2018-10-13">2018-10-13</h2>
|
||||
<ul>
|
||||
<li>Run all system updates on DSpace Test (linode19) and reboot it</li>
|
||||
<li>Look through Peter's list of 746 author corrections in OpenRefine</li>
|
||||
<li>Look through Peter’s list of 746 author corrections in OpenRefine</li>
|
||||
<li>I first facet by blank, trim whitespace, and then check for weird characters that might be indicative of encoding issues with this GREL:</li>
|
||||
</ul>
|
||||
<pre><code>or(
|
||||
@ -307,13 +307,13 @@ COPY 10000
|
||||
</ul>
|
||||
<pre><code>$ ./fix-metadata-values.py -i 2018-10-11-top-authors.csv -db dspace -u dspace -p 'fuuu' -f dc.contributor.author -t CORRECT -m 3
|
||||
</code></pre><ul>
|
||||
<li>I will apply these on CGSpace when I do the other updates tomorrow, as well as double check the high scoring ones to see if they are correct in Sisay's author controlled vocabulary</li>
|
||||
<li>I will apply these on CGSpace when I do the other updates tomorrow, as well as double check the high scoring ones to see if they are correct in Sisay’s author controlled vocabulary</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-14">2018-10-14</h2>
|
||||
<ul>
|
||||
<li>Merge the authors controlled vocabulary (<a href="https://github.com/ilri/DSpace/pull/393">#393</a>), usage rights (<a href="https://github.com/ilri/DSpace/pull/394">#394</a>), and the upstream DSpace 5.x cherry-picks (<a href="https://github.com/ilri/DSpace/pull/395">#394</a>) into our <code>5_x-prod</code> branch</li>
|
||||
<li>Switch to new CGIAR LDAP server on CGSpace, as it's been running (at least for authentication) on DSpace Test for the last few weeks, and I think they old one will be deprecated soon (today?)</li>
|
||||
<li>Apply Peter's 746 author corrections on CGSpace and DSpace Test using my <a href="https://gist.github.com/alanorth/df92cbfb54d762ba21b28f7cd83b6897">fix-metadata-values.py</a> script:</li>
|
||||
<li>Switch to new CGIAR LDAP server on CGSpace, as it’s been running (at least for authentication) on DSpace Test for the last few weeks, and I think they old one will be deprecated soon (today?)</li>
|
||||
<li>Apply Peter’s 746 author corrections on CGSpace and DSpace Test using my <a href="https://gist.github.com/alanorth/df92cbfb54d762ba21b28f7cd83b6897">fix-metadata-values.py</a> script:</li>
|
||||
</ul>
|
||||
<pre><code>$ ./fix-metadata-values.py -i /tmp/2018-10-11-top-authors.csv -f dc.contributor.author -t CORRECT -m 3 -db dspace -u dspace -p 'fuuu'
|
||||
</code></pre><ul>
|
||||
@ -322,21 +322,21 @@ COPY 10000
|
||||
<li>Restarting the service with systemd works for a few seconds, then the java process quits</li>
|
||||
<li>I suspect that the systemd service type needs to be <code>forking</code> rather than <code>simple</code>, because the service calls the default DSpace <code>start-handle-server</code> shell script, which uses <code>nohup</code> and <code>&</code> to background the java process</li>
|
||||
<li>It would be nice if there was a cleaner way to start the service and then just log to the systemd journal rather than all this hiding and log redirecting</li>
|
||||
<li>Email the Landportal.org people to ask if they would consider Dublin Core metadata tags in their page's header, rather than the HTML properties they are using in their body</li>
|
||||
<li>Email the Landportal.org people to ask if they would consider Dublin Core metadata tags in their page’s header, rather than the HTML properties they are using in their body</li>
|
||||
<li>Peter pointed out that some thumbnails were still not getting generated
|
||||
<ul>
|
||||
<li>When I tried to generate them manually I noticed that the path to the CMYK profile had changed because Ubuntu upgraded Ghostscript from 9.18 to 9.25 last week… WTF?</li>
|
||||
<li>Looks like I can use <code>/usr/share/ghostscript/current</code> instead of <code>/usr/share/ghostscript/9.25</code>…</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>I limited the tall thumbnails even further to 170px because Peter said CTA's were still too tall at 200px (<a href="https://github.com/ilri/DSpace/pull/396">#396</a>)</li>
|
||||
<li>I limited the tall thumbnails even further to 170px because Peter said CTA’s were still too tall at 200px (<a href="https://github.com/ilri/DSpace/pull/396">#396</a>)</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-15">2018-10-15</h2>
|
||||
<ul>
|
||||
<li>Tomcat on DSpace Test (linode19) has somehow stopped running all the DSpace applications</li>
|
||||
<li>I don't see anything in the Catalina logs or <code>dmesg</code>, and the Tomcat manager shows XMLUI, REST, OAI, etc all “Running: false”</li>
|
||||
<li>I don’t see anything in the Catalina logs or <code>dmesg</code>, and the Tomcat manager shows XMLUI, REST, OAI, etc all “Running: false”</li>
|
||||
<li>Actually, now I remember that yesterday when I deployed the latest changes from git on DSpace Test I noticed a syntax error in one XML file when I was doing the discovery reindexing</li>
|
||||
<li>I fixed it so that I could reindex, but I guess the rest of DSpace actually didn't start up…</li>
|
||||
<li>I fixed it so that I could reindex, but I guess the rest of DSpace actually didn’t start up…</li>
|
||||
<li>Create an account on DSpace Test for Felix from Earlham so he can test COPO submission
|
||||
<ul>
|
||||
<li>I created a new collection and added him as the administrator so he can test submission</li>
|
||||
@ -360,8 +360,8 @@ $ psql -h localhost -U postgres dspacetest -c 'alter user dspacetest nosuperuser
|
||||
</ul>
|
||||
<pre><code>dspace=# \copy (SELECT (CASE when metadata_schema_id=1 THEN 'dc' WHEN metadata_schema_id=2 THEN 'cg' END) AS schema, element, qualifier, scope_note FROM metadatafieldregistry where metadata_schema_id IN (1,2)) TO /tmp/cgspace-schema.csv WITH CSV HEADER;
|
||||
</code></pre><ul>
|
||||
<li>Talking to the CodeObia guys about the REST API I started to wonder why it's so slow and how I can quantify it in order to ask the dspace-tech mailing list for help profiling it</li>
|
||||
<li>Interestingly, the speed doesn't get better after you request the same thing multiple times–it's consistently bad on both CGSpace and DSpace Test!</li>
|
||||
<li>Talking to the CodeObia guys about the REST API I started to wonder why it’s so slow and how I can quantify it in order to ask the dspace-tech mailing list for help profiling it</li>
|
||||
<li>Interestingly, the speed doesn’t get better after you request the same thing multiple times–it’s consistently bad on both CGSpace and DSpace Test!</li>
|
||||
</ul>
|
||||
<pre><code>$ time http --print h 'https://cgspace.cgiar.org/rest/items?expand=metadata,bitstreams,parentCommunityList&limit=100&offset=0'
|
||||
...
|
||||
@ -441,13 +441,13 @@ $ tidy -xml -utf8 -iq -m -w 0 dspace/config/controlled-vocabularies/cg-creator-i
|
||||
<pre><code>Looking up the names associated with ORCID iD: 0000-0001-7930-5752
|
||||
Given Names Deactivated Family Name Deactivated: 0000-0001-7930-5752
|
||||
</code></pre><ul>
|
||||
<li>So I need to handle that situation in the script for sure, but I'm not sure what to do organizationally or ethically, since that user disabled their name! Do we remove him from the list?</li>
|
||||
<li>So I need to handle that situation in the script for sure, but I’m not sure what to do organizationally or ethically, since that user disabled their name! Do we remove him from the list?</li>
|
||||
<li>I made a pull request and merged the ORCID updates into the <code>5_x-prod</code> branch (<a href="https://github.com/ilri/DSpace/pull/397">#397</a>)</li>
|
||||
<li>Improve the logic of name checking in my <a href="https://gist.github.com/alanorth/57a88379126d844563c1410bd7b8d12b">resolve-orcids.py</a> script</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-18">2018-10-18</h2>
|
||||
<ul>
|
||||
<li>I granted MEL's deposit user admin access to IITA, CIP, Bioversity, and RTB communities on DSpace Test so they can start testing real depositing</li>
|
||||
<li>I granted MEL’s deposit user admin access to IITA, CIP, Bioversity, and RTB communities on DSpace Test so they can start testing real depositing</li>
|
||||
<li>After they do some tests and we check the values Enrico will send a formal email to Peter et al to ask that they start depositing officially</li>
|
||||
<li>I upgraded PostgreSQL to 9.6 on DSpace Test using Ansible, then had to manually <a href="https://wiki.postgresql.org/wiki/Using_pg_upgrade_on_Ubuntu/Debian">migrate from 9.5 to 9.6</a>:</li>
|
||||
</ul>
|
||||
@ -474,12 +474,12 @@ $ exit
|
||||
1629 66.249.64.91
|
||||
1758 5.9.6.51
|
||||
</code></pre><ul>
|
||||
<li>5.9.6.51 is MegaIndex, which I've seen before…</li>
|
||||
<li>5.9.6.51 is MegaIndex, which I’ve seen before…</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-20">2018-10-20</h2>
|
||||
<ul>
|
||||
<li>I was going to try to run Solr in Docker because I learned I can run Docker on Travis-CI (for testing my dspace-statistics-api), but the oldest official Solr images are for 5.5, and DSpace's Solr configuration is for 4.9</li>
|
||||
<li>This means our existing Solr configuration doesn't run in Solr 5.5:</li>
|
||||
<li>I was going to try to run Solr in Docker because I learned I can run Docker on Travis-CI (for testing my dspace-statistics-api), but the oldest official Solr images are for 5.5, and DSpace’s Solr configuration is for 4.9</li>
|
||||
<li>This means our existing Solr configuration doesn’t run in Solr 5.5:</li>
|
||||
</ul>
|
||||
<pre><code>$ sudo docker pull solr:5
|
||||
$ sudo docker run --name my_solr -v ~/dspace/solr/statistics/conf:/tmp/conf -d -p 8983:8983 -t solr:5
|
||||
@ -488,7 +488,7 @@ $ sudo docker logs my_solr
|
||||
ERROR: Error CREATEing SolrCore 'statistics': Unable to create core [statistics] Caused by: solr.IntField
|
||||
</code></pre><ul>
|
||||
<li>Apparently a bunch of variable types were removed in <a href="https://issues.apache.org/jira/browse/SOLR-5936">Solr 5</a></li>
|
||||
<li>So for now it's actually a huge pain in the ass to run the tests for my dspace-statistics-api</li>
|
||||
<li>So for now it’s actually a huge pain in the ass to run the tests for my dspace-statistics-api</li>
|
||||
<li>Linode sent a message that the CPU usage was high on CGSpace (linode18) last night</li>
|
||||
<li>According to the nginx logs around that time it was 5.9.6.51 (MegaIndex) again:</li>
|
||||
</ul>
|
||||
@ -517,11 +517,11 @@ ERROR: Error CREATEing SolrCore 'statistics': Unable to create core [statistics]
|
||||
# grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=5.9.6.51' dspace.log.2018-10-20 | sort | uniq
|
||||
8915
|
||||
</code></pre><ul>
|
||||
<li>Last month I added “crawl” to the Tomcat Crawler Session Manager Valve's regular expression matching, and it seems to be working for MegaIndex's user agent:</li>
|
||||
<li>Last month I added “crawl” to the Tomcat Crawler Session Manager Valve’s regular expression matching, and it seems to be working for MegaIndex’s user agent:</li>
|
||||
</ul>
|
||||
<pre><code>$ http --print Hh 'https://dspacetest.cgiar.org/handle/10568/1' User-Agent:'"Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"'
|
||||
</code></pre><ul>
|
||||
<li>So I'm not sure why this bot uses so many sessions — is it because it requests very slowly?</li>
|
||||
<li>So I’m not sure why this bot uses so many sessions — is it because it requests very slowly?</li>
|
||||
</ul>
|
||||
<h2 id="2018-10-21">2018-10-21</h2>
|
||||
<ul>
|
||||
@ -552,7 +552,7 @@ UPDATE 76608
|
||||
<li>Improve the usage rights (dc.rights) on CGSpace again by adding the long names in the submission form, as well as adding versio 3.0 and Creative Commons Zero (CC0) public domain license (<a href="https://github.com/ilri/DSpace/pull/399">#399</a>)</li>
|
||||
<li>Add “usage rights” to the XMLUI item display (<a href="https://github.com/ilri/DSpace/pull/400">#400</a>)</li>
|
||||
<li>I emailed the MARLO guys to ask if they can send us a dump of rights data and Handles from their system so we can tag our older items on CGSpace</li>
|
||||
<li>Testing REST login and logout via httpie because Felix from Earlham says he's having issues:</li>
|
||||
<li>Testing REST login and logout via httpie because Felix from Earlham says he’s having issues:</li>
|
||||
</ul>
|
||||
<pre><code>$ http --print b POST 'https://dspacetest.cgiar.org/rest/login' email='testdeposit@cgiar.org' password=deposit
|
||||
acef8a4a-41f3-4392-b870-e873790f696b
|
||||
@ -576,8 +576,8 @@ $ curl -X GET -H "Content-Type: application/json" -H "Accept: app
|
||||
<ul>
|
||||
<li>I deployed the new Creative Commons choices to the usage rights on the CGSpace submission form</li>
|
||||
<li>Also, I deployed the changes to show usage rights on the item view</li>
|
||||
<li>Re-work the <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a> to use Python's native json instead of ujson to make it easier to deploy in places where we don't have — or don't want to have — Python headers and a compiler (like containers)</li>
|
||||
<li>Re-work the deployment of the API to use systemd's <code>EnvironmentFile</code> to read the environment variables instead of <code>Environment</code> in the <a href="https://github.com/ilri/rmg-ansible-public">RMG Ansible infrastructure scripts</a></li>
|
||||
<li>Re-work the <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-api</a> to use Python’s native json instead of ujson to make it easier to deploy in places where we don’t have — or don’t want to have — Python headers and a compiler (like containers)</li>
|
||||
<li>Re-work the deployment of the API to use systemd’s <code>EnvironmentFile</code> to read the environment variables instead of <code>Environment</code> in the <a href="https://github.com/ilri/rmg-ansible-public">RMG Ansible infrastructure scripts</a></li>
|
||||
</ul>
|
||||
<h2 id="2018-10-25">2018-10-25</h2>
|
||||
<ul>
|
||||
@ -602,7 +602,7 @@ $ curl -X GET -H "Content-Type: application/json" -H "Accept: app
|
||||
<li>Then I re-generated the <code>requirements.txt</code> in the <a href="https://github.com/alanorth/dspace-statistics-api">dspace-statistics-library</a> and released version 0.5.2</li>
|
||||
<li>Then I re-deployed the API on DSpace Test, ran all system updates on the server, and rebooted it</li>
|
||||
<li>I tested my hack of depositing to one collection where the default item and bistream READ policies are restricted and then mapping the item to another collection, but the item retains its default policies so Anonymous cannot see them in the mapped collection either</li>
|
||||
<li>Perhaps we need to try moving the item and inheriting the target collection's policies?</li>
|
||||
<li>Perhaps we need to try moving the item and inheriting the target collection’s policies?</li>
|
||||
<li>I merged the changes for adding publisher (<code>dc.publisher</code>) to the advanced search to the <code>5_x-prod</code> branch (<a href="https://github.com/ilri/DSpace/pull/402">#402</a>)</li>
|
||||
<li>I merged the changes for adding versionless Creative Commons licenses to the submission form to the <code>5_x-prod</code> branch (<a href="https://github.com/ilri/DSpace/pull/403">#403</a>)</li>
|
||||
<li>I will deploy them later this week</li>
|
||||
@ -617,7 +617,7 @@ $ curl -X GET -H "Content-Type: application/json" -H "Accept: app
|
||||
<li>Meet with the COPO guys to walk them through the CGSpace submission workflow and discuss CG core, REST API, etc
|
||||
<ul>
|
||||
<li>I suggested that they look into submitting via the <a href="https://wiki.duraspace.org/display/DSDOC5x/SWORDv2+Server">SWORDv2</a> protocol because it respects the workflows</li>
|
||||
<li>They said that they're not too worried about the hierarchical CG core schema, that they would just flatten metadata like affiliations when depositing to a DSpace repository</li>
|
||||
<li>They said that they’re not too worried about the hierarchical CG core schema, that they would just flatten metadata like affiliations when depositing to a DSpace repository</li>
|
||||
<li>I said that it might be time to engage the DSpace community to add support for more advanced schemas in DSpace 7+ (perhaps partnership with Atmire?)</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
Reference in New Issue
Block a user