mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2025-01-27 05:49:12 +01:00
Update notes for 2018-11-08
This commit is contained in:
@ -14,7 +14,7 @@
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
|
||||
<meta name="generator" content="Hugo 0.50" />
|
||||
<meta name="generator" content="Hugo 0.51" />
|
||||
|
||||
|
||||
|
||||
@ -111,8 +111,6 @@
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -136,8 +134,6 @@
|
||||
<li>Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items</li>
|
||||
<li>I created a GitHub issue to track this <a href="https://github.com/ilri/DSpace/issues/389">#389</a>, because I’m super busy in Nairobi right now</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-10/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -163,8 +159,6 @@
|
||||
<li>Also, I’ll re-run the <code>postgresql</code> tasks because the custom PostgreSQL variables are dynamic according to the system’s RAM, and we never re-ran them after migrating to larger Linodes last month</li>
|
||||
<li>I’m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I’m getting those autowire errors in Tomcat 8.5.30 again:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -201,8 +195,6 @@
|
||||
<li>The server only has 8GB of RAM so we’ll eventually need to upgrade to a larger one because we’ll start starving the OS, PostgreSQL, and command line batch processes</li>
|
||||
<li>I ran all system updates on DSpace Test and rebooted it</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-08/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -235,8 +227,6 @@
|
||||
|
||||
<pre><code>There is insufficient memory for the Java Runtime Environment to continue.
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-07/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -280,8 +270,6 @@ real 74m42.646s
|
||||
user 8m5.056s
|
||||
sys 2m7.289s
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -311,8 +299,6 @@ sys 2m7.289s
|
||||
<li>Then I reduced the JVM heap size from 6144 back to 5120m</li>
|
||||
<li>Also, I switched it to use OpenJDK instead of Oracle Java, as well as re-worked the <a href="https://github.com/ilri/rmg-ansible-public">Ansible infrastructure scripts</a> to support hosts choosing which distribution they want to use</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-05/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -336,8 +322,6 @@ sys 2m7.289s
|
||||
<li>I tried to test something on DSpace Test but noticed that it’s down since god knows when</li>
|
||||
<li>Catalina logs at least show some memory errors yesterday:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -360,8 +344,6 @@ sys 2m7.289s
|
||||
<ul>
|
||||
<li>Export a CSV of the IITA community metadata for Martin Mueller</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-03/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -387,8 +369,6 @@ sys 2m7.289s
|
||||
<li>Yesterday I figured out how to monitor DSpace sessions using JMX</li>
|
||||
<li>I copied the logic in the <code>jmx_tomcat_dbpools</code> provided by Ubuntu’s <code>munin-plugins-java</code> package and used the stuff I discovered about JMX <a href="/cgspace-notes/2018-01/">in 2018-01</a></li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-02/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
@ -29,9 +29,7 @@
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -45,9 +43,7 @@
|
||||
<ul>
|
||||
<li>Phil Thornton got an ORCID identifier so we need to add it to the list on CGSpace and tag his existing items</li>
|
||||
<li>I created a GitHub issue to track this <a href="https://github.com/ilri/DSpace/issues/389">#389</a>, because I&rsquo;m super busy in Nairobi right now</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -63,9 +59,7 @@
|
||||
<li>I&rsquo;ll update the DSpace role in our <a href="https://github.com/ilri/rmg-ansible-public">Ansible infrastructure playbooks</a> and run the updated playbooks on CGSpace and DSpace Test</li>
|
||||
<li>Also, I&rsquo;ll re-run the <code>postgresql</code> tasks because the custom PostgreSQL variables are dynamic according to the system&rsquo;s RAM, and we never re-ran them after migrating to larger Linodes last month</li>
|
||||
<li>I&rsquo;m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I&rsquo;m getting those autowire errors in Tomcat 8.5.30 again:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -92,9 +86,7 @@
|
||||
<li>Anyways, perhaps I should increase the JVM heap from 5120m to 6144m like we did a few months ago when we tried to run the whole CGSpace Solr core</li>
|
||||
<li>The server only has 8GB of RAM so we&rsquo;ll eventually need to upgrade to a larger one because we&rsquo;ll start starving the OS, PostgreSQL, and command line batch processes</li>
|
||||
<li>I ran all system updates on DSpace Test and rebooted it</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -117,9 +109,7 @@
|
||||
</ul>
|
||||
|
||||
<pre><code>There is insufficient memory for the Java Runtime Environment to continue.
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -153,9 +143,7 @@
|
||||
real 74m42.646s
|
||||
user 8m5.056s
|
||||
sys 2m7.289s
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -175,9 +163,7 @@ sys 2m7.289s
|
||||
</ul></li>
|
||||
<li>Then I reduced the JVM heap size from 6144 back to 5120m</li>
|
||||
<li>Also, I switched it to use OpenJDK instead of Oracle Java, as well as re-worked the <a href="https://github.com/ilri/rmg-ansible-public">Ansible infrastructure scripts</a> to support hosts choosing which distribution they want to use</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -191,9 +177,7 @@ sys 2m7.289s
|
||||
<ul>
|
||||
<li>I tried to test something on DSpace Test but noticed that it&rsquo;s down since god knows when</li>
|
||||
<li>Catalina logs at least show some memory errors yesterday:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -206,9 +190,7 @@ sys 2m7.289s
|
||||
|
||||
<ul>
|
||||
<li>Export a CSV of the IITA community metadata for Martin Mueller</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -224,9 +206,7 @@ sys 2m7.289s
|
||||
<li>We don&rsquo;t need to distinguish between internal and external works, so that makes it just a simple list</li>
|
||||
<li>Yesterday I figured out how to monitor DSpace sessions using JMX</li>
|
||||
<li>I copied the logic in the <code>jmx_tomcat_dbpools</code> provided by Ubuntu&rsquo;s <code>munin-plugins-java</code> package and used the stuff I discovered about JMX <a href="https://alanorth.github.io/cgspace-notes/cgspace-notes/2018-01/">in 2018-01</a></li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -311,9 +291,7 @@ dspace.log.2018-01-02:34
|
||||
|
||||
<ul>
|
||||
<li>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let&rsquo;s Encrypt if it&rsquo;s just a handful of domains</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -329,9 +307,7 @@ dspace.log.2018-01-02:34
|
||||
<li>The logs say &ldquo;Timeout waiting for idle object&rdquo;</li>
|
||||
<li>PostgreSQL activity says there are 115 connections currently</li>
|
||||
<li>The list of connections to XMLUI and REST API for today:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -362,9 +338,7 @@ dspace.log.2018-01-02:34
|
||||
|
||||
<pre><code>dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element = 'contributor' and qualifier = 'author') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/authors.csv with csv;
|
||||
COPY 54701
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -385,9 +359,7 @@ COPY 54701
|
||||
<ul>
|
||||
<li>There appears to be a pattern but I&rsquo;ll have to look a bit closer and try to clean them up automatically, either in SQL or in OpenRefine</li>
|
||||
<li>Add Katherine Lutz to the groups for content submission and edit steps of the CGIAR System collections</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -396,9 +368,7 @@ COPY 54701
|
||||
<pubDate>Mon, 18 Sep 2017 16:38:35 +0300</pubDate>
|
||||
|
||||
<guid>https://alanorth.github.io/cgspace-notes/cgiar-library-migration/</guid>
|
||||
<description><p>Rough notes for importing the CGIAR Library content. It was decided that this content would go to a new top-level community called <em>CGIAR System Organization</em>.</p>
|
||||
|
||||
<p></p></description>
|
||||
<description><p>Rough notes for importing the CGIAR Library content. It was decided that this content would go to a new top-level community called <em>CGIAR System Organization</em>.</p></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -417,9 +387,7 @@ COPY 54701
|
||||
|
||||
<ul>
|
||||
<li>Ask Sisay to clean up the WLE approvers a bit, as Marianne&rsquo;s user account is both in the approvers step as well as the group</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -450,9 +418,7 @@ COPY 54701
|
||||
<li>This was due to newline characters in the <code>dc.description.abstract</code> column, which caused OpenRefine to choke when exporting the CSV</li>
|
||||
<li>I exported a new CSV from the collection on DSpace Test and then manually removed the characters in vim using <code>g/^$/d</code></li>
|
||||
<li>Then I cleaned up the author authorities and HTML characters in OpenRefine and sent the file back to Abenet</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -473,9 +439,7 @@ COPY 54701
|
||||
<li>Merge changes for WLE Phase II theme rename (<a href="https://github.com/ilri/DSpace/pull/329">#329</a>)</li>
|
||||
<li>Looking at extracting the metadata registries from ICARDA&rsquo;s MEL DSpace database so we can compare fields with CGSpace</li>
|
||||
<li>We can use PostgreSQL&rsquo;s extended output format (<code>-x</code>) plus <code>sed</code> to format the output into quasi XML:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -517,9 +481,7 @@ COPY 54701
|
||||
</ul>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p &quot;ImageMagick PDF Thumbnail&quot; -v &gt;&amp; /tmp/filter-media-cmyk.txt
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -550,9 +512,7 @@ COPY 54701
|
||||
|
||||
<pre><code>$ identify ~/Desktop/alc_contrastes_desafios.jpg
|
||||
/Users/aorth/Desktop/alc_contrastes_desafios.jpg JPEG 464x600 464x600+0+0 8-bit CMYK 168KB 0.000u 0:00.000
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -581,9 +541,7 @@ DELETE 1
|
||||
<ul>
|
||||
<li>Create issue on GitHub to track the addition of CCAFS Phase II project tags (<a href="https://github.com/ilri/DSpace/issues/301">#301</a>)</li>
|
||||
<li>Looks like we&rsquo;ll be using <code>cg.identifier.ccafsprojectpii</code> as the field name</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -598,9 +556,7 @@ DELETE 1
|
||||
<li>I checked to see if the Solr sharding task that is supposed to run on January 1st had run and saw there was an error</li>
|
||||
<li>I tested on DSpace Test as well and it doesn&rsquo;t work there either</li>
|
||||
<li>I asked on the dspace-tech mailing list because it seems to be broken, and actually now I&rsquo;m not sure if we&rsquo;ve ever had the sharding task run successfully over all these years</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -627,9 +583,7 @@ DELETE 1
|
||||
<li>I see thousands of them in the logs for the last few months, so it&rsquo;s not related to the DSpace 5.5 upgrade</li>
|
||||
<li>I&rsquo;ve raised a ticket with Atmire to ask</li>
|
||||
<li>Another worrying error from dspace.log is:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -644,9 +598,7 @@ DELETE 1
|
||||
<li>Add <code>dc.type</code> to the output options for Atmire&rsquo;s Listings and Reports module (<a href="https://github.com/ilri/DSpace/pull/286">#286</a>)</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="https://alanorth.github.io/cgspace-notes/cgspace-notes/2016/11/listings-and-reports.png" alt="Listings and Reports with output type" /></p>
|
||||
|
||||
<p></p></description>
|
||||
<p><img src="https://alanorth.github.io/cgspace-notes/cgspace-notes/2016/11/listings-and-reports.png" alt="Listings and Reports with output type" /></p></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -669,9 +621,7 @@ DELETE 1
|
||||
</ul>
|
||||
|
||||
<pre><code>0000-0002-6115-0956||0000-0002-3812-8793||0000-0001-7462-405X
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -690,9 +640,7 @@ DELETE 1
|
||||
</ul>
|
||||
|
||||
<pre><code>$ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b &quot;dc=cgiarad,dc=org&quot; -D &quot;admigration1@cgiarad.org&quot; -W &quot;(sAMAccountName=admigration1)&quot;
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -715,9 +663,7 @@ DELETE 1
|
||||
<pre><code>$ git checkout -b 55new 5_x-prod
|
||||
$ git reset --hard ilri/5_x-prod
|
||||
$ git rebase -i dspace-5.5
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -743,9 +689,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
|
||||
<ul>
|
||||
<li>In this case the select query was showing 95 results before the update</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -763,9 +707,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -784,9 +726,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
|
||||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||||
3168
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -803,9 +743,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<li>After running DSpace for over five years I&rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year!</li>
|
||||
<li>This will save us a few gigs of backup space we&rsquo;re paying for on S3</li>
|
||||
<li>Also, I noticed the <code>checker</code> log has some errors we should pay attention to:</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -820,9 +758,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<li>Looking at issues with author authorities on CGSpace</li>
|
||||
<li>For some reason we still have the <code>index-lucene-update</code> cron job active on CGSpace, but I&rsquo;m pretty sure we don&rsquo;t need it as of the latest few versions of Atmire&rsquo;s Listings and Reports module</li>
|
||||
<li>Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Java JDK 1.7 to match environment on CGSpace server</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -844,9 +780,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<ul>
|
||||
<li>Not only are there 49,000 countries, we have some blanks (25)&hellip;</li>
|
||||
<li>Also, lots of things like &ldquo;COTE D`LVOIRE&rdquo; and &ldquo;COTE D IVOIRE&rdquo;</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -861,9 +795,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<li>Move ILRI collection <code>10568/12503</code> from <code>10568/27869</code> to <code>10568/27629</code> using the <a href="https://gist.github.com/alanorth/392c4660e8b022d99dfa">move_collections.sh</a> script I wrote last year.</li>
|
||||
<li>I realized it is only necessary to clear the Cocoon cache after moving collections—rather than reindexing—as no metadata has changed, and therefore no search or browse indexes need to be updated.</li>
|
||||
<li>Update GitHub wiki for documentation of <a href="https://github.com/ilri/DSpace/wiki/Maintenance-Tasks">maintenance tasks</a>.</li>
|
||||
</ul>
|
||||
|
||||
<p></p></description>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -883,9 +815,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
-rw-rw-r-- 1 tomcat7 tomcat7 2.0M Nov 18 23:59 dspace.log.2015-11-18
|
||||
-rw-rw-r-- 1 tomcat7 tomcat7 387K Nov 18 23:59 dspace.log.2015-11-18.lzo
|
||||
-rw-rw-r-- 1 tomcat7 tomcat7 169K Nov 18 23:59 dspace.log.2015-11-18.xz
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
@ -904,9 +834,7 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
|
||||
<pre><code>$ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspace
|
||||
78
|
||||
</code></pre>
|
||||
|
||||
<p></p></description>
|
||||
</code></pre></description>
|
||||
</item>
|
||||
|
||||
</channel>
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
|
||||
<meta name="generator" content="Hugo 0.50" />
|
||||
<meta name="generator" content="Hugo 0.51" />
|
||||
|
||||
|
||||
|
||||
@ -175,8 +175,6 @@ dspace.log.2018-01-02:34
|
||||
<ul>
|
||||
<li>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -202,8 +200,6 @@ dspace.log.2018-01-02:34
|
||||
<li>PostgreSQL activity says there are 115 connections currently</li>
|
||||
<li>The list of connections to XMLUI and REST API for today:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-12/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -244,8 +240,6 @@ dspace.log.2018-01-02:34
|
||||
<pre><code>dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element = 'contributor' and qualifier = 'author') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/authors.csv with csv;
|
||||
COPY 54701
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -276,8 +270,6 @@ COPY 54701
|
||||
<li>There appears to be a pattern but I’ll have to look a bit closer and try to clean them up automatically, either in SQL or in OpenRefine</li>
|
||||
<li>Add Katherine Lutz to the groups for content submission and edit steps of the CGIAR System collections</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-10/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -296,8 +288,6 @@ COPY 54701
|
||||
</p>
|
||||
</header>
|
||||
<p>Rough notes for importing the CGIAR Library content. It was decided that this content would go to a new top-level community called <em>CGIAR System Organization</em>.</p>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/cgiar-library-migration/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -326,8 +316,6 @@ COPY 54701
|
||||
<ul>
|
||||
<li>Ask Sisay to clean up the WLE approvers a bit, as Marianne’s user account is both in the approvers step as well as the group</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -368,8 +356,6 @@ COPY 54701
|
||||
<li>I exported a new CSV from the collection on DSpace Test and then manually removed the characters in vim using <code>g/^$/d</code></li>
|
||||
<li>Then I cleaned up the author authorities and HTML characters in OpenRefine and sent the file back to Abenet</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-08/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -400,8 +386,6 @@ COPY 54701
|
||||
<li>Looking at extracting the metadata registries from ICARDA’s MEL DSpace database so we can compare fields with CGSpace</li>
|
||||
<li>We can use PostgreSQL’s extended output format (<code>-x</code>) plus <code>sed</code> to format the output into quasi XML:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-07/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
|
||||
<meta name="generator" content="Hugo 0.50" />
|
||||
<meta name="generator" content="Hugo 0.51" />
|
||||
|
||||
|
||||
|
||||
@ -114,8 +114,6 @@
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -156,8 +154,6 @@
|
||||
<pre><code>$ identify ~/Desktop/alc_contrastes_desafios.jpg
|
||||
/Users/aorth/Desktop/alc_contrastes_desafios.jpg JPEG 464x600 464x600+0+0 8-bit CMYK 168KB 0.000u 0:00.000
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-03/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -196,8 +192,6 @@ DELETE 1
|
||||
<li>Create issue on GitHub to track the addition of CCAFS Phase II project tags (<a href="https://github.com/ilri/DSpace/issues/301">#301</a>)</li>
|
||||
<li>Looks like we’ll be using <code>cg.identifier.ccafsprojectpii</code> as the field name</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-02/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -222,8 +216,6 @@ DELETE 1
|
||||
<li>I tested on DSpace Test as well and it doesn’t work there either</li>
|
||||
<li>I asked on the dspace-tech mailing list because it seems to be broken, and actually now I’m not sure if we’ve ever had the sharding task run successfully over all these years</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -260,8 +252,6 @@ DELETE 1
|
||||
<li>I’ve raised a ticket with Atmire to ask</li>
|
||||
<li>Another worrying error from dspace.log is:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-12/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -286,8 +276,6 @@ DELETE 1
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2016/11/listings-and-reports.png" alt="Listings and Reports with output type" /></p>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -320,8 +308,6 @@ DELETE 1
|
||||
|
||||
<pre><code>0000-0002-6115-0956||0000-0002-3812-8793||0000-0001-7462-405X
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-10/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -350,8 +336,6 @@ DELETE 1
|
||||
|
||||
<pre><code>$ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=org" -D "admigration1@cgiarad.org" -W "(sAMAccountName=admigration1)"
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -384,8 +368,6 @@ DELETE 1
|
||||
$ git reset --hard ilri/5_x-prod
|
||||
$ git rebase -i dspace-5.5
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-08/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -421,8 +403,6 @@ dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and
|
||||
<ul>
|
||||
<li>In this case the select query was showing 95 results before the update</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-07/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
|
||||
<meta name="generator" content="Hugo 0.50" />
|
||||
<meta name="generator" content="Hugo 0.51" />
|
||||
|
||||
|
||||
|
||||
@ -108,8 +108,6 @@
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -138,8 +136,6 @@
|
||||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||||
3168
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-05/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -166,8 +162,6 @@
|
||||
<li>This will save us a few gigs of backup space we’re paying for on S3</li>
|
||||
<li>Also, I noticed the <code>checker</code> log has some errors we should pay attention to:</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -192,8 +186,6 @@
|
||||
<li>For some reason we still have the <code>index-lucene-update</code> cron job active on CGSpace, but I’m pretty sure we don’t need it as of the latest few versions of Atmire’s Listings and Reports module</li>
|
||||
<li>Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Java JDK 1.7 to match environment on CGSpace server</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-03/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -225,8 +217,6 @@
|
||||
<li>Not only are there 49,000 countries, we have some blanks (25)…</li>
|
||||
<li>Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE”</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-02/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -251,8 +241,6 @@
|
||||
<li>I realized it is only necessary to clear the Cocoon cache after moving collections—rather than reindexing—as no metadata has changed, and therefore no search or browse indexes need to be updated.</li>
|
||||
<li>Update GitHub wiki for documentation of <a href="https://github.com/ilri/DSpace/wiki/Maintenance-Tasks">maintenance tasks</a>.</li>
|
||||
</ul>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -282,8 +270,6 @@
|
||||
-rw-rw-r-- 1 tomcat7 tomcat7 387K Nov 18 23:59 dspace.log.2015-11-18.lzo
|
||||
-rw-rw-r-- 1 tomcat7 tomcat7 169K Nov 18 23:59 dspace.log.2015-11-18.xz
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2015-12/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
@ -312,8 +298,6 @@
|
||||
<pre><code>$ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspace
|
||||
78
|
||||
</code></pre>
|
||||
|
||||
<p></p>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2015-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user