mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2025-01-27 05:49:12 +01:00
Add notes for 2020-01-27
This commit is contained in:
@ -17,8 +17,8 @@ While looking in the logs for errors, I see tons of warnings about Atmire MQM:
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=ADD, SubjectType=BUNDLE, SubjectID=70316, Obje ctType=BITSTREAM, ObjectID=86715, TimeStamp=1480647632318, dispatcher=1544803905, detail="-1", transactionID="TX157907838689377964651674089851855413607")
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=MODIFY, SubjectType=ITEM, SubjectID=80044, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632351, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607")
|
||||
|
||||
I see thousands of them in the logs for the last few months, so it's not related to the DSpace 5.5 upgrade
|
||||
I've raised a ticket with Atmire to ask
|
||||
I see thousands of them in the logs for the last few months, so it’s not related to the DSpace 5.5 upgrade
|
||||
I’ve raised a ticket with Atmire to ask
|
||||
Another worrying error from dspace.log is:
|
||||
" />
|
||||
<meta property="og:type" content="article" />
|
||||
@ -39,11 +39,11 @@ While looking in the logs for errors, I see tons of warnings about Atmire MQM:
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=ADD, SubjectType=BUNDLE, SubjectID=70316, Obje ctType=BITSTREAM, ObjectID=86715, TimeStamp=1480647632318, dispatcher=1544803905, detail="-1", transactionID="TX157907838689377964651674089851855413607")
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=MODIFY, SubjectType=ITEM, SubjectID=80044, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632351, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607")
|
||||
|
||||
I see thousands of them in the logs for the last few months, so it's not related to the DSpace 5.5 upgrade
|
||||
I've raised a ticket with Atmire to ask
|
||||
I see thousands of them in the logs for the last few months, so it’s not related to the DSpace 5.5 upgrade
|
||||
I’ve raised a ticket with Atmire to ask
|
||||
Another worrying error from dspace.log is:
|
||||
"/>
|
||||
<meta name="generator" content="Hugo 0.62.2" />
|
||||
<meta name="generator" content="Hugo 0.63.1" />
|
||||
|
||||
|
||||
|
||||
@ -73,7 +73,7 @@ Another worrying error from dspace.log is:
|
||||
|
||||
<!-- combined, minified CSS -->
|
||||
|
||||
<link href="https://alanorth.github.io/cgspace-notes/css/style.a20c1a4367639632cdb341d23c27ca44fedcc75b0f8b3cbea6203010da153d3c.css" rel="stylesheet" integrity="sha256-ogwaQ2djljLNs0HSPCfKRP7cx1sPizy+piAwENoVPTw=" crossorigin="anonymous">
|
||||
<link href="https://alanorth.github.io/cgspace-notes/css/style.23e2c3298bcc8c1136c19aba330c211ec94c36f7c4454ea15cf4d3548370042a.css" rel="stylesheet" integrity="sha256-I+LDKYvMjBE2wZq6MwwhHslMNvfERU6hXPTTVINwBCo=" crossorigin="anonymous">
|
||||
|
||||
|
||||
<!-- RSS 2.0 feed -->
|
||||
@ -121,7 +121,7 @@ Another worrying error from dspace.log is:
|
||||
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2016-12/">December, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-12-02T10:43:00+03:00">Fri Dec 02, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
<span class="fas fa-tag" aria-hidden="true"></span> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
@ -136,8 +136,8 @@ Another worrying error from dspace.log is:
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=ADD, SubjectType=BUNDLE, SubjectID=70316, Obje ctType=BITSTREAM, ObjectID=86715, TimeStamp=1480647632318, dispatcher=1544803905, detail="-1", transactionID="TX157907838689377964651674089851855413607")
|
||||
2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=MODIFY, SubjectType=ITEM, SubjectID=80044, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632351, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607")
|
||||
</code></pre><ul>
|
||||
<li>I see thousands of them in the logs for the last few months, so it's not related to the DSpace 5.5 upgrade</li>
|
||||
<li>I've raised a ticket with Atmire to ask</li>
|
||||
<li>I see thousands of them in the logs for the last few months, so it’s not related to the DSpace 5.5 upgrade</li>
|
||||
<li>I’ve raised a ticket with Atmire to ask</li>
|
||||
<li>Another worrying error from dspace.log is:</li>
|
||||
</ul>
|
||||
<pre><code>org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.NoSuchMethodError: com.atmire.statistics.generator.DSpaceObjectDatasetGenerator.toDatasetQuery(Lorg/dspace/core/Context;)Lcom/atmire/statistics/content/DatasetQuery;
|
||||
@ -232,16 +232,16 @@ Caused by: java.lang.NoSuchMethodError: com.atmire.statistics.generator.DSpaceOb
|
||||
<pre><code>2016-12-02 03:00:46,656 ERROR org.dspace.authority.AuthorityValueFinder @ anonymous::Error while retrieving AuthorityValue from solr:query\colon; id\colon;"b0b541c1-ec15-48bf-9209-6dbe8e338cdc"
|
||||
org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://localhost:8081/solr/authority
|
||||
</code></pre><ul>
|
||||
<li>Looking through DSpace's solr log I see that about 20 seconds before this, there were a few 30+ KiB solr queries</li>
|
||||
<li>Looking through DSpace’s solr log I see that about 20 seconds before this, there were a few 30+ KiB solr queries</li>
|
||||
<li>The last logs here right before Solr became unresponsive (and right after I restarted it five hours later) were:</li>
|
||||
</ul>
|
||||
<pre><code>2016-12-02 03:00:42,606 INFO org.apache.solr.core.SolrCore @ [statistics] webapp=/solr path=/select params={q=containerItem:72828+AND+type:0&shards=localhost:8081/solr/statistics-2010,localhost:8081/solr/statistics&fq=-isInternal:true&fq=-(author_mtdt:"CGIAR\+Institutional\+Learning\+and\+Change\+Initiative"++AND+subject_mtdt:"PARTNERSHIPS"+AND+subject_mtdt:"RESEARCH"+AND+subject_mtdt:"AGRICULTURE"+AND+subject_mtdt:"DEVELOPMENT"++AND+iso_mtdt:"en"+)&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=19
|
||||
2016-12-02 08:28:23,908 INFO org.apache.solr.servlet.SolrDispatchFilter @ SolrDispatchFilter.init()
|
||||
</code></pre><ul>
|
||||
<li>DSpace's own Solr logs don't give IP addresses, so I will have to enable Nginx's logging of <code>/solr</code> so I can see where this request came from</li>
|
||||
<li>I enabled logging of <code>/rest/</code> and I think I'll leave it on for good</li>
|
||||
<li>Also, the disk is nearly full because of log file issues, so I'm running some compression on DSpace logs</li>
|
||||
<li>Normally these stay uncompressed for a month just in case we need to look at them, so now I've just compressed anything older than 2 weeks so we can get some disk space back</li>
|
||||
<li>DSpace’s own Solr logs don’t give IP addresses, so I will have to enable Nginx’s logging of <code>/solr</code> so I can see where this request came from</li>
|
||||
<li>I enabled logging of <code>/rest/</code> and I think I’ll leave it on for good</li>
|
||||
<li>Also, the disk is nearly full because of log file issues, so I’m running some compression on DSpace logs</li>
|
||||
<li>Normally these stay uncompressed for a month just in case we need to look at them, so now I’ve just compressed anything older than 2 weeks so we can get some disk space back</li>
|
||||
</ul>
|
||||
<h2 id="2016-12-04">2016-12-04</h2>
|
||||
<ul>
|
||||
@ -266,10 +266,10 @@ Checksum Calculated =
|
||||
Result = The bitstream could not be found
|
||||
-----------------------------------------------
|
||||
</code></pre><ul>
|
||||
<li>The first one seems ok, but I don't know what to make of the second one…</li>
|
||||
<li>The first one seems ok, but I don’t know what to make of the second one…</li>
|
||||
<li>I had a look and there is indeed no file with the second checksum in the assetstore (ie, looking in <code>[dspace-dir]/assetstore/99/59/30/...</code>)</li>
|
||||
<li>For what it's worth, there is no item on DSpace Test or S3 backups with that checksum either…</li>
|
||||
<li>In other news, I'm looking at JVM settings from the Solr 4.10.2 release, from <code>bin/solr.in.sh</code>:</li>
|
||||
<li>For what it’s worth, there is no item on DSpace Test or S3 backups with that checksum either…</li>
|
||||
<li>In other news, I’m looking at JVM settings from the Solr 4.10.2 release, from <code>bin/solr.in.sh</code>:</li>
|
||||
</ul>
|
||||
<pre><code># These GC settings have shown to work well for a number of common Solr workloads
|
||||
GC_TUNE="-XX:-UseSuperWord \
|
||||
@ -292,21 +292,21 @@ GC_TUNE="-XX:-UseSuperWord \
|
||||
-XX:+AggressiveOpts"
|
||||
</code></pre><ul>
|
||||
<li>I need to try these because they are recommended by the Solr project itself</li>
|
||||
<li>Also, as always, I need to read <a href="https://wiki.apache.org/solr/ShawnHeisey">Shawn Heisey's wiki page on Solr</a></li>
|
||||
<li>Also, as always, I need to read <a href="https://wiki.apache.org/solr/ShawnHeisey">Shawn Heisey’s wiki page on Solr</a></li>
|
||||
</ul>
|
||||
<h2 id="2016-12-05">2016-12-05</h2>
|
||||
<ul>
|
||||
<li>I did some basic benchmarking on a local DSpace before and after the JVM settings above, but there wasn't anything amazingly obvious</li>
|
||||
<li>I did some basic benchmarking on a local DSpace before and after the JVM settings above, but there wasn’t anything amazingly obvious</li>
|
||||
<li>I want to make the changes on DSpace Test and monitor the JVM heap graphs for a few days to see if they change the JVM GC patterns or anything (munin graphs)</li>
|
||||
<li>Spin up new CGSpace server on Linode</li>
|
||||
<li>I did a few traceroutes from Jordan and Kenya and it seems that Linode's Frankfurt datacenter is a few less hops and perhaps less packet loss than the London one, so I put the new server in Frankfurt</li>
|
||||
<li>I did a few traceroutes from Jordan and Kenya and it seems that Linode’s Frankfurt datacenter is a few less hops and perhaps less packet loss than the London one, so I put the new server in Frankfurt</li>
|
||||
<li>Do initial provisioning</li>
|
||||
<li>Atmire responded about the MQM warnings in the DSpace logs</li>
|
||||
<li>Apparently we need to change the batch edit consumers in <code>dspace/config/dspace.cfg</code>:</li>
|
||||
</ul>
|
||||
<pre><code>event.consumer.batchedit.filters = Community|Collection+Create
|
||||
</code></pre><ul>
|
||||
<li>I haven't tested it yet, but I created a pull request: <a href="https://github.com/ilri/DSpace/pull/289">#289</a></li>
|
||||
<li>I haven’t tested it yet, but I created a pull request: <a href="https://github.com/ilri/DSpace/pull/289">#289</a></li>
|
||||
</ul>
|
||||
<h2 id="2016-12-06">2016-12-06</h2>
|
||||
<ul>
|
||||
@ -333,7 +333,7 @@ UPDATE 561
|
||||
<li>Paola from CCAFS mentioned she also has the “take task” bug on CGSpace</li>
|
||||
<li>Reading about <a href="https://www.postgresql.org/docs/9.5/static/runtime-config-resource.html"><code>shared_buffers</code> in PostgreSQL configuration</a> (default is 128MB)</li>
|
||||
<li>Looks like we have ~5GB of memory used by caches on the test server (after OS and JVM heap!), so we might as well bump up the buffers for Postgres</li>
|
||||
<li>The docs say a good starting point for a dedicated server is 25% of the system RAM, and our server isn't dedicated (also runs Solr, which can benefit from OS cache) so let's try 1024MB</li>
|
||||
<li>The docs say a good starting point for a dedicated server is 25% of the system RAM, and our server isn’t dedicated (also runs Solr, which can benefit from OS cache) so let’s try 1024MB</li>
|
||||
<li>In other news, the authority reindexing keeps crashing (I was manually running it after the author updates above):</li>
|
||||
</ul>
|
||||
<pre><code>$ time JAVA_OPTS="-Xms768m -Xmx768m -Dfile.encoding=UTF-8" /home/dspacetest.cgiar.org/bin/dspace index-authority
|
||||
@ -363,9 +363,9 @@ user 1m54.190s
|
||||
sys 0m22.647s
|
||||
</code></pre><h2 id="2016-12-07">2016-12-07</h2>
|
||||
<ul>
|
||||
<li>For what it's worth, after running the same SQL updates on my local test server, <code>index-authority</code> runs and completes just fine</li>
|
||||
<li>For what it’s worth, after running the same SQL updates on my local test server, <code>index-authority</code> runs and completes just fine</li>
|
||||
<li>I will have to test more</li>
|
||||
<li>Anyways, I noticed that some of the authority values I set actually have versions of author names we don't want, ie “Grace, D.”</li>
|
||||
<li>Anyways, I noticed that some of the authority values I set actually have versions of author names we don’t want, ie “Grace, D.”</li>
|
||||
<li>For example, do a Solr query for “first_name:Grace” and look at the results</li>
|
||||
<li>Querying that ID shows the fields that need to be changed:</li>
|
||||
</ul>
|
||||
@ -400,7 +400,7 @@ sys 0m22.647s
|
||||
}
|
||||
</code></pre><ul>
|
||||
<li>I think I can just update the <code>value</code>, <code>first_name</code>, and <code>last_name</code> fields…</li>
|
||||
<li>The update syntax should be something like this, but I'm getting errors from Solr:</li>
|
||||
<li>The update syntax should be something like this, but I’m getting errors from Solr:</li>
|
||||
</ul>
|
||||
<pre><code>$ curl 'localhost:8081/solr/authority/update?commit=true&wt=json&indent=true' -H 'Content-type:application/json' -d '[{"id":"1","price":{"set":100}}]'
|
||||
{
|
||||
@ -417,7 +417,7 @@ sys 0m22.647s
|
||||
<pre><code>dspace=# update metadatavalue set authority=null, confidence=-1 where resource_type_id=2 and metadata_field_id=3 and text_value like 'Grace, D%';
|
||||
UPDATE 561
|
||||
</code></pre><ul>
|
||||
<li>Then I'll reindex discovery and authority and see how the authority Solr core looks</li>
|
||||
<li>Then I’ll reindex discovery and authority and see how the authority Solr core looks</li>
|
||||
<li>After this, now there are authorities for some of the “Grace, D.” and “Grace, Delia” text_values in the database (the first version is actually the same authority that already exists in the core, so it was just added back to some text_values, but the second one is new):</li>
|
||||
</ul>
|
||||
<pre><code>$ curl 'localhost:8081/solr/authority/select?q=id%3A18ea1525-2513-430a-8817-a834cd733fbc&wt=json&indent=true'
|
||||
@ -462,7 +462,7 @@ update metadatavalue set authority='0d8369bb-57f7-4b2f-92aa-af820b183aca', confi
|
||||
update metadatavalue set text_value='Grace, Delia', authority='bfa61d7c-7583-4175-991c-2e7315000f0c', confidence=600 where resource_type_id=2 and metadata_field_id=3 and text_value like 'Grace, D%';
|
||||
</code></pre><h2 id="2016-12-08">2016-12-08</h2>
|
||||
<ul>
|
||||
<li>Something weird happened and Peter Thorne's names all ended up as “Thorne”, I guess because the original authority had that as its name value:</li>
|
||||
<li>Something weird happened and Peter Thorne’s names all ended up as “Thorne”, I guess because the original authority had that as its name value:</li>
|
||||
</ul>
|
||||
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like 'Thorne%';
|
||||
text_value | authority | confidence
|
||||
@ -480,7 +480,7 @@ update metadatavalue set text_value='Grace, Delia', authority='bfa61d7c-7583-417
|
||||
<pre><code>dspace=# update metadatavalue set authority='b2f7603d-2fb5-4018-923a-c4ec8d85b3bb', text_value='Thorne, P.J.' where resource_type_id=2 and metadata_field_id=3 and authority='18349f29-61b1-44d7-ac60-89e55546e812';
|
||||
UPDATE 43
|
||||
</code></pre><ul>
|
||||
<li>Apparently we also need to normalize Phil Thornton's names to <code>Thornton, Philip K.</code>:</li>
|
||||
<li>Apparently we also need to normalize Phil Thornton’s names to <code>Thornton, Philip K.</code>:</li>
|
||||
</ul>
|
||||
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value ~ '^Thornton[,\.]? P.*';
|
||||
text_value | authority | confidence
|
||||
@ -504,13 +504,13 @@ UPDATE 362
|
||||
</code></pre><ul>
|
||||
<li>It seems that, when you are messing with authority and author text values in the database, it is better to run authority reindex first (postgres→solr authority core) and then Discovery reindex (postgres→solr Discovery core)</li>
|
||||
<li>Everything looks ok after authority and discovery reindex</li>
|
||||
<li>In other news, I think we should really be using more RAM for PostgreSQL's <code>shared_buffers</code></li>
|
||||
<li>The <a href="https://www.postgresql.org/docs/9.5/static/runtime-config-resource.html">PostgreSQL documentation</a> recommends using 25% of the system's RAM on dedicated systems, but we should use a bit less since we also have a massive JVM heap and also benefit from some RAM being used by the OS cache</li>
|
||||
<li>In other news, I think we should really be using more RAM for PostgreSQL’s <code>shared_buffers</code></li>
|
||||
<li>The <a href="https://www.postgresql.org/docs/9.5/static/runtime-config-resource.html">PostgreSQL documentation</a> recommends using 25% of the system’s RAM on dedicated systems, but we should use a bit less since we also have a massive JVM heap and also benefit from some RAM being used by the OS cache</li>
|
||||
</ul>
|
||||
<h2 id="2016-12-09">2016-12-09</h2>
|
||||
<ul>
|
||||
<li>More work on finishing rough draft of KM4Dev article</li>
|
||||
<li>Set PostgreSQL's <code>shared_buffers</code> on CGSpace to 10% of system RAM (1200MB)</li>
|
||||
<li>Set PostgreSQL’s <code>shared_buffers</code> on CGSpace to 10% of system RAM (1200MB)</li>
|
||||
<li>Run the following author corrections on CGSpace:</li>
|
||||
</ul>
|
||||
<pre><code>dspace=# update metadatavalue set authority='34df639a-42d8-4867-a3f2-1892075fcb3f', text_value='Thorne, P.J.' where resource_type_id=2 and metadata_field_id=3 and authority='18349f29-61b1-44d7-ac60-89e55546e812' or authority='021cd183-946b-42bb-964e-522ebff02993';
|
||||
@ -520,7 +520,7 @@ dspace=# update metadatavalue set authority='2df8136e-d8f4-4142-b58c-562337cab76
|
||||
</ul>
|
||||
<h2 id="2016-12-11">2016-12-11</h2>
|
||||
<ul>
|
||||
<li>After enabling a sizable <code>shared_buffers</code> for CGSpace's PostgreSQL configuration the number of connections to the database dropped significantly</li>
|
||||
<li>After enabling a sizable <code>shared_buffers</code> for CGSpace’s PostgreSQL configuration the number of connections to the database dropped significantly</li>
|
||||
</ul>
|
||||
<p><img src="/cgspace-notes/2016/12/postgres_bgwriter-week.png" alt="postgres_bgwriter-week">
|
||||
<img src="/cgspace-notes/2016/12/postgres_connections_ALL-week.png" alt="postgres_connections_ALL-week"></p>
|
||||
@ -563,12 +563,12 @@ UPDATE 35
|
||||
<ul>
|
||||
<li>Looking at logs, it seems we need to evaluate which logs we keep and for how long</li>
|
||||
<li>Basically the only ones we <em>need</em> are <code>dspace.log</code> because those are used for legacy statistics (need to keep for 1 month)</li>
|
||||
<li>Other logs will be an issue because they don't have date stamps</li>
|
||||
<li>I will add date stamps to the logs we're storing from the tomcat7 user's cron jobs at least, using: <code>$(date --iso-8601)</code></li>
|
||||
<li>Other logs will be an issue because they don’t have date stamps</li>
|
||||
<li>I will add date stamps to the logs we’re storing from the tomcat7 user’s cron jobs at least, using: <code>$(date --iso-8601)</code></li>
|
||||
<li>Would probably be better to make custom logrotate files for them in the future</li>
|
||||
<li>Clean up some unneeded log files from 2014 (they weren't large, just don't need them)</li>
|
||||
<li>Clean up some unneeded log files from 2014 (they weren’t large, just don’t need them)</li>
|
||||
<li>So basically, new cron jobs for logs should look something like this:</li>
|
||||
<li>Find any file named <code>*.log*</code> that isn't <code>dspace.log*</code>, isn't already zipped, and is older than one day, and zip it:</li>
|
||||
<li>Find any file named <code>*.log*</code> that isn’t <code>dspace.log*</code>, isn’t already zipped, and is older than one day, and zip it:</li>
|
||||
</ul>
|
||||
<pre><code># find /home/dspacetest.cgiar.org/log -regextype posix-extended -iregex ".*\.log.*" ! -iregex ".*dspace\.log.*" ! -iregex ".*\.(gz|lrz|lzo|xz)" ! -newermt "Yesterday" -exec schedtool -B -e ionice -c2 -n7 xz {} \;
|
||||
</code></pre><ul>
|
||||
@ -582,7 +582,7 @@ PID 17049: PRIO 0, POLICY B: SCHED_BATCH , NICE 0, AFFINITY 0xf
|
||||
best-effort: prio 7
|
||||
</code></pre><ul>
|
||||
<li>All in all this should free up a few gigs (we were at 9.3GB free when I started)</li>
|
||||
<li>Next thing to look at is whether we need Tomcat's access logs</li>
|
||||
<li>Next thing to look at is whether we need Tomcat’s access logs</li>
|
||||
<li>I just looked and it seems that we saved 10GB by zipping these logs</li>
|
||||
<li>Some users pointed out issues with the “most popular” stats on a community or collection</li>
|
||||
<li>This error appears in the logs when you try to view them:</li>
|
||||
@ -645,20 +645,20 @@ Caused by: java.lang.NoSuchMethodError: com.atmire.statistics.generator.DSpaceOb
|
||||
<ul>
|
||||
<li>Atmire sent a quick fix for the <code>last-update.txt</code> file not found error</li>
|
||||
<li>After applying pull request <a href="https://github.com/ilri/DSpace/pull/291">#291</a> on DSpace Test I no longer see the error in the logs after the <code>UpdateSolrStorageReports</code> task runs</li>
|
||||
<li>Also, I'm toying with the idea of moving the <code>tomcat7</code> user's cron jobs to <code>/etc/cron.d</code> so we can manage them in Ansible</li>
|
||||
<li>Also, I’m toying with the idea of moving the <code>tomcat7</code> user’s cron jobs to <code>/etc/cron.d</code> so we can manage them in Ansible</li>
|
||||
<li>Made a pull request with a template for the cron jobs (<a href="https://github.com/ilri/rmg-ansible-public/pull/75">#75</a>)</li>
|
||||
<li>Testing SMTP from the new CGSpace server and it's not working, I'll have to tell James</li>
|
||||
<li>Testing SMTP from the new CGSpace server and it’s not working, I’ll have to tell James</li>
|
||||
</ul>
|
||||
<h2 id="2016-12-15">2016-12-15</h2>
|
||||
<ul>
|
||||
<li>Start planning for server migration this weekend, letting users know</li>
|
||||
<li>I am trying to figure out what the process is to <a href="http://handle.net/hnr_support.html">update the server's IP in the Handle system</a>, and emailing the hdladmin account bounces(!)</li>
|
||||
<li>I will contact the Jane Euler directly as I know I've corresponded with her in the past</li>
|
||||
<li>I am trying to figure out what the process is to <a href="http://handle.net/hnr_support.html">update the server’s IP in the Handle system</a>, and emailing the hdladmin account bounces(!)</li>
|
||||
<li>I will contact the Jane Euler directly as I know I’ve corresponded with her in the past</li>
|
||||
<li>She said that I should indeed just re-run the <code>[dspace]/bin/dspace make-handle-config</code> command and submit the new <code>sitebndl.zip</code> file to the CNRI website</li>
|
||||
<li>Also I was troubleshooting some workflow issues from Bizuwork</li>
|
||||
<li>I re-created the same scenario by adding a non-admin account and submitting an item, but I was able to successfully approve and commit it</li>
|
||||
<li>So it turns out it's not a bug, it's just that Peter was added as a reviewer/admin AFTER the items were submitted</li>
|
||||
<li>This is how DSpace works, and I need to ask if there is a way to override someone's submission, as the other reviewer seems to not be paying attention, or has perhaps taken the item from the task pool?</li>
|
||||
<li>So it turns out it’s not a bug, it’s just that Peter was added as a reviewer/admin AFTER the items were submitted</li>
|
||||
<li>This is how DSpace works, and I need to ask if there is a way to override someone’s submission, as the other reviewer seems to not be paying attention, or has perhaps taken the item from the task pool?</li>
|
||||
<li>Run a batch edit to add “RANGELANDS” ILRI subject to all items containing the word “RANGELANDS” in their metadata for Peter Ballantyne</li>
|
||||
</ul>
|
||||
<p><img src="/cgspace-notes/2016/12/batch-edit1.png" alt="Select all items with “rangelands” in metadata">
|
||||
@ -666,9 +666,9 @@ Caused by: java.lang.NoSuchMethodError: com.atmire.statistics.generator.DSpaceOb
|
||||
<h2 id="2016-12-18">2016-12-18</h2>
|
||||
<ul>
|
||||
<li>Add four new CRP subjects for 2017 and sort the input forms alphabetically (<a href="https://github.com/ilri/DSpace/pull/294">#294</a>)</li>
|
||||
<li>Test the SMTP on the new server and it's working</li>
|
||||
<li>Test the SMTP on the new server and it’s working</li>
|
||||
<li>Last week, when we asked CGNET to update the DNS records this weekend, they misunderstood and did it immediately</li>
|
||||
<li>We quickly told them to undo it, but I just realized they didn't undo the IPv6 AAAA record!</li>
|
||||
<li>We quickly told them to undo it, but I just realized they didn’t undo the IPv6 AAAA record!</li>
|
||||
<li>None of our users in African institutes will have IPv6, but some Europeans might, so I need to check if any submissions have been added since then</li>
|
||||
<li>Update some names and authorities in the database:</li>
|
||||
</ul>
|
||||
@ -680,7 +680,7 @@ dspace=# update metadatavalue set authority='f840da02-26e7-4a74-b7ba-3e2b723f368
|
||||
UPDATE 140
|
||||
</code></pre><ul>
|
||||
<li>Generated a new UUID for Ben using <code>uuidgen | tr [A-Z] [a-z]</code> as the one in Solr had his ORCID but the name format was incorrect</li>
|
||||
<li>In theory DSpace should be able to check names from ORCID and update the records in the database, but I find that this doesn't work (see Jira bug <a href="https://jira.duraspace.org/browse/DS-3302">DS-3302</a>)</li>
|
||||
<li>In theory DSpace should be able to check names from ORCID and update the records in the database, but I find that this doesn’t work (see Jira bug <a href="https://jira.duraspace.org/browse/DS-3302">DS-3302</a>)</li>
|
||||
<li>I need to run these updates along with the other one for CIAT that I found last week</li>
|
||||
<li>Enable OCSP stapling for hosts >= Ubuntu 16.04 in our Ansible playbooks (<a href="https://github.com/ilri/rmg-ansible-public/pull/76">#76</a>)</li>
|
||||
<li>Working for DSpace Test on the second response:</li>
|
||||
@ -729,7 +729,7 @@ $ exit
|
||||
</code></pre><ul>
|
||||
<li>It took about twenty minutes and afterwards I had to check a few things, like:
|
||||
<ul>
|
||||
<li>check and enable systemd timer for let's encrypt</li>
|
||||
<li>check and enable systemd timer for let’s encrypt</li>
|
||||
<li>enable root cron jobs</li>
|
||||
<li>disable root cron jobs on old server after!</li>
|
||||
<li>enable tomcat7 cron jobs</li>
|
||||
@ -740,13 +740,13 @@ $ exit
|
||||
</ul>
|
||||
<h2 id="2016-12-22">2016-12-22</h2>
|
||||
<ul>
|
||||
<li>Abenet wanted a CSV of the IITA community, but the web export doesn't include the <code>dc.date.accessioned</code> field</li>
|
||||
<li>Abenet wanted a CSV of the IITA community, but the web export doesn’t include the <code>dc.date.accessioned</code> field</li>
|
||||
<li>I had to export it from the command line using the <code>-a</code> flag:</li>
|
||||
</ul>
|
||||
<pre><code>$ [dspace]/bin/dspace metadata-export -a -f /tmp/iita.csv -i 10568/68616
|
||||
</code></pre><h2 id="2016-12-28">2016-12-28</h2>
|
||||
<ul>
|
||||
<li>We've been getting two alerts per day about CPU usage on the new server from Linode</li>
|
||||
<li>We’ve been getting two alerts per day about CPU usage on the new server from Linode</li>
|
||||
<li>These are caused by the batch jobs for Solr etc that run in the early morning hours</li>
|
||||
<li>The Linode default is to alert at 90% CPU usage for two hours, but I see the old server was at 150%, so maybe we just need to adjust it</li>
|
||||
<li>Speaking of the old server (linode01), I think we can decommission it now</li>
|
||||
|
||||
Reference in New Issue
Block a user