mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-22 14:45:03 +01:00
402 lines
15 KiB
HTML
402 lines
15 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
||
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
|
||
|
||
<meta property="og:title" content="CGSpace Notes" />
|
||
<meta property="og:description" content="" />
|
||
<meta property="og:type" content="website" />
|
||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||
|
||
|
||
<meta property="og:updated_time" content="2017-01-02T10:43:00+03:00"/>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<meta itemprop="name" content="CGSpace Notes">
|
||
<meta itemprop="description" content="">
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<meta name="generator" content="Hugo 0.18.1" />
|
||
|
||
|
||
<link rel="canonical" href="https://alanorth.github.io/cgspace-notes/">
|
||
|
||
<title>CGSpace Notes</title>
|
||
|
||
<!-- combined, minified CSS -->
|
||
<link href="https://alanorth.github.io/cgspace-notes/css/style.css" rel="stylesheet">
|
||
|
||
|
||
<!-- RSS 2.0 feed -->
|
||
<link href="https://alanorth.github.io/cgspace-notes/index.xml" rel="alternate" type="application/rss+xml" title="CGSpace Notes" />
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</head>
|
||
|
||
<body>
|
||
|
||
<div class="blog-masthead">
|
||
<div class="container">
|
||
<nav class="nav blog-nav">
|
||
<a class="nav-link active" href="https://alanorth.github.io/cgspace-notes/">Home</a>
|
||
|
||
|
||
|
||
</nav>
|
||
</div>
|
||
</div>
|
||
|
||
<header class="blog-header">
|
||
<div class="container">
|
||
<h1 class="blog-title"><a href="https://alanorth.github.io/cgspace-notes/" rel="home">CGSpace Notes</a></h1>
|
||
|
||
</div>
|
||
</header>
|
||
|
||
<div class="container">
|
||
<div class="row">
|
||
<div class="col-sm-8 blog-main">
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-01/">January, 2017</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2017-01-02T10:43:00+03:00">Mon Jan 02, 2017</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2017-01-02 I checked to see if the Solr sharding task that is supposed to run on January 1st had run and saw there was an error I tested on DSpace Test as well and it doesn’t work there either I asked on the dspace-tech mailing list because it seems to be broken, and actually now I’m not sure if we’ve ever had the sharding task run successfully over all these years 2017-01-04 I tried to shard my local dev instance and it fails the same way: $ JAVA_OPTS="-Xms768m -Xmx768m -Dfile.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2017-01/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-12/">December, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-12-02T10:43:00+03:00">Fri Dec 02, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2016-12-02 CGSpace was down for five hours in the morning while I was sleeping While looking in the logs for errors, I see tons of warnings about Atmire MQM: 2016-12-02 03:00:32,352 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=CREATE, SubjectType=BUNDLE, SubjectID=70316, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632305, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607") 2016-12-02 03:00:32,353 WARN com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-12/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-11/">November, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-11-01T09:21:00+03:00">Tue Nov 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2016-11-01 Add dc.type to the output options for Atmire’s Listings and Reports module (#286) 2016-11-02 Migrate DSpace Test to DSpace 5.5 (notes) Run all updates on DSpace Test and reboot the server Looks like the OAI bug from DSpace 5.1 that caused validation at Base Search to fail is now fixed and DSpace Test passes validation! (#63) Indexing Discovery on DSpace Test took 332 minutes, which is like five times as long as it usually takes At the end it appeared to finish correctly but there were lots of errors right after it finished: 2016-11-02 15:09:48,578 INFO com.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-11/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-10/">October, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-10-03T15:53:00+03:00">Mon Oct 03, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2016-10-03 Testing adding ORCIDs to a CSV file for a single item to see if the author orders get messed up Need to test the following scenarios to see how author order is affected: ORCIDs only ORCIDs plus normal authors I exported a random item’s metadata as CSV, deleted all columns except id and collection, and made a new coloum called ORCID:dc.contributor.author with the following random ORCIDs from the ORCID registry: 0000-0002-6115-0956||0000-0002-3812-8793||0000-0001-7462-405X Hmm, with the dc.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-10/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-09/">September, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-09-01T15:53:00+03:00">Thu Sep 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-09-01">2016-09-01</h2>
|
||
|
||
<ul>
|
||
<li>Discuss helping CCAFS with some batch tagging of ORCID IDs for their authors</li>
|
||
<li>Discuss how the migration of CGIAR’s Active Directory to a flat structure will break our LDAP groups in DSpace</li>
|
||
<li>We had been using <code>DC=ILRI</code> to determine whether a user was ILRI or not</li>
|
||
<li>It looks like we might be able to use OUs now, instead of DCs:</li>
|
||
</ul>
|
||
|
||
<pre><code>$ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=org" -D "admigration1@cgiarad.org" -W "(sAMAccountName=admigration1)"
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-09/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-08/">August, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-08-01T15:53:00+03:00">Mon Aug 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-08-01">2016-08-01</h2>
|
||
|
||
<ul>
|
||
<li>Add updated distribution license from Sisay (<a href="https://github.com/ilri/DSpace/issues/259">#259</a>)</li>
|
||
<li>Play with upgrading Mirage 2 dependencies in <code>bower.json</code> because most are several versions of out date</li>
|
||
<li>Bootstrap is at 3.3.0 but upstream is at 3.3.7, and upgrading to anything beyond 3.3.1 breaks glyphicons and probably more</li>
|
||
<li>bower stuff is a dead end, waste of time, too many issues</li>
|
||
<li>Anything after Bootstrap 3.3.1 makes glyphicons disappear (HTTP 404 trying to access from incorrect path of <code>fonts</code>)</li>
|
||
<li>Start working on DSpace 5.1 → 5.5 port:</li>
|
||
</ul>
|
||
|
||
<pre><code>$ git checkout -b 55new 5_x-prod
|
||
$ git reset --hard ilri/5_x-prod
|
||
$ git rebase -i dspace-5.5
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-08/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-07/">July, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-07-01T10:53:00+03:00">Fri Jul 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-07-01">2016-07-01</h2>
|
||
|
||
<ul>
|
||
<li>Add <code>dc.description.sponsorship</code> to Discovery sidebar facets and make investors clickable in item view (<a href="https://github.com/ilri/DSpace/issues/232">#232</a>)</li>
|
||
<li>I think this query should find and replace all authors that have “,” at the end of their names:</li>
|
||
</ul>
|
||
|
||
<pre><code>dspacetest=# update metadatavalue set text_value = regexp_replace(text_value, '(^.+?),$', '\1') where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
|
||
UPDATE 95
|
||
dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
|
||
text_value
|
||
------------
|
||
(0 rows)
|
||
</code></pre>
|
||
|
||
<ul>
|
||
<li>In this case the select query was showing 95 results before the update</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-07/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-06-01">2016-06-01</h2>
|
||
|
||
<ul>
|
||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-05-01T23:06:00+03:00">Sun May 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-05-01">2016-05-01</h2>
|
||
|
||
<ul>
|
||
<li>Since yesterday there have been 10,000 REST errors and the site has been unstable again</li>
|
||
<li>I have blocked access to the API now</li>
|
||
<li>There are 3,000 IPs accessing the REST API in a 24-hour period!</li>
|
||
</ul>
|
||
|
||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||
3168
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-05/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-04/">April, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-04-04T11:06:00+03:00">Mon Apr 04, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-04-04">2016-04-04</h2>
|
||
|
||
<ul>
|
||
<li>Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit</li>
|
||
<li>We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc</li>
|
||
<li>After running DSpace for over five years I’ve never needed to look in any other log file than dspace.log, leave alone one from last year!</li>
|
||
<li>This will save us a few gigs of backup space we’re paying for on S3</li>
|
||
<li>Also, I noticed the <code>checker</code> log has some errors we should pay attention to:</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-04/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<nav class="blog-pagination">
|
||
|
||
|
||
|
||
<a class="btn btn-outline-primary disabled" href="#" role="button" aria-disabled="true">Previous page</a>
|
||
<a class="btn btn-outline-primary" href="/cgspace-notes/page/2/" role="button">Next page</a>
|
||
|
||
</nav>
|
||
|
||
|
||
|
||
|
||
</div> <!-- /.blog-main -->
|
||
|
||
<aside class="col-sm-3 offset-sm-1 blog-sidebar">
|
||
|
||
|
||
|
||
|
||
<section class="sidebar-module">
|
||
<h4>Recent Posts</h4>
|
||
<ol class="list-unstyled">
|
||
|
||
<li><a href="/cgspace-notes/2017-01/">January, 2017</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-12/">December, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-11/">November, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-10/">October, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-09/">September, 2016</a></li>
|
||
|
||
</ol>
|
||
</section>
|
||
|
||
|
||
<section class="sidebar-module">
|
||
<h4>Links</h4>
|
||
<ol class="list-unstyled">
|
||
|
||
<li><a href="https://cgspace.cgiar.org">CGSpace</a></li>
|
||
|
||
<li><a href="https://dspacetest.cgiar.org">DSpace Test</a></li>
|
||
|
||
<li><a href="https://github.com/ilri/DSpace">CGSpace @ GitHub</a></li>
|
||
|
||
</ol>
|
||
</section>
|
||
|
||
</aside>
|
||
|
||
|
||
</div> <!-- /.row -->
|
||
</div> <!-- /.container -->
|
||
|
||
<footer class="blog-footer">
|
||
<p>
|
||
|
||
Blog template created by <a href="https://twitter.com/mdo">@mdo</a>, ported to Hugo by <a href='https://twitter.com/mralanorth'>@mralanorth</a>.
|
||
|
||
</p>
|
||
<p>
|
||
<a href="#">Back to top</a>
|
||
</p>
|
||
</footer>
|
||
|
||
</body>
|
||
|
||
</html>
|