mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-22 06:35:03 +01:00
427 lines
15 KiB
HTML
427 lines
15 KiB
HTML
<!DOCTYPE html>
|
||
<html lang="en">
|
||
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta http-equiv="X-UA-Compatible" content="IE=edge">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
||
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
|
||
|
||
<meta property="og:title" content="CGSpace Notes" />
|
||
<meta property="og:description" content="" />
|
||
<meta property="og:type" content="website" />
|
||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||
|
||
|
||
<meta property="og:updated_time" content="2016-11-01T09:21:00+03:00"/>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<meta itemprop="name" content="CGSpace Notes">
|
||
<meta itemprop="description" content="">
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<meta name="generator" content="Hugo 0.17" />
|
||
|
||
|
||
<base href="https://alanorth.github.io/cgspace-notes/">
|
||
<link rel="canonical" href="https://alanorth.github.io/cgspace-notes/">
|
||
|
||
<title>CGSpace Notes</title>
|
||
|
||
<!-- combined, minified CSS -->
|
||
<link href="https://alanorth.github.io/cgspace-notes/css/style.css" rel="stylesheet">
|
||
|
||
|
||
<!-- RSS 2.0 feed -->
|
||
<link href="https://alanorth.github.io/cgspace-notes/index.xml" rel="alternate" type="application/rss+xml" title="CGSpace Notes" />
|
||
<link href="https://alanorth.github.io/cgspace-notes/index.xml" rel="feed" type="application/rss+xml" title="CGSpace Notes" />
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</head>
|
||
|
||
<body>
|
||
|
||
<div class="blog-masthead">
|
||
<div class="container">
|
||
<nav class="nav blog-nav">
|
||
<a class="nav-link active" href="https://alanorth.github.io/cgspace-notes/">Home</a>
|
||
|
||
|
||
|
||
</nav>
|
||
</div>
|
||
</div>
|
||
|
||
<header class="blog-header">
|
||
<div class="container">
|
||
<h1 class="blog-title"><a href="https://alanorth.github.io/cgspace-notes/" rel="home">CGSpace Notes</a></h1>
|
||
|
||
</div>
|
||
</header>
|
||
|
||
<div class="container">
|
||
<div class="row">
|
||
<div class="col-sm-8 blog-main">
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-11/">November, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-11-01T09:21:00+03:00">Tue Nov 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2016-11-01 Add dc.type to the output options for Atmire’s Listings and Reports module (#286) 2016-11-02 Migrate DSpace Test to DSpace 5.5 (notes) Run all updates on DSpace Test and reboot the server Looks like the OAI bug from DSpace 5.1 that caused validation at Base Search to fail is now fixed and DSpace Test passes validation! (#63) Indexing Discovery on DSpace Test took 332 minutes, which is like five times as long as it usually takes At the end it appeared to finish correctly but there were lots of errors right after it finished: 2016-11-02 15:09:48,578 INFO com.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-11/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-10/">October, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-10-03T15:53:00+03:00">Mon Oct 03, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
2016-10-03 Testing adding ORCIDs to a CSV file for a single item to see if the author orders get messed up Need to test the following scenarios to see how author order is affected: ORCIDs only ORCIDs plus normal authors I exported a random item’s metadata as CSV, deleted all columns except id and collection, and made a new coloum called ORCID:dc.contributor.author with the following random ORCIDs from the ORCID registry: 0000-0002-6115-0956||0000-0002-3812-8793||0000-0001-7462-405X Hmm, with the dc.
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-10/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-09/">September, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-09-01T15:53:00+03:00">Thu Sep 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-09-01">2016-09-01</h2>
|
||
|
||
<ul>
|
||
<li>Discuss helping CCAFS with some batch tagging of ORCID IDs for their authors</li>
|
||
<li>Discuss how the migration of CGIAR’s Active Directory to a flat structure will break our LDAP groups in DSpace</li>
|
||
<li>We had been using <code>DC=ILRI</code> to determine whether a user was ILRI or not</li>
|
||
<li>It looks like we might be able to use OUs now, instead of DCs:</li>
|
||
</ul>
|
||
|
||
<pre><code>$ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=org" -D "admigration1@cgiarad.org" -W "(sAMAccountName=admigration1)"
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-09/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-08/">August, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-08-01T15:53:00+03:00">Mon Aug 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-08-01">2016-08-01</h2>
|
||
|
||
<ul>
|
||
<li>Add updated distribution license from Sisay (<a href="https://github.com/ilri/DSpace/issues/259">#259</a>)</li>
|
||
<li>Play with upgrading Mirage 2 dependencies in <code>bower.json</code> because most are several versions of out date</li>
|
||
<li>Bootstrap is at 3.3.0 but upstream is at 3.3.7, and upgrading to anything beyond 3.3.1 breaks glyphicons and probably more</li>
|
||
<li>bower stuff is a dead end, waste of time, too many issues</li>
|
||
<li>Anything after Bootstrap 3.3.1 makes glyphicons disappear (HTTP 404 trying to access from incorrect path of <code>fonts</code>)</li>
|
||
<li>Start working on DSpace 5.1 → 5.5 port:</li>
|
||
</ul>
|
||
|
||
<pre><code>$ git checkout -b 55new 5_x-prod
|
||
$ git reset --hard ilri/5_x-prod
|
||
$ git rebase -i dspace-5.5
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-08/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-07/">July, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-07-01T10:53:00+03:00">Fri Jul 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-07-01">2016-07-01</h2>
|
||
|
||
<ul>
|
||
<li>Add <code>dc.description.sponsorship</code> to Discovery sidebar facets and make investors clickable in item view (<a href="https://github.com/ilri/DSpace/issues/232">#232</a>)</li>
|
||
<li>I think this query should find and replace all authors that have “,” at the end of their names:</li>
|
||
</ul>
|
||
|
||
<pre><code>dspacetest=# update metadatavalue set text_value = regexp_replace(text_value, '(^.+?),$', '\1') where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
|
||
UPDATE 95
|
||
dspacetest=# select text_value from metadatavalue where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
|
||
text_value
|
||
------------
|
||
(0 rows)
|
||
</code></pre>
|
||
|
||
<ul>
|
||
<li>In this case the select query was showing 95 results before the update</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-07/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-06-01">2016-06-01</h2>
|
||
|
||
<ul>
|
||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-05-01T23:06:00+03:00">Sun May 01, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-05-01">2016-05-01</h2>
|
||
|
||
<ul>
|
||
<li>Since yesterday there have been 10,000 REST errors and the site has been unstable again</li>
|
||
<li>I have blocked access to the API now</li>
|
||
<li>There are 3,000 IPs accessing the REST API in a 24-hour period!</li>
|
||
</ul>
|
||
|
||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||
3168
|
||
</code></pre>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-05/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-04/">April, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-04-04T11:06:00+03:00">Mon Apr 04, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-04-04">2016-04-04</h2>
|
||
|
||
<ul>
|
||
<li>Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit</li>
|
||
<li>We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc</li>
|
||
<li>After running DSpace for over five years I’ve never needed to look in any other log file than dspace.log, leave alone one from last year!</li>
|
||
<li>This will save us a few gigs of backup space we’re paying for on S3</li>
|
||
<li>Also, I noticed the <code>checker</code> log has some errors we should pay attention to:</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-04/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-03/">March, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-03-02T16:50:00+03:00">Wed Mar 02, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-03-02">2016-03-02</h2>
|
||
|
||
<ul>
|
||
<li>Looking at issues with author authorities on CGSpace</li>
|
||
<li>For some reason we still have the <code>index-lucene-update</code> cron job active on CGSpace, but I’m pretty sure we don’t need it as of the latest few versions of Atmire’s Listings and Reports module</li>
|
||
<li>Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Java JDK 1.7 to match environment on CGSpace server</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-03/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<article class="blog-post">
|
||
<header>
|
||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-02/">February, 2016</a></h2>
|
||
<p class="blog-post-meta"><time datetime="2016-02-05T13:18:00+03:00">Fri Feb 05, 2016</time> by Alan Orth in
|
||
|
||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||
|
||
</p>
|
||
</header>
|
||
<h2 id="2016-02-05">2016-02-05</h2>
|
||
|
||
<ul>
|
||
<li>Looking at some DAGRIS data for Abenet Yabowork</li>
|
||
<li>Lots of issues with spaces, newlines, etc causing the import to fail</li>
|
||
<li>I noticed we have a very <em>interesting</em> list of countries on CGSpace:</li>
|
||
</ul>
|
||
|
||
<p><img src="2016/02/cgspace-countries.png" alt="CGSpace country list" /></p>
|
||
|
||
<ul>
|
||
<li>Not only are there 49,000 countries, we have some blanks (25)…</li>
|
||
<li>Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE”</li>
|
||
</ul>
|
||
|
||
<p></p>
|
||
<a href='https://alanorth.github.io/cgspace-notes/2016-02/'>Read more →</a>
|
||
</article>
|
||
|
||
|
||
|
||
|
||
<nav class="blog-pagination">
|
||
|
||
|
||
|
||
<a class="btn btn-outline-primary disabled" href="#" role="button" aria-disabled="true">Previous page</a>
|
||
<a class="btn btn-outline-primary" href="/page/2/" role="button">Next page</a>
|
||
|
||
</nav>
|
||
|
||
|
||
|
||
|
||
</div> <!-- /.blog-main -->
|
||
|
||
<aside class="col-sm-3 offset-sm-1 blog-sidebar">
|
||
|
||
|
||
|
||
|
||
<section class="sidebar-module">
|
||
<h4>Recent Posts</h4>
|
||
<ol class="list-unstyled">
|
||
|
||
<li><a href="/cgspace-notes/2016-11/">November, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-10/">October, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-09/">September, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-08/">August, 2016</a></li>
|
||
|
||
<li><a href="/cgspace-notes/2016-07/">July, 2016</a></li>
|
||
|
||
</ol>
|
||
</section>
|
||
|
||
|
||
<section class="sidebar-module">
|
||
<h4>Links</h4>
|
||
<ol class="list-unstyled">
|
||
|
||
<li><a href="https://cgspace.cgiar.org">CGSpace</a></li>
|
||
|
||
<li><a href="https://dspacetest.cgiar.org">DSpace Test</a></li>
|
||
|
||
<li><a href="https://github.com/ilri/DSpace">CGSpace @ GitHub</a></li>
|
||
|
||
</ol>
|
||
</section>
|
||
|
||
</aside>
|
||
|
||
|
||
</div> <!-- /.row -->
|
||
</div> <!-- /.container -->
|
||
|
||
<footer class="blog-footer">
|
||
<p>
|
||
|
||
Blog template created by <a href="https://twitter.com/mdo">@mdo</a>, ported to Hugo by <a href='https://twitter.com/mralanorth'>@mralanorth</a>.
|
||
|
||
</p>
|
||
<p>
|
||
<a href="#">Back to top</a>
|
||
</p>
|
||
</footer>
|
||
|
||
</body>
|
||
|
||
</html>
|