alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-05 06:43:00 +01:00
Code Issues Projects Releases Wiki Activity
fc3235ed84
cgspace-notes/public/2016-05/index.html

280 lines
9.2 KiB
HTML
Raw Normal View History

Add notes for 2016-05-01
2016-05-02 00:33:35 +02:00
<!DOCTYPE html>
<html lang="en-us">
<head prefix="og: http://ogp.me/ns#">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1" />
<meta property="og:title" content=" May, 2016 &middot; CGSpace Notes" />
<meta property="og:site_name" content="CGSpace Notes" />
<meta property="og:url" content="/cgspace-notes/2016-05/" />
<meta property="og:type" content="article" />
<meta property="og:article:published_time" content="2016-05-01T23:06:00&#43;03:00" />
<meta property="og:article:tag" content="notes" />
<title>
May, 2016 &middot; CGSpace Notes
</title>
<link rel="stylesheet" href="/cgspace-notes/css/bootstrap.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/main.css" />
<link rel="stylesheet" href="/cgspace-notes/css/font-awesome.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/github.css" />
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Source+Sans+Pro:200,300,400" type="text/css">
<link rel="shortcut icon" href="/cgspace-notes/images/favicon.ico" />
<link rel="apple-touch-icon" href="/cgspace-notes/images/apple-touch-icon.png" />
</head>
<body>
<header class="global-header" style="background-image:url(../images/bg.jpg )">
<section class="header-text">
<h1><a href="/cgspace-notes/">CGSpace Notes</a></h1>
<div class="sns-links hidden-print">
</div>
<a href="/cgspace-notes/" class="btn-header btn-back hidden-xs">
<i class="fa fa-angle-left" aria-hidden="true"></i>
&nbsp;Home
</a>
</section>
</header>
<main class="container">
<article>
<header>
<h1 class="text-primary">May, 2016</h1>
<div class="post-meta clearfix">
<div class="post-date pull-left">
Posted on
<time datetime="2016-05-01T23:06:00&#43;03:00">
May 1, 2016
</time>
</div>
<div class="pull-right">
<span class="post-tag small"><a href="/cgspace-notes//tags/notes">#notes</a></span>
</div>
</div>
</header>
<section>
<h2 id="2016-05-01:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-01</h2>
<ul>
<li>Since yesterday there have been 10,000 REST errors and the site has been unstable again</li>
<li>I have blocked access to the API now</li>
<li>There are 3,000 IPs accessing the REST API in a 24-hour period!</li>
</ul>
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
3168
</code></pre>
<ul>
<li>The two most often requesters are in Ethiopia and Colombia: 213.55.99.121 and 181.118.144.29</li>
<li>100% of the requests coming from Ethiopia are like this and result in an HTTP 500:</li>
</ul>
<pre><code>GET /rest/handle/10568/NaN?expand=parentCommunityList,metadata HTTP/1.1
</code></pre>
<ul>
<li>For now I&rsquo;ll block just the Ethiopian IP</li>
Add notes for 2016-05-06
2016-05-06 16:23:30 +02:00
<li>The owner of that application has said that the <code>NaN</code> (not a number) is an error in his code and he&rsquo;ll fix it</li>
Add notes for 2016-05-03
2016-05-03 11:05:03 +02:00
</ul>
<h2 id="2016-05-03:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-03</h2>
<ul>
<li>Update nginx to 1.10.x branch on CGSpace</li>
Update notes for 2016-05-03
2016-05-03 13:33:21 +02:00
<li>Fix a reference to <code>dc.type.output</code> in Discovery that I had missed when we migrated to <code>dc.type</code> last month (<a href="https://github.com/ilri/DSpace/pull/223">#223</a>)</li>
Add notes for 2016-05-01
2016-05-02 00:33:35 +02:00
</ul>
Update notes for 2016-05-03
2016-05-03 13:33:21 +02:00
<p><img src="../images/2016/05/discovery-types.png" alt="Item type in Discovery results" /></p>
Add notes for 2016-05-06
2016-05-06 16:23:30 +02:00
<h2 id="2016-05-06:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-06</h2>
<ul>
<li>DSpace Test is down, <code>catalina.out</code> has lots of messages about heap space from some time yesterday (!)</li>
<li>It looks like Sisay was doing some batch imports</li>
<li>Hmm, also disk space is full</li>
<li>I decided to blow away the solr indexes, since they are 50GB and we don&rsquo;t really need all the Atmire stuff there right now</li>
<li>I will re-generate the Discovery indexes after re-deploying</li>
Update notes for 2016-05-06
2016-05-06 17:00:59 +02:00
<li>Testing <code>renew-letsencrypt.sh</code> script for nginx</li>
</ul>
<pre><code>#!/usr/bin/env bash
readonly SERVICE_BIN=/usr/sbin/service
readonly LETSENCRYPT_BIN=/opt/letsencrypt/letsencrypt-auto
# stop nginx so LE can listen on port 443
$SERVICE_BIN nginx stop
$LETSENCRYPT_BIN renew -nvv --standalone --standalone-supported-challenges tls-sni-01 &gt; /var/log/letsencrypt/renew.log 2&gt;&amp;1
LE_RESULT=$?
$SERVICE_BIN nginx start
if [[ &quot;$LE_RESULT&quot; != 0 ]]; then
echo 'Automated renewal failed:'
cat /var/log/letsencrypt/renew.log
exit 1
fi
</code></pre>
<ul>
<li>Seems to work well</li>
Add notes for 2016-05-10
2016-05-10 13:22:31 +02:00
</ul>
<h2 id="2016-05-10:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-10</h2>
<ul>
<li>Start looking at more metadata migrations</li>
<li>There are lots of fields in <code>dcterms</code> namespace that look interesting, like:
<ul>
<li>dcterms.type</li>
<li>dcterms.spatial</li>
</ul></li>
<li>Not sure what <code>dcterms</code> is&hellip;</li>
<li>Looks like these were <a href="https://wiki.duraspace.org/display/DSDOC5x/Metadata+and+Bitstream+Format+Registries#MetadataandBitstreamFormatRegistries-DublinCoreTermsRegistry(DCTERMS)">added in DSpace 4</a> to allow for future work to make DSpace more flexible</li>
Update notes for 2016-05-10
2016-05-10 17:05:55 +02:00
<li>CGSpace&rsquo;s <code>dc</code> registry has 96 items, and the default DSpace one has 73.</li>
Add notes for 2016-05-11
2016-05-11 11:48:28 +02:00
</ul>
<h2 id="2016-05-11:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-11</h2>
<ul>
Update notes for 2016-05-11
2016-05-11 16:51:09 +02:00
<li><p>Identify and propose the next phase of CGSpace fields to migrate:</p>
Add notes for 2016-05-11
2016-05-11 11:48:28 +02:00
<ul>
<li>dc.title.jtitle → cg.title.journal</li>
<li>dc.identifier.status → cg.identifier.status</li>
<li>dc.river.basin → cg.river.basin</li>
<li>dc.Species → cg.species</li>
<li>dc.targetaudience → cg.targetaudience</li>
<li>dc.fulltextstatus → cg.fulltextstatus</li>
<li>dc.editon → cg.edition</li>
<li>dc.isijournal → cg.isijournal</li>
</ul></li>
Update notes for 2016-05-11
2016-05-11 16:51:09 +02:00
<li><p>Start a test rebase of the <code>5_x-prod</code> branch on top of the <code>dspace-5.5</code> tag</p></li>
<li><p>There were a handful of conflicts that I didn&rsquo;t understand</p></li>
<li><p>After completing the rebase I tried to build with the module versions Atmire had indicated as being 5.5 ready but I got this error:</p></li>
</ul>
<pre><code>[ERROR] Failed to execute goal on project additions: Could not resolve dependencies for project org.dspace.modules:additions:jar:5.5: Could not find artifact com.atmire:atmire-metadata-quality-api:jar:5.5-2.10.1-0 in sonatype-releases (https://oss.sonatype.org/content/repositories/releases/) -&gt; [Help 1]
</code></pre>
<ul>
<li>I&rsquo;ve sent them a question about it</li>
Update notes for 2016-05-11
2016-05-11 16:58:26 +02:00
<li>A user mentioned having problems with uploading a 33 MB PDF</li>
<li>I told her I would increase the limit temporarily tomorrow morning</li>
Add notes for 2016-05-12
2016-05-12 17:40:59 +02:00
<li>Turns out she was able to decrease the size of the PDF so we didn&rsquo;t have to do anything</li>
Add notes for 2016-05-06
2016-05-06 16:23:30 +02:00
</ul>
Add notes for 2016-05-12
2016-05-12 17:40:59 +02:00
<h2 id="2016-05-12:b7bf1a0f8f2415a40e1e11e343b04c0d">2016-05-12</h2>
<ul>
<li>Looks like the issue that Abenet was having a few days ago with &ldquo;Connection Reset&rdquo; in Firefox might be due to a Firefox 46 issue: <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1268775">https://bugzilla.mozilla.org/show_bug.cgi?id=1268775</a></li>
<li>I finally found a copy of the latest CG Core metadata guidelines and it looks like we can add a few more fields to our next migration:
<ul>
<li>dc.rplace.region → cg.coverage.region</li>
<li>dc.cplace.country → cg.coverage.country</li>
</ul></li>
<li>Questions for CG people:
<ul>
<li>Our <code>dc.place</code> and <code>dc.srplace.subregion</code> could both map to <code>cg.coverage.admin-unit</code>?</li>
<li>Should we use <code>dc.contributor.crp</code> or <code>cg.contributor.crp</code> for the CRP (ours is <code>dc.crsubject.crpsubject</code>)?</li>
<li>Our <code>dc.contributor.affiliation</code> and <code>dc.contributor.corporate</code> could both map to <code>dc.contributor</code> and possibly <code>dc.contributor.center</code> depending on if it&rsquo;s a CG center or not</li>
<li><code>dc.title.jtitle</code> could either map to <code>dc.publisher</code> or <code>dc.source</code> depending on how you read things</li>
</ul></li>
<li>Found ~200 messed up CIAT values in <code>dc.publisher</code>:</li>
</ul>
<pre><code># select text_value from metadatavalue where resource_type_id=2 and metadata_field_id=39 and text_value similar to &quot;% %&quot;;
</code></pre>
Add notes for 2016-05-01
2016-05-02 00:33:35 +02:00
</section>
<footer>
<section class="author-info row">
<div class="author-avatar col-md-2">
</div>
<div class="author-meta col-md-6">
<h1 class="author-name text-primary">Alan Orth</h1>
</div>
</section>
<ul class="pager">
<li class="previous"><a href="/cgspace-notes/2016-04/"><span aria-hidden="true">&larr;</span> Older</a></li>
<li class="next disabled"><a href="#">Newer <span aria-hidden="true">&rarr;</span></a></li>
</ul>
</footer>
</article>
</main>
<footer class="container global-footer">
<div class="copyright-note pull-left">
</div>
<div class="sns-links hidden-print">
</div>
</footer>
<script src="/cgspace-notes/js/highlight.pack.js"></script>
<script>
hljs.initHighlightingOnLoad();
</script>
</body>
</html>
Reference in New Issue Copy Permalink