cgspace-notes/public/2016-02/index.html

246 lines
7.7 KiB
HTML
Raw Normal View History

<!DOCTYPE html>
<html lang="en-us">
<head prefix="og: http://ogp.me/ns#">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1" />
<meta property="og:title" content=" February, 2016 &middot; CGSpace Notes" />
<meta property="og:site_name" content="CGSpace Notes" />
<meta property="og:url" content="/cgspace-notes/2016-02/" />
<meta property="og:type" content="article" />
<meta property="og:article:published_time" content="2016-02-05T13:18:00&#43;03:00" />
<meta property="og:article:tag" content="notes" />
<title>
February, 2016 &middot; CGSpace Notes
</title>
<link rel="stylesheet" href="/cgspace-notes/css/bootstrap.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/main.css" />
<link rel="stylesheet" href="/cgspace-notes/css/font-awesome.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/github.css" />
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Source+Sans+Pro:200,300,400" type="text/css">
<link rel="shortcut icon" href="/cgspace-notes/images/favicon.ico" />
<link rel="apple-touch-icon" href="/cgspace-notes/images/apple-touch-icon.png" />
</head>
<body>
<header class="global-header" style="background-image:url(../images/bg.jpg )">
<section class="header-text">
<h1><a href="/cgspace-notes/">CGSpace Notes</a></h1>
<div class="sns-links hidden-print">
</div>
<a href="/cgspace-notes/" class="btn-header btn-back hidden-xs">
<i class="fa fa-angle-left" aria-hidden="true"></i>
&nbsp;Home
</a>
</section>
</header>
<main class="container">
<article>
<header>
<h1 class="text-primary">February, 2016</h1>
<div class="post-meta clearfix">
<div class="post-date pull-left">
Posted on
<time datetime="2016-02-05T13:18:00&#43;03:00">
Feb 5, 2016
</time>
</div>
<div class="pull-right">
<span class="post-tag small"><a href="/cgspace-notes//tags/notes">#notes</a></span>
</div>
</div>
</header>
<section>
<h2 id="2016-02-05:124a59adbaa8ef13e1518d003fc03981">2016-02-05</h2>
<ul>
<li>Looking at some DAGRIS data for Abenet Yabowork</li>
<li>Lots of issues with spaces, newlines, etc causing the import to fail</li>
<li>I noticed we have a very <em>interesting</em> list of countries on CGSpace:</li>
</ul>
<p><img src="../images/2016/02/cgspace-countries.png" alt="CGSpace country list" /></p>
<ul>
<li>Not only are there 49,000 countries, we have some blanks (25)&hellip;</li>
<li>Also, lots of things like &ldquo;COTE D`LVOIRE&rdquo; and &ldquo;COTE D IVOIRE&rdquo;</li>
</ul>
<h2 id="2016-02-06:124a59adbaa8ef13e1518d003fc03981">2016-02-06</h2>
<ul>
<li>Found a way to get items with null/empty metadata values from SQL</li>
<li>First, find the <code>metadata_field_id</code> for the field you want from the <code>metadatafieldregistry</code> table:</li>
</ul>
<pre><code>dspacetest=# select * from metadatafieldregistry;
</code></pre>
<ul>
<li>In this case our country field is 78</li>
<li>Now find all resources with type 2 (item) that have null/empty values for that field:</li>
</ul>
<pre><code>dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL);
</code></pre>
<ul>
<li>Then you can find the handle that owns it from its <code>resource_id</code>:</li>
</ul>
<pre><code>dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678';
</code></pre>
<ul>
<li>It&rsquo;s 25 items so editing in the web UI is annoying, let&rsquo;s try SQL!</li>
</ul>
<pre><code>dspacetest=# delete from metadatavalue where metadata_field_id=78 and text_value='';
DELETE 25
</code></pre>
<ul>
<li>After that perhaps a regular <code>dspace index-discovery</code> (no -b) <em>should</em> suffice&hellip;</li>
<li>Hmm, I indexed, cleared the Cocoon cache, and restarted Tomcat but the 25 &ldquo;|||&rdquo; countries are still there</li>
<li>Maybe I need to do a full re-index&hellip;</li>
<li>Yep! The full re-index seems to work.</li>
<li>Process the empty countries on CGSpace</li>
</ul>
<h2 id="2016-02-07:124a59adbaa8ef13e1518d003fc03981">2016-02-07</h2>
<ul>
<li>Working on cleaning up Abenet&rsquo;s DAGRIS data with OpenRefine</li>
<li>I discovered two really nice functions in OpenRefine: <code>value.trim()</code> and <code>value.escape(&quot;javascript&quot;)</code> which shows whitespace characters like <code>\r\n</code>!</li>
<li>For some reason when you import an Excel file into OpenRefine it exports dates like 1949 to 1949.0 in the CSV</li>
<li>I re-import the resulting CSV and run a GREL on the date issued column: <code>value.replace(&quot;\.0&quot;, &quot;&quot;)</code></li>
<li>I need to start running DSpace in Mac OS X instead of a Linux VM</li>
<li>Install PostgreSQL from homebrew, then configure and import CGSpace database dump:</li>
</ul>
<pre><code>$ postgres -D /opt/brew/var/postgres
$ createuser --superuser postgres
$ createuser --pwprompt dspacetest
$ createdb -O dspacetest --encoding=UNICODE dspacetest
$ psql postgres
postgres=# alter user dspacetest createuser;
postgres=# \q
$ pg_restore -O -U dspacetest -d dspacetest ~/Downloads/cgspace_2016-02-07.backup
$ psql postgres
postgres=# alter user dspacetest nocreateuser;
postgres=# \q
$ vacuumdb dspacetest
$ psql -U dspacetest -f ~/src/git/DSpace/dspace/etc/postgres/update-sequences.sql dspacetest -h localhost
</code></pre>
<ul>
<li>After building and running a <code>fresh_install</code> I symlinked the webapps into Tomcat&rsquo;s webapps folder:</li>
</ul>
<pre><code>$ mv /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/ROOT /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/ROOT.orig
$ ln -sfv ~/dspace/webapps/xmlui /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/ROOT
$ ln -sfv ~/dspace/webapps/rest /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/rest
$ ln -sfv ~/dspace/webapps/jspui /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/jspui
$ ln -sfv ~/dspace/webapps/oai /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/oai
$ ln -sfv ~/dspace/webapps/solr /opt/brew/Cellar/tomcat/8.0.30/libexec/webapps/solr
$ /opt/brew/Cellar/tomcat/8.0.30/bin/catalina start
</code></pre>
<ul>
<li>Add CATALINA_OPTS in <code>/opt/brew/Cellar/tomcat/8.0.30/libexec/bin/setenv.sh</code>, as this script is sourced by the <code>catalina</code> startup script</li>
<li>For example:</li>
</ul>
<pre><code>CATALINA_OPTS=&quot;-Djava.awt.headless=true -Xms2048m -Xmx2048m -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF-8&quot;
</code></pre>
<ul>
<li>After verifying that the site is working, start a full index:</li>
</ul>
<pre><code>$ ~/dspace/bin/dspace index-discovery -b
</code></pre>
</section>
<footer>
<section class="author-info row">
<div class="author-avatar col-md-2">
</div>
<div class="author-meta col-md-6">
<h1 class="author-name text-primary">Alan Orth</h1>
</div>
</section>
<ul class="pager">
<li class="previous"><a href="/cgspace-notes/2016-01/"><span aria-hidden="true">&larr;</span> Older</a></li>
<li class="next disabled"><a href="#">Newer <span aria-hidden="true">&rarr;</span></a></li>
</ul>
</footer>
</article>
</main>
<footer class="container global-footer">
<div class="copyright-note pull-left">
</div>
<div class="sns-links hidden-print">
</div>
</footer>
<script src="/cgspace-notes/js/highlight.pack.js"></script>
<script>
hljs.initHighlightingOnLoad();
</script>
</body>
</html>