mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2025-01-27 05:49:12 +01:00
Add notes for 2019-12-17
This commit is contained in:
@ -43,7 +43,7 @@ Most worryingly, there are encoding errors in the abstracts for eleven items, fo
|
||||
|
||||
I think I will need to ask Udana to re-copy and paste the abstracts with more care using Google Docs
|
||||
"/>
|
||||
<meta name="generator" content="Hugo 0.60.1" />
|
||||
<meta name="generator" content="Hugo 0.61.0" />
|
||||
|
||||
|
||||
|
||||
@ -124,7 +124,7 @@ I think I will need to ask Udana to re-copy and paste the abstracts with more ca
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="20190301">2019-03-01</h2>
|
||||
<h2 id="2019-03-01">2019-03-01</h2>
|
||||
<ul>
|
||||
<li>I checked IITA's 259 Feb 14 records from last month for duplicates using Atmire's Duplicate Checker on a fresh snapshot of CGSpace on my local machine and everything looks good</li>
|
||||
<li>I am now only waiting to hear from her about where the items should go, though I assume Journal Articles go to IITA Journal Articles collection, etc…</li>
|
||||
@ -139,7 +139,7 @@ I think I will need to ask Udana to re-copy and paste the abstracts with more ca
|
||||
</li>
|
||||
<li>I think I will need to ask Udana to re-copy and paste the abstracts with more care using Google Docs</li>
|
||||
</ul>
|
||||
<h2 id="20190303">2019-03-03</h2>
|
||||
<h2 id="2019-03-03">2019-03-03</h2>
|
||||
<ul>
|
||||
<li>Trying to finally upload IITA's 259 Feb 14 items to CGSpace so I exported them from DSpace Test:</li>
|
||||
</ul>
|
||||
@ -166,7 +166,7 @@ $ dspace export -i 10568/108684 -t COLLECTION -m -n 0 -d 2019-03-03-IITA-Feb14
|
||||
</li>
|
||||
<li>Deploy Tomcat 7.0.93 on CGSpace (linode18) after having tested it on DSpace Test (linode19) for a week</li>
|
||||
</ul>
|
||||
<h2 id="20190306">2019-03-06</h2>
|
||||
<h2 id="2019-03-06">2019-03-06</h2>
|
||||
<ul>
|
||||
<li>Abenet was having problems with a CIP user account, I think that the user could not register</li>
|
||||
<li>I suspect it's related to the email issue that ICT hasn't responded about since last week</li>
|
||||
@ -184,7 +184,7 @@ Error sending email:
|
||||
</code></pre><ul>
|
||||
<li>I will send a follow-up to ICT to ask them to reset the password</li>
|
||||
</ul>
|
||||
<h2 id="20190307">2019-03-07</h2>
|
||||
<h2 id="2019-03-07">2019-03-07</h2>
|
||||
<ul>
|
||||
<li>ICT reset the email password and I confirmed that it is working now</li>
|
||||
<li>Generate a controlled vocabulary of 1187 AGROVOC subjects from the top 1500 that I checked last month, dumping the terms themselves using <code>csvcut</code> and then applying XML controlled vocabulary format in vim and then checking with tidy for good measure:</li>
|
||||
@ -200,7 +200,7 @@ $ tidy -xml -utf8 -iq -m -w 0 dspace/config/controlled-vocabularies/dc-subject.x
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190308">2019-03-08</h2>
|
||||
<h2 id="2019-03-08">2019-03-08</h2>
|
||||
<ul>
|
||||
<li>There's an issue with CGSpace right now where all items are giving a blank page in the XMLUI
|
||||
<ul>
|
||||
@ -223,7 +223,7 @@ $ tidy -xml -utf8 -iq -m -w 0 dspace/config/controlled-vocabularies/dc-subject.x
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190309">2019-03-09</h2>
|
||||
<h2 id="2019-03-09">2019-03-09</h2>
|
||||
<ul>
|
||||
<li>I shared a post on Yammer informing our editors to try to AGROVOC controlled list</li>
|
||||
<li>The SPDX legal committee had a meeting and discussed the addition of CC-BY-ND-3.0-IGO and other IGO licenses to their list, but it seems unlikely (<a href="https://github.com/spdx/license-list-XML/issues/767#issuecomment-470709673">spdx/license-list-XML/issues/767</a>)</li>
|
||||
@ -241,7 +241,7 @@ UPDATE 44
|
||||
</code></pre><ul>
|
||||
<li>I ran the corrections on CGSpace and DSpace Test</li>
|
||||
</ul>
|
||||
<h2 id="20190310">2019-03-10</h2>
|
||||
<h2 id="2019-03-10">2019-03-10</h2>
|
||||
<ul>
|
||||
<li>Working on tagging IITA's items with their new research theme (<code>cg.identifier.iitatheme</code>) based on their existing IITA subjects (see <a href="/cgspace-notes/2018-02/">notes from 2019-02</a>)</li>
|
||||
<li>I exported the entire IITA community from CGSpace and then used <code>csvcut</code> to extract only the needed fields:</li>
|
||||
@ -261,15 +261,15 @@ UPDATE 44
|
||||
<li>In total this would add research themes to 1,755 items</li>
|
||||
<li>I want to double check one last time with Bosede that they would like to do this, because I also see that this will tag a few hundred items from the 1970s and 1980s</li>
|
||||
</ul>
|
||||
<h2 id="20190311">2019-03-11</h2>
|
||||
<h2 id="2019-03-11">2019-03-11</h2>
|
||||
<ul>
|
||||
<li>Bosede said that she would like the IITA research theme tagging only for items since 2015, which would be 256 items</li>
|
||||
</ul>
|
||||
<h2 id="20190312">2019-03-12</h2>
|
||||
<h2 id="2019-03-12">2019-03-12</h2>
|
||||
<ul>
|
||||
<li>I imported the changes to 256 of IITA's records on CGSpace</li>
|
||||
</ul>
|
||||
<h2 id="20190314">2019-03-14</h2>
|
||||
<h2 id="2019-03-14">2019-03-14</h2>
|
||||
<ul>
|
||||
<li>CGSpace had the same issue with blank items like earlier this month and I restarted Tomcat to fix it</li>
|
||||
<li>Create a pull request to change Swaziland to Eswatini and Macedonia to North Macedonia (<a href="https://github.com/ilri/DSpace/pull/414">#414</a>)
|
||||
@ -301,7 +301,7 @@ done
|
||||
<li>Run all system updates and reboot linode20</li>
|
||||
<li>Follow up with Felix from Earlham to see if he's done testing DSpace Test with COPO so I can re-sync the server from CGSpace</li>
|
||||
</ul>
|
||||
<h2 id="20190315">2019-03-15</h2>
|
||||
<h2 id="2019-03-15">2019-03-15</h2>
|
||||
<ul>
|
||||
<li>CGSpace (linode18) has the blank page error again</li>
|
||||
<li>I'm not sure if it's related, but I see the following error in DSpace's log:</li>
|
||||
@ -402,7 +402,7 @@ java.util.EmptyStackException
|
||||
</code></pre><ul>
|
||||
<li>For now I will just restart Tomcat…</li>
|
||||
</ul>
|
||||
<h2 id="20190317">2019-03-17</h2>
|
||||
<h2 id="2019-03-17">2019-03-17</h2>
|
||||
<ul>
|
||||
<li>Last week Felix from Earlham said that they finished testing on DSpace Test (linode19) so I made backups of some things there and re-deployed the system on Ubuntu 18.04
|
||||
<ul>
|
||||
@ -437,7 +437,7 @@ Error: ERROR: update or delete on table "bitstream" violates foreign k
|
||||
<pre><code># su - postgres
|
||||
$ psql dspace -c 'update bundle set primary_bitstream_id=NULL where primary_bitstream_id in (164496);'
|
||||
UPDATE 1
|
||||
</code></pre><h2 id="20190318">2019-03-18</h2>
|
||||
</code></pre><h2 id="2019-03-18">2019-03-18</h2>
|
||||
<ul>
|
||||
<li>I noticed that the regular expression for validating lines from input files in my <code>agrovoc-lookup.py</code> script was skipping characters with accents, etc, so I changed it to use the <code>\w</code> character class for words instead of trying to match <code>[A-Z]</code> etc…
|
||||
<ul>
|
||||
@ -568,7 +568,7 @@ $ psql -c 'select * from pg_stat_activity' | grep -o -E '(dspaceWeb|dspaceApi|ds
|
||||
</code></pre><ul>
|
||||
<li>I'm not sure if it's cocoon or that's just a symptom of something else</li>
|
||||
</ul>
|
||||
<h2 id="20190319">2019-03-19</h2>
|
||||
<h2 id="2019-03-19">2019-03-19</h2>
|
||||
<ul>
|
||||
<li>I found a handful of AGROVOC subjects that use a non-breaking space (0x00a0) instead of a regular space, which makes for a pretty confusing debugging…</li>
|
||||
<li>I will replace these in the database immediately to save myself the headache later:</li>
|
||||
@ -640,7 +640,7 @@ Max realtime timeout unlimited unlimited us
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190320">2019-03-20</h2>
|
||||
<h2 id="2019-03-20">2019-03-20</h2>
|
||||
<ul>
|
||||
<li>Create a branch for Solr 4.10.4 changes so I can test on DSpace Test (linode19)
|
||||
<ul>
|
||||
@ -648,7 +648,7 @@ Max realtime timeout unlimited unlimited us
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190321">2019-03-21</h2>
|
||||
<h2 id="2019-03-21">2019-03-21</h2>
|
||||
<ul>
|
||||
<li>It's been two days since we had the blank page issue on CGSpace, and looking in the Cocoon logs I see very low numbers of the errors that we were seeing the last time the issue occurred:</li>
|
||||
</ul>
|
||||
@ -687,12 +687,12 @@ $ grep 'Can not load requested doc' cocoon.log.2019-03-21 | grep -oE '2019-03-21
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190322">2019-03-22</h2>
|
||||
<h2 id="2019-03-22">2019-03-22</h2>
|
||||
<ul>
|
||||
<li>Share the initial list of invalid AGROVOC terms on Yammer to ask the editors for help in correcting them</li>
|
||||
<li>Advise Phanuel Ayuka from IITA about using controlled vocabularies in DSpace</li>
|
||||
</ul>
|
||||
<h2 id="20190323">2019-03-23</h2>
|
||||
<h2 id="2019-03-23">2019-03-23</h2>
|
||||
<ul>
|
||||
<li>CGSpace (linode18) is having the blank page issue again and it seems to have started last night around 21:00:</li>
|
||||
</ul>
|
||||
@ -811,7 +811,7 @@ org.postgresql.util.PSQLException: This statement has been closed.
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190324">2019-03-24</h2>
|
||||
<h2 id="2019-03-24">2019-03-24</h2>
|
||||
<ul>
|
||||
<li>I did some more tests with the <a href="https://github.com/gnosly/TomcatJdbcConnectionTest">TomcatJdbcConnectionTest</a> thing and while monitoring the number of active connections in jconsole and after adjusting the limits quite low I eventually saw some connections get abandoned</li>
|
||||
<li>I forgot that to connect to a remote JMX session with jconsole you need to use a dynamic SSH SOCKS proxy (as I originally <a href="/cgspace-notes/2017-11/">discovered in 2017-11</a>:</li>
|
||||
@ -831,7 +831,7 @@ org.postgresql.util.PSQLException: This statement has been closed.
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190325">2019-03-25</h2>
|
||||
<h2 id="2019-03-25">2019-03-25</h2>
|
||||
<ul>
|
||||
<li>Finish looking over the 175 invalid AGROVOC terms
|
||||
<ul>
|
||||
@ -918,7 +918,7 @@ $ grep -o -E 'session_id=[A-Z0-9]{32}' dspace.log.2019-03-22 | sort -u | wc -l
|
||||
</li>
|
||||
<li>According the Uptime Robot the server was up and down a few more times over the next hour so I restarted Tomcat again</li>
|
||||
</ul>
|
||||
<h2 id="20190326">2019-03-26</h2>
|
||||
<h2 id="2019-03-26">2019-03-26</h2>
|
||||
<ul>
|
||||
<li>UptimeRobot says CGSpace went down again and I see the load is again at 14.0!</li>
|
||||
<li>Here are the top IPs in nginx logs in the last hour:</li>
|
||||
@ -1032,7 +1032,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-03-26-AGROVOC-79-deletions.csv -db ds
|
||||
</ul>
|
||||
<pre><code>$ grep -I -c 45.5.184.72 dspace.log.2019-03-26
|
||||
0
|
||||
</code></pre><h2 id="20190328">2019-03-28</h2>
|
||||
</code></pre><h2 id="2019-03-28">2019-03-28</h2>
|
||||
<ul>
|
||||
<li>Run the corrections and deletions to AGROVOC (dc.subject) on DSpace Test and CGSpace, and then start a full re-index of Discovery</li>
|
||||
<li>What the hell is going on with this CTA publication?</li>
|
||||
@ -1074,7 +1074,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-03-26-AGROVOC-79-deletions.csv -db ds
|
||||
</code></pre><ul>
|
||||
<li>In other other news I see that DSpace has no statistics for years before 2019 currently, yet when I connect to Solr I see all the cores up</li>
|
||||
</ul>
|
||||
<h2 id="20190329">2019-03-29</h2>
|
||||
<h2 id="2019-03-29">2019-03-29</h2>
|
||||
<ul>
|
||||
<li>Sent Linode more information from <code>top</code> and <code>iostat</code> about the resource usage on linode18
|
||||
<ul>
|
||||
@ -1088,7 +1088,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-03-26-AGROVOC-79-deletions.csv -db ds
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="20190331">2019-03-31</h2>
|
||||
<h2 id="2019-03-31">2019-03-31</h2>
|
||||
<ul>
|
||||
<li>After a few days of the CGSpace VM (linode18) being migrated to a new host the CPU steal is gone and the site is much more responsive</li>
|
||||
</ul>
|
||||
|
||||
Reference in New Issue
Block a user