alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2019-12-17

Browse Source
This commit is contained in:
Alan Orth
2019-12-17 14:49:24 +02:00
parent d83c951532
commit d54e5b69f1
90 changed files with 1420 additions and 1377 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
docs/2019-01/index.html
View File

@ -47,7 +47,7 @@ I don't see anything interesting in the web server logs around that time tho
357 207.46.13.1
903 54.70.40.11
"/>
<meta name="generator" content="Hugo 0.60.1" />
<meta name="generator" content="Hugo 0.61.0" />
@ -128,7 +128,7 @@ I don&#39;t see anything interesting in the web server logs around that time tho
</p>
</header>
<h2 id="20190102">2019-01-02</h2>
<h2 id="2019-01-02">2019-01-02</h2>
<ul>
<li>Linode alerted that CGSpace (linode18) had a higher outbound traffic rate than normal early this morning</li>
<li>I don't see anything interesting in the web server logs around that time though:</li>
@ -173,7 +173,7 @@ Moving: 18497180 into core statistics-2018
<li>This could by why the outbound traffic rate was high, due to the S3 backup that run at 3:30AM&hellip;</li>
<li>Run all system updates on DSpace Test (linode19) and reboot the server</li>
</ul>
<h2 id="20190103">2019-01-03</h2>
<h2 id="2019-01-03">2019-01-03</h2>
<ul>
<li>Update local Docker image for DSpace PostgreSQL, re-using the existing data volume:</li>
</ul>
@ -271,7 +271,7 @@ org.apache.jasper.JasperException: /home.jsp (line: [214], column: [1]) /discove
</li>
<li>I sent a message to the dspace-tech mailing list to ask</li>
</ul>
<h2 id="20190104">2019-01-04</h2>
<h2 id="2019-01-04">2019-01-04</h2>
<ul>
<li>Linode sent a message last night that CGSpace (linode18) had high CPU usage, but I don't see anything around that time in the web server logs:</li>
</ul>
@ -403,7 +403,7 @@ In [14]: for row in result.fetchone():
</code></pre><ul>
<li>The SPARQL query comes from my notes in <a href="/cgspace-notes/2017-08/">2017-08</a></li>
</ul>
<h2 id="20190106">2019-01-06</h2>
<h2 id="2019-01-06">2019-01-06</h2>
<ul>
<li>I built a clean DSpace 5.8 installation from the upstream <code>dspace-5.8</code> tag and the issue with the XMLUI/JSPUI login is still there with Tomcat 8.5.37
<ul>
@ -413,7 +413,7 @@ In [14]: for row in result.fetchone():
</ul>
</li>
</ul>
<h2 id="20190107">2019-01-07</h2>
<h2 id="2019-01-07">2019-01-07</h2>
<ul>
<li>I built a clean DSpace 6.3 installation from the upstream <code>dspace-6.3</code> tag and the issue with the XMLUI/JSPUI login is still there with Tomcat 8.5.37
<ul>
@ -423,7 +423,7 @@ In [14]: for row in result.fetchone():
</ul>
</li>
</ul>
<h2 id="20190108">2019-01-08</h2>
<h2 id="2019-01-08">2019-01-08</h2>
<ul>
<li>Tim Donohue responded to my thread about the cookies on the dspace-tech mailing list
<ul>
@ -433,7 +433,7 @@ In [14]: for row in result.fetchone():
</ul>
</li>
</ul>
<h2 id="20190111">2019-01-11</h2>
<h2 id="2019-01-11">2019-01-11</h2>
<ul>
<li>Tezira wrote to say she has stopped receiving the <code>DSpace Submission Approved and Archived</code> emails from CGSpace as of January 2nd
<ul>
@ -442,11 +442,11 @@ In [14]: for row in result.fetchone():
</ul>
</li>
</ul>
<h2 id="20190114">2019-01-14</h2>
<h2 id="2019-01-14">2019-01-14</h2>
<ul>
<li>Day one of CGSpace AReS meeting in Amman</li>
</ul>
<h2 id="20190115">2019-01-15</h2>
<h2 id="2019-01-15">2019-01-15</h2>
<ul>
<li>Day two of CGSpace AReS meeting in Amman
<ul>
@ -477,7 +477,7 @@ In [14]: for row in result.fetchone():
1211 35.237.175.180
1830 66.249.64.155
2482 45.5.186.2
</code></pre><h2 id="20190116">2019-01-16</h2>
</code></pre><h2 id="2019-01-16">2019-01-16</h2>
<ul>
<li>Day three of CGSpace AReS meeting in Amman
<ul>
@ -719,7 +719,7 @@ Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed
real 0m17.161s
user 0m16.205s
sys 0m2.396s
</code></pre><h2 id="20190117">2019-01-17</h2>
</code></pre><h2 id="2019-01-17">2019-01-17</h2>
<ul>
<li>Send reminder to Atmire about purchasing the <a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=657">MQM module</a></li>
<li>Trying to decide the solid action points for CGSpace on the CG Core 2.0 metadata&hellip;</li>
@ -758,7 +758,7 @@ sys 0m2.396s
</ul>
</li>
</ul>
<h2 id="20190119">2019-01-19</h2>
<h2 id="2019-01-19">2019-01-19</h2>
<ul>
<li>
<p>There's no official set of Dublin Core qualifiers so I can't tell if things like <code>dc.contributor.author</code> that are used by DSpace are official</p>
@ -774,7 +774,7 @@ sys 0m2.396s
<p>These terms conform with the DCMI Abstract Model and may be used in DCMI application profiles. DCMI endorses their use with Dublin Core elements as indicated.</p>
</li>
</ul>
<h2 id="20190120">2019-01-20</h2>
<h2 id="2019-01-20">2019-01-20</h2>
<ul>
<li>That's weird, I logged into DSpace Test (linode19) and it says it has been up for 213 days:</li>
</ul>
@ -790,7 +790,7 @@ sys 0m2.396s
<li>The query currently shows 3023 items, but a <a href="https://cgspace.cgiar.org/discover?filtertype_1=crpsubject&amp;filter_relational_operator_1=equals&amp;filter_1=Livestock&amp;submit_apply_filter=&amp;query=">Discovery search for Livestock CRP only returns 858 items</a></li>
<li>That query seems to return items tagged with <code>Livestock and Fish</code> CRP as well&hellip; hmm.</li>
</ul>
<h2 id="20190121">2019-01-21</h2>
<h2 id="2019-01-21">2019-01-21</h2>
<ul>
<li>Investigating running Tomcat 7 on Ubuntu 18.04 with the tarball and a custom systemd package instead of waiting for our DSpace to get compatible with Ubuntu 18.04's Tomcat 8.5</li>
<li>I could either run with a simple <code>tomcat7.service</code> like this:</li>
@ -909,7 +909,7 @@ $ http 'http://localhost:3000/solr/statistics/select?&amp;shards=localhost:8081/
&lt;result name=&quot;response&quot; numFound=&quot;275&quot; start=&quot;0&quot; maxScore=&quot;12.205825&quot;&gt;
$ http 'http://localhost:8081/solr/statistics/select?indent=on&amp;rows=0&amp;q=type:2+id:11576&amp;fq=isBot:false&amp;fq=statistics_type:view&amp;shards=localhost:8081/solr/statistics-2018' | grep numFound
&lt;result name=&quot;response&quot; numFound=&quot;241&quot; start=&quot;0&quot; maxScore=&quot;12.205825&quot;&gt;
</code></pre><h2 id="20190122">2019-01-22</h2>
</code></pre><h2 id="2019-01-22">2019-01-22</h2>
<ul>
<li>Release <a href="https://github.com/ilri/dspace-statistics-api/releases/tag/v0.9.0">version 0.9.0 of the dspace-statistics-api</a> to address the issue of querying multiple Solr statistics shards</li>
<li>I deployed it on DSpace Test (linode19) and restarted the indexer and now it shows all the stats from 2018 as well (756 pages of views, intead of 6)</li>
@ -937,7 +937,7 @@ $ http 'http://localhost:8081/solr/statistics/select?indent=on&amp;rows=0&amp;q=
<li>Another interesting one is 154.113.73.30, which is apparently at IITA Nigeria and uses the user agent:</li>
</ul>
<pre><code>Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
</code></pre><h2 id="20190123">2019-01-23</h2>
</code></pre><h2 id="2019-01-23">2019-01-23</h2>
<ul>
<li>Peter noticed that some goo.gl links in our tweets from Feedburner are broken, for example this one from last week:</li>
</ul>
@ -1019,7 +1019,7 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 /home/cgspace.cgiar.org/bin/dspace fi
<li>I think this Launchpad discussion is relevant: <a href="https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1806517">https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1806517</a></li>
<li>As well as the original Ghostscript bug report: <a href="https://bugs.ghostscript.com/show_bug.cgi?id=699815">https://bugs.ghostscript.com/show_bug.cgi?id=699815</a></li>
</ul>
<h2 id="20190124">2019-01-24</h2>
<h2 id="2019-01-24">2019-01-24</h2>
<ul>
<li>I noticed Ubuntu's Ghostscript 9.26 works on some troublesome PDFs where Arch's Ghostscript 9.26 doesn't, so the fix for the first/last page crash is not the patch I found yesterday</li>
<li>Ubuntu's Ghostscript uses another <a href="http://git.ghostscript.com/?p=ghostpdl.git;h=fae21f1668d2b44b18b84cf0923a1d5f3008a696">patch from Ghostscript git</a> (<a href="https://bugs.ghostscript.com/show_bug.cgi?id=700315">upstream bug report</a>)</li>
@ -1078,7 +1078,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
</li>
<li>I sent a message titled &ldquo;<a href="https://groups.google.com/forum/#!topic/dspace-tech/phV_t51TGuE">DC, QDC, and DCTERMS: reviewing our metadata practices</a>&rdquo; to the dspace-tech mailing list to ask about some of this</li>
</ul>
<h2 id="20190125">2019-01-25</h2>
<h2 id="2019-01-25">2019-01-25</h2>
<ul>
<li>A little bit more work on getting Tomcat to run from a tarball on our <a href="https://github.com/ilri/rmg-ansible-public">Ansible infrastructure playbooks</a>
<ul>
@ -1090,7 +1090,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
</ul>
</li>
</ul>
<h2 id="20190127">2019-01-27</h2>
<h2 id="2019-01-27">2019-01-27</h2>
<ul>
<li>Linode sent an email that the server was using a lot of CPU this morning, and these were the top IPs in the web server logs at the time:</li>
</ul>
@ -1113,7 +1113,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
</ul>
</li>
</ul>
<h2 id="20190128">2019-01-28</h2>
<h2 id="2019-01-28">2019-01-28</h2>
<ul>
<li>Udana from WLE asked me about the interaction between their publication website and their items on CGSpace
<ul>
@ -1161,7 +1161,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
<li><code>199.47.87.140</code> and <code>199.47.87.141</code> is TurnItIn with the following user agent:</li>
</ul>
<pre><code>TurnitinBot (https://turnitin.com/robot/crawlerinfo.html)
</code></pre><h2 id="20190129">2019-01-29</h2>
</code></pre><h2 id="2019-01-29">2019-01-29</h2>
<ul>
<li>Linode sent an alert about CGSpace (linode18) CPU usage this morning, here are the top IPs in the web server logs just before, during, and after the alert:</li>
</ul>
@ -1186,7 +1186,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
</ul>
</li>
</ul>
<h2 id="20190130">2019-01-30</h2>
<h2 id="2019-01-30">2019-01-30</h2>
<ul>
<li>Got another alert from Linode about CGSpace (linode18) this morning, here are the top IPs before, during, and after the alert:</li>
</ul>
@ -1204,7 +1204,7 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
</code></pre><ul>
<li>I might need to adjust the threshold again, because the load average this morning was 296% and the activity looks pretty normal (as always recently)</li>
</ul>
<h2 id="20190131">2019-01-31</h2>
<h2 id="2019-01-31">2019-01-31</h2>
<ul>
<li>Linode sent alerts about CGSpace (linode18) last night and this morning, here are the top IPs before, during, and after those times:</li>
</ul>