Add notes for 2019-01-24

This commit is contained in:
Alan Orth 2019-01-24 10:59:03 +02:00
parent 93126c8157
commit 0a112d16bf
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
4 changed files with 136 additions and 14 deletions

View File

@ -835,4 +835,62 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 /home/cgspace.cgiar.org/bin/dspace fi
- I think this Launchpad discussion is relevant: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1806517 - I think this Launchpad discussion is relevant: https://bugs.launchpad.net/ubuntu/+source/ghostscript/+bug/1806517
- As well as the original Ghostscript bug report: https://bugs.ghostscript.com/show_bug.cgi?id=699815 - As well as the original Ghostscript bug report: https://bugs.ghostscript.com/show_bug.cgi?id=699815
## 2019-01-24
- I noticed Ubuntu's Ghostscript 9.26 works on some troublesome PDFs where Arch's Ghostscript 9.26 doesn't, so the fix for the first/last page crash is not the patch I found yesterday
- Ubuntu's Ghostscript uses another [patch from Ghostscript git](http://git.ghostscript.com/?p=ghostpdl.git;h=fae21f1668d2b44b18b84cf0923a1d5f3008a696) ([upstream bug report](https://bugs.ghostscript.com/show_bug.cgi?id=700315))
- I re-compiled Arch's ghostscript with the patch and then I was able to generate a thumbnail from one of the [troublesome PDFs](https://cgspace.cgiar.org/handle/10568/98390)
- Before and after:
```
$ identify Food\ safety\ Kenya\ fruits.pdf\[0\]
zsh: abort (core dumped) identify Food\ safety\ Kenya\ fruits.pdf\[0\]
$ identify Food\ safety\ Kenya\ fruits.pdf\[0\]
Food safety Kenya fruits.pdf[0]=>Food safety Kenya fruits.pdf PDF 612x792 612x792+0+0 16-bit sRGB 64626B 0.000u 0:00.000
identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/1747.
```
- I reported it to the Arch Linux bug tracker ([61513](https://bugs.archlinux.org/task/61513))
- I told Atmire to go ahead with the Metadata Quality Module addition based on our `5_x-dev` branch ([657](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=657))
- Linode sent alerts last night to say that CGSpace (linode18) was using high CPU last night, here are the top ten IPs from the nginx logs around that time:
```
# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "23/Jan/2019:(18|19|20)" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
305 3.81.136.184
306 3.83.14.11
306 52.54.252.47
325 54.221.57.180
378 66.249.64.157
424 54.70.40.11
497 47.29.247.74
783 35.237.175.180
1108 66.249.64.155
2378 45.5.186.2
```
- 45.5.186.2 is CIAT and 66.249.64.155 is Google... hmmm.
- Linode sent another alert this morning, here are the top ten IPs active during that time:
```
# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "24/Jan/2019:0(4|5|6)" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
360 3.89.134.93
362 34.230.15.139
366 100.24.48.177
369 18.212.208.240
377 3.81.136.184
404 54.221.57.180
506 66.249.64.155
4642 70.32.83.92
4643 205.186.128.185
8593 45.5.186.2
```
- Just double checking what CIAT is doing, they are mainly hitting the REST API:
```
# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "24/Jan/2019:" | grep 45.5.186.2 | grep -Eo "GET /(handle|bitstream|rest|oai)/" | sort | uniq -c | sort -n
```
- CIAT's community currently has 12,000 items in it so this is normal
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -27,7 +27,7 @@ I don&rsquo;t see anything interesting in the web server logs around that time t
" /> " />
<meta property="og:type" content="article" /> <meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-01/" /><meta property="article:published_time" content="2019-01-02T09:48:30&#43;02:00"/> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-01/" /><meta property="article:published_time" content="2019-01-02T09:48:30&#43;02:00"/>
<meta property="article:modified_time" content="2019-01-23T17:27:09&#43;02:00"/> <meta property="article:modified_time" content="2019-01-23T18:21:06&#43;02:00"/>
<meta name="twitter:card" content="summary"/> <meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="January, 2019"/> <meta name="twitter:title" content="January, 2019"/>
@ -60,9 +60,9 @@ I don&rsquo;t see anything interesting in the web server logs around that time t
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "January, 2019", "headline": "January, 2019",
"url": "https://alanorth.github.io/cgspace-notes/2019-01/", "url": "https://alanorth.github.io/cgspace-notes/2019-01/",
"wordCount": "4210", "wordCount": "4540",
"datePublished": "2019-01-02T09:48:30&#43;02:00", "datePublished": "2019-01-02T09:48:30&#43;02:00",
"dateModified": "2019-01-23T17:27:09&#43;02:00", "dateModified": "2019-01-23T18:21:06&#43;02:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -1122,6 +1122,70 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 /home/cgspace.cgiar.org/bin/dspace fi
<li>As well as the original Ghostscript bug report: <a href="https://bugs.ghostscript.com/show_bug.cgi?id=699815">https://bugs.ghostscript.com/show_bug.cgi?id=699815</a></li> <li>As well as the original Ghostscript bug report: <a href="https://bugs.ghostscript.com/show_bug.cgi?id=699815">https://bugs.ghostscript.com/show_bug.cgi?id=699815</a></li>
</ul> </ul>
<h2 id="2019-01-24">2019-01-24</h2>
<ul>
<li>I noticed Ubuntu&rsquo;s Ghostscript 9.26 works on some troublesome PDFs where Arch&rsquo;s Ghostscript 9.26 doesn&rsquo;t, so the fix for the first/last page crash is not the patch I found yesterday</li>
<li>Ubuntu&rsquo;s Ghostscript uses another <a href="http://git.ghostscript.com/?p=ghostpdl.git;h=fae21f1668d2b44b18b84cf0923a1d5f3008a696">patch from Ghostscript git</a> (<a href="https://bugs.ghostscript.com/show_bug.cgi?id=700315">upstream bug report</a>)</li>
<li>I re-compiled Arch&rsquo;s ghostscript with the patch and then I was able to generate a thumbnail from one of the <a href="https://cgspace.cgiar.org/handle/10568/98390">troublesome PDFs</a></li>
<li>Before and after:</li>
</ul>
<pre><code>$ identify Food\ safety\ Kenya\ fruits.pdf\[0\]
zsh: abort (core dumped) identify Food\ safety\ Kenya\ fruits.pdf\[0\]
$ identify Food\ safety\ Kenya\ fruits.pdf\[0\]
Food safety Kenya fruits.pdf[0]=&gt;Food safety Kenya fruits.pdf PDF 612x792 612x792+0+0 16-bit sRGB 64626B 0.000u 0:00.000
identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/1747.
</code></pre>
<ul>
<li>I reported it to the Arch Linux bug tracker (<a href="https://bugs.archlinux.org/task/61513">61513</a>)</li>
<li>I told Atmire to go ahead with the Metadata Quality Module addition based on our <code>5_x-dev</code> branch (<a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=657">657</a>)</li>
<li>Linode sent alerts last night to say that CGSpace (linode18) was using high CPU last night, here are the top ten IPs from the nginx logs around that time:</li>
</ul>
<pre><code># zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E &quot;23/Jan/2019:(18|19|20)&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
305 3.81.136.184
306 3.83.14.11
306 52.54.252.47
325 54.221.57.180
378 66.249.64.157
424 54.70.40.11
497 47.29.247.74
783 35.237.175.180
1108 66.249.64.155
2378 45.5.186.2
</code></pre>
<ul>
<li>45.5.186.2 is CIAT and 66.249.64.155 is Google&hellip; hmmm.</li>
<li>Linode sent another alert this morning, here are the top ten IPs active during that time:</li>
</ul>
<pre><code># zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E &quot;24/Jan/2019:0(4|5|6)&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
360 3.89.134.93
362 34.230.15.139
366 100.24.48.177
369 18.212.208.240
377 3.81.136.184
404 54.221.57.180
506 66.249.64.155
4642 70.32.83.92
4643 205.186.128.185
8593 45.5.186.2
</code></pre>
<ul>
<li>Just double checking what CIAT is doing, they are mainly hitting the REST API:</li>
</ul>
<pre><code># zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E &quot;24/Jan/2019:&quot; | grep 45.5.186.2 | grep -Eo &quot;GET /(handle|bitstream|rest|oai)/&quot; | sort | uniq -c | sort -n
</code></pre>
<ul>
<li>CIAT&rsquo;s community currently has 12,000 items in it so this is normal</li>
</ul>
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -43,7 +43,7 @@ Disallow: /cgspace-notes/2015-12/
Disallow: /cgspace-notes/2015-11/ Disallow: /cgspace-notes/2015-11/
Disallow: /cgspace-notes/ Disallow: /cgspace-notes/
Disallow: /cgspace-notes/categories/ Disallow: /cgspace-notes/categories/
Disallow: /cgspace-notes/categories/notes/
Disallow: /cgspace-notes/tags/notes/ Disallow: /cgspace-notes/tags/notes/
Disallow: /cgspace-notes/categories/notes/
Disallow: /cgspace-notes/posts/ Disallow: /cgspace-notes/posts/
Disallow: /cgspace-notes/tags/ Disallow: /cgspace-notes/tags/

View File

@ -4,7 +4,7 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/2019-01/</loc> <loc>https://alanorth.github.io/cgspace-notes/2019-01/</loc>
<lastmod>2019-01-23T17:27:09+02:00</lastmod> <lastmod>2019-01-23T18:21:06+02:00</lastmod>
</url> </url>
<url> <url>
@ -204,7 +204,7 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2019-01-23T17:27:09+02:00</lastmod> <lastmod>2019-01-23T18:21:06+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
@ -213,27 +213,27 @@
<priority>0</priority> <priority>0</priority>
</url> </url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2019-01-23T18:21:06+02:00</lastmod>
<priority>0</priority>
</url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2018-03-09T22:10:33+02:00</lastmod> <lastmod>2018-03-09T22:10:33+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2019-01-23T17:27:09+02:00</lastmod>
<priority>0</priority>
</url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc> <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2019-01-23T17:27:09+02:00</lastmod> <lastmod>2019-01-23T18:21:06+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2019-01-23T17:27:09+02:00</lastmod> <lastmod>2019-01-23T18:21:06+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>