alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Update notes for 2019-04-07

Browse Source
This commit is contained in:
Alan Orth
2019-04-07 21:05:52 +03:00
parent c9685770ab
commit 370807e83c
4 changed files with 94 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
docs/2019-04/index.html
View File

@ -38,7 +38,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-02-21-delete-1-region.csv -db dspace
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-04/" />
<meta property="article:published_time" content="2019-04-01T09:00:43&#43;03:00"/>
<meta property="article:modified_time" content="2019-04-07T11:45:34&#43;03:00"/>
<meta property="article:modified_time" content="2019-04-07T18:08:38&#43;03:00"/>
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="April, 2019"/>
@ -81,9 +81,9 @@ $ ./delete-metadata-values.py -i /tmp/2019-02-21-delete-1-region.csv -db dspace
"@type": "BlogPosting",
"headline": "April, 2019",
"url": "https://alanorth.github.io/cgspace-notes/2019-04/",
"wordCount": "1954",
"wordCount": "2174",
"datePublished": "2019-04-01T09:00:43&#43;03:00",
"dateModified": "2019-04-07T11:45:34&#43;03:00",
"dateModified": "2019-04-07T18:08:38&#43;03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -569,6 +569,47 @@ X-XSS-Protection: 1; mode=block
<li>DSpace 5.10 upgraded to use GeoIP2, but we are on 5.8 so I just copied the missing database file from another server because it has been <em>removed</em> from MaxMind&rsquo;s server as of 2018-04-01</li>
<li>Now I made 100 requests and I see them in the Solr statistics&hellip; fuck my life for wasting five hours debugging this</li>
</ul></li>
<li>UptimeRobot said CGSpace went down and up a few times tonight, and my first instict was to check <code>iostat 1 10</code> and I saw that CPU steal is around 10–30 percent right now&hellip;</li>
<li>The load average is super high right now, as I&rsquo;ve noticed the last few times UptimeRobot said that CGSpace went down:</li>
</ul>
<pre><code>$ cat /proc/loadavg
10.70 9.17 8.85 18/633 4198
</code></pre>
<ul>
<li>According to the server logs there is actually not much going on right now:</li>
</ul>
<pre><code># zcat --force /var/log/nginx/{access,library-access}.log /var/log/nginx/{access,library-access}.log.1 | grep -E &quot;07/Apr/2019:(18|19)&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
116 40.77.167.128
128 207.46.13.219
129 167.114.64.100
159 207.46.13.129
179 207.46.13.33
188 2408:8214:7a00:868f:7c1e:e0f3:20c6:c142
195 66.249.79.59
363 40.77.167.21
740 2a01:4f8:140:3192::2
3711 45.5.184.72
# zcat --force /var/log/nginx/{rest,oai}.log /var/log/nginx/{rest,oai}.log.1 | grep -E &quot;07/Apr/2019:(18|19)&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
3 2a02:1206:4576:bae0:95ff:8bbd:a274:6223
3 66.249.79.62
3 66.249.83.196
5 82.145.222.150
6 2a01:4f9:2b:1263::2
6 41.204.190.40
7 35.174.176.49
10 40.77.167.21
11 194.246.119.6
11 66.249.79.59
</code></pre>
<ul>
<li><code>45.5.184.72</code> is CIAT, who I already blocked and am waiting to hear from</li>
<li><code>2a01:4f8:140:3192::2</code> is BLEXbot, which should be handled by the Tomcat Crawler Session Manager Valve</li>
<li><code>2408:8214:7a00:868f:7c1e:e0f3:20c6:c142</code> is some stupid Chinese bot making malicious POST requests</li>
<li>Anyways, it seems that the issue with CGSpace being &ldquo;down&rdquo; is actually because of CPU steal again!!!</li>
</ul>
<!-- vim: set sw=2 ts=2: -->