alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2020-05-07

Browse Source
This commit is contained in:
Alan Orth
2020-05-07 11:45:25 +03:00
parent b48d9c686d
commit ead47c2269
92 changed files with 150 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
docs/2020-05/index.html
View File

@ -18,7 +18,7 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-05/" />
<meta property="article:published_time" content="2020-05-02T09:52:04+03:00" />
<meta property="article:modified_time" content="2020-05-03T16:10:21+03:00" />
<meta property="article:modified_time" content="2020-05-06T16:03:29+03:00" />
<meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="May, 2020"/>
@ -31,7 +31,7 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2
"/>
<meta name="generator" content="Hugo 0.69.2" />
<meta name="generator" content="Hugo 0.70.0" />
@ -41,9 +41,9 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2
"@type": "BlogPosting",
"headline": "May, 2020",
"url": "https://alanorth.github.io/cgspace-notes/2020-05/",
"wordCount": "253",
"wordCount": "432",
"datePublished": "2020-05-02T09:52:04+03:00",
"dateModified": "2020-05-03T16:10:21+03:00",
"dateModified": "2020-05-06T16:03:29+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -155,6 +155,33 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2
</ul>
</li>
</ul>
<h2 id="2020-05-07">2020-05-07</h2>
<ul>
<li>Linode sent an alert that there was high CPU usage on CGSpace (linode18) early this morning
<ul>
<li>I looked at the nginx logs using goaccess and I found a few IPs making lots of requests around then:</li>
</ul>
</li>
</ul>
<pre><code># cat /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E &quot;07/May/2020:(01|03|04)&quot; | goaccess --log-format=COMBINED -
</code></pre><ul>
<li>The two main IPs making requests around then are 188.134.31.88 and 212.34.8.188
<ul>
<li>The first is in Russia and it is hitting mostly XMLUI Discover links using <em>dozens</em> of different user agents, a total of 20,000 requests this week</li>
<li>The second IP is CodeObia testing AReS, a total of 171,000 hits this month</li>
<li>I will purge both of those IPs from the Solr stats using my <code>check-spider-ip-hits.sh</code> script:</li>
</ul>
</li>
</ul>
<pre><code>$ ./check-spider-ip-hits.sh -f /tmp/ips -s statistics -p
Purging 171641 hits from 212.34.8.188 in statistics
Purging 20691 hits from 188.134.31.88 in statistics
Total number of bot hits purged: 192332
</code></pre><ul>
<li>And then I will add 188.134.31.88 to the nginx bad bot list and tell CodeObia to please use a &ldquo;bot&rdquo; user agent</li>
<li>I also changed the nginx config to block requests with blank user agents</li>
</ul>
<!-- raw HTML omitted -->