Add notes for 2018-02-22

This commit is contained in:
Alan Orth 2018-02-22 18:10:11 +02:00
parent 1cc5208bc1
commit def64b0799
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 120 additions and 8 deletions

View File

@ -756,3 +756,56 @@ $ cat orcid-test-values.txt
- I told them that it's their JavaScript that is fucked up - I told them that it's their JavaScript that is fucked up
- Remove CPWF project number and Humidtropics subject from submission form ([#3](https://github.com/alanorth/DSpace/pull/3)) - Remove CPWF project number and Humidtropics subject from submission form ([#3](https://github.com/alanorth/DSpace/pull/3))
- I accidentally merged it into my own repository, oops - I accidentally merged it into my own repository, oops
## 2018-02-22
- CGSpace was apparently down today around 13:00 server time and I didn't get any emails on my phone, but saw them later on the computer
- It looks like Sisay restarted Tomcat because I was offline
- There was absolutely nothing interesting going on at 13:00 on the server, WTF?
```
# cat /var/log/nginx/*.log | grep -E "22/Feb/2018:13" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
55 192.99.39.235
60 207.46.13.26
62 40.77.167.38
65 207.46.13.23
103 41.57.108.208
120 104.196.152.243
133 104.154.216.0
145 68.180.228.117
159 54.92.197.82
231 5.9.6.51
```
- Otherwise there was pretty normal traffic the rest of the day:
```
# zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "22/Feb/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
839 216.244.66.245
1074 68.180.228.117
1114 157.55.39.100
1162 207.46.13.26
1178 207.46.13.23
2749 104.196.152.243
3109 50.116.102.77
4199 70.32.83.92
5208 5.9.6.51
8686 45.5.184.196
```
- While looking at the logs I see some new bot:
```
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
```
- It seems to re-use its user agent but makes tons of useless requests and I wonder if I should add ".*spider.*" to the Tomcat Crawler Session Manager valve?
- This is definitely not good:
```
# grep -c 'org.apache.tomcat.jdbc.pool.ConnectionPool abandon' /var/log/tomcat7/catalina.out
729
```
- I think the `removeAbandonedTimeout` might still be too low (I increased it from 60 to 90 seconds last week)
- Perhaps something more like a few minutes is better?

View File

@ -23,7 +23,7 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl
<meta property="article:published_time" content="2018-02-01T16:28:54&#43;02:00"/> <meta property="article:published_time" content="2018-02-01T16:28:54&#43;02:00"/>
<meta property="article:modified_time" content="2018-02-20T14:47:28&#43;02:00"/> <meta property="article:modified_time" content="2018-02-20T15:32:50&#43;02:00"/>
@ -57,9 +57,9 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu&rsquo;s munin-pl
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "February, 2018", "headline": "February, 2018",
"url": "https://alanorth.github.io/cgspace-notes/2018-02/", "url": "https://alanorth.github.io/cgspace-notes/2018-02/",
"wordCount": "4708", "wordCount": "4946",
"datePublished": "2018-02-01T16:28:54&#43;02:00", "datePublished": "2018-02-01T16:28:54&#43;02:00",
"dateModified": "2018-02-20T14:47:28&#43;02:00", "dateModified": "2018-02-20T15:32:50&#43;02:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -972,6 +972,65 @@ TypeError: 'NoneType' object is not subscriptable
<li>I accidentally merged it into my own repository, oops</li> <li>I accidentally merged it into my own repository, oops</li>
</ul> </ul>
<h2 id="2018-02-22">2018-02-22</h2>
<ul>
<li>CGSpace was apparently down today around 13:00 server time and I didn&rsquo;t get any emails on my phone, but saw them later on the computer</li>
<li>It looks like Sisay restarted Tomcat because I was offline</li>
<li>There was absolutely nothing interesting going on at 13:00 on the server, WTF?</li>
</ul>
<pre><code># cat /var/log/nginx/*.log | grep -E &quot;22/Feb/2018:13&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
55 192.99.39.235
60 207.46.13.26
62 40.77.167.38
65 207.46.13.23
103 41.57.108.208
120 104.196.152.243
133 104.154.216.0
145 68.180.228.117
159 54.92.197.82
231 5.9.6.51
</code></pre>
<ul>
<li>Otherwise there was pretty normal traffic the rest of the day:</li>
</ul>
<pre><code># zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E &quot;22/Feb/2018&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
839 216.244.66.245
1074 68.180.228.117
1114 157.55.39.100
1162 207.46.13.26
1178 207.46.13.23
2749 104.196.152.243
3109 50.116.102.77
4199 70.32.83.92
5208 5.9.6.51
8686 45.5.184.196
</code></pre>
<ul>
<li>While looking at the logs I see some new bot:</li>
</ul>
<pre><code>Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.4.2661.102 Safari/537.36; 360Spider
</code></pre>
<ul>
<li>It seems to re-use its user agent but makes tons of useless requests and I wonder if I should add &ldquo;.<em>spider.</em>&rdquo; to the Tomcat Crawler Session Manager valve?</li>
<li>This is definitely not good:</li>
</ul>
<pre><code># grep -c 'org.apache.tomcat.jdbc.pool.ConnectionPool abandon' /var/log/tomcat7/catalina.out
729
</code></pre>
<ul>
<li>I think the <code>removeAbandonedTimeout</code> might still be too low (I increased it from 60 to 90 seconds last week)</li>
<li>Perhaps something more like a few minutes is better?</li>
</ul>

View File

@ -4,7 +4,7 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/2018-02/</loc> <loc>https://alanorth.github.io/cgspace-notes/2018-02/</loc>
<lastmod>2018-02-20T14:47:28+02:00</lastmod> <lastmod>2018-02-20T15:32:50+02:00</lastmod>
</url> </url>
<url> <url>
@ -149,7 +149,7 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2018-02-20T14:47:28+02:00</lastmod> <lastmod>2018-02-20T15:32:50+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
@ -160,7 +160,7 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2018-02-20T14:47:28+02:00</lastmod> <lastmod>2018-02-20T15:32:50+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
@ -172,13 +172,13 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/post/</loc> <loc>https://alanorth.github.io/cgspace-notes/post/</loc>
<lastmod>2018-02-20T14:47:28+02:00</lastmod> <lastmod>2018-02-20T15:32:50+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2018-02-20T14:47:28+02:00</lastmod> <lastmod>2018-02-20T15:32:50+02:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>