Add notes for 2017-11-04

content/post/2017-11.md

@@ -47,3 +47,46 @@ COPY 54701
 
 - Atmire got back to us to say that they estimate it will take two days of labor to implement the change to Listings and Reports
 - I said I'd ask Abenet if she wants that feature
+
+## 2017-11-04
+
+- I finished looking through Sisay's CIAT records for the "Alianzas de Aprendizaje" data
+- I corrected about half of the authors to standardize them
+- Linode emailed this morning to say that the CPU usage was high again, this time at 6:14AM
+- It's the first time in a few days that this has happened
+- I had a look to see what was going on, but it isn't the CORE bot:
+
+```
+# awk '{print $1}' /var/log/nginx/access.log | sort -n | uniq -c | sort -h | tail
+    306 68.180.229.31
+    323 61.148.244.116
+    414 66.249.66.91
+    507 40.77.167.16
+    618 157.55.39.161
+    652 207.46.13.103
+    666 157.55.39.254
+   1173 104.196.152.243
+   1737 66.249.66.90
+  23101 138.201.52.218
+```
+
+- 138.201.52.218 is from some Hetzner server, and I see it making 40,000 requests yesterday too, but none before that:
+
+```
+# zgrep -c 138.201.52.218 /var/log/nginx/access.log*
+/var/log/nginx/access.log:24403
+/var/log/nginx/access.log.1:45958
+/var/log/nginx/access.log.2.gz:0
+/var/log/nginx/access.log.3.gz:0
+/var/log/nginx/access.log.4.gz:0
+/var/log/nginx/access.log.5.gz:0
+/var/log/nginx/access.log.6.gz:0
+```
+
+- It's clearly a bot as it's making tens of thousands of requests, but it's using a "normal" user agent:
+
+```
+Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
+```
+
+- For now I don't know what this user is!

public/2017-11/index.html

@@ -38,7 +38,7 @@ COPY 54701
 
 <meta property="article:published_time" content="2017-11-02T09:37:54&#43;02:00"/>
 
-<meta property="article:modified_time" content="2017-11-02T17:30:54&#43;02:00"/>
+<meta property="article:modified_time" content="2017-11-03T18:15:40&#43;02:00"/>
 
 
 
@@ -86,9 +86,9 @@ COPY 54701
   "@type": "BlogPosting",
   "headline": "November, 2017",
   "url": "https://alanorth.github.io/cgspace-notes/2017-11/",
-  "wordCount": "297",
+  "wordCount": "472",
   "datePublished": "2017-11-02T09:37:54&#43;02:00",
-  "dateModified": "2017-11-02T17:30:54&#43;02:00",
+  "dateModified": "2017-11-03T18:15:40&#43;02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -203,6 +203,54 @@ COPY 54701
 <li>I said I&rsquo;d ask Abenet if she wants that feature</li>
 </ul>
 
+<h2 id="2017-11-04">2017-11-04</h2>
+
+<ul>
+<li>I finished looking through Sisay&rsquo;s CIAT records for the &ldquo;Alianzas de Aprendizaje&rdquo; data</li>
+<li>I corrected about half of the authors to standardize them</li>
+<li>Linode emailed this morning to say that the CPU usage was high again, this time at 6:14AM</li>
+<li>It&rsquo;s the first time in a few days that this has happened</li>
+<li>I had a look to see what was going on, but it isn&rsquo;t the CORE bot:</li>
+</ul>
+
+<pre><code># awk '{print $1}' /var/log/nginx/access.log | sort -n | uniq -c | sort -h | tail
+    306 68.180.229.31
+    323 61.148.244.116
+    414 66.249.66.91
+    507 40.77.167.16
+    618 157.55.39.161
+    652 207.46.13.103
+    666 157.55.39.254
+   1173 104.196.152.243
+   1737 66.249.66.90
+  23101 138.201.52.218
+</code></pre>
+
+<ul>
+<li>138.201.52.218 is from some Hetzner server, and I see it making 40,000 requests yesterday too, but none before that:</li>
+</ul>
+
+<pre><code># zgrep -c 138.201.52.218 /var/log/nginx/access.log*
+/var/log/nginx/access.log:24403
+/var/log/nginx/access.log.1:45958
+/var/log/nginx/access.log.2.gz:0
+/var/log/nginx/access.log.3.gz:0
+/var/log/nginx/access.log.4.gz:0
+/var/log/nginx/access.log.5.gz:0
+/var/log/nginx/access.log.6.gz:0
+</code></pre>
+
+<ul>
+<li>It&rsquo;s clearly a bot as it&rsquo;s making tens of thousands of requests, but it&rsquo;s using a &ldquo;normal&rdquo; user agent:</li>
+</ul>
+
+<pre><code>Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36
+</code></pre>
+
+<ul>
+<li>For now I don&rsquo;t know what this user is!</li>
+</ul>
+
   
 
   

public/sitemap.xml

@@ -4,7 +4,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2017-11/</loc>
-    <lastmod>2017-11-02T17:30:54+02:00</lastmod>
+    <lastmod>2017-11-03T18:15:40+02:00</lastmod>
   </url>
   
   <url>
@@ -134,7 +134,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2017-11-02T17:30:54+02:00</lastmod>
+    <lastmod>2017-11-03T18:15:40+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -145,7 +145,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2017-11-02T17:30:54+02:00</lastmod>
+    <lastmod>2017-11-03T18:15:40+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -157,13 +157,13 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/post/</loc>
-    <lastmod>2017-11-02T17:30:54+02:00</lastmod>
+    <lastmod>2017-11-03T18:15:40+02:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2017-11-02T17:30:54+02:00</lastmod>
+    <lastmod>2017-11-03T18:15:40+02:00</lastmod>
     <priority>0</priority>
   </url>