Add notes for 2019-11-08

content/posts/2019-11.md

@@ -235,4 +235,15 @@ $ curl -s 'http://localhost:8081/solr/statistics/select?facet=true&facet.field=d
 
 - That answers Peter's question about why the stats jumped in October...
 
+## 2019-11-08
+
+- I saw a bunch of user agents that have the literal string `User-Agent` in their user agent HTTP header, for example:
+  - `User-Agent: Drupal (+http://drupal.org/)`
+  - `User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31`
+  - `User-Agent:Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0) IKU/7.0.5.9226;IKUCID/IKU;`
+  - `User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)`
+  - `User-Agent:User-Agent:Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB7.5; .NET4.0C)IKU/6.7.6.12189;IKUCID/IKU;IKU/6.7.6.12189;IKUCID/IKU;`
+  - `User-Agent:Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0) IKU/7.0.5.9226;IKUCID/IKU;`
+- I filed [an issue](https://github.com/atmire/COUNTER-Robots/issues/27) on the COUNTER-Robots project to see if they agree to add `User-Agent:` to the list of robot user agents
+
 <!-- vim: set sw=2 ts=2: -->

docs/2019-11/index.html

@@ -34,7 +34,7 @@ Let’s see how many of the REST API requests were for bitstreams (because t
 <meta property="og:type" content="article" />
 <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-11/" />
 <meta property="article:published_time" content="2019-11-04T12:20:30+02:00" />
-<meta property="article:modified_time" content="2019-11-07T18:22:19+02:00" />
+<meta property="article:modified_time" content="2019-11-08T09:27:13+02:00" />
 
 <meta name="twitter:card" content="summary"/>
 <meta name="twitter:title" content="November, 2019"/>
@@ -73,9 +73,9 @@ Let&rsquo;s see how many of the REST API requests were for bitstreams (because t
   "@type": "BlogPosting",
   "headline": "November, 2019",
   "url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-11\/",
-  "wordCount": "1343",
+  "wordCount": "1442",
   "datePublished": "2019-11-04T12:20:30+02:00",
-  "dateModified": "2019-11-07T18:22:19+02:00",
+  "dateModified": "2019-11-08T09:27:13+02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -397,6 +397,22 @@ $ http &ndash;print b &lsquo;<a href="http://localhost:8081/solr/statistics-2018
 <li>That answers Peter&rsquo;s question about why the stats jumped in October&hellip;</li>
 </ul>
 
+<h2 id="2019-11-08">2019-11-08</h2>
+
+<ul>
+<li>I saw a bunch of user agents that have the literal string <code>User-Agent</code> in their user agent HTTP header, for example:
+
+<ul>
+<li><code>User-Agent: Drupal (+http://drupal.org/)</code></li>
+<li><code>User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31</code></li>
+<li><code>User-Agent:Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0) IKU/7.0.5.9226;IKUCID/IKU;</code></li>
+<li><code>User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 360SE)</code></li>
+<li><code>User-Agent:User-Agent:Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB7.5; .NET4.0C)IKU/6.7.6.12189;IKUCID/IKU;IKU/6.7.6.12189;IKUCID/IKU;</code></li>
+<li><code>User-Agent:Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0) IKU/7.0.5.9226;IKUCID/IKU;</code></li>
+</ul></li>
+<li>I filed <a href="https://github.com/atmire/COUNTER-Robots/issues/27">an issue</a> on the COUNTER-Robots project to see if they agree to add <code>User-Agent:</code> to the list of robot user agents</li>
+</ul>
+
 <!-- vim: set sw=2 ts=2: -->
 
   

docs/sitemap.xml

@@ -4,27 +4,27 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
-    <lastmod>2019-11-07T18:22:19+02:00</lastmod>
+    <lastmod>2019-11-08T09:27:13+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2019-11-07T18:22:19+02:00</lastmod>
+    <lastmod>2019-11-08T09:27:13+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
-    <lastmod>2019-11-07T18:22:19+02:00</lastmod>
+    <lastmod>2019-11-08T09:27:13+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2019-11/</loc>
-    <lastmod>2019-11-07T18:22:19+02:00</lastmod>
+    <lastmod>2019-11-08T09:27:13+02:00</lastmod>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2019-11-07T18:22:19+02:00</lastmod>
+    <lastmod>2019-11-08T09:27:13+02:00</lastmod>
   </url>
   
   <url>