alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-17 20:27:05 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2017-11-09

Browse Source
This commit is contained in:
Alan Orth 2017-11-09 17:41:14 +02:00
parent bb4b63bd96
commit 25cef07639
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 64 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
content/post/2017-11.md
View File

@ -476,3 +476,29 @@ proxy_set_header User-Agent $ua;
- I will whip up something in nginx later
- Run system updates on CGSpace and reboot the server
- Re-deploy latest `5_x-prod` branch on CGSpace and DSpace Test (includes the clickable thumbnails, CCAFS phase II project tags, and updated news text)
## 2017-11-09
- Awesome, it seems my bot mapping stuff in nginx actually reduced the number of Tomcat sessions used by the CIAT scraper today, total requests and unique sessions:
```
# cat /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep '09/Nov/2017' | grep -c 104.196.152.243
5769
$ grep 104.196.152.243 dspace.log.2017-11-09 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
223
```
- Versus the same stats for yesterday and the day before:
```
# zcat -f -- /var/log/nginx/access.log.1 /var/log/nginx/access.log.2.gz | grep '08/Nov/2017' | grep -c 104.196.152.243
10216
$ grep 104.196.152.243 dspace.log.2017-11-08 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
2592
# zcat -f -- /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz | grep '07/Nov/2017' | grep -c 104.196.152.243
8120
$ grep 104.196.152.243 dspace.log.2017-11-07 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
3506
```
- The number of total requests vary by a few thousand, but the number of sessions is over *ten times less*!

36
public/2017-11/index.html
View File

@ -38,7 +38,7 @@ COPY 54701
<meta property="article:published_time" content="2017-11-02T09:37:54&#43;02:00"/>
<meta property="article:modified_time" content="2017-11-08T22:26:37&#43;02:00"/>
<meta property="article:modified_time" content="2017-11-08T22:36:15&#43;02:00"/>
@ -86,9 +86,9 @@ COPY 54701
"@type": "BlogPosting",
"headline": "November, 2017",
"url": "https://alanorth.github.io/cgspace-notes/2017-11/",
"wordCount": "2725",
"wordCount": "2877",
"datePublished": "2017-11-02T09:37:54&#43;02:00",
"dateModified": "2017-11-08T22:26:37&#43;02:00",
"dateModified": "2017-11-08T22:36:15&#43;02:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -678,6 +678,36 @@ proxy_set_header User-Agent $ua;
<li>Re-deploy latest <code>5_x-prod</code> branch on CGSpace and DSpace Test (includes the clickable thumbnails, CCAFS phase II project tags, and updated news text)</li>
</ul>
<h2 id="2017-11-09">2017-11-09</h2>
<ul>
<li>Awesome, it seems my bot mapping stuff in nginx actually reduced the number of Tomcat sessions used by the CIAT scraper today, total requests and unique sessions:</li>
</ul>
<pre><code># cat /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep '09/Nov/2017' | grep -c 104.196.152.243
5769
$ grep 104.196.152.243 dspace.log.2017-11-09 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
223
</code></pre>
<ul>
<li>Versus the same stats for yesterday and the day before:</li>
</ul>
<pre><code># zcat -f -- /var/log/nginx/access.log.1 /var/log/nginx/access.log.2.gz | grep '08/Nov/2017' | grep -c 104.196.152.243
10216
$ grep 104.196.152.243 dspace.log.2017-11-08 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
2592
# zcat -f -- /var/log/nginx/access.log.2.gz /var/log/nginx/access.log.3.gz | grep '07/Nov/2017' | grep -c 104.196.152.243
8120
$ grep 104.196.152.243 dspace.log.2017-11-07 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | uniq | wc -l
3506
</code></pre>
<ul>
<li>The number of total requests vary by a few thousand, but the number of sessions is over <em>ten times less</em>!</li>
</ul>

10
public/sitemap.xml
View File

@ -4,7 +4,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/2017-11/</loc>
<lastmod>2017-11-08T22:26:37+02:00</lastmod>
<lastmod>2017-11-08T22:36:15+02:00</lastmod>
</url>
<url>
@ -134,7 +134,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2017-11-08T22:26:37+02:00</lastmod>
<lastmod>2017-11-08T22:36:15+02:00</lastmod>
<priority>0</priority>
</url>
@ -145,7 +145,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2017-11-08T22:26:37+02:00</lastmod>
<lastmod>2017-11-08T22:36:15+02:00</lastmod>
<priority>0</priority>
</url>
@ -157,13 +157,13 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/post/</loc>
<lastmod>2017-11-08T22:26:37+02:00</lastmod>
<lastmod>2017-11-08T22:36:15+02:00</lastmod>
<priority>0</priority>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2017-11-08T22:26:37+02:00</lastmod>
<lastmod>2017-11-08T22:36:15+02:00</lastmod>
<priority>0</priority>
</url>