alanorth
/
cgspace-notes
mirror of https://github.com/alanorth/cgspace-notes.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add notes for 2022-09-19

This commit is contained in:
Alan Orth 2022-09-19 15:58:41 +03:00
parent 3b78d2f7e4
commit fbf08b7003
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
29 changed files with 77 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
content/posts/2022-09.md
View File

@ -242,4 +242,25 @@ COMMIT
- Deploy the `org.apache.cocoon.uploads.autosave=false` change on CGSpace
- Start a harvest on AReS
## 2022-09-19
- Deploy the nginx proxy cache for /rest requests on CGSpace
- I had tested this last week on DSpace Test
- By my counts on CGSpace yesterday (Sunday, a busy day for the REST API), we had 5,654 URLs that were requested more than twice, and it tails off after that towards two, three, four, etc:
```console
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 > 1' | wc -l
5654
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 == 2' | wc -l
4710
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 == 3' | wc -l
814
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 == 4' | wc -l
86
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 == 5' | wc -l
39
```
- For now I guess requests that were done two or three times by different clients will be cached and that's a win, and I expect more and more REST API activity soon when initiatives and One CGIAR stuff picks up
<!-- vim: set sw=2 ts=2: -->

28
docs/2022-09/index.html
View File

@ -25,7 +25,7 @@ I also fixed a few bugs and improved the region-matching logic
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-09/" />
<meta property="article:published_time" content="2022-09-01T09:41:36+03:00" />
<meta property="article:modified_time" content="2022-09-16T17:09:32+03:00" />
<meta property="article:modified_time" content="2022-09-18T21:04:01+03:00" />
@ -56,9 +56,9 @@ I also fixed a few bugs and improved the region-matching logic
"@type": "BlogPosting",
"headline": "September, 2022",
"url": "https://alanorth.github.io/cgspace-notes/2022-09/",
"wordCount": "1450",
"wordCount": "1660",
"datePublished": "2022-09-01T09:41:36+03:00",
"dateModified": "2022-09-16T17:09:32+03:00",
"dateModified": "2022-09-18T21:04:01+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -401,6 +401,28 @@ I also fixed a few bugs and improved the region-matching logic
<li>Deploy the <code>org.apache.cocoon.uploads.autosave=false</code> change on CGSpace</li>
<li>Start a harvest on AReS</li>
</ul>
<h2 id="2022-09-19">2022-09-19</h2>
<ul>
<li>Deploy the nginx proxy cache for /rest requests on CGSpace
<ul>
<li>I had tested this last week on DSpace Test</li>
<li>By my counts on CGSpace yesterday (Sunday, a busy day for the REST API), we had 5,654 URLs that were requested more than twice, and it tails off after that towards two, three, four, etc:</li>
</ul>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span># awk <span style="color:#e6db74">&#39;{print $7}&#39;</span> /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk <span style="color:#e6db74">&#39;$1 &gt; 1&#39;</span> | wc -l
</span></span><span style="display:flex;"><span>5654
</span></span><span style="display:flex;"><span># awk <span style="color:#e6db74">&#39;{print $7}&#39;</span> /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk <span style="color:#e6db74">&#39;$1 == 2&#39;</span> | wc -l
</span></span><span style="display:flex;"><span>4710
</span></span><span style="display:flex;"><span># awk <span style="color:#e6db74">&#39;{print $7}&#39;</span> /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk <span style="color:#e6db74">&#39;$1 == 3&#39;</span> | wc -l
</span></span><span style="display:flex;"><span>814
</span></span><span style="display:flex;"><span># awk <span style="color:#e6db74">&#39;{print $7}&#39;</span> /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk <span style="color:#e6db74">&#39;$1 == 4&#39;</span> | wc -l
</span></span><span style="display:flex;"><span>86
</span></span><span style="display:flex;"><span># awk <span style="color:#e6db74">&#39;{print $7}&#39;</span> /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk <span style="color:#e6db74">&#39;$1 == 5&#39;</span> | wc -l
</span></span><span style="display:flex;"><span>39
</span></span></code></pre></div><ul>
<li>For now I guess requests that were done two or three times by different clients will be cached and that&rsquo;s a win, and I expect more and more REST API activity soon when initiatives and One CGIAR stuff picks up</li>
</ul>
<!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/categories/notes/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

2
docs/posts/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2022-09-16T17:09:32+03:00" />
<meta property="og:updated_time" content="2022-09-18T21:04:01+03:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2022-09-16T17:09:32+03:00</lastmod>
<lastmod>2022-09-18T21:04:01+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2022-09-16T17:09:32+03:00</lastmod>
<lastmod>2022-09-18T21:04:01+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2022-09-16T17:09:32+03:00</lastmod>
<lastmod>2022-09-18T21:04:01+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2022-09-16T17:09:32+03:00</lastmod>
<lastmod>2022-09-18T21:04:01+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-09/</loc>
<lastmod>2022-09-16T17:09:32+03:00</lastmod>
<lastmod>2022-09-18T21:04:01+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2022-08/</loc>
<lastmod>2022-08-31T17:37:28+03:00</lastmod>