alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-25 06:24:29 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2024-09-14

Browse Source
This commit is contained in:
Alan Orth 2024-09-14 23:02:16 +03:00
parent a5298945a3
commit 79708bd30c
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
39 changed files with 112 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
content/posts/2024-09.md
View File

@ -53,4 +53,37 @@ $ pm2 status
- I put some of these changes in on DSpace Test and will monitor this week
## 2024-09-10
- Some bot in South Africa made a ton of requests on the API and made the load hit the roof:
```
# grep -E '10/Sep/2024:[10-11]' /var/log/nginx/api-access.log | awk '{print $1}' | sort | uniq -c | sort -h
...
149720 102.182.38.90
```
- They are using several user agents so are obviously a bot:
```
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/111.0
Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:11.0) Gecko/20100101 Firefox/11.0
```
- I added them to the list of bot networks in nginx and the load went down
## 2024-09-11
- Upgrade DSpace 7 Test to Ubuntu 24.04
- I did some minor maintenance to test dspace-statistics-api with Python 3.12
- I tagged version 1.4.4 and released it on GitHub
## 2024-09-14
- Noticed a persistent higher than usual load on CGSpace and checked the server logs
- Found some new data center subnets to block because they were making thousands of requests with normal user agents
- I enabled HTTP/3 in nginx
- I enabled the SSR patch in Angular: https://github.com/DSpace/dspace-angular/issues/3110
<!-- vim: set sw=2 ts=2: -->

41
docs/2024-09/index.html
View File

@ -14,7 +14,7 @@ Upgrade CGSpace to DSpace 7.6.2
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2024-09/" />
<meta property="article:published_time" content="2024-09-01T21:16:00-07:00" />
<meta property="article:modified_time" content="2024-09-01T21:16:00-07:00" />
<meta property="article:modified_time" content="2024-09-09T10:20:09+03:00" />
@ -34,9 +34,9 @@ Upgrade CGSpace to DSpace 7.6.2
"@type": "BlogPosting",
"headline": "September, 2024",
"url": "https://alanorth.github.io/cgspace-notes/2024-09/",
"wordCount": "353",
"wordCount": "523",
"datePublished": "2024-09-01T21:16:00-07:00",
"dateModified": "2024-09-01T21:16:00-07:00",
"dateModified": "2024-09-09T10:20:09+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -157,6 +157,41 @@ Sep 06 00:00:25 server systemd[1]: tomcat9.service: Failed with result &#39;oom-
<ul>
<li>I put some of these changes in on DSpace Test and will monitor this week</li>
</ul>
<h2 id="2024-09-10">2024-09-10</h2>
<ul>
<li>Some bot in South Africa made a ton of requests on the API and made the load hit the roof:</li>
</ul>
<pre tabindex="0"><code># grep -E &#39;10/Sep/2024:[10-11]&#39; /var/log/nginx/api-access.log | awk &#39;{print $1}&#39; | sort | uniq -c | sort -h
...
149720 102.182.38.90
</code></pre><ul>
<li>They are using several user agents so are obviously a bot:</li>
</ul>
<pre tabindex="0"><code>Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/111.0
Mozilla/5.0 (Windows; U; Windows NT 6.1; rv:11.0) Gecko/20100101 Firefox/11.0
</code></pre><ul>
<li>I added them to the list of bot networks in nginx and the load went down</li>
</ul>
<h2 id="2024-09-11">2024-09-11</h2>
<ul>
<li>Upgrade DSpace 7 Test to Ubuntu 24.04</li>
<li>I did some minor maintenance to test dspace-statistics-api with Python 3.12
<ul>
<li>I tagged version 1.4.4 and released it on GitHub</li>
</ul>
</li>
</ul>
<h2 id="2024-09-14">2024-09-14</h2>
<ul>
<li>Noticed a persistent higher than usual load on CGSpace and checked the server logs
<ul>
<li>Found some new data center subnets to block because they were making thousands of requests with normal user agents</li>
</ul>
</li>
<li>I enabled HTTP/3 in nginx</li>
<li>I enabled the SSR patch in Angular: <a href="https://github.com/DSpace/dspace-angular/issues/3110">https://github.com/DSpace/dspace-angular/issues/3110</a></li>
</ul>
<!-- raw HTML omitted -->

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/index.xml
View File

@ -6,7 +6,7 @@
<description>Recent content in Categories on CGSpace Notes</description>
<generator>Hugo</generator>
<language>en-us</language>
<lastBuildDate>Sun, 01 Sep 2024 21:16:00 -0700</lastBuildDate>
<lastBuildDate>Mon, 09 Sep 2024 10:20:09 +0300</lastBuildDate>
<atom:link href="https://alanorth.github.io/cgspace-notes/categories/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Notes</title>

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/index.xml
View File

@ -6,7 +6,7 @@
<description>Recent content in Notes on CGSpace Notes</description>
<generator>Hugo</generator>
<language>en-us</language>
<lastBuildDate>Sun, 01 Sep 2024 21:16:00 -0700</lastBuildDate>
<lastBuildDate>Mon, 09 Sep 2024 10:20:09 +0300</lastBuildDate>
<atom:link href="https://alanorth.github.io/cgspace-notes/categories/notes/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>September, 2024</title>

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/categories/notes/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/index.xml
View File

@ -6,7 +6,7 @@
<description>Recent content on CGSpace Notes</description>
<generator>Hugo</generator>
<language>en-us</language>
<lastBuildDate>Sun, 01 Sep 2024 21:16:00 -0700</lastBuildDate>
<lastBuildDate>Mon, 09 Sep 2024 10:20:09 +0300</lastBuildDate>
<atom:link href="https://alanorth.github.io/cgspace-notes/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>September, 2024</title>

2
docs/page/10/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/11/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/index.xml
View File

@ -6,7 +6,7 @@
<description>Recent content in Posts on CGSpace Notes</description>
<generator>Hugo</generator>
<language>en-us</language>
<lastBuildDate>Sun, 01 Sep 2024 21:16:00 -0700</lastBuildDate>
<lastBuildDate>Mon, 09 Sep 2024 10:20:09 +0300</lastBuildDate>
<atom:link href="https://alanorth.github.io/cgspace-notes/posts/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>September, 2024</title>

2
docs/posts/page/10/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/11/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/8/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

2
docs/posts/page/9/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-09-01T21:16:00-07:00" />
<meta property="og:updated_time" content="2024-09-09T10:20:09+03:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2024-09-01T21:16:00-07:00</lastmod>
<lastmod>2024-09-09T10:20:09+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2024-09-01T21:16:00-07:00</lastmod>
<lastmod>2024-09-09T10:20:09+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2024-09-01T21:16:00-07:00</lastmod>
<lastmod>2024-09-09T10:20:09+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2024-09-01T21:16:00-07:00</lastmod>
<lastmod>2024-09-09T10:20:09+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2024-09/</loc>
<lastmod>2024-09-01T21:16:00-07:00</lastmod>
<lastmod>2024-09-09T10:20:09+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2024-08/</loc>
<lastmod>2024-08-28T11:35:05+03:00</lastmod>