alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2022-07-17

Browse Source
This commit is contained in:
Alan Orth
2022-07-17 22:45:16 +03:00
parent 05bf1fa02d
commit 6fb5aa2be0
33 changed files with 108 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
docs/2022-07/index.html
View File

@ -19,7 +19,7 @@ Also, the trgm functions I’ve used before are case insensitive, but Levens
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2022-07/" />
<meta property="article:published_time" content="2022-07-02T14:07:36+03:00" />
<meta property="article:modified_time" content="2022-07-12T12:15:17+03:00" />
<meta property="article:modified_time" content="2022-07-14T16:46:24+03:00" />
@ -44,9 +44,9 @@ Also, the trgm functions I&rsquo;ve used before are case insensitive, but Levens
"@type": "BlogPosting",
"headline": "July, 2022",
"url": "https://alanorth.github.io/cgspace-notes/2022-07/",
"wordCount": "1743",
"wordCount": "1959",
"datePublished": "2022-07-02T14:07:36+03:00",
"dateModified": "2022-07-12T12:15:17+03:00",
"dateModified": "2022-07-14T16:46:24+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -445,6 +445,46 @@ Also, the trgm functions I&rsquo;ve used before are case insensitive, but Levens
</li>
<li>Then I was getting more errors starting the backend server in Tomcat, but the issue was that the backend server needs Solr to be up first!</li>
</ul>
<h2 id="2022-07-17">2022-07-17</h2>
<ul>
<li>Start a harvest on AReS around 3:30PM</li>
<li>Later in the evening I see CGSpace was going down and up (not as bad as last Sunday) with around 18.0 load&hellip;</li>
<li>I see very high CPU usage:</li>
</ul>
<p><img src="/cgspace-notes/2022/07/cpu-day2.png" alt="CPU day"></p>
<ul>
<li>But DSpace sessions are normal (not like last weekend):</li>
</ul>
<p><img src="/cgspace-notes/2022/07/jmx_dspace_sessions-week2.png" alt="DSpace sessions week"></p>
<ul>
<li>I see some Hetzner IPs in the top users today, but most of the requests are getting HTTP 503 because of the changes I made last week</li>
<li>I see 137.184.159.211, which is on Digital Ocean, and the DNS is apparently iitawpsite.iita.org
<ul>
<li>I&rsquo;ve seen their user agent before, but I don&rsquo;t think I knew it was IITA: &ldquo;GuzzleHttp/6.3.3 curl/7.84.0 PHP/7.4.30&rdquo;</li>
<li>I already have something in nginx to mark Guzzle as a bot, but interestingly it shows up in Solr as <code>$http_user_agent</code> so there is a logic error in my nginx config</li>
</ul>
</li>
<li>Ouch, the logic error seems to be this:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>geo $ua {
</span></span><span style="display:flex;"><span> default $http_user_agent;
</span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010">
</span></span></span><span style="display:flex;"><span><span style="color:#960050;background-color:#1e0010"></span> include /etc/nginx/bot-networks.conf;
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><ul>
<li>After some testing on DSpace Test I see that this is actually setting the default user agent to a literal <code>$http_user_agent</code></li>
<li>The <a href="http://nginx.org/en/docs/http/ngx_http_map_module.html">nginx map docs</a> say:</li>
</ul>
<blockquote>
<p>The resulting value can contain text, variable (0.9.0), and their combination (1.11.0).</p>
</blockquote>
<ul>
<li>But I can&rsquo;t get it to work, neither for the default value or for matching my IP&hellip;
<ul>
<li>I will have to ask on the nginx mailing list</li>
</ul>
</li>
</ul>
<!-- raw HTML omitted -->