Add notes

This commit is contained in:
Alan Orth 2024-01-18 15:59:49 +03:00
parent 20ace46614
commit 57fe0587a4
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
34 changed files with 372 additions and 39 deletions

View File

@ -210,4 +210,166 @@ Time: 240.041 ms
- I think we can move those to a new `cg.identifier.project` if we create one - I think we can move those to a new `cg.identifier.project` if we create one
- The `cg.identifier.cpwfproject` field is similarly sparse, but the CCAFS ones are widely used - The `cg.identifier.cpwfproject` field is similarly sparse, but the CCAFS ones are widely used
## 2024-01-12
- Export a list of affiliations to do some cleanup:
```console
localhost/dspace7= ☘ \COPY (SELECT DISTINCT text_value AS "cg.contributor.affiliation", count(*) FROM metadatavalue WHERE dspace_object_id in (SELECT dspace_object_id FROM item) AND metadata_field_id = 211 GROUP BY "cg.contributor.affiliation" ORDER BY count DESC) to /tmp/2024-01-affiliations.csv WITH CSV HEADER;
COPY 11719
```
- I first did some clustering and editing in OpenRefine, then I'll import those back into CGSpace and then do another export
- Troubleshooting the statistics pages that aren't working on DSpace 7
- On a hunch, I queried for for Solr statistics documents that **did not have an `id` matching the 36-character UUID pattern**:
```console
$ curl 'http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&rows=0'
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"-id:/.{36}/",
"rows":"0"}},
"response":{"numFound":800167,"start":0,"numFoundExact":true,"docs":[]
}}
```
- They seem to come mostly from 2020, 2023, and 2024:
```console
$ curl 'http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&facet.range=time&facet=true&facet.range.start=2010-01-01T00:00:00Z&facet.range.end=NOW&facet.range.gap=%2B1YEAR&rows=0'
{
"responseHeader":{
"status":0,
"QTime":13,
"params":{
"facet.range":"time",
"q":"-id:/.{36}/",
"facet.range.gap":"+1YEAR",
"rows":"0",
"facet":"true",
"facet.range.start":"2010-01-01T00:00:00Z",
"facet.range.end":"NOW"}},
"response":{"numFound":800168,"start":0,"numFoundExact":true,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{},
"facet_ranges":{
"time":{
"counts":[
"2010-01-01T00:00:00Z",0,
"2011-01-01T00:00:00Z",0,
"2012-01-01T00:00:00Z",0,
"2013-01-01T00:00:00Z",0,
"2014-01-01T00:00:00Z",0,
"2015-01-01T00:00:00Z",89,
"2016-01-01T00:00:00Z",11,
"2017-01-01T00:00:00Z",0,
"2018-01-01T00:00:00Z",0,
"2019-01-01T00:00:00Z",0,
"2020-01-01T00:00:00Z",1339,
"2021-01-01T00:00:00Z",0,
"2022-01-01T00:00:00Z",0,
"2023-01-01T00:00:00Z",653736,
"2024-01-01T00:00:00Z",144993],
"gap":"+1YEAR",
"start":"2010-01-01T00:00:00Z",
"end":"2025-01-01T00:00:00Z"}},
"facet_intervals":{},
"facet_heatmaps":{}}}
```
- They seem to come from 2023-08 until now (so way before we migrated to DSpace 7):
```console
$ curl 'http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&facet.range=time&facet=true&facet.range.start=2023-01-01T00:00:00Z&facet.range.end=NOW&facet.range.gap=%2B1MONTH&rows=0'
{
"responseHeader":{
"status":0,
"QTime":196,
"params":{
"facet.range":"time",
"q":"-id:/.{36}/",
"facet.range.gap":"+1MONTH",
"rows":"0",
"facet":"true",
"facet.range.start":"2023-01-01T00:00:00Z",
"facet.range.end":"NOW"}},
"response":{"numFound":800168,"start":0,"numFoundExact":true,"docs":[]
},
"facet_counts":{
"facet_queries":{},
"facet_fields":{},
"facet_ranges":{
"time":{
"counts":[
"2023-01-01T00:00:00Z",1,
"2023-02-01T00:00:00Z",0,
"2023-03-01T00:00:00Z",0,
"2023-04-01T00:00:00Z",0,
"2023-05-01T00:00:00Z",0,
"2023-06-01T00:00:00Z",0,
"2023-07-01T00:00:00Z",0,
"2023-08-01T00:00:00Z",27621,
"2023-09-01T00:00:00Z",59165,
"2023-10-01T00:00:00Z",115338,
"2023-11-01T00:00:00Z",96147,
"2023-12-01T00:00:00Z",355464,
"2024-01-01T00:00:00Z",125429],
"gap":"+1MONTH",
"start":"2023-01-01T00:00:00Z",
"end":"2024-02-01T00:00:00Z"}},
"facet_intervals":{},
"facet_heatmaps":{}}}
```
- I see that we had 31,744 statistic events yesterday, and 799 have no `id`!
- I asked about this on Slack and will file an issue on GitHub if someone else also finds such records
- Several people said they have them, so it's a bug of some sort in DSpace, not our configuration
## 2024-01-13
- Yesterday alone we had 37,000 unique IPs making requests to nginx
- I looked up the ASNs and found 6,000 IPs from this network in Amazon Singapore: 47.128.0.0/14
## 2024-01-15
- Investigating the CSS selector warning that I've seen in PM2 logs:
```console
0|dspace-ui | 1 rules skipped due to selector errors:
0|dspace-ui | .custom-file-input:lang(en)~.custom-file-label -> unmatched pseudo-class :lang
```
- It seems to be a bug in Angular, as this selector comes from Bootstrap 4.6.x and is not invalid
- But that led me to a more interesting issue with `inlineCritical` optimization for styles in Angular SSR that might be responsible for causing high load in the frontend
- See: https://github.com/angular/angular/issues/42098
- See: https://github.com/angular/universal/issues/2106
- See: https://github.com/GoogleChromeLabs/critters/issues/78
- Since the production site was flapping a lot I decided to try disabling inlineCriticalCss
- There have been on and off load issues with the Angular frontend today
- I think I will just block all data center network blocks for now
- In the last week I see almost 200,000 unique IPs:
```console
# zcat -f /var/log/nginx/*access.log /var/log/nginx/*access.log.1 /var/log/nginx/*access.log.2.gz /var/log/nginx/*access.log.3.gz /var/log/nginx/*access.log.4.gz /var/log/nginx/*access.log.5.gz /var/log/nginx/*access.log.6.gz | awk '{print $1}' | sort -u |
tee /tmp/ips.txt | wc -l
196493
```
- Looking these IPs up I see there are 18,000 coming from Comcast, 10,000 from AT&T, 4110 from Charter, 3500 from Cox and dozens of other residential IPs
- I highly doubt these are home users browsing CGSpace... seems super fishy
- Also, over 1,000 IPs from SpaceX Starlink in the last week. RIGHT
- I will temporarily add a few new datacenter ISP network blocks to our rate limit:
- 16509 Amazon-02
- 701 UUNET
- 8075 Microsoft
- 15169 Google
- 14618 Amazon-AES
- 396982 Google Cloud
- The load on the server *immediately* dropped
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -22,7 +22,7 @@ Work on IFPRI ISNAR archive cleanup
<meta property="og:type" content="article" /> <meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2024-01/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2024-01/" />
<meta property="article:published_time" content="2024-01-02T10:08:00+03:00" /> <meta property="article:published_time" content="2024-01-02T10:08:00+03:00" />
<meta property="article:modified_time" content="2024-01-10T08:34:16+03:00" /> <meta property="article:modified_time" content="2024-01-10T17:21:12+03:00" />
@ -50,9 +50,9 @@ Work on IFPRI ISNAR archive cleanup
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "January, 2024", "headline": "January, 2024",
"url": "https://alanorth.github.io/cgspace-notes/2024-01/", "url": "https://alanorth.github.io/cgspace-notes/2024-01/",
"wordCount": "1306", "wordCount": "1847",
"datePublished": "2024-01-02T10:08:00+03:00", "datePublished": "2024-01-02T10:08:00+03:00",
"dateModified": "2024-01-10T08:34:16+03:00", "dateModified": "2024-01-10T17:21:12+03:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -340,6 +340,177 @@ Work on IFPRI ISNAR archive cleanup
<li>I think we can move those to a new <code>cg.identifier.project</code> if we create one</li> <li>I think we can move those to a new <code>cg.identifier.project</code> if we create one</li>
<li>The <code>cg.identifier.cpwfproject</code> field is similarly sparse, but the CCAFS ones are widely used</li> <li>The <code>cg.identifier.cpwfproject</code> field is similarly sparse, but the CCAFS ones are widely used</li>
</ul> </ul>
<h2 id="2024-01-12">2024-01-12</h2>
<ul>
<li>Export a list of affiliations to do some cleanup:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>localhost/dspace7= ☘ \COPY (SELECT DISTINCT text_value AS &#34;cg.contributor.affiliation&#34;, count(*) FROM metadatavalue WHERE dspace_object_id in (SELECT dspace_object_id FROM item) AND metadata_field_id = 211 GROUP BY &#34;cg.contributor.affiliation&#34; ORDER BY count DESC) to /tmp/2024-01-affiliations.csv WITH CSV HEADER;
</span></span><span style="display:flex;"><span>COPY 11719
</span></span></code></pre></div><ul>
<li>I first did some clustering and editing in OpenRefine, then I&rsquo;ll import those back into CGSpace and then do another export</li>
<li>Troubleshooting the statistics pages that aren&rsquo;t working on DSpace 7
<ul>
<li>On a hunch, I queried for for Solr statistics documents that <strong>did not have an <code>id</code> matching the 36-character UUID pattern</strong>:</li>
</ul>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ curl <span style="color:#e6db74">&#39;http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&amp;rows=0&#39;</span>
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> &#34;responseHeader&#34;:{
</span></span><span style="display:flex;"><span> &#34;status&#34;:0,
</span></span><span style="display:flex;"><span> &#34;QTime&#34;:0,
</span></span><span style="display:flex;"><span> &#34;params&#34;:{
</span></span><span style="display:flex;"><span> &#34;q&#34;:&#34;-id:/.{36}/&#34;,
</span></span><span style="display:flex;"><span> &#34;rows&#34;:&#34;0&#34;}},
</span></span><span style="display:flex;"><span> &#34;response&#34;:{&#34;numFound&#34;:800167,&#34;start&#34;:0,&#34;numFoundExact&#34;:true,&#34;docs&#34;:[]
</span></span><span style="display:flex;"><span> }}
</span></span></code></pre></div><ul>
<li>They seem to come mostly from 2020, 2023, and 2024:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ curl <span style="color:#e6db74">&#39;http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&amp;facet.range=time&amp;facet=true&amp;facet.range.start=2010-01-01T00:00:00Z&amp;facet.range.end=NOW&amp;facet.range.gap=%2B1YEAR&amp;rows=0&#39;</span>
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> &#34;responseHeader&#34;:{
</span></span><span style="display:flex;"><span> &#34;status&#34;:0,
</span></span><span style="display:flex;"><span> &#34;QTime&#34;:13,
</span></span><span style="display:flex;"><span> &#34;params&#34;:{
</span></span><span style="display:flex;"><span> &#34;facet.range&#34;:&#34;time&#34;,
</span></span><span style="display:flex;"><span> &#34;q&#34;:&#34;-id:/.{36}/&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.gap&#34;:&#34;+1YEAR&#34;,
</span></span><span style="display:flex;"><span> &#34;rows&#34;:&#34;0&#34;,
</span></span><span style="display:flex;"><span> &#34;facet&#34;:&#34;true&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.start&#34;:&#34;2010-01-01T00:00:00Z&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.end&#34;:&#34;NOW&#34;}},
</span></span><span style="display:flex;"><span> &#34;response&#34;:{&#34;numFound&#34;:800168,&#34;start&#34;:0,&#34;numFoundExact&#34;:true,&#34;docs&#34;:[]
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> &#34;facet_counts&#34;:{
</span></span><span style="display:flex;"><span> &#34;facet_queries&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_fields&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_ranges&#34;:{
</span></span><span style="display:flex;"><span> &#34;time&#34;:{
</span></span><span style="display:flex;"><span> &#34;counts&#34;:[
</span></span><span style="display:flex;"><span> &#34;2010-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2011-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2012-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2013-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2014-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2015-01-01T00:00:00Z&#34;,89,
</span></span><span style="display:flex;"><span> &#34;2016-01-01T00:00:00Z&#34;,11,
</span></span><span style="display:flex;"><span> &#34;2017-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2018-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2019-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2020-01-01T00:00:00Z&#34;,1339,
</span></span><span style="display:flex;"><span> &#34;2021-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2022-01-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-01-01T00:00:00Z&#34;,653736,
</span></span><span style="display:flex;"><span> &#34;2024-01-01T00:00:00Z&#34;,144993],
</span></span><span style="display:flex;"><span> &#34;gap&#34;:&#34;+1YEAR&#34;,
</span></span><span style="display:flex;"><span> &#34;start&#34;:&#34;2010-01-01T00:00:00Z&#34;,
</span></span><span style="display:flex;"><span> &#34;end&#34;:&#34;2025-01-01T00:00:00Z&#34;}},
</span></span><span style="display:flex;"><span> &#34;facet_intervals&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_heatmaps&#34;:{}}}
</span></span></code></pre></div><ul>
<li>They seem to come from 2023-08 until now (so way before we migrated to DSpace 7):</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>$ curl <span style="color:#e6db74">&#39;http://localhost:8983/solr/statistics/select?q=-id%3A%2F.\{36\}%2F&amp;facet.range=time&amp;facet=true&amp;facet.range.start=2023-01-01T00:00:00Z&amp;facet.range.end=NOW&amp;facet.range.gap=%2B1MONTH&amp;rows=0&#39;</span>
</span></span><span style="display:flex;"><span>{
</span></span><span style="display:flex;"><span> &#34;responseHeader&#34;:{
</span></span><span style="display:flex;"><span> &#34;status&#34;:0,
</span></span><span style="display:flex;"><span> &#34;QTime&#34;:196,
</span></span><span style="display:flex;"><span> &#34;params&#34;:{
</span></span><span style="display:flex;"><span> &#34;facet.range&#34;:&#34;time&#34;,
</span></span><span style="display:flex;"><span> &#34;q&#34;:&#34;-id:/.{36}/&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.gap&#34;:&#34;+1MONTH&#34;,
</span></span><span style="display:flex;"><span> &#34;rows&#34;:&#34;0&#34;,
</span></span><span style="display:flex;"><span> &#34;facet&#34;:&#34;true&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.start&#34;:&#34;2023-01-01T00:00:00Z&#34;,
</span></span><span style="display:flex;"><span> &#34;facet.range.end&#34;:&#34;NOW&#34;}},
</span></span><span style="display:flex;"><span> &#34;response&#34;:{&#34;numFound&#34;:800168,&#34;start&#34;:0,&#34;numFoundExact&#34;:true,&#34;docs&#34;:[]
</span></span><span style="display:flex;"><span> },
</span></span><span style="display:flex;"><span> &#34;facet_counts&#34;:{
</span></span><span style="display:flex;"><span> &#34;facet_queries&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_fields&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_ranges&#34;:{
</span></span><span style="display:flex;"><span> &#34;time&#34;:{
</span></span><span style="display:flex;"><span> &#34;counts&#34;:[
</span></span><span style="display:flex;"><span> &#34;2023-01-01T00:00:00Z&#34;,1,
</span></span><span style="display:flex;"><span> &#34;2023-02-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-03-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-04-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-05-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-06-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-07-01T00:00:00Z&#34;,0,
</span></span><span style="display:flex;"><span> &#34;2023-08-01T00:00:00Z&#34;,27621,
</span></span><span style="display:flex;"><span> &#34;2023-09-01T00:00:00Z&#34;,59165,
</span></span><span style="display:flex;"><span> &#34;2023-10-01T00:00:00Z&#34;,115338,
</span></span><span style="display:flex;"><span> &#34;2023-11-01T00:00:00Z&#34;,96147,
</span></span><span style="display:flex;"><span> &#34;2023-12-01T00:00:00Z&#34;,355464,
</span></span><span style="display:flex;"><span> &#34;2024-01-01T00:00:00Z&#34;,125429],
</span></span><span style="display:flex;"><span> &#34;gap&#34;:&#34;+1MONTH&#34;,
</span></span><span style="display:flex;"><span> &#34;start&#34;:&#34;2023-01-01T00:00:00Z&#34;,
</span></span><span style="display:flex;"><span> &#34;end&#34;:&#34;2024-02-01T00:00:00Z&#34;}},
</span></span><span style="display:flex;"><span> &#34;facet_intervals&#34;:{},
</span></span><span style="display:flex;"><span> &#34;facet_heatmaps&#34;:{}}}
</span></span></code></pre></div><ul>
<li>I see that we had 31,744 statistic events yesterday, and 799 have no <code>id</code>!</li>
<li>I asked about this on Slack and will file an issue on GitHub if someone else also finds such records
<ul>
<li>Several people said they have them, so it&rsquo;s a bug of some sort in DSpace, not our configuration</li>
</ul>
</li>
</ul>
<h2 id="2024-01-13">2024-01-13</h2>
<ul>
<li>Yesterday alone we had 37,000 unique IPs making requests to nginx
<ul>
<li>I looked up the ASNs and found 6,000 IPs from this network in Amazon Singapore: 47.128.0.0/14</li>
</ul>
</li>
</ul>
<h2 id="2024-01-15">2024-01-15</h2>
<ul>
<li>Investigating the CSS selector warning that I&rsquo;ve seen in PM2 logs:</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span>0|dspace-ui | 1 rules skipped due to selector errors:
</span></span><span style="display:flex;"><span>0|dspace-ui | .custom-file-input:lang(en)~.custom-file-label -&gt; unmatched pseudo-class :lang
</span></span></code></pre></div><ul>
<li>It seems to be a bug in Angular, as this selector comes from Bootstrap 4.6.x and is not invalid
<ul>
<li>But that led me to a more interesting issue with <code>inlineCritical</code> optimization for styles in Angular SSR that might be responsible for causing high load in the frontend</li>
<li>See: <a href="https://github.com/angular/angular/issues/42098">https://github.com/angular/angular/issues/42098</a></li>
<li>See: <a href="https://github.com/angular/universal/issues/2106">https://github.com/angular/universal/issues/2106</a></li>
<li>See: <a href="https://github.com/GoogleChromeLabs/critters/issues/78">https://github.com/GoogleChromeLabs/critters/issues/78</a></li>
</ul>
</li>
<li>Since the production site was flapping a lot I decided to try disabling inlineCriticalCss</li>
<li>There have been on and off load issues with the Angular frontend today
<ul>
<li>I think I will just block all data center network blocks for now</li>
<li>In the last week I see almost 200,000 unique IPs:</li>
</ul>
</li>
</ul>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-console" data-lang="console"><span style="display:flex;"><span># zcat -f /var/log/nginx/*access.log /var/log/nginx/*access.log.1 /var/log/nginx/*access.log.2.gz /var/log/nginx/*access.log.3.gz /var/log/nginx/*access.log.4.gz /var/log/nginx/*access.log.5.gz /var/log/nginx/*access.log.6.gz | awk <span style="color:#e6db74">&#39;{print $1}&#39;</span> | sort -u |
</span></span><span style="display:flex;"><span>tee /tmp/ips.txt | wc -l
</span></span><span style="display:flex;"><span>196493
</span></span></code></pre></div><ul>
<li>Looking these IPs up I see there are 18,000 coming from Comcast, 10,000 from AT&amp;T, 4110 from Charter, 3500 from Cox and dozens of other residential IPs
<ul>
<li>I highly doubt these are home users browsing CGSpace&hellip; seems super fishy</li>
<li>Also, over 1,000 IPs from SpaceX Starlink in the last week. RIGHT</li>
<li>I will temporarily add a few new datacenter ISP network blocks to our rate limit:
<ul>
<li>16509 Amazon-02</li>
<li>701 UUNET</li>
<li>8075 Microsoft</li>
<li>15169 Google</li>
<li>14618 Amazon-AES</li>
<li>396982 Google Cloud</li>
</ul>
</li>
<li>The load on the server <em>immediately</em> dropped</li>
</ul>
</li>
</ul>
<!-- raw HTML omitted --> <!-- raw HTML omitted -->

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." /> <meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" /> <meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" /> <meta property="og:updated_time" content="2024-01-10T17:21:12+03:00" />

View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml"> xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc> <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2024-01-10T08:34:16+03:00</lastmod> <lastmod>2024-01-10T17:21:12+03:00</lastmod>
</url><url> </url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2024-01-10T08:34:16+03:00</lastmod> <lastmod>2024-01-10T17:21:12+03:00</lastmod>
</url><url> </url><url>
<loc>https://alanorth.github.io/cgspace-notes/2024-01/</loc> <loc>https://alanorth.github.io/cgspace-notes/2024-01/</loc>
<lastmod>2024-01-10T08:34:16+03:00</lastmod> <lastmod>2024-01-10T17:21:12+03:00</lastmod>
</url><url> </url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2024-01-10T08:34:16+03:00</lastmod> <lastmod>2024-01-10T17:21:12+03:00</lastmod>
</url><url> </url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc> <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2024-01-10T08:34:16+03:00</lastmod> <lastmod>2024-01-10T17:21:12+03:00</lastmod>
</url><url> </url><url>
<loc>https://alanorth.github.io/cgspace-notes/2023-12/</loc> <loc>https://alanorth.github.io/cgspace-notes/2023-12/</loc>
<lastmod>2023-12-29T12:08:57+03:00</lastmod> <lastmod>2023-12-29T12:08:57+03:00</lastmod>