alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-12-22 13:12:19 +01:00
Code Issues Projects Releases Wiki Activity

Add notes

Browse Source
This commit is contained in:
Alan Orth 2021-04-09 15:00:21 +03:00
parent 8016270024
commit 1551e0ed7e
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
29 changed files with 314 additions and 28 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

149
content/posts/2021-04.md
View File

@ -331,5 +331,154 @@ $ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid =
- I deleted the ilri/AReS repository on GitHub since we haven't updated it in two years
- All development is happening in https://github.com/ilri/openRXV now
- 10PM and the server is down again, with locks through the roof:
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12198
```
- I see that there are tons of PostgreSQL connections getting abandoned today, compared to very few in the past few weeks:
```console
$ journalctl -u tomcat7 --since=today | grep -c 'ConnectionPool abandon'
1838
$ journalctl -u tomcat7 --since=2021-03-20 --until=2021-04-05 | grep -c 'ConnectionPool abandon'
3
```
- I even restarted the server and connections were low for a few minutes until they shot back up:
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
13
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
8651
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
8940
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
10504
```
- I had to go to bed and I bet it will crash and be down for hours until I wake up...
- What the hell is this user agent?
```
54.197.119.143 - - [06/Apr/2021:19:18:11 +0200] "GET /handle/10568/16499 HTTP/1.1" 499 0 "-" "GetUrl/1.0 wdestiny@umich.edu (Linux)"
```
## 2021-04-07
- CGSpace was still down from last night of course, with tons of database locks:
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12168
```
- I restarted the server again and the locks came back
- Atmire responded to the message from yesterday
- The noticed something in the logs about emails failing to be sent
- There appears to be an issue sending mails on workflow tasks when a user in that group has an invalid email address:
```console
2021-04-01 12:45:11,414 WARN org.dspace.workflowbasic.BasicWorkflowServiceImpl @ a.akwarandu@cgiar.org:session_id=2F20F20D4A8C36DB53D42DE45DFA3CCE:notifyGroupofTask:cannot email user group_id=aecf811b-b7e9-4b6f-8776-3d372e6a048b workflow_item_id=33085\colon; Invalid Addresses (com.sun.mail.smtp.SMTPAddressFailedException\colon; 501 5.1.3 Invalid address
```
- The issue is not the named user above, but a member of the group...
- And the group does have users with invalid email addresses (probably accounts created automatically after authenticating with LDAP):
![DSpace group](/cgspace-notes/2021/04/group-invalid-email.png)
- I extracted all the group IDs from recent logs that had users with invalid email addresses:
```console
$ grep -a -E 'email user group_id=\b[0-9a-f]{8}\b-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-\b[0-9a-f]{12}\b' /home/cgspace.cgiar.org/log/dspace.log.* | grep -o -E '\b[0-9a-f]{8}\b-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-\b[0-9a-f]{12}\b' | sort | uniq
0a30d6ae-74a6-4eee-a8f5-ee5d15192ee6
1769137c-36d4-42b2-8fec-60585e110db7
203c8614-8a97-4ac8-9686-d9d62cb52acc
294603de-3d09-464e-a5b0-09e452c6b5ab
35878555-9623-4679-beb8-bb3395fdf26e
3d8a5efa-5509-4bf9-9374-2bc714aceb99
4238208a-f848-47cb-9dd2-43f9f954a4af
44939b84-1894-41e7-b3e6-8c8d1781057b
49ba087e-75a3-45ce-805c-69eeda0f786b
4a6606ce-0284-421d-bf80-4dafddba2d42
527de6aa-9cd0-4988-bf5f-c9c92ba2ac10
54cd1b16-65bf-4041-9d84-fb2ea3301d6d
58982847-5f7c-4b8b-a7b0-4d4de702136e
5f0b85be-bd23-47de-927d-bca368fa1fbc
646ada17-e4ef-49f6-9378-af7e58596ce1
7e2f4bf8-fbc9-4b2f-97a4-75e5427bef90
8029fd53-f9f5-4107-bfc3-8815507265cf
81faa934-c602-4608-bf45-de91845dfea7
8611a462-210c-4be1-a5bb-f87a065e6113
8855c903-ef86-433c-b0be-c12300eb0f84
8c7ece98-3598-4de7-a885-d61fd033bea8
8c9a0d01-2d12-4a99-84f9-cdc25ac072f9
8f9f888a-b501-41f3-a462-4da16150eebf
94168f0e-9f45-4112-ac8d-3ba9be917842
96998038-f381-47dc-8488-ff7252703627
9768f4a8-3018-44e9-bf58-beba4296327c
9a99e8d2-558e-4fc1-8011-e4411f658414
a34e6400-78ed-45c0-a751-abc039eed2e6
a9da5af3-4ec7-4a9b-becb-6e3d028d594d
abf5201c-8be5-4dee-b461-132203dd51cb
adb5658c-cef3-402f-87b6-b498f580351c
aecf811b-b7e9-4b6f-8776-3d372e6a048b
ba5aae61-ea34-4ac1-9490-4645acf2382f
bf7f3638-c7c6-4a8f-893d-891a6d3dafff
c617ada0-09d1-40ed-b479-1c4860a4f724
cff91d44-a855-458c-89e5-bd48c17d1a54
e65171ae-a2bf-4043-8f54-f8457bc9174b
e7098b40-4701-4ca2-b9a9-3a1282f67044
e904f122-71dc-439b-b877-313ef62486d7
ede59734-adac-4c01-8691-b45f19088d37
f88bd6bb-f93f-41cb-872f-ff26f6237068
f985f5fb-be5c-430b-a8f1-cf86ae4fc49a
fe800006-aaec-4f9e-9ab4-f9475b4cbdc3
```
## 2021-04-08
- I can't believe it but the server has been down for twelve hours or so
- The locks have not changed since I went to bed last night:
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12070
```
- I restarted PostgreSQL and Tomcat and the locks go straight back up!
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
13
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
986
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1194
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1212
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1489
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
2124
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
5934
```
## 2021-04-09
- Atmire managed to get CGSpace back up by killing all the PostgreSQL connections yesterday
- I don't know how they did it...
- They also think it's weird that restarting PostgreSQL didn't kill the connections
- They asked some more questions, like for example if there were also issues on DSpace Test
- Strangely enough, I checked DSpace Test and notice a clear spike in PostgreSQL locks on the morning of April 6th as well!
![PostgreSQL locks week CGSpace](/cgspace-notes/2021/04/postgres_locks_ALL-week-PROD.png)
![PostgreSQL locks week DSpace Test](/cgspace-notes/2021/04/postgres_locks_ALL-week-TEST.png)
- I definitely need to look into that!
<!-- vim: set sw=2 ts=2: -->

143
docs/2021-04/index.html
View File

@ -24,7 +24,7 @@ Perhaps one of the containers crashed, I should have looked closer but I was in
<meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2021-04/" />
<meta property="article:published_time" content="2021-04-01T09:50:54+03:00" />
<meta property="article:modified_time" content="2021-04-06T22:33:43+03:00" />
<meta property="article:modified_time" content="2021-04-06T22:48:44+03:00" />
@ -54,9 +54,9 @@ Perhaps one of the containers crashed, I should have looked closer but I was in
"@type": "BlogPosting",
"headline": "April, 2021",
"url": "https://alanorth.github.io/cgspace-notes/2021-04/",
"wordCount": "1824",
"wordCount": "2530",
"datePublished": "2021-04-01T09:50:54+03:00",
"dateModified": "2021-04-06T22:33:43+03:00",
"dateModified": "2021-04-06T22:48:44+03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -462,6 +462,143 @@ $ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid =
<li>All development is happening in <a href="https://github.com/ilri/openRXV">https://github.com/ilri/openRXV</a> now</li>
</ul>
</li>
<li>10PM and the server is down again, with locks through the roof:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12198
</code></pre><ul>
<li>I see that there are tons of PostgreSQL connections getting abandoned today, compared to very few in the past few weeks:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ journalctl -u tomcat7 --since=today | grep -c 'ConnectionPool abandon'
1838
$ journalctl -u tomcat7 --since=2021-03-20 --until=2021-04-05 | grep -c 'ConnectionPool abandon'
3
</code></pre><ul>
<li>I even restarted the server and connections were low for a few minutes until they shot back up:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
13
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
8651
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
8940
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
10504
</code></pre><ul>
<li>I had to go to bed and I bet it will crash and be down for hours until I wake up&hellip;</li>
<li>What the hell is this user agent?</li>
</ul>
<pre><code>54.197.119.143 - - [06/Apr/2021:19:18:11 +0200] &quot;GET /handle/10568/16499 HTTP/1.1&quot; 499 0 &quot;-&quot; &quot;GetUrl/1.0 wdestiny@umich.edu (Linux)&quot;
</code></pre><h2 id="2021-04-07">2021-04-07</h2>
<ul>
<li>CGSpace was still down from last night of course, with tons of database locks:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12168
</code></pre><ul>
<li>I restarted the server again and the locks came back</li>
<li>Atmire responded to the message from yesterday
<ul>
<li>The noticed something in the logs about emails failing to be sent</li>
<li>There appears to be an issue sending mails on workflow tasks when a user in that group has an invalid email address:</li>
</ul>
</li>
</ul>
<pre><code class="language-console" data-lang="console">2021-04-01 12:45:11,414 WARN org.dspace.workflowbasic.BasicWorkflowServiceImpl @ a.akwarandu@cgiar.org:session_id=2F20F20D4A8C36DB53D42DE45DFA3CCE:notifyGroupofTask:cannot email user group_id=aecf811b-b7e9-4b6f-8776-3d372e6a048b workflow_item_id=33085\colon; Invalid Addresses (com.sun.mail.smtp.SMTPAddressFailedException\colon; 501 5.1.3 Invalid address
</code></pre><ul>
<li>The issue is not the named user above, but a member of the group&hellip;</li>
<li>And the group does have users with invalid email addresses (probably accounts created automatically after authenticating with LDAP):</li>
</ul>
<p><img src="/cgspace-notes/2021/04/group-invalid-email.png" alt="DSpace group"></p>
<ul>
<li>I extracted all the group IDs from recent logs that had users with invalid email addresses:</li>
</ul>
<pre><code class="language-console" data-lang="console">$ grep -a -E 'email user group_id=\b[0-9a-f]{8}\b-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-\b[0-9a-f]{12}\b' /home/cgspace.cgiar.org/log/dspace.log.* | grep -o -E '\b[0-9a-f]{8}\b-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-\b[0-9a-f]{12}\b' | sort | uniq
0a30d6ae-74a6-4eee-a8f5-ee5d15192ee6
1769137c-36d4-42b2-8fec-60585e110db7
203c8614-8a97-4ac8-9686-d9d62cb52acc
294603de-3d09-464e-a5b0-09e452c6b5ab
35878555-9623-4679-beb8-bb3395fdf26e
3d8a5efa-5509-4bf9-9374-2bc714aceb99
4238208a-f848-47cb-9dd2-43f9f954a4af
44939b84-1894-41e7-b3e6-8c8d1781057b
49ba087e-75a3-45ce-805c-69eeda0f786b
4a6606ce-0284-421d-bf80-4dafddba2d42
527de6aa-9cd0-4988-bf5f-c9c92ba2ac10
54cd1b16-65bf-4041-9d84-fb2ea3301d6d
58982847-5f7c-4b8b-a7b0-4d4de702136e
5f0b85be-bd23-47de-927d-bca368fa1fbc
646ada17-e4ef-49f6-9378-af7e58596ce1
7e2f4bf8-fbc9-4b2f-97a4-75e5427bef90
8029fd53-f9f5-4107-bfc3-8815507265cf
81faa934-c602-4608-bf45-de91845dfea7
8611a462-210c-4be1-a5bb-f87a065e6113
8855c903-ef86-433c-b0be-c12300eb0f84
8c7ece98-3598-4de7-a885-d61fd033bea8
8c9a0d01-2d12-4a99-84f9-cdc25ac072f9
8f9f888a-b501-41f3-a462-4da16150eebf
94168f0e-9f45-4112-ac8d-3ba9be917842
96998038-f381-47dc-8488-ff7252703627
9768f4a8-3018-44e9-bf58-beba4296327c
9a99e8d2-558e-4fc1-8011-e4411f658414
a34e6400-78ed-45c0-a751-abc039eed2e6
a9da5af3-4ec7-4a9b-becb-6e3d028d594d
abf5201c-8be5-4dee-b461-132203dd51cb
adb5658c-cef3-402f-87b6-b498f580351c
aecf811b-b7e9-4b6f-8776-3d372e6a048b
ba5aae61-ea34-4ac1-9490-4645acf2382f
bf7f3638-c7c6-4a8f-893d-891a6d3dafff
c617ada0-09d1-40ed-b479-1c4860a4f724
cff91d44-a855-458c-89e5-bd48c17d1a54
e65171ae-a2bf-4043-8f54-f8457bc9174b
e7098b40-4701-4ca2-b9a9-3a1282f67044
e904f122-71dc-439b-b877-313ef62486d7
ede59734-adac-4c01-8691-b45f19088d37
f88bd6bb-f93f-41cb-872f-ff26f6237068
f985f5fb-be5c-430b-a8f1-cf86ae4fc49a
fe800006-aaec-4f9e-9ab4-f9475b4cbdc3
</code></pre><h2 id="2021-04-08">2021-04-08</h2>
<ul>
<li>I can&rsquo;t believe it but the server has been down for twelve hours or so
<ul>
<li>The locks have not changed since I went to bed last night:</li>
</ul>
</li>
</ul>
<pre><code class="language-console" data-lang="console">$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
12070
</code></pre><ul>
<li>I restarted PostgreSQL and Tomcat and the locks go straight back up!</li>
</ul>
<pre><code class="language-console" data-lang="console">$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
13
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
986
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1194
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1212
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
1489
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
2124
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
5934
</code></pre><h2 id="2021-04-09">2021-04-09</h2>
<ul>
<li>Atmire managed to get CGSpace back up by killing all the PostgreSQL connections yesterday
<ul>
<li>I don&rsquo;t know how they did it&hellip;</li>
<li>They also think it&rsquo;s weird that restarting PostgreSQL didn&rsquo;t kill the connections</li>
<li>They asked some more questions, like for example if there were also issues on DSpace Test</li>
<li>Strangely enough, I checked DSpace Test and notice a clear spike in PostgreSQL locks on the morning of April 6th as well!</li>
</ul>
</li>
</ul>
<p><img src="/cgspace-notes/2021/04/postgres_locks_ALL-week-PROD.png" alt="PostgreSQL locks week CGSpace">
<img src="/cgspace-notes/2021/04/postgres_locks_ALL-week-TEST.png" alt="PostgreSQL locks week DSpace Test"></p>
<ul>
<li>I definitely need to look into that!</li>
</ul>
<!-- raw HTML omitted -->

BIN
docs/2021/04/group-invalid-email.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
docs/2021/04/postgres_locks_ALL-week-PROD.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
docs/2021/04/postgres_locks_ALL-week-TEST.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.7 KiB

2
docs/categories/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/categories/notes/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/categories/notes/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/categories/notes/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/categories/notes/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/categories/notes/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/2/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/3/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/4/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/5/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/6/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

2
docs/posts/page/7/index.html
View File

@ -10,7 +10,7 @@
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
<meta property="og:type" content="website" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
<meta property="og:updated_time" content="2021-04-06T22:33:43+03:00" />
<meta property="og:updated_time" content="2021-04-06T22:48:44+03:00" />

10
docs/sitemap.xml
View File

@ -3,19 +3,19 @@
xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>https://alanorth.github.io/cgspace-notes/2021-04/</loc>
<lastmod>2021-04-06T22:33:43+03:00</lastmod>
<lastmod>2021-04-06T22:48:44+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2021-04-06T22:33:43+03:00</lastmod>
<lastmod>2021-04-06T22:48:44+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2021-04-06T22:33:43+03:00</lastmod>
<lastmod>2021-04-06T22:48:44+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2021-04-06T22:33:43+03:00</lastmod>
<lastmod>2021-04-06T22:48:44+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2021-04-06T22:33:43+03:00</lastmod>
<lastmod>2021-04-06T22:48:44+03:00</lastmod>
</url><url>
<loc>https://alanorth.github.io/cgspace-notes/2021-03/</loc>
<lastmod>2021-04-05T19:36:44+03:00</lastmod>

BIN
static/2021/04/group-invalid-email.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
static/2021/04/postgres_locks_ALL-week-PROD.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
static/2021/04/postgres_locks_ALL-week-TEST.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.7 KiB