Add notes for 2022-12-21

content/posts/2022-12.md

@@ -234,4 +234,23 @@ $ ./ilri/move-metadata-values.py -i /tmp/data.txt -db dspace -u dspace -p 'dom@i
 - RStudio is already in the ILRI bot overrides for DSpace so it shouldn't be causing any extra hits, but I'll put an HTTP 403 in the nginx config to tell the user to use the REST API
 - Start a harvest on AReS
 
+## 2022-12-21
+
+- I saw that load on CGSpace was over 20.0 for several hours
+  - I saw there were some stuck locks in PostgreSQL:
+
+```console
+$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | grep -o -E '(dspaceWeb|dspaceApi|dspaceCli)' | sort | uniq -c
+    948 dspaceApi
+     30 dspaceCli
+   1237 dspaceWeb
+```
+
+- Ah, it's likely there is something stuck because I see the load high since yesterday at 6AM, which is 24 hours now:
+
+![CPU load day](/cgspace-notes/2022/12/cpu-day.png)
+![PostgreSQL locks week](/cgspace-notes/2022/12/postgres_locks_ALL-week.png)
+
+- I ran all updates and restarted the server
+
 <!-- vim: set sw=2 ts=2: -->