From 147ad863758a892ef18730721c142682d1e90264 Mon Sep 17 00:00:00 2001 From: Alan Orth Date: Mon, 12 Sep 2022 11:35:57 +0300 Subject: [PATCH] Add notes for 2022-09-12 --- content/posts/2022-09.md | 68 ++++++++++++++++++ docs/2015-11/index.html | 2 +- docs/2015-12/index.html | 2 +- docs/2016-01/index.html | 2 +- docs/2016-02/index.html | 2 +- docs/2016-03/index.html | 2 +- docs/2016-04/index.html | 2 +- docs/2016-05/index.html | 2 +- docs/2016-06/index.html | 2 +- docs/2016-07/index.html | 2 +- docs/2016-08/index.html | 2 +- docs/2016-09/index.html | 2 +- docs/2016-10/index.html | 2 +- docs/2016-11/index.html | 2 +- docs/2016-12/index.html | 2 +- docs/2017-01/index.html | 2 +- docs/2017-02/index.html | 2 +- docs/2017-03/index.html | 2 +- docs/2017-04/index.html | 2 +- docs/2017-05/index.html | 2 +- docs/2017-06/index.html | 2 +- docs/2017-07/index.html | 2 +- docs/2017-08/index.html | 2 +- docs/2017-09/index.html | 2 +- docs/2017-10/index.html | 2 +- docs/2017-11/index.html | 2 +- docs/2017-12/index.html | 2 +- docs/2018-01/index.html | 2 +- docs/2018-02/index.html | 2 +- docs/2018-03/index.html | 2 +- docs/2018-04/index.html | 2 +- docs/2018-05/index.html | 2 +- docs/2018-06/index.html | 2 +- docs/2018-07/index.html | 2 +- docs/2018-08/index.html | 2 +- docs/2018-09/index.html | 2 +- docs/2018-10/index.html | 2 +- docs/2018-11/index.html | 2 +- docs/2018-12/index.html | 2 +- docs/2019-01/index.html | 2 +- docs/2019-02/index.html | 2 +- docs/2019-03/index.html | 2 +- docs/2019-04/index.html | 2 +- docs/2019-05/index.html | 2 +- docs/2019-06/index.html | 2 +- docs/2019-07/index.html | 2 +- docs/2019-08/index.html | 2 +- docs/2019-09/index.html | 2 +- docs/2019-10/index.html | 2 +- docs/2019-11/index.html | 2 +- docs/2019-12/index.html | 2 +- docs/2020-01/index.html | 2 +- docs/2020-02/index.html | 2 +- docs/2020-03/index.html | 2 +- docs/2020-04/index.html | 2 +- docs/2020-05/index.html | 2 +- docs/2020-06/index.html | 2 +- docs/2020-07/index.html | 2 +- docs/2020-08/index.html | 2 +- docs/2020-09/index.html | 2 +- docs/2020-10/index.html | 2 +- docs/2020-11/index.html | 2 +- docs/2020-12/index.html | 2 +- docs/2021-01/index.html | 2 +- docs/2021-02/index.html | 2 +- docs/2021-03/index.html | 2 +- docs/2021-04/index.html | 2 +- docs/2021-05/index.html | 2 +- docs/2021-06/index.html | 2 +- docs/2021-07/index.html | 2 +- docs/2021-08/index.html | 2 +- docs/2021-09/index.html | 2 +- docs/2021-10/index.html | 2 +- docs/2021-11/index.html | 2 +- docs/2021-12/index.html | 2 +- docs/2022-01/index.html | 82 ++++++++++++++++++++-- docs/2022-02/index.html | 2 +- docs/2022-03/index.html | 2 +- docs/2022-04/index.html | 2 +- docs/2022-05/index.html | 2 +- docs/2022-06/index.html | 2 +- docs/2022-07/index.html | 2 +- docs/2022-08/index.html | 2 +- docs/404.html | 2 +- docs/categories/index.html | 4 +- docs/categories/notes/index.html | 4 +- docs/categories/notes/page/2/index.html | 4 +- docs/categories/notes/page/3/index.html | 4 +- docs/categories/notes/page/4/index.html | 4 +- docs/categories/notes/page/5/index.html | 4 +- docs/categories/notes/page/6/index.html | 4 +- docs/categories/notes/page/7/index.html | 4 +- docs/cgiar-library-migration/index.html | 2 +- docs/cgspace-cgcorev2-migration/index.html | 2 +- docs/cgspace-dspace6-upgrade/index.html | 2 +- docs/index.html | 4 +- docs/page/2/index.html | 4 +- docs/page/3/index.html | 4 +- docs/page/4/index.html | 4 +- docs/page/5/index.html | 4 +- docs/page/6/index.html | 4 +- docs/page/7/index.html | 4 +- docs/page/8/index.html | 4 +- docs/page/9/index.html | 4 +- docs/posts/index.html | 4 +- docs/posts/page/2/index.html | 4 +- docs/posts/page/3/index.html | 4 +- docs/posts/page/4/index.html | 4 +- docs/posts/page/5/index.html | 4 +- docs/posts/page/6/index.html | 4 +- docs/posts/page/7/index.html | 4 +- docs/posts/page/8/index.html | 4 +- docs/posts/page/9/index.html | 4 +- docs/sitemap.xml | 10 +-- docs/tags/index.html | 2 +- docs/tags/migration/index.html | 2 +- docs/tags/notes/index.html | 2 +- docs/tags/notes/page/2/index.html | 2 +- docs/tags/notes/page/3/index.html | 2 +- themes/hugo-theme-bootstrap4-blog | 2 +- 120 files changed, 294 insertions(+), 152 deletions(-) diff --git a/content/posts/2022-09.md b/content/posts/2022-09.md index 62b048822..eabf4b743 100644 --- a/content/posts/2022-09.md +++ b/content/posts/2022-09.md @@ -137,4 +137,72 @@ COMMIT - Start a full Discovery index on CGSpace to catch these changes in the Discovery +## 2022-09-11 + +- Today is Sunday and I see the load on the server is high + - Google and a bunch of other bots have been blocked on XMLUI for the past two weeks so it's not from them! + - Looking at the top IPs this morning: + +```console +# cat /var/log/nginx/{access,library-access,oai,rest}.log /var/log/nginx/{access,library-access,oai,rest}.log.1 | grep '11/Sep/2022' | awk '{print $1}' | sort | uniq -c | sort -h | tail -n 40 +... + 165 64.233.172.79 + 166 87.250.224.34 + 200 69.162.124.231 + 202 216.244.66.198 + 385 207.46.13.149 + 398 207.46.13.147 + 421 66.249.64.185 + 422 157.55.39.81 + 442 2a01:4f8:1c17:5550::1 + 451 64.124.8.36 + 578 137.184.159.211 + 597 136.243.228.195 + 1185 66.249.64.183 + 1201 157.55.39.80 + 3135 80.248.237.167 + 4794 54.195.118.125 + 5486 45.5.186.2 + 6322 2a01:7e00::f03c:91ff:fe9a:3a37 + 9556 66.249.64.181 +``` + +- The top is still Google, but all the requests are HTTP 503 because I classified them as bots for XMLUI at least +- Then there's 80.248.237.167, which is using a normal user agent and scraping Discovery + - That IP is on Internet Vikings aka Internetbolaget and we are already marking that subnet as 'bot' for XMLUI so most of these requests are HTTP 503 +- On another note, I'm curious to explore enabling caching of certain REST API responses + - For example, where the use is for harvesting rather than actual clients getting bitstreams or thumbnails, it seems there might be a benefit to speeding these up for subsequent requestors: + +```console +# awk '{print $7}' /var/log/nginx/rest.log | grep -v retrieve | sort | uniq -c | sort -h | tail -n 10 + 4 /rest/items/3f692ddd-7856-4bf0-a587-99fb3df0688a/bitstreams + 4 /rest/items/3f692ddd-7856-4bf0-a587-99fb3df0688a/metadata + 4 /rest/items/b014e36f-b496-43d8-9148-cc9db8a6efac/bitstreams + 4 /rest/items/b014e36f-b496-43d8-9148-cc9db8a6efac/metadata + 5 /rest/handle/10568/110310?expand=all + 5 /rest/handle/10568/89980?expand=all + 5 /rest/handle/10568/97614?expand=all + 6 /rest/handle/10568/107086?expand=all + 6 /rest/handle/10568/108503?expand=all + 6 /rest/handle/10568/98424?expand=all +``` + +- I specifically have to not cache things like requests for bitstreams because those are from actual users and we need to keep the real requests so we get the statistics hit + - Will be interesting to check the results above as the day goes on (now 10AM) + - To estimate the potential savings from caching I will check how many non-bitstream requests are made versus how many are made more than once (updated the next morning using yesterday's log): + +```console +# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort -u | wc -l +33733 +# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 > 1' | wc -l +5637 +``` + +- In the afternoon I started a harvest on AReS (which should affect the numbers above also) +- I enabled an nginx proxy cache on DSpace Test for this location regex: `location ~ /rest/(handle|items|collections|communities)/.+` + +## 2022-09-12 + +- I am testing harvesting DSpace Test via AReS with the nginx proxy cache enabled + diff --git a/docs/2015-11/index.html b/docs/2015-11/index.html index 28c39375a..93d043292 100644 --- a/docs/2015-11/index.html +++ b/docs/2015-11/index.html @@ -64,7 +64,7 @@ $ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspac - + diff --git a/docs/2015-12/index.html b/docs/2015-12/index.html index 70946375b..c9e0edfa0 100644 --- a/docs/2015-12/index.html +++ b/docs/2015-12/index.html @@ -66,7 +66,7 @@ Replace lzop with xz in log compression cron jobs on DSpace Test—it uses less - + diff --git a/docs/2016-01/index.html b/docs/2016-01/index.html index bdb0ed3aa..d1df24510 100644 --- a/docs/2016-01/index.html +++ b/docs/2016-01/index.html @@ -58,7 +58,7 @@ Update GitHub wiki for documentation of maintenance tasks. - + diff --git a/docs/2016-02/index.html b/docs/2016-02/index.html index bebaa8b3d..3f987b8f3 100644 --- a/docs/2016-02/index.html +++ b/docs/2016-02/index.html @@ -68,7 +68,7 @@ Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE&r - + diff --git a/docs/2016-03/index.html b/docs/2016-03/index.html index dfd977b37..fc5dacc53 100644 --- a/docs/2016-03/index.html +++ b/docs/2016-03/index.html @@ -58,7 +58,7 @@ Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Ja - + diff --git a/docs/2016-04/index.html b/docs/2016-04/index.html index 579ea5652..4d41cdf9f 100644 --- a/docs/2016-04/index.html +++ b/docs/2016-04/index.html @@ -62,7 +62,7 @@ Also, I noticed the checker log has some errors we should pay attention to: - + diff --git a/docs/2016-05/index.html b/docs/2016-05/index.html index e3eda06bd..f37a8cd50 100644 --- a/docs/2016-05/index.html +++ b/docs/2016-05/index.html @@ -64,7 +64,7 @@ There are 3,000 IPs accessing the REST API in a 24-hour period! - + diff --git a/docs/2016-06/index.html b/docs/2016-06/index.html index 41024c25d..b3658a4fc 100644 --- a/docs/2016-06/index.html +++ b/docs/2016-06/index.html @@ -64,7 +64,7 @@ Working on second phase of metadata migration, looks like this will work for mov - + diff --git a/docs/2016-07/index.html b/docs/2016-07/index.html index ca0f729fd..f8abee1c8 100644 --- a/docs/2016-07/index.html +++ b/docs/2016-07/index.html @@ -74,7 +74,7 @@ In this case the select query was showing 95 results before the update - + diff --git a/docs/2016-08/index.html b/docs/2016-08/index.html index c5bb99151..b87a4d18e 100644 --- a/docs/2016-08/index.html +++ b/docs/2016-08/index.html @@ -72,7 +72,7 @@ $ git rebase -i dspace-5.5 - + diff --git a/docs/2016-09/index.html b/docs/2016-09/index.html index edbaf193a..46326c8eb 100644 --- a/docs/2016-09/index.html +++ b/docs/2016-09/index.html @@ -64,7 +64,7 @@ $ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=org - + diff --git a/docs/2016-10/index.html b/docs/2016-10/index.html index 85eedca5d..0a123814b 100644 --- a/docs/2016-10/index.html +++ b/docs/2016-10/index.html @@ -72,7 +72,7 @@ I exported a random item’s metadata as CSV, deleted all columns except id - + diff --git a/docs/2016-11/index.html b/docs/2016-11/index.html index 0240d7c54..343235d9f 100644 --- a/docs/2016-11/index.html +++ b/docs/2016-11/index.html @@ -56,7 +56,7 @@ Add dc.type to the output options for Atmire’s Listings and Reports module - + diff --git a/docs/2016-12/index.html b/docs/2016-12/index.html index 76dafba1a..ea66a3c1a 100644 --- a/docs/2016-12/index.html +++ b/docs/2016-12/index.html @@ -76,7 +76,7 @@ Another worrying error from dspace.log is: - + diff --git a/docs/2017-01/index.html b/docs/2017-01/index.html index 8e2d5a0d3..44f81dae7 100644 --- a/docs/2017-01/index.html +++ b/docs/2017-01/index.html @@ -58,7 +58,7 @@ I asked on the dspace-tech mailing list because it seems to be broken, and actua - + diff --git a/docs/2017-02/index.html b/docs/2017-02/index.html index cd5249c34..0ac98c8e0 100644 --- a/docs/2017-02/index.html +++ b/docs/2017-02/index.html @@ -80,7 +80,7 @@ Looks like we’ll be using cg.identifier.ccafsprojectpii as the field name - + diff --git a/docs/2017-03/index.html b/docs/2017-03/index.html index 49d5d10d4..a7f12c035 100644 --- a/docs/2017-03/index.html +++ b/docs/2017-03/index.html @@ -84,7 +84,7 @@ $ identify ~/Desktop/alc_contrastes_desafios.jpg - + diff --git a/docs/2017-04/index.html b/docs/2017-04/index.html index b6518bebb..43cd53f02 100644 --- a/docs/2017-04/index.html +++ b/docs/2017-04/index.html @@ -70,7 +70,7 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thu - + diff --git a/docs/2017-05/index.html b/docs/2017-05/index.html index 5796e9479..19fd73b7d 100644 --- a/docs/2017-05/index.html +++ b/docs/2017-05/index.html @@ -48,7 +48,7 @@ - + diff --git a/docs/2017-06/index.html b/docs/2017-06/index.html index 2ca6fdd73..39669166d 100644 --- a/docs/2017-06/index.html +++ b/docs/2017-06/index.html @@ -48,7 +48,7 @@ - + diff --git a/docs/2017-07/index.html b/docs/2017-07/index.html index 9868648f0..5699fd43c 100644 --- a/docs/2017-07/index.html +++ b/docs/2017-07/index.html @@ -66,7 +66,7 @@ We can use PostgreSQL’s extended output format (-x) plus sed to format the - + diff --git a/docs/2017-08/index.html b/docs/2017-08/index.html index 19f8499fb..519dea5ea 100644 --- a/docs/2017-08/index.html +++ b/docs/2017-08/index.html @@ -90,7 +90,7 @@ Then I cleaned up the author authorities and HTML characters in OpenRefine and s - + diff --git a/docs/2017-09/index.html b/docs/2017-09/index.html index 7286aca8f..4c311fb6b 100644 --- a/docs/2017-09/index.html +++ b/docs/2017-09/index.html @@ -62,7 +62,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne’s user account - + diff --git a/docs/2017-10/index.html b/docs/2017-10/index.html index 4bffbf16d..173b30ea3 100644 --- a/docs/2017-10/index.html +++ b/docs/2017-10/index.html @@ -64,7 +64,7 @@ Add Katherine Lutz to the groups for content submission and edit steps of the CG - + diff --git a/docs/2017-11/index.html b/docs/2017-11/index.html index 5a11554cf..40b386e0e 100644 --- a/docs/2017-11/index.html +++ b/docs/2017-11/index.html @@ -78,7 +78,7 @@ COPY 54701 - + diff --git a/docs/2017-12/index.html b/docs/2017-12/index.html index 363f0aa94..bd334c2dc 100644 --- a/docs/2017-12/index.html +++ b/docs/2017-12/index.html @@ -60,7 +60,7 @@ The list of connections to XMLUI and REST API for today: - + diff --git a/docs/2018-01/index.html b/docs/2018-01/index.html index 261e7cbd7..9377e2d45 100644 --- a/docs/2018-01/index.html +++ b/docs/2018-01/index.html @@ -180,7 +180,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv - + diff --git a/docs/2018-02/index.html b/docs/2018-02/index.html index 884d2cc16..30c634f1f 100644 --- a/docs/2018-02/index.html +++ b/docs/2018-02/index.html @@ -60,7 +60,7 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl - + diff --git a/docs/2018-03/index.html b/docs/2018-03/index.html index f2992c219..0fc8ecb9e 100644 --- a/docs/2018-03/index.html +++ b/docs/2018-03/index.html @@ -54,7 +54,7 @@ Export a CSV of the IITA community metadata for Martin Mueller - + diff --git a/docs/2018-04/index.html b/docs/2018-04/index.html index 1520776a0..5c69835a0 100644 --- a/docs/2018-04/index.html +++ b/docs/2018-04/index.html @@ -56,7 +56,7 @@ Catalina logs at least show some memory errors yesterday: - + diff --git a/docs/2018-05/index.html b/docs/2018-05/index.html index b7a4e7e8b..878ef7656 100644 --- a/docs/2018-05/index.html +++ b/docs/2018-05/index.html @@ -68,7 +68,7 @@ Also, I switched it to use OpenJDK instead of Oracle Java, as well as re-worked - + diff --git a/docs/2018-06/index.html b/docs/2018-06/index.html index 483f3bd0c..11ccccc68 100644 --- a/docs/2018-06/index.html +++ b/docs/2018-06/index.html @@ -88,7 +88,7 @@ sys 2m7.289s - + diff --git a/docs/2018-07/index.html b/docs/2018-07/index.html index 674b58afc..38b2da271 100644 --- a/docs/2018-07/index.html +++ b/docs/2018-07/index.html @@ -66,7 +66,7 @@ There is insufficient memory for the Java Runtime Environment to continue. - + diff --git a/docs/2018-08/index.html b/docs/2018-08/index.html index c4e64eba9..7ecc188d9 100644 --- a/docs/2018-08/index.html +++ b/docs/2018-08/index.html @@ -76,7 +76,7 @@ I ran all system updates on DSpace Test and rebooted it - + diff --git a/docs/2018-09/index.html b/docs/2018-09/index.html index 868cf1be6..2b9187862 100644 --- a/docs/2018-09/index.html +++ b/docs/2018-09/index.html @@ -60,7 +60,7 @@ I’m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I - + diff --git a/docs/2018-10/index.html b/docs/2018-10/index.html index 66adf972f..e409cc526 100644 --- a/docs/2018-10/index.html +++ b/docs/2018-10/index.html @@ -56,7 +56,7 @@ I created a GitHub issue to track this #389, because I’m super busy in Nai - + diff --git a/docs/2018-11/index.html b/docs/2018-11/index.html index 722e6e9d8..17efc4fd6 100644 --- a/docs/2018-11/index.html +++ b/docs/2018-11/index.html @@ -66,7 +66,7 @@ Today these are the top 10 IPs: - + diff --git a/docs/2018-12/index.html b/docs/2018-12/index.html index f02c61581..43f6088e5 100644 --- a/docs/2018-12/index.html +++ b/docs/2018-12/index.html @@ -66,7 +66,7 @@ I noticed that there is another issue with PDF thumbnails on CGSpace, and I see - + diff --git a/docs/2019-01/index.html b/docs/2019-01/index.html index f95e69602..00572e294 100644 --- a/docs/2019-01/index.html +++ b/docs/2019-01/index.html @@ -80,7 +80,7 @@ I don’t see anything interesting in the web server logs around that time t - + diff --git a/docs/2019-02/index.html b/docs/2019-02/index.html index 11b0c1458..1e33aea75 100644 --- a/docs/2019-02/index.html +++ b/docs/2019-02/index.html @@ -102,7 +102,7 @@ sys 0m1.979s - + diff --git a/docs/2019-03/index.html b/docs/2019-03/index.html index 1d2d1befc..fa836e607 100644 --- a/docs/2019-03/index.html +++ b/docs/2019-03/index.html @@ -76,7 +76,7 @@ I think I will need to ask Udana to re-copy and paste the abstracts with more ca - + diff --git a/docs/2019-04/index.html b/docs/2019-04/index.html index de7523592..3cd334f1d 100644 --- a/docs/2019-04/index.html +++ b/docs/2019-04/index.html @@ -94,7 +94,7 @@ $ ./delete-metadata-values.py -i /tmp/2019-02-21-delete-1-region.csv -db dspace - + diff --git a/docs/2019-05/index.html b/docs/2019-05/index.html index c6da2582a..05d66a661 100644 --- a/docs/2019-05/index.html +++ b/docs/2019-05/index.html @@ -78,7 +78,7 @@ But after this I tried to delete the item from the XMLUI and it is still present - + diff --git a/docs/2019-06/index.html b/docs/2019-06/index.html index 97e481701..b53ae8a2c 100644 --- a/docs/2019-06/index.html +++ b/docs/2019-06/index.html @@ -64,7 +64,7 @@ Skype with Marie-Angélique and Abenet about CG Core v2 - + diff --git a/docs/2019-07/index.html b/docs/2019-07/index.html index 09d3a186f..a30bfa35c 100644 --- a/docs/2019-07/index.html +++ b/docs/2019-07/index.html @@ -68,7 +68,7 @@ Abenet had another similar issue a few days ago when trying to find the stats fo - + diff --git a/docs/2019-08/index.html b/docs/2019-08/index.html index b910ef884..2548601bd 100644 --- a/docs/2019-08/index.html +++ b/docs/2019-08/index.html @@ -76,7 +76,7 @@ Run system updates on DSpace Test (linode19) and reboot it - + diff --git a/docs/2019-09/index.html b/docs/2019-09/index.html index ca171ee79..6839a9fd3 100644 --- a/docs/2019-09/index.html +++ b/docs/2019-09/index.html @@ -102,7 +102,7 @@ Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning: - + diff --git a/docs/2019-10/index.html b/docs/2019-10/index.html index 0c9280942..4b26f5523 100644 --- a/docs/2019-10/index.html +++ b/docs/2019-10/index.html @@ -48,7 +48,7 @@ - + diff --git a/docs/2019-11/index.html b/docs/2019-11/index.html index e180b1b54..d8db39d16 100644 --- a/docs/2019-11/index.html +++ b/docs/2019-11/index.html @@ -88,7 +88,7 @@ Let’s see how many of the REST API requests were for bitstreams (because t - + diff --git a/docs/2019-12/index.html b/docs/2019-12/index.html index 56dd99888..a8f519b14 100644 --- a/docs/2019-12/index.html +++ b/docs/2019-12/index.html @@ -76,7 +76,7 @@ Make sure all packages are up to date and the package manager is up to date, the - + diff --git a/docs/2020-01/index.html b/docs/2020-01/index.html index 1ffc706e3..2e6450606 100644 --- a/docs/2020-01/index.html +++ b/docs/2020-01/index.html @@ -86,7 +86,7 @@ I tweeted the CGSpace repository link - + diff --git a/docs/2020-02/index.html b/docs/2020-02/index.html index 260a33c2b..97f2163b8 100644 --- a/docs/2020-02/index.html +++ b/docs/2020-02/index.html @@ -68,7 +68,7 @@ The code finally builds and runs with a fresh install - + diff --git a/docs/2020-03/index.html b/docs/2020-03/index.html index be6a6998c..f7ce58705 100644 --- a/docs/2020-03/index.html +++ b/docs/2020-03/index.html @@ -72,7 +72,7 @@ You need to download this into the DSpace 6.x source and compile it - + diff --git a/docs/2020-04/index.html b/docs/2020-04/index.html index 159d20122..6ca75099b 100644 --- a/docs/2020-04/index.html +++ b/docs/2020-04/index.html @@ -78,7 +78,7 @@ On the same note, the one item Abenet pointed out last week now has a donut with - + diff --git a/docs/2020-05/index.html b/docs/2020-05/index.html index 8ae72a11e..da953f1d0 100644 --- a/docs/2020-05/index.html +++ b/docs/2020-05/index.html @@ -64,7 +64,7 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2 - + diff --git a/docs/2020-06/index.html b/docs/2020-06/index.html index 5b6656d8a..4b29cc630 100644 --- a/docs/2020-06/index.html +++ b/docs/2020-06/index.html @@ -66,7 +66,7 @@ I tried to build the OAI registry on the freshly migrated DSpace 6 on DSpace Tes - + diff --git a/docs/2020-07/index.html b/docs/2020-07/index.html index 18c6c54ca..5bcdf3a8d 100644 --- a/docs/2020-07/index.html +++ b/docs/2020-07/index.html @@ -68,7 +68,7 @@ Since I was restarting Tomcat anyways I decided to redeploy the latest changes f - + diff --git a/docs/2020-08/index.html b/docs/2020-08/index.html index 7dc4a6bf4..b8b0b4859 100644 --- a/docs/2020-08/index.html +++ b/docs/2020-08/index.html @@ -66,7 +66,7 @@ It is class based so I can easily add support for other vocabularies, and the te - + diff --git a/docs/2020-09/index.html b/docs/2020-09/index.html index 3e84d25cf..92c87099a 100644 --- a/docs/2020-09/index.html +++ b/docs/2020-09/index.html @@ -78,7 +78,7 @@ I filed an issue on OpenRXV to make some minor edits to the admin UI: https://gi - + diff --git a/docs/2020-10/index.html b/docs/2020-10/index.html index b5d9bc25b..8934dce55 100644 --- a/docs/2020-10/index.html +++ b/docs/2020-10/index.html @@ -74,7 +74,7 @@ During the FlywayDB migration I got an error: - + diff --git a/docs/2020-11/index.html b/docs/2020-11/index.html index b290dd05d..cf792838c 100644 --- a/docs/2020-11/index.html +++ b/docs/2020-11/index.html @@ -62,7 +62,7 @@ So far we’ve spent at least fifty hours to process the statistics and stat - + diff --git a/docs/2020-12/index.html b/docs/2020-12/index.html index 105d5ff9e..d496b460c 100644 --- a/docs/2020-12/index.html +++ b/docs/2020-12/index.html @@ -66,7 +66,7 @@ I started processing those (about 411,000 records): - + diff --git a/docs/2021-01/index.html b/docs/2021-01/index.html index 3d793b4de..93c57410d 100644 --- a/docs/2021-01/index.html +++ b/docs/2021-01/index.html @@ -80,7 +80,7 @@ For example, this item has 51 views on CGSpace, but 0 on AReS - + diff --git a/docs/2021-02/index.html b/docs/2021-02/index.html index 39d821f3d..6ac6bf6e0 100644 --- a/docs/2021-02/index.html +++ b/docs/2021-02/index.html @@ -90,7 +90,7 @@ $ curl -s 'http://localhost:9200/openrxv-items-temp/_count?q=*&pretty - + diff --git a/docs/2021-03/index.html b/docs/2021-03/index.html index 07ef6e553..85f894dc1 100644 --- a/docs/2021-03/index.html +++ b/docs/2021-03/index.html @@ -64,7 +64,7 @@ Also, we found some issues building and running OpenRXV currently due to ecosyst - + diff --git a/docs/2021-04/index.html b/docs/2021-04/index.html index 8c058d3ff..7dcbed45a 100644 --- a/docs/2021-04/index.html +++ b/docs/2021-04/index.html @@ -74,7 +74,7 @@ Perhaps one of the containers crashed, I should have looked closer but I was in - + diff --git a/docs/2021-05/index.html b/docs/2021-05/index.html index 9d663054b..495363f6e 100644 --- a/docs/2021-05/index.html +++ b/docs/2021-05/index.html @@ -66,7 +66,7 @@ I will add the RI/1.0 pattern to our DSpace agents overload and purge them from - + diff --git a/docs/2021-06/index.html b/docs/2021-06/index.html index 69d2e6e22..0a88834a3 100644 --- a/docs/2021-06/index.html +++ b/docs/2021-06/index.html @@ -66,7 +66,7 @@ I simply started it and AReS was running again: - + diff --git a/docs/2021-07/index.html b/docs/2021-07/index.html index 0c5435df1..5518992ae 100644 --- a/docs/2021-07/index.html +++ b/docs/2021-07/index.html @@ -60,7 +60,7 @@ COPY 20994 - + diff --git a/docs/2021-08/index.html b/docs/2021-08/index.html index c414a65b9..c9b8df777 100644 --- a/docs/2021-08/index.html +++ b/docs/2021-08/index.html @@ -62,7 +62,7 @@ I decided to upgrade linode20 from Ubuntu 18.04 to 20.04 - + diff --git a/docs/2021-09/index.html b/docs/2021-09/index.html index 7a1d9b5cb..448776b19 100644 --- a/docs/2021-09/index.html +++ b/docs/2021-09/index.html @@ -78,7 +78,7 @@ The syntax Moayad showed me last month doesn’t seem to honor the search qu - + diff --git a/docs/2021-10/index.html b/docs/2021-10/index.html index 962afbd7c..b2ecb38ee 100644 --- a/docs/2021-10/index.html +++ b/docs/2021-10/index.html @@ -76,7 +76,7 @@ So we have 1879/7100 (26.46%) matching already - + diff --git a/docs/2021-11/index.html b/docs/2021-11/index.html index bd4ed9f8e..015003b68 100644 --- a/docs/2021-11/index.html +++ b/docs/2021-11/index.html @@ -62,7 +62,7 @@ $ zstd statistics-2019.json - + diff --git a/docs/2021-12/index.html b/docs/2021-12/index.html index 3e401e191..646cfba60 100644 --- a/docs/2021-12/index.html +++ b/docs/2021-12/index.html @@ -70,7 +70,7 @@ Total number of bot hits purged: 3679 - + diff --git a/docs/2022-01/index.html b/docs/2022-01/index.html index 8fecca31e..a38e7cccf 100644 --- a/docs/2022-01/index.html +++ b/docs/2022-01/index.html @@ -25,7 +25,7 @@ I also fixed a few bugs and improved the region-matching logic - + @@ -56,9 +56,9 @@ I also fixed a few bugs and improved the region-matching logic "@type": "BlogPosting", "headline": "September, 2022", "url": "https://alanorth.github.io/cgspace-notes/2022-01/", - "wordCount": "844", + "wordCount": "1259", "datePublished": "2022-01-01T09:41:36+03:00", - "dateModified": "2022-09-08T17:47:25+03:00", + "dateModified": "2022-09-09T17:29:51+03:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -76,7 +76,7 @@ I also fixed a few bugs and improved the region-matching logic - + @@ -283,6 +283,80 @@ I also fixed a few bugs and improved the region-matching logic +

2022-09-11

+ +
# cat /var/log/nginx/{access,library-access,oai,rest}.log /var/log/nginx/{access,library-access,oai,rest}.log.1 | grep '11/Sep/2022' | awk '{print $1}' | sort | uniq -c | sort -h | tail -n 40
+...
+    165 64.233.172.79
+    166 87.250.224.34
+    200 69.162.124.231
+    202 216.244.66.198
+    385 207.46.13.149
+    398 207.46.13.147
+    421 66.249.64.185
+    422 157.55.39.81
+    442 2a01:4f8:1c17:5550::1
+    451 64.124.8.36
+    578 137.184.159.211
+    597 136.243.228.195
+   1185 66.249.64.183
+   1201 157.55.39.80
+   3135 80.248.237.167
+   4794 54.195.118.125
+   5486 45.5.186.2
+   6322 2a01:7e00::f03c:91ff:fe9a:3a37
+   9556 66.249.64.181
+
+
# awk '{print $7}' /var/log/nginx/rest.log | grep -v retrieve | sort | uniq -c | sort -h | tail -n 10
+      4 /rest/items/3f692ddd-7856-4bf0-a587-99fb3df0688a/bitstreams
+      4 /rest/items/3f692ddd-7856-4bf0-a587-99fb3df0688a/metadata
+      4 /rest/items/b014e36f-b496-43d8-9148-cc9db8a6efac/bitstreams
+      4 /rest/items/b014e36f-b496-43d8-9148-cc9db8a6efac/metadata
+      5 /rest/handle/10568/110310?expand=all
+      5 /rest/handle/10568/89980?expand=all
+      5 /rest/handle/10568/97614?expand=all
+      6 /rest/handle/10568/107086?expand=all
+      6 /rest/handle/10568/108503?expand=all
+      6 /rest/handle/10568/98424?expand=all
+
+
# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort -u | wc -l
+33733
+# awk '{print $7}' /var/log/nginx/rest.log.1 | grep -v retrieve | sort | uniq -c | awk '$1 > 1' | wc -l
+5637
+
+

2022-09-12

+ diff --git a/docs/2022-02/index.html b/docs/2022-02/index.html index 554dc3ddb..562966bc8 100644 --- a/docs/2022-02/index.html +++ b/docs/2022-02/index.html @@ -68,7 +68,7 @@ We agreed to try to do more alignment of affiliations/funders with ROR - + diff --git a/docs/2022-03/index.html b/docs/2022-03/index.html index 9a9c5ccae..ab72902bb 100644 --- a/docs/2022-03/index.html +++ b/docs/2022-03/index.html @@ -64,7 +64,7 @@ $ csvjoin -c id /tmp/2022-03-01-tac-batch4-701-980.csv /tmp/tac4-filenames.csv & - + diff --git a/docs/2022-04/index.html b/docs/2022-04/index.html index 239c16d17..5d1d20266 100644 --- a/docs/2022-04/index.html +++ b/docs/2022-04/index.html @@ -48,7 +48,7 @@ - + diff --git a/docs/2022-05/index.html b/docs/2022-05/index.html index 5ad0c98f1..6b5d3458f 100644 --- a/docs/2022-05/index.html +++ b/docs/2022-05/index.html @@ -96,7 +96,7 @@ I purged 93,974 hits from these IPs using my check-spider-ip-hits.sh script - + diff --git a/docs/2022-06/index.html b/docs/2022-06/index.html index 450ecb2c9..39d2dffd4 100644 --- a/docs/2022-06/index.html +++ b/docs/2022-06/index.html @@ -78,7 +78,7 @@ There seem to be many more of these: - + diff --git a/docs/2022-07/index.html b/docs/2022-07/index.html index b0a4b38b1..7552edef7 100644 --- a/docs/2022-07/index.html +++ b/docs/2022-07/index.html @@ -64,7 +64,7 @@ Also, the trgm functions I’ve used before are case insensitive, but Levens - + diff --git a/docs/2022-08/index.html b/docs/2022-08/index.html index a9b9b682e..fe64a95d1 100644 --- a/docs/2022-08/index.html +++ b/docs/2022-08/index.html @@ -54,7 +54,7 @@ Our request to add CC-BY-3.0-IGO to SPDX was approved a few weeks ago - + diff --git a/docs/404.html b/docs/404.html index fbca6a95a..e255b464f 100644 --- a/docs/404.html +++ b/docs/404.html @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/index.html b/docs/categories/index.html index 3dcc06f46..0b3b12c87 100644 --- a/docs/categories/index.html +++ b/docs/categories/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/index.html b/docs/categories/notes/index.html index 825cd632a..e00c58278 100644 --- a/docs/categories/notes/index.html +++ b/docs/categories/notes/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/2/index.html b/docs/categories/notes/page/2/index.html index 649a34b02..5aade2d3e 100644 --- a/docs/categories/notes/page/2/index.html +++ b/docs/categories/notes/page/2/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/3/index.html b/docs/categories/notes/page/3/index.html index 12b2bc8d7..0bc2aba2e 100644 --- a/docs/categories/notes/page/3/index.html +++ b/docs/categories/notes/page/3/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/4/index.html b/docs/categories/notes/page/4/index.html index 1dffd9b68..feeaacb32 100644 --- a/docs/categories/notes/page/4/index.html +++ b/docs/categories/notes/page/4/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/5/index.html b/docs/categories/notes/page/5/index.html index de405ee36..446baa424 100644 --- a/docs/categories/notes/page/5/index.html +++ b/docs/categories/notes/page/5/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/6/index.html b/docs/categories/notes/page/6/index.html index 23aab7224..32011f5b9 100644 --- a/docs/categories/notes/page/6/index.html +++ b/docs/categories/notes/page/6/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/categories/notes/page/7/index.html b/docs/categories/notes/page/7/index.html index aff9cdf7b..d97908dfa 100644 --- a/docs/categories/notes/page/7/index.html +++ b/docs/categories/notes/page/7/index.html @@ -10,7 +10,7 @@ - + @@ -31,7 +31,7 @@ - + diff --git a/docs/cgiar-library-migration/index.html b/docs/cgiar-library-migration/index.html index 4a23525c1..1b59c0315 100644 --- a/docs/cgiar-library-migration/index.html +++ b/docs/cgiar-library-migration/index.html @@ -49,7 +49,7 @@ - + diff --git a/docs/cgspace-cgcorev2-migration/index.html b/docs/cgspace-cgcorev2-migration/index.html index b68fd14fb..c06fb560c 100644 --- a/docs/cgspace-cgcorev2-migration/index.html +++ b/docs/cgspace-cgcorev2-migration/index.html @@ -49,7 +49,7 @@ - + diff --git a/docs/cgspace-dspace6-upgrade/index.html b/docs/cgspace-dspace6-upgrade/index.html index 6d9ab2f68..6ab5b1aa2 100644 --- a/docs/cgspace-dspace6-upgrade/index.html +++ b/docs/cgspace-dspace6-upgrade/index.html @@ -49,7 +49,7 @@ - + diff --git a/docs/index.html b/docs/index.html index d8e42905c..aa6b83724 100644 --- a/docs/index.html +++ b/docs/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/2/index.html b/docs/page/2/index.html index 1d3a4d007..feefd27af 100644 --- a/docs/page/2/index.html +++ b/docs/page/2/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/3/index.html b/docs/page/3/index.html index 3e0113241..73fa2bd61 100644 --- a/docs/page/3/index.html +++ b/docs/page/3/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/4/index.html b/docs/page/4/index.html index 1e3f4e4ce..1c3655f68 100644 --- a/docs/page/4/index.html +++ b/docs/page/4/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/5/index.html b/docs/page/5/index.html index 714d4516c..e6fc1caa1 100644 --- a/docs/page/5/index.html +++ b/docs/page/5/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/6/index.html b/docs/page/6/index.html index 3a2b314ba..d47e95bfe 100644 --- a/docs/page/6/index.html +++ b/docs/page/6/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/7/index.html b/docs/page/7/index.html index a16a317af..a818b9535 100644 --- a/docs/page/7/index.html +++ b/docs/page/7/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/8/index.html b/docs/page/8/index.html index 423551698..2b0bfa2d7 100644 --- a/docs/page/8/index.html +++ b/docs/page/8/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/page/9/index.html b/docs/page/9/index.html index 869a4c5d0..a130824a5 100644 --- a/docs/page/9/index.html +++ b/docs/page/9/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/index.html b/docs/posts/index.html index da68c7f17..0b7aaa35e 100644 --- a/docs/posts/index.html +++ b/docs/posts/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/2/index.html b/docs/posts/page/2/index.html index 4fa6c56bf..64b311923 100644 --- a/docs/posts/page/2/index.html +++ b/docs/posts/page/2/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/3/index.html b/docs/posts/page/3/index.html index 017ac360a..bf627683e 100644 --- a/docs/posts/page/3/index.html +++ b/docs/posts/page/3/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/4/index.html b/docs/posts/page/4/index.html index a7545f203..7c859b204 100644 --- a/docs/posts/page/4/index.html +++ b/docs/posts/page/4/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/5/index.html b/docs/posts/page/5/index.html index 748531933..7b536e6ad 100644 --- a/docs/posts/page/5/index.html +++ b/docs/posts/page/5/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/6/index.html b/docs/posts/page/6/index.html index 8585b1ec0..f3f9a139a 100644 --- a/docs/posts/page/6/index.html +++ b/docs/posts/page/6/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/7/index.html b/docs/posts/page/7/index.html index a9dab238f..94b17513b 100644 --- a/docs/posts/page/7/index.html +++ b/docs/posts/page/7/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/8/index.html b/docs/posts/page/8/index.html index 635575cfc..99dfb38b1 100644 --- a/docs/posts/page/8/index.html +++ b/docs/posts/page/8/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/posts/page/9/index.html b/docs/posts/page/9/index.html index 9101c2eff..4aa45522e 100644 --- a/docs/posts/page/9/index.html +++ b/docs/posts/page/9/index.html @@ -10,7 +10,7 @@ - + @@ -45,7 +45,7 @@ - + diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 8ecf56845..067809bc9 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -6,16 +6,16 @@ 2022-08-31T17:37:28+03:00 https://alanorth.github.io/cgspace-notes/categories/ - 2022-09-08T17:47:25+03:00 + 2022-09-09T17:29:51+03:00 https://alanorth.github.io/cgspace-notes/ - 2022-09-08T17:47:25+03:00 + 2022-09-09T17:29:51+03:00 https://alanorth.github.io/cgspace-notes/categories/notes/ - 2022-09-08T17:47:25+03:00 + 2022-09-09T17:29:51+03:00 https://alanorth.github.io/cgspace-notes/posts/ - 2022-09-08T17:47:25+03:00 + 2022-09-09T17:29:51+03:00 https://alanorth.github.io/cgspace-notes/2022-07/ 2022-07-31T15:49:35+03:00 @@ -39,7 +39,7 @@ 2022-05-12T12:51:45+03:00 https://alanorth.github.io/cgspace-notes/2022-01/ - 2022-09-08T17:47:25+03:00 + 2022-09-09T17:29:51+03:00 https://alanorth.github.io/cgspace-notes/2021-12/ 2022-01-09T10:39:51+02:00 diff --git a/docs/tags/index.html b/docs/tags/index.html index 68ca2d7ef..11e616da8 100644 --- a/docs/tags/index.html +++ b/docs/tags/index.html @@ -31,7 +31,7 @@ - + diff --git a/docs/tags/migration/index.html b/docs/tags/migration/index.html index 476bf88e7..088b3d954 100644 --- a/docs/tags/migration/index.html +++ b/docs/tags/migration/index.html @@ -31,7 +31,7 @@ - + diff --git a/docs/tags/notes/index.html b/docs/tags/notes/index.html index 5fe26db57..eab22f52a 100644 --- a/docs/tags/notes/index.html +++ b/docs/tags/notes/index.html @@ -31,7 +31,7 @@ - + diff --git a/docs/tags/notes/page/2/index.html b/docs/tags/notes/page/2/index.html index 6126cf7b5..cb8c62c5c 100644 --- a/docs/tags/notes/page/2/index.html +++ b/docs/tags/notes/page/2/index.html @@ -31,7 +31,7 @@ - + diff --git a/docs/tags/notes/page/3/index.html b/docs/tags/notes/page/3/index.html index 9794ebeb9..c642f647d 100644 --- a/docs/tags/notes/page/3/index.html +++ b/docs/tags/notes/page/3/index.html @@ -31,7 +31,7 @@ - + diff --git a/themes/hugo-theme-bootstrap4-blog b/themes/hugo-theme-bootstrap4-blog index 99f1a6725..a8ff91e1e 160000 --- a/themes/hugo-theme-bootstrap4-blog +++ b/themes/hugo-theme-bootstrap4-blog @@ -1 +1 @@ -Subproject commit 99f1a6725c97a01179c305b077cb46f34feee693 +Subproject commit a8ff91e1ea162877f7f00192070a5a75db77fba7