--- title: "April, 2022" date: 2022-03-01T10:53:39+03:00 author: "Alan Orth" categories: ["Notes"] --- ## 2022-04-01 - I did G1GC tests on DSpace Test (linode26) to compliment the CMS tests I did yesterday - The Discovery indexing took this long: ```console real 334m33.625s user 227m51.331s sys 3m43.037s ``` ## 2022-04-04 - Start a full harvest on AReS - Help Marianne with submit/approve access on a new collection on CGSpace - Go back in Gaia's batch reports to find records that she indicated for replacing on CGSpace (ie, those with better new copies, new versions, etc) - Looking at the Solr statistics for 2022-03 on CGSpace - I see 54.229.218.204 on Amazon AWS made 49,000 requests, some of which with this user agent: `Apache-HttpClient/4.5.9 (Java/1.8.0_322)`, and many others with a normal browser agent, so that's fishy! - The DSpace agent pattern `http.?agent` seems to have caught the first ones, but I'll purge the IP ones - I see 40.77.167.80 is Bing or MSN Bot, but using a normal browser user agent, and if I search Solr for `dns:*msnbot* AND dns:*.msn.com.` I see over 100,000, which is a problem I noticed a few months ago too... - I extracted the MSN Bot IPs from Solr using an IP facet, then used the `check-spider-ip-hits.sh` script to purge them ## 2022-04-10 - Start a full harvest on AReS ## 2022-04-13 - UptimeRobot mailed to say that CGSpace was down - I looked and found the load at 44... - There seem to be a lot of locks from the XMLUI: ```console $ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | grep -o -E '(dspaceWeb|dspaceApi)' | sort | uniq -c | sort -n 3173 dspaceWeb ``` - Looking at the top IPs in nginx's access log one IP in particular stands out: ```console 941 66.249.66.222 1224 95.108.213.28 2074 157.90.209.76 3064 66.249.66.221 95743 185.192.69.15 ``` - 185.192.69.15 is in the UK - I added a block for that IP in nginx and the load went down...