alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-06-26 16:13:48 +02:00
Code Issues Projects Releases Wiki Activity
d7abff0a5b
cgspace-notes/content/posts/2022-04.md

1.3 KiB
Raw Blame History

title date author categories
April, 2022 2022-03-01T10:53:39+03:00 Alan Orth
Notes

2022-04-01

  • I did G1GC tests on DSpace Test (linode26) to compliment the CMS tests I did yesterday
    • The Discovery indexing took this long:
real    334m33.625s
user    227m51.331s
sys     3m43.037s

2022-04-04

  • Start a full harvest on AReS
  • Help Marianne with submit/approve access on a new collection on CGSpace
  • Go back in Gaia's batch reports to find records that she indicated for replacing on CGSpace (ie, those with better new copies, new versions, etc)
  • Looking at the Solr statistics for 2022-03 on CGSpace
    • I see 54.229.218.204 on Amazon AWS made 49,000 requests, some of which with this user agent: Apache-HttpClient/4.5.9 (Java/1.8.0_322), and many others with a normal browser agent, so that's fishy!
    • The DSpace agent pattern http.?agent seems to have caught the first ones, but I'll purge the IP ones
    • I see 40.77.167.80 is Bing or MSN Bot, but using a normal browser user agent, and if I search Solr for dns:*msnbot* AND dns:*.msn.com. I see over 100,000, which is a problem I noticed a few months ago too...
    • I extracted the MSN Bot IPs from Solr using an IP facet, then used the check-spider-ip-hits.sh script to purge them