alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2022-04-10

Browse Source
This commit is contained in:
Alan Orth
2022-04-10 23:38:31 +03:00
parent d7abff0a5b
commit bad7d9bb7f
27 changed files with 34 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
content/posts/2022-04.md
View File

@ -26,6 +26,9 @@ sys 3m43.037s
- The DSpace agent pattern `http.?agent` seems to have caught the first ones, but I'll purge the IP ones
- I see 40.77.167.80 is Bing or MSN Bot, but using a normal browser user agent, and if I search Solr for `dns:*msnbot* AND dns:*.msn.com.` I see over 100,000, which is a problem I noticed a few months ago too...
- I extracted the MSN Bot IPs from Solr using an IP facet, then used the `check-spider-ip-hits.sh` script to purge them
-
## 2022-04-10
- Start a full harvest on AReS
<!-- vim: set sw=2 ts=2: -->