alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2021-10-05

Browse Source
This commit is contained in:
Alan Orth
2021-10-05 18:54:39 +03:00
parent 4c11bc1c1e
commit 63500a8837
26 changed files with 59 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
content/posts/2021-10.md
View File

@ -87,7 +87,7 @@ $ csvgrep -c asn -m 14618 /tmp/mozilla-4.0-ips.csv | csvcut -c ip | sed 1d | tee
290382 GET /handle/10568/83389
```
- Before I purge all those I will ask someone Samuel Stacey from the System office to hopefully get an insight...
- Before I purge all those I will ask someone Samuel Stacey from the System Office to hopefully get an insight...
- Meeting with Michael Victor, Peter, Jane, and Abenet about the future of repositories in the One CGIAR
- Meeting with Michelle from Altmetric about their new CSV upload system
- I sent her some examples of Handles that have DOIs, but no linked score (yet) to see if an association will be created when she uploads them
@ -107,4 +107,17 @@ $ ./ilri/agrovoc-lookup.py -i /tmp/agrovoc-sorted.txt -o /tmp/agrovoc-matches.cs
$ csvgrep -c 'number of matches' -m '0' /tmp/agrovoc-matches.csv | csvcut -c 1 > /tmp/invalid-agrovoc.csv
```
## 2021-10-05
- Sam put me in touch with Dodi from the System Office web team and he confirmed that the Amazon requests are not theirs
- I added `Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)` to the list of bad bots in nginx
- I purged all the Amazon IPs using this user agent, as well as the few other IPs I identified yesterday
```console
$ ./ilri/check-spider-ip-hits.sh -f /tmp/robot-ips.txt -p
...
Total number of bot hits purged: 465119
```
<!-- vim: set sw=2 ts=2: -->