alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2021-11-07

Browse Source
This commit is contained in:
Alan Orth
2021-11-07 11:26:32 +02:00
parent 2ca9096495
commit b3df4ff58f
26 changed files with 108 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
content/posts/2021-11.md
View File

@ -36,4 +36,41 @@ $ ./run.sh -s http://localhost:8081/solr/statistics-2019 -a import -o statistics
- I checked on CGSpace's and I can't find them there either, but I see them in Solr when I query in the admin UI
- I need to debug that, but it doesn't seem to be related to the sharding...
## 2021-11-04
- I spent a little bit of time debugging the Solr bug with the statistics-2019 shard but couldn't reproduce it for the few items I tested
- So that's good, it seems the sharding worked
- Linode alerted me to high CPU usage on CGSpace (linode18) yesterday
- Looking at the Solr hits from yesterday I see 91.213.50.11 making 2,300 requests
- According to AbuseIPDB.com this is owned by Registrarus LLC (registrarus.ru) and it has been reported for malicious activity by several users
- The ASN is 50340 (SELECTEL-MSK, RU)
- They are attempting SQL injection:
```console
91.213.50.11 - - [03/Nov/2021:06:47:20 +0100] "HEAD /bitstream/handle/10568/106239/U19ArtSimonikovaChromosomeInthomNodev.pdf?sequence=1%60%20WHERE%206158%3D6158%20AND%204894%3D4741--%20kIlq&isAllowed=y HTTP/1.1" 200 0 "https://cgspace.cgiar.org:443/bitstream/handle/10568/106239/U19ArtSimonikovaChromosomeInthomNodev.pdf" "Mozilla/5.0 (X11; U; Linux i686; en-CA; rv:1.8.0.10) Gecko/20070223 Fedora/1.5.0.10-1.fc5 Firefox/1.5.0.10"
```
- Another is in China, and they grabbed 1,200 PDFs from the REST API in under an hour:
```console
# zgrep 222.129.53.160 /var/log/nginx/rest.log.2.gz | wc -l
1178
```
- I will continue to split the Solr statistics back into year-shards on DSpace Test (linode26)
- Today I did all 2018 stats...
- I want to see if there is a noticeable change in JVM memory, Solr response time, etc
## 2021-11-07
- Update all Docker containers on AReS and rebuild OpenRXV:
```console
$ docker images | grep -v ^REPO | sed 's/ \+/:/g' | cut -d: -f1,2 | xargs -L1 docker pull
$ docker-compose build
```
- Then restart the server and start a fresh harvest
- Continue splitting the Solr statistics into yearly shards on DSpace Test (doing 2017 today)
<!-- vim: set sw=2 ts=2: -->