2022-06-06
- Look at the Solr statistics on CGSpace
- I see 167,000 hits from a bunch of Microsoft IPs with reverse DNS “msnbot-” using the Solr query
dns:*msnbot* AND dns:*.msn.com
- I purged these first so I could see the other “real” IPs in the Solr facets
- I see 47,500 hits from 80.248.237.167 on a data center ISP in Sweden, using a normal user agent
- I see 13,000 hits from 163.237.216.11 on a data center ISP in Australia, using a normal user agent
- I see 7,300 hits from 208.185.238.57 from Britanica, using a normal user agent
- There seem to be many more of these:
# zcat --force /var/log/nginx/access.log* | grep 208.185.238. | awk '{print $1}' | sort | uniq -c | sort -h
2 208.185.238.1
166 208.185.238.54
1293 208.185.238.51
2587 208.185.238.59
4692 208.185.238.56
5480 208.185.238.53
6277 208.185.238.52
6400 208.185.238.58
8261 208.185.238.55
17549 208.185.238.57
- I see 3,000 hits from 178.208.75.33 by a Russian-owned IP in the Netherlands that is making a GET to / every one minute, using a normal user agent
- I see 3,000 hits from 134.122.124.196 on Digital Ocean to the REST API with a normal user agent
- I purged all these hits from IPs for a total of about 265,000
- Then I faceted by user agent and found
- 1,000 hits by
insomnia/2022.2.1, which I also saw last month and submitted to COUNTER-Robots
- 265 hits by
omgili/0.5 +http://omgili.com
- 150 hits by
Vizzit
- 132 hits by
MetaInspector/5.7.0 (+https://github.com/jaimeiniesta/metainspector)
- 73 hits by
Scoop.it
- 62 hits by
bitdiscovery
- 59 hits by
Asana/1.4.0 WebsiteMetadataRetriever
- 32 hits by
Sprout Social (Link Attachment)
- 29 hits by
CyotekWebCopy/1.9 CyotekHTTP/6.2
- 20 hits by
Hootsuite-Authoring/1.0
- I purged about 4,100 hits from these user agents
- Run all system updates on AReS server (linode20) and reboot
- I want to try to update some of the build dependencies of OpenRXV since Node.js 12 is no longer supported
- Upgrade linode20 to Ubuntu 22.04 and start an AReS harvest
- I merged the Mirage 2 build fix to
dspace-6_x for DSpace 6.4
2022-06-07
- I tested Node.js 14 one more time with vanilla DSpace 6.4-SNAPSHOT and with the CGSpace source and it worked well
- I made a pull request to DSpace to use Node.js 14 for Mirage 2
- I even tested Node.js 16 and it works, but that is enough for now…
2022-06-08
- Work on AReS a bit since I wasn’t able to harvest after doing the updates on the server and in the containers a few days ago
- I don’t know what the problem was really, but on the server I had to enable IPv4 forwarding so the frontend container would build
- Once I downed and upped AReS with docker-compose I was able to start a new harvest
- I also did some tests to enable ES2020 target in the backend because we’re on Node.js 14 there now
2022-06-13
- Create a user for Mohammed Salem to test MEL submission on DSpace Test:
$ dspace user -a -m mel-submit@cgiar.org -g MEL -s Submit -p 'owwwwwwww'
- According to my notes from 2020-10 the account must be in the admin group in order to submit via the REST API
2022-06-14
2022-06-16
- Francesca asked us to add the CC-BY-3.0-IGO license to the submission form on CGSpace
- I remember I had requested SPDX to add CC-BY-NC-ND-3.0-IGO in 2019-02, and they finally merged it in 2020-07, but I never added it to CGSpace
- I will add the full suite of CC 3.0 IGO licenses to CGSpace and then make a request to SPDX for the others:
- CC-BY-3.0-IGO
- CC-BY-SA-3.0-IGO
- CC-BY-ND-3.0-IGO
- CC-BY-NC-3.0-IGO
- CC-BY-NC-SA-3.0-IGO
- CC-BY-NC-ND-3.0-IGO
- I filed an issue asking for SPDX to add CC-BY-3.0-IGO
- Meeting with Moayad from CodeObia to discuss OpenRXV
- He added the ability to use multiple indexes / dashboards, and to be able to embed them in iframes
- Add
cg.contributor.initiative with a controlled vocabulary based on CLARISA’s list to the CGSpace submission form