cgspace-notes/content/posts/2021-08.md

---
title: "August, 2021"
date: 2021-08-01T09:01:07+03:00
author: "Alan Orth"
categories: ["Notes"]
---

## 2021-08-01

- Update Docker images on AReS server (linode20) and reboot the server:

```console
# docker images | grep -v ^REPO | sed 's/ \+/:/g' | cut -d: -f1,2 | grep -v none | xargs -L1 docker pull
```

- I decided to upgrade linode20 from Ubuntu 18.04 to 20.04

<!--more-->

- First running all existing updates, taking some backups, checking for broken packages, and then rebooting:

```console
# apt update && apt dist-upgrade
# apt autoremove && apt autoclean
# check for any packages with residual configs we can purge
# dpkg -l | grep -E '^rc' | awk '{print $2}'
# dpkg -l | grep -E '^rc' | awk '{print $2}' | xargs dpkg -P
# dpkg -C
# dpkg -l > 2021-08-01-linode20-dpkg.txt
# tar -I zstd -cvf 2021-08-01-etc.tar.zst /etc
# reboot
# sed -i 's/bionic/focal/' /etc/apt/sources.list.d/*.list
# do-release-upgrade
```
- ... but of course it hit [the libxcrypt bug](https://bugs.launchpad.net/ubuntu/+source/libxcrypt/+bug/1903838)
- I had to get a copy of libcrypt.so.1.1.0 from a working Ubuntu 20.04 system and finish the upgrade manually

```console
# apt install -f
# apt dist-upgrade
# reboot
```

- After rebooting I purged all packages with residual configs and cleaned up again:

```console
# dpkg -l | grep -E '^rc' | awk '{print $2}' | xargs dpkg -P
# apt autoremove && apt autoclean
```

- Then I cleared my local Ansible fact cache and re-ran the [infrastructure playbooks](https://github.com/ilri/rmg-ansible-public)
- Open [an issue for the value mappings global replacement bug in OpenRXV](https://github.com/ilri/OpenRXV/issues/111)
- Advise Peter and Abenet on expected CGSpace budget for 2022
- Start a fresh harvesting on AReS (linode20)

## 2021-08-02

- Help Udana with OAI validation on CGSpace
  - He was checking the OAI base URL on OpenArchives and I had to verify the results in order to proceed to Step 2
  - Now it seems to be verified (all green): https://www.openarchives.org/Register/ValidateSite?log=R23ZWX85
  - We are listed in the OpenArchives list of databases conforming to OAI 2.0

## 2021-08-03

- Run fresh re-harvest on AReS

## 2021-08-05

- Have a quick call with Mishell Portilla from CIP about a journal article that was flagged as being in a predatory journal (Beall's List)
  - We agreed to unmap it from RTB's collection for now, and I asked for advice from Peter and Abenet for what to do in the future
- A developer from the Alliance asked for access to the CGSpace database so they can make some integration with PowerBI
  - I told them we don't allow direct database access, and that it would be tricky anyways (that's what APIs are for!)
- I'm curious if there are still any requests coming in to CGSpace from the abusive Russian networks
  - I extracted all the unique IPs that nginx processed in the last week:

```console
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/access.log.2 /var/log/nginx/access.log.3 /var/log/nginx/access.log.4 /var/log/nginx/access.log.5 /var/log/nginx/access.log.6 /var/log/nginx/access.log.7 /var/log/nginx/access.log.8 | grep -E " (200|499) " | grep -v -E "(mahider|Googlebot|Turnitin|Grammarly|Unpaywall|UptimeRobot|bot)" | awk '{print $1}' | sort | uniq > /tmp/2021-08-05-all-ips.txt
# wc -l /tmp/2021-08-05-all-ips.txt
43428 /tmp/2021-08-05-all-ips.txt
```

- Already I can see that the total is much less than during the attack on one weekend last month (over 50,000!)
  - Indeed, now I see that there are no IPs from those networks coming in now:

```console
$ ./ilri/resolve-addresses-geoip2.py -i /tmp/2021-08-05-all-ips.txt -o /tmp/2021-08-05-all-ips.csv
$ csvgrep -c asn -r '^(49453|46844|206485|62282|36352|35913|35624|8100)$' /tmp/2021-08-05-all-ips.csv | csvcut -c ip | sed 1d | sort | uniq > /tmp/2021-08-05-all-ips-to-purge.csv
$ wc -l /tmp/2021-08-05-all-ips-to-purge.csv
0 /tmp/2021-08-05-all-ips-to-purge.csv
```

<!-- vim: set sw=2 ts=2: -->