For some reason the nftables set support in firewalld doesn't seem
to be working. I see that sets (aka ipsets in nftables lingo) are
created, but they are empty. For now I will just leave these tasks
as they are to revert the behavior on current hosts (should do no
change on new installed, as the regexp won't match).
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" -H "Accept: text/plain" | sort > /tmp/ips.txt
I manually remove the IPv6 addresses and save them to a different
filr, then I add the XML formatting to files and run them through
tidy.
This is active banning of IPs that are brute forcing login attempts
to SSH, versus the passive banning of 10,000 abusive IPs from the
abuseipdb.com blacklist. For now I am banning IPs that fail to log
in successfully more than twelve times in a one-hour period, but
these settings might change, and I can override them at the group
and host level if needed.
Currently this works for CentOS 7, Ubuntu 16.04, and Ubuntu 18.04,
with minor differences in the systemd configuration due to older
versions on some distributions.
You can see the status of the jail like this:
# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
| |- Currently failed: 0
| |- Total failed: 0
| `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
|- Currently banned: 1
|- Total banned: 1
`- Banned IP list: 106.13.112.20
You can unban IPs like this:
# fail2ban-client set sshd unbanip 106.13.112.20
Seems to work around an issue when firewalld is using the nftables
backend with iptables 1.8.2 on Debian 10. Alternatively I could go
back to using the iptables backend... hmm.
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914694
This causes problems every time I re-run the Ansible tag because the
nightly apt security uses a different sources.list and the indexes
are then missing buster-backports. I could add a cache update to the
task, but actually I think the original bug I was trying to solve is
finally fixed, and I'm going to switch to nftables anyways.
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" -H "Accept: text/plain" | sort > /tmp/ips.txt
Then I add the XML formatting to the file and run it through tidy.
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" -H "Accept: text/plain" | sort > /tmp/ips.txt
Then I add the XML formatting to the file and run it through tidy.
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" -H "Accept: text/plain" | sort > /tmp/ips.txt
Then I add the XML formatting to the file and run it through tidy.
Now that I'm blocking ~10,000 malicious IPs from AbuseIPDB I feel
more comfortable using a more relaxed rate limit for SSH. A limit
of 12 per minute is about one every five seconds.
I updated the list with a few dozen more hosts that we brute forcing
SSH but failed to even negotiate a connection because they are using
old ciphers. I will still block them because they attempted 100+ co-
nnections.
This uses the ipsets feature of the Linux kernel to create lists of
IPs (though could be MACs, IP:port, etc) that we can block via the
existing firewalld zone we are already using. In my testing it works
on CentOS 7, Ubuntu 16.04, and Ubuntu 18.04.
The list of abusive IPs currently comes from HPC's systemd journal,
where I filtered for hosts that had attempted and failed to log in
over 100 times. The list is formatted with tidy, for example:
$ tidy -xml -iq -m -w 0 roles/common/files/abusers-ipv4.xml
See: https://firewalld.org/2015/12/ipset-support
If a user has RSA, ECDSA, and ED25519 private keys present on their
system then the ssh client will offer all of these to the server
and they may not get a chance to try password auth before it fails.
There is a bug in iptables 1.8.2 in Debian 10 "Buster" that causes
firewalld to fail when restoring rules. The bug has been fixed in
iptables 1.8.3, which is currently in buster-backports.
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914694
For some reason the key ID I had here was wrong. According to the
Tarsnap website the key ID is 0x6D97F5A4CA38CF33.
ee: https://www.tarsnap.com/pkg-deb.html
Debian 10 comes with Python 2 and Python 3 (at least from the ISO),
so we should prefer the Python 3 version of pycurl. We'll see whet-
her cloud providers like Linode and Digital Ocean ship with Python
3 or not in their default image.
We can register changes when adding repositories and keys and then
update the apt package index conditionally. This should make it be
more consistent between initial host setup and subsequent re-runs.
Ansible errors on adding the tarsnap signing key because it is not
valid (expired a month ago). I contacted Colin Percival about this
on Twitter but he did not seem worried for some reason.
