Some hosts can use fail2ban's nginx-botsearch filter to ban anyone
making requests to non-existent files like wp-login.php. There is
no reason to request such files naively and anyone found doing so
can be banned immediately.
In theory I should report them to AbuseIPDB.com, but that will take
a little more wiring up.
For now I am still manually updating this, as we can only hit their
API five times per day, so it is not possible to have each host get
the list themselves every day with our one API key.
This adds Abuse.sh's list of IPs using blacklisted SSL certificates
to nftables. These IPs are high confidence indicators of compromise
and we should not route them. The list is updated daily by a systemd
timer.
See: https://sslbl.abuse.ch/blacklist/
This opens TCP port 22 on all hosts, TCP ports 80 and 443 on hosts
in the web group, and allows configuration of "extra" rules in the
host or group vars.
I will try using nftables directly instead of via firewalld as of
Debian 11 as it is the replacement for the iptables/ipset stack in
recent years and is easier to work with.
This also includes a systemd service, timer, and script to update
the spamhaus DROP lists as nftables sets.
Still need to add fail2ban support.
Recommended by ssh-audit, but also generally the concensus for a few
years that Encrypt-and-MAC is hard to get right. OpenSSH has several
Encrypt-then-MAC schemes available so we can use those.
See: https://www.daemonology.net/blog/2009-06-24-encrypt-then-mac.html
This configures the recommended DROP, EDROP, and DROPv6 lists from
Spamhaus as ipsets in firewalld. First we copy an empty placeholder
ipset to seed firewalld, then we use a shell script to download the
real lists and activate them. The same shell script is run daily as
a service (update-spamhaus-lists.service) by a systemd timer.
I am strictly avoiding any direct ipset commands here because I want
to make sure that this works on older hosts where ipsets is used as
well as newer hosts that have moved to nftables such as Ubuntu 20.04.
So far I have tested this on Ubuntu 16.04, 18.04, and 20.04, but ev-
entually I need to abstract the tasks and run them on CentOS 7+ as
well.
See: https://www.spamhaus.org/drop/
This is active banning of IPs that are brute forcing login attempts
to SSH, versus the passive banning of 10,000 abusive IPs from the
abuseipdb.com blacklist. For now I am banning IPs that fail to log
in successfully more than twelve times in a one-hour period, but
these settings might change, and I can override them at the group
and host level if needed.
Currently this works for CentOS 7, Ubuntu 16.04, and Ubuntu 18.04,
with minor differences in the systemd configuration due to older
versions on some distributions.
You can see the status of the jail like this:
# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
| |- Currently failed: 0
| |- Total failed: 0
| `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
|- Currently banned: 1
|- Total banned: 1
`- Banned IP list: 106.13.112.20
You can unban IPs like this:
# fail2ban-client set sshd unbanip 106.13.112.20
Now that I'm blocking ~10,000 malicious IPs from AbuseIPDB I feel
more comfortable using a more relaxed rate limit for SSH. A limit
of 12 per minute is about one every five seconds.
If a user has RSA, ECDSA, and ED25519 private keys present on their
system then the ssh client will offer all of these to the server
and they may not get a chance to try password auth before it fails.
There is a bug in iptables 1.8.2 in Debian 10 "Buster" that causes
firewalld to fail when restoring rules. The bug has been fixed in
iptables 1.8.3, which is currently in buster-backports.
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914694
The default in later OpenSSH is 6, which seems too high. If you can't
get your password correct after 3 tries then I think you need help.
Eventually I'd like an easy way to enable blocking of repeated login
attempts at the firewall level. I think it's possible in firewalld.
Vanilla Ubuntu (and Debian actually) defaults to using the official
mirror for security updates rather than country or regional mirrors.
Also, for what it's worth, Ubuntu mirrors didn't always sync these
security archives. I'd prefer to stay closer to vanilla Ubuntu but
also it kinda makes sense to get security updates from the official
source than a mirror (in case of delay or errors).
