According the the Debian docs for third-party repositories we must
create this manually on distros before Debian 12 and Ubuntu 22.04.
This is due to changes in apt-secure and the deprecation of apt-key.
See: https://wiki.debian.org/DebianRepository/UseThirdParty
According to Ansible we can use yes, true, True, "or any quoted st-
ring" for a boolean true, but ansible-lint wants us to use either
true or false.
See: https://chronicler.tech/red-hat-ansible-yes-no-and/
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
Then I formatted the nftables files manually. Meh...
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
Then I formatted the nftables files manually. Meh...
This seems to be automatically selected, but on some other servers
I notice it is not. I will set it here explicitly so fail2ban does
not fall back to the inefficient "polling" or incorrect "systemd"
backends.
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
Then I created the nftables files manually. Meh...
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
The WeakDH team showed (in 2015) that Diffie-Hellman key exchange
with prime number groups of 1024 bits or less were weaker than we
previously thought, and well within the reach of nation states. They
recommended (in 2015) using 2048-bit or higher prime groups.
The SSH audit project recommends that we should use 3072-bit now.
See: https://weakdh.org/
See: https://github.com/jtesta/ssh-audit/
We should always restart fail2ban after updating the firewall. Also
note that the order of execution of handlers depends on how they are
defined in the handler config, not on the order they are listed in
the task's notify statement.
See: https://docs.ansible.com/ansible/latest/user_guide/playbooks_handlers.html
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
Note: there were no IPv6 addresses in the top 10,000 this time so I
used a dummy address for the nftables set so the syntax was valid.
It is a standalone package on (at least) Ubuntu 20.04 and Debian 11
and some cloud images do not have it installed by default (for exa-
mple Scaleway).
We have to force these because they are not updated on the host like
the other lists (API limit of five requests per day!). We update the
list periodically here in git.
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:
$ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
-H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
-e '/:/d' > /tmp/ipv4.txt
I manually add the XML formatting to each file and run them through
tidy:
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
$ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
According to jail.conf we actually need to separate multiple values
with spaces instead of commas. On some versions of fail2ban this is
a fatal error:
> CRITICAL Unhandled exception in Fail2Ban:
> Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/fail2ban/server/jailthread.py", line 66, in run_with_except_hook
> run(*args, **kwargs)
> File "/usr/lib/python3/dist-packages/fail2ban/server/filtersystemd.py", line 246, in run
> *self.formatJournalEntry(logentry))
> File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 432, in processLineAndAdd
> if self.inIgnoreIPList(ip, log_ignore=True):
> File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 371, in inIgnoreIPList
> "(?<=b)1+", bin(DNSUtils.addr2bin(s[1]))).group())
> File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 928, in addr2bin
> return struct.unpack("!L", socket.inet_aton(ipstring))[0]
> OSError: illegal IP address string passed to inet_aton
This affects (at least) fail2ban 0.9.3 on Ubuntu 16.04, but I never
noticed.
Some hosts can use fail2ban's nginx-botsearch filter to ban anyone
making requests to non-existent files like wp-login.php. There is
no reason to request such files naively and anyone found doing so
can be banned immediately.
In theory I should report them to AbuseIPDB.com, but that will take
a little more wiring up.
For now I am still manually updating this, as we can only hit their
API five times per day, so it is not possible to have each host get
the list themselves every day with our one API key.
This adds Abuse.sh's list of IPs using blacklisted SSL certificates
to nftables. These IPs are high confidence indicators of compromise
and we should not route them. The list is updated daily by a systemd
timer.
See: https://sslbl.abuse.ch/blacklist/
We should only try to start the nftables service after we finish
copying all the config files just in case there is some unclean
state in one of them. On a first run this shouldn't matter, but
after nftables and some abuse list update scripts have run this
can happen (mostly in testing!).
cron-apt updates the system against the security-only databases at
night so many packages are "missing" unless you run apt update. We
need to update the cache on all apt tasks actually because I might
be running them by their tag and they currently only get updated at
the beginning of the playbook.
This opens TCP port 22 on all hosts, TCP ports 80 and 443 on hosts
in the web group, and allows configuration of "extra" rules in the
host or group vars.
I will try using nftables directly instead of via firewalld as of
Debian 11 as it is the replacement for the iptables/ipset stack in
recent years and is easier to work with.
This also includes a systemd service, timer, and script to update
the spamhaus DROP lists as nftables sets.
Still need to add fail2ban support.
