Commit Graph

947 Commits

Author SHA1 Message Date
37901da5b5
roles/common: update AbuseIPDB lists for nftables 2021-09-08 09:57:58 +03:00
e36ae3b11e
roles/common: Update list of abusive IP addresses
This comes from the AbuseIPDB with a confidence level of 95%. I use
the following command to download and sort the IPs:

  $ curl -G https://api.abuseipdb.com/api/v2/blacklist -d \
    confidenceMinimum=95 -H "Key: $ABUSEIPDB_API_KEY" \
    -H "Accept: text/plain" | sort | sed -e '/:/w /tmp/ipv6.txt' \
    -e '/:/d' > /tmp/ipv4.txt

I manually add the XML formatting to each file and run them through
tidy:

  $ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv4.xml
  $ tidy -xml -utf8 -m -iq -w 0 roles/common/files/abusers-ipv6.xml
2021-09-08 09:35:36 +03:00
81c1231a28
roles/php-fpm: Fix logic
First, we cannot do a global check for has_wordpress or needs_php,
as those are defined per nginx vhost. Second, I realized that this
was only working in the past because vhosts that had WordPress or
needed PHP were listed first in the nginx_vhosts dict.

This changes the logic to first check if any vhosts have WordPress
or need PHP, then sets a fact that we can use to decide whether to
run php-fpm tasks or not.
2021-09-08 09:32:06 +03:00
bb6f058025
roles/php-fpm: whitespace 2021-09-07 20:12:31 +03:00
547395b26e
roles/nginx: Use php7.4-fpm socket on Debian 11 as well 2021-09-07 17:51:54 +03:00
15208241d3
roles/common: Add git-lfs to base packages 2021-09-07 17:51:33 +03:00
0fd05d496e
roles/nginx: Set mode of downloaded acme.sh 2021-09-07 17:10:35 +03:00
023a0d48ba
roles/nginx: Remove old comment 2021-09-07 17:07:53 +03:00
c687b7a91a
roles/nginx: Run Let's Encrypt on Debian 11 too 2021-09-07 17:07:33 +03:00
bd4ae36bb6
roles/mariadb: use socket for all operations
Otherwise Ansible will try to connect with host 'localhost', which
we do not use (and we have disabled name resolution anyways).
2021-09-07 16:48:15 +03:00
b60637c7d9
roles/mariadb: Update comments for Ansible module 2021-09-07 16:47:47 +03:00
479127a5e4
roles/common: Fix nftables handler in Debian firewall
We used to use reload, but now the idempotent thing to do is to use
restart instead of reload.
2021-09-07 15:43:33 +03:00
d261f81642
roles/php-fpm: Use Ubuntu 20.04 configs on Debian 11
They both use PHP 7.4.
2021-09-06 21:19:57 +03:00
6bc044d454
host_vars: remove mosh rules
They are in roles/common now.
2021-09-05 16:33:45 +03:00
9e07e27fbe
host_vars/web19: remove extra mosh rules
These are now in the common role for all hosts.
2021-09-05 16:24:28 +03:00
575a9fdfe6
roles/common: Add mosh ports to common
These have been in each hosts's "extra" rules lists forever and I
use them on every single host so they might as well be in the base
rules.
2021-09-05 16:23:42 +03:00
35fa3b0d72
roles/common: Fix typo in handlers 2021-09-05 16:19:31 +03:00
ba5760bf8c
host_vars/web19: WordPress 5.8 2021-09-05 15:55:58 +03:00
5e918da88e
Pipfile.lock: run pipenv update 2021-09-05 15:38:51 +03:00
f7e87ea7be
roles/common: Fix fail2ban ignoreip
According to jail.conf we actually need to separate multiple values
with spaces instead of commas. On some versions of fail2ban this is
a fatal error:

> CRITICAL Unhandled exception in Fail2Ban:
> Traceback (most recent call last):
>   File "/usr/lib/python3/dist-packages/fail2ban/server/jailthread.py", line 66, in run_with_except_hook
>     run(*args, **kwargs)
>   File "/usr/lib/python3/dist-packages/fail2ban/server/filtersystemd.py", line 246, in run
>     *self.formatJournalEntry(logentry))
>   File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 432, in processLineAndAdd
>     if self.inIgnoreIPList(ip, log_ignore=True):
>   File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 371, in inIgnoreIPList
>     "(?<=b)1+", bin(DNSUtils.addr2bin(s[1]))).group())
>   File "/usr/lib/python3/dist-packages/fail2ban/server/filter.py", line 928, in addr2bin
>     return struct.unpack("!L", socket.inet_aton(ipstring))[0]
> OSError: illegal IP address string passed to inet_aton

This affects (at least) fail2ban 0.9.3 on Ubuntu 16.04, but I never
noticed.
2021-08-12 15:24:50 +03:00
7b233eb31d
Pipfile.lock: Run pipenv update
Ansible 4.3.0
2021-08-01 16:14:42 +03:00
b5ea575d8d
roles/common: Always restart nftables service
The "reload" capability only exists on Ubuntu, and it is exactly
the same as the "restart" functionality.
2021-08-01 14:23:00 +03:00
98cc3a8c2e
Add nginx filter for fail2ban
Some hosts can use fail2ban's nginx-botsearch filter to ban anyone
making requests to non-existent files like wp-login.php. There is
no reason to request such files naively and anyone found doing so
can be banned immediately.

In theory I should report them to AbuseIPDB.com, but that will take
a little more wiring up.
2021-08-01 09:56:43 +03:00
a67d901641
roles/common: Use AbuseIPDB.com list in nftables
For now I am still manually updating this, as we can only hit their
API five times per day, so it is not possible to have each host get
the list themselves every day with our one API key.
2021-07-31 21:46:50 +03:00
7ae100faeb
roles/common: Add comments to nftables.conf 2021-07-30 09:37:30 +03:00
debcb21161
roles/common: Install curl for Abuse.ch update scripts 2021-07-29 10:24:32 +03:00
8dd7663b3c
roles/common: Use Abuse.ch's SSL Blacklist in nftables
This adds Abuse.sh's list of IPs using blacklisted SSL certificates
to nftables. These IPs are high confidence indicators of compromise
and we should not route them. The list is updated daily by a systemd
timer.

See: https://sslbl.abuse.ch/blacklist/
2021-07-29 10:16:00 +03:00
cba2a7a996
roles/common: Fix nftables in Debian firewall
The previous commit meant to move the service start, not the config
copying task.
2021-07-29 10:10:04 +03:00
197bdf7666
roles/common: Start nftables service later
We should only try to start the nftables service after we finish
copying all the config files just in case there is some unclean
state in one of them. On a first run this shouldn't matter, but
after nftables and some abuse list update scripts have run this
can happen (mostly in testing!).
2021-07-29 10:05:15 +03:00
46fc2ce3d4
roles/common: Move cleanup to a one-off play
We only need to run this once on existing hosts that are using the
old firewalld/ipsets setup before applying the new nftables config.
2021-07-29 10:00:30 +03:00
b4d50166f4
roles/common: Fix loop in firewall cleanup 2021-07-28 23:46:53 +03:00
c336b217c5
Remove extra TCP ports from firewall rules
Now all web hosts get TCP 80 and 443 open automatically.
2021-07-28 14:49:50 +03:00
af6c3dd12a
roles/common: Update cache in firewall playbook
cron-apt updates the system against the security-only databases at
night so many packages are "missing" unless you run apt update. We
need to update the cache on all apt tasks actually because I might
be running them by their tag and they currently only get updated at
the beginning of the playbook.
2021-07-28 14:46:58 +03:00
b66c724109
roles/common: Use nftables on Ubuntu 20.04 as well
This mostly copies the Debian 11 nftables setup and includes a play
to clean up the old firewalld settings, timers, etc.
2021-07-28 14:18:41 +03:00
8bc2b6f493
roles/common: Retab nftables.conf.j2 2021-07-27 22:03:23 +03:00
a74d6dfc08
roles/common: Don't overwrite spamhaus nft sets
The ones in this repo are only placeholders that get updated by the
update-spamhaus-nftables service, so we shouldn't overwrite them if
they exist.
2021-07-27 22:01:57 +03:00
d3922e7878
roles/common: Port configurable firewall logic to nftables
This opens TCP port 22 on all hosts, TCP ports 80 and 443 on hosts
in the web group, and allows configuration of "extra" rules in the
host or group vars.
2021-07-27 21:22:32 +03:00
14814aa5d9
roles/common: Wire up fail2ban
The nftables support works easily and creates the table, chains, and
sets on demand.
2021-07-26 22:07:31 +03:00
3b053167b1
roles/common: Fix sources.list for Debian 11 Bullseye
Seems the path to the security updates repo changed.
2021-07-26 21:12:05 +03:00
9bba0d96bb
roles/common: Add initial support for nftables on Debian 11
I will try using nftables directly instead of via firewalld as of
Debian 11 as it is the replacement for the iptables/ipset stack in
recent years and is easier to work with.

This also includes a systemd service, timer, and script to update
the spamhaus DROP lists as nftables sets.

Still need to add fail2ban support.
2021-07-26 13:09:41 +03:00
38c333045b
roles/common: bring Ubuntu firewall changes to Debian 11
Note that there is currently an issue loading the spamhaus rules on
Debian 11 when using ipsets with firewalld and the nftables backend.
The bug is apparently caused by overlapping CIDR segments, and the
solution appears to be that we need to manually aggregate them with
a tool like aggregate6 (Python).

See: https://bugzilla.redhat.com/show_bug.cgi?id=1836571
See: https://wiki.fysik.dtu.dk/it/Linux_firewall_configuration#using-ipsets-in-firewalld-on-rhel-centos-8
See: https://github.com/job/aggregate6
2021-07-24 23:09:33 +03:00
d4ede33099
roles/common: Don't configure apt sources on ARM
I was using this on Ubuntu, but might as well bring it here too so
that I can run Debian on Scaleway's ARM instances, for example.
2021-07-24 22:32:20 +03:00
0bad75788d
roles/common: Add encryption settings to Debian 11 sshd_config
Mostly based on the ssh-audit policy for OpenSSH 8.4, but with any
algorithms using less than 256 bits removed. NSA's Suite B removed
these long ago, and the new CNSA suite only uses 256 and up.

See: https://github.com/jtesta/ssh-audit/blob/master/src/ssh_audit/policy.py
See: https://en.wikipedia.org/wiki/Commercial_National_Security_Algorithm_Suite
2021-07-24 22:28:59 +03:00
892033b880
roles/common: port common settings to Debian 11 sshd_config
Still need to add the encryption settings.
2021-07-22 14:16:20 +03:00
7c6ab2a652 roles/common: Add sshd_config from Debian 11 RC2 2021-07-22 14:15:00 +03:00
1c95c1faa8
roles/common: Update KexAlgorithms in Ubuntu 20.04 sshd_config
Recommended by ssh-audit. Note that curve25519-sha256 is the new name
for the previously private implementation in libssh.
2021-07-22 12:57:31 +03:00
9ea14de6f5
roles/common: Remove Encrypt-and-MAC modes from Ubuntu 20.04 sshd_config
Recommended by ssh-audit, but also generally the concensus for a few
years that Encrypt-and-MAC is hard to get right. OpenSSH has several
Encrypt-then-MAC schemes available so we can use those.

See: https://www.daemonology.net/blog/2009-06-24-encrypt-then-mac.html
2021-07-22 12:48:12 +03:00
9b7a31ebf9
roles/common: Remove 00-persistent-journal.conf
This was to enable the persistent systemd journal, but it is no lo-
nger needed as of Ubuntu 18.04 and Debian 11. I had removed the ta-
asks long ago, but forgot to remove this file.
2021-07-21 10:02:33 +03:00
d7c34a30a3
roles/common: Add Spamhaus DROP lists to firewalld ipsets
This configures the recommended DROP, EDROP, and DROPv6 lists from
Spamhaus as ipsets in firewalld. First we copy an empty placeholder
ipset to seed firewalld, then we use a shell script to download the
real lists and activate them. The same shell script is run daily as
a service (update-spamhaus-lists.service) by a systemd timer.

I am strictly avoiding any direct ipset commands here because I want
to make sure that this works on older hosts where ipsets is used as
well as newer hosts that have moved to nftables such as Ubuntu 20.04.
So far I have tested this on Ubuntu 16.04, 18.04, and 20.04, but ev-
entually I need to abstract the tasks and run them on CentOS 7+ as
well.

See: https://www.spamhaus.org/drop/
2021-07-21 09:34:51 +03:00
ee5f4cdf74
host_vars: don't hard code python path
We now use auto mode in ansible.cfg, so we don't need to hard code
this in every single host's vars file.

See: https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html
2021-07-07 12:23:05 +03:00