--- title: "January, 2019" date: 2019-01-02T09:48:30+02:00 author: "Alan Orth" tags: ["Notes"] --- ## 2019-01-02 - Linode alerted that CGSpace (linode18) had a higher outbound traffic rate than normal early this morning - I don't see anything interesting in the web server logs around that time though: ``` # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "02/Jan/2019:0(1|2|3)" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10 92 40.77.167.4 99 210.7.29.100 120 38.126.157.45 177 35.237.175.180 177 40.77.167.32 216 66.249.75.219 225 18.203.76.93 261 46.101.86.248 357 207.46.13.1 903 54.70.40.11 ``` - Analyzing the types of requests made by the top few IPs during that time: ``` # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "02/Jan/2019:0(1|2|3)" | grep 54.70.40.11 | grep -o -E "(bitstream|discover|handle)" | sort | uniq -c 30 bitstream 534 discover 352 handle # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "02/Jan/2019:0(1|2|3)" | grep 207.46.13.1 | grep -o -E "(bitstream|discover|handle)" | sort | uniq -c 194 bitstream 345 handle # zcat --force /var/log/nginx/*.log /var/log/nginx/*.log.1 | grep -E "02/Jan/2019:0(1|2|3)" | grep 46.101.86.248 | grep -o -E "(bitstream|discover|handle)" | sort | uniq -c 261 handle ``` - It's not clear to me what was causing the outbound traffic spike - Oh nice! The once-per-year cron job for rotating the Solr statistics actually worked now (for the first time ever!): ``` Moving: 81742 into core statistics-2010 Moving: 1837285 into core statistics-2011 Moving: 3764612 into core statistics-2012 Moving: 4557946 into core statistics-2013 Moving: 5483684 into core statistics-2014 Moving: 2941736 into core statistics-2015 Moving: 5926070 into core statistics-2016 Moving: 10562554 into core statistics-2017 Moving: 18497180 into core statistics-2018 ``` - This could by why the outbound traffic rate was high, due to the S3 backup that run at 3:30AM... - Run all system updates on DSpace Test (linode19) and reboot the server