diff --git a/content/2016-04.md b/content/2016-04.md index 9f0e68867..66fda625e 100644 --- a/content/2016-04.md +++ b/content/2016-04.md @@ -43,3 +43,17 @@ java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290 - For what it's worth, we have been setting the actual Tomcat 7 process' limit to 16384 for a few years (in `/etc/default/tomcat7`) - Looks like cron will read limits from `/etc/security/limits.*` so we can do something for the tomcat7 user there - Submit pull request for Tomcat 7 limits in Ansible dspace role ([#30](https://github.com/ilri/rmg-ansible-public/pull/30)) + +## 2016-04-05 + +- Reduce Amazon S3 storage used for logs from 46 GB to 6GB by deleting a bunch of logs we don't need! + +``` +# s3cmd ls s3://cgspace.cgiar.org/log/ > /tmp/s3-logs.txt +# grep checker.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep cocoon.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep handle-plugin.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep solr.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +``` + +- Also, adjust the cron jobs for backups so they only backup `dspace.log` and some stats files (.dat) diff --git a/public/2016-04/index.html b/public/2016-04/index.html index 0a11455f4..4102c4a9f 100644 --- a/public/2016-04/index.html +++ b/public/2016-04/index.html @@ -118,6 +118,23 @@ java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290
  • For what it’s worth, we have been setting the actual Tomcat 7 process’ limit to 16384 for a few years (in /etc/default/tomcat7)
  • Looks like cron will read limits from /etc/security/limits.* so we can do something for the tomcat7 user there
  • Submit pull request for Tomcat 7 limits in Ansible dspace role (#30)
  • + + +

    2016-04-05

    + + + +
    # s3cmd ls s3://cgspace.cgiar.org/log/ > /tmp/s3-logs.txt
    +# grep checker.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del
    +# grep cocoon.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del
    +# grep handle-plugin.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del
    +# grep solr.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del
    +
    + + diff --git a/public/index.xml b/public/index.xml index a675676da..99ca4df9a 100644 --- a/public/index.xml +++ b/public/index.xml @@ -57,6 +57,23 @@ java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290 <li>Looks like cron will read limits from <code>/etc/security/limits.*</code> so we can do something for the tomcat7 user there</li> <li>Submit pull request for Tomcat 7 limits in Ansible dspace role (<a href="https://github.com/ilri/rmg-ansible-public/pull/30">#30</a>)</li> </ul> + +<h2 id="2016-04-05:c88be15f5b2f07c85f7742556a955e47">2016-04-05</h2> + +<ul> +<li>Reduce Amazon S3 storage used for logs from 46 GB to 6GB by deleting a bunch of logs we don&rsquo;t need!</li> +</ul> + +<pre><code># s3cmd ls s3://cgspace.cgiar.org/log/ &gt; /tmp/s3-logs.txt +# grep checker.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep cocoon.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep handle-plugin.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep solr.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +</code></pre> + +<ul> +<li>Also, adjust the cron jobs for backups so they only backup <code>dspace.log</code> and some stats files (.dat)</li> +</ul> diff --git a/public/tags/notes/index.xml b/public/tags/notes/index.xml index 6ede670cf..b3f9cfca8 100644 --- a/public/tags/notes/index.xml +++ b/public/tags/notes/index.xml @@ -57,6 +57,23 @@ java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290 <li>Looks like cron will read limits from <code>/etc/security/limits.*</code> so we can do something for the tomcat7 user there</li> <li>Submit pull request for Tomcat 7 limits in Ansible dspace role (<a href="https://github.com/ilri/rmg-ansible-public/pull/30">#30</a>)</li> </ul> + +<h2 id="2016-04-05:c88be15f5b2f07c85f7742556a955e47">2016-04-05</h2> + +<ul> +<li>Reduce Amazon S3 storage used for logs from 46 GB to 6GB by deleting a bunch of logs we don&rsquo;t need!</li> +</ul> + +<pre><code># s3cmd ls s3://cgspace.cgiar.org/log/ &gt; /tmp/s3-logs.txt +# grep checker.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep cocoon.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep handle-plugin.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +# grep solr.log /tmp/s3-logs.txt | awk '{print $4}' | xargs s3cmd del +</code></pre> + +<ul> +<li>Also, adjust the cron jobs for backups so they only backup <code>dspace.log</code> and some stats files (.dat)</li> +</ul>