Add notes for 2016-04-04

Signed-off-by: Alan Orth <alan.orth@gmail.com>
This commit is contained in:
Alan Orth 2016-04-04 12:37:05 +03:00
parent e272cddd99
commit 1467eefde0
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
9 changed files with 387 additions and 5 deletions

45
content/2016-04.md Normal file
View File

@ -0,0 +1,45 @@
+++
date = "2016-04-04T11:06:00+03:00"
author = "Alan Orth"
title = "April, 2016"
tags = ["notes"]
image = "../images/bg.jpg"
+++
## 2016-04-04
- Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit
- We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc
- After running DSpace for over five years I've never needed to look in any other log file than dspace.log, leave alone one from last year!
- This will save us a few gigs of backup space we're paying for on S3
- Also, I noticed the `checker` log has some errors we should pay attention to:
```
Run start time: 03/06/2016 04:00:22
Error retrieving bitstream ID 71274 from asset store.
java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290601546459645925328536011917633626 (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at edu.sdsc.grid.io.local.LocalFileInputStream.open(LocalFileInputStream.java:171)
at edu.sdsc.grid.io.GeneralFileInputStream.<init>(GeneralFileInputStream.java:145)
at edu.sdsc.grid.io.local.LocalFileInputStream.<init>(LocalFileInputStream.java:139)
at edu.sdsc.grid.io.FileFactory.newFileInputStream(FileFactory.java:630)
at org.dspace.storage.bitstore.BitstreamStorageManager.retrieve(BitstreamStorageManager.java:525)
at org.dspace.checker.BitstreamDAO.getBitstream(BitstreamDAO.java:60)
at org.dspace.checker.CheckerCommand.processBitstream(CheckerCommand.java:303)
at org.dspace.checker.CheckerCommand.checkBitstream(CheckerCommand.java:171)
at org.dspace.checker.CheckerCommand.process(CheckerCommand.java:120)
at org.dspace.app.checker.ChecksumChecker.main(ChecksumChecker.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
******************************************************
```
- So this would be the `tomcat7` Unix user, who seems to have a default limit of 1024 files in its shell
- For what it's worth, we have been setting the actual Tomcat 7 process' limit to 16384 for a few years (in `/etc/default/tomcat7`)
- Looks like cron will read limits from `/etc/security/limits.*` so we can do something for the tomcat7 user there
- Submit pull request for Tomcat 7 limits in Ansible dspace role ([#30](https://github.com/ilri/rmg-ansible-public/pull/30))

View File

@ -337,7 +337,7 @@
<li class="previous"><a href="/cgspace-notes/2016-02/"><span aria-hidden="true">&larr;</span> Older</a></li>
<li class="next disabled"><a href="#">Newer <span aria-hidden="true">&rarr;</span></a></li>
<li class="next"><a href="/cgspace-notes/2016-04/">Newer <span aria-hidden="true">&rarr;</span></a></li>
</ul>
</footer>

176
public/2016-04/index.html Normal file
View File

@ -0,0 +1,176 @@
<!DOCTYPE html>
<html lang="en-us">
<head prefix="og: http://ogp.me/ns#">
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1" />
<meta property="og:title" content=" April, 2016 &middot; CGSpace Notes" />
<meta property="og:site_name" content="CGSpace Notes" />
<meta property="og:url" content="/cgspace-notes/2016-04/" />
<meta property="og:type" content="article" />
<meta property="og:article:published_time" content="2016-04-04T11:06:00&#43;03:00" />
<meta property="og:article:tag" content="notes" />
<title>
April, 2016 &middot; CGSpace Notes
</title>
<link rel="stylesheet" href="/cgspace-notes/css/bootstrap.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/main.css" />
<link rel="stylesheet" href="/cgspace-notes/css/font-awesome.min.css" />
<link rel="stylesheet" href="/cgspace-notes/css/github.css" />
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Source+Sans+Pro:200,300,400" type="text/css">
<link rel="shortcut icon" href="/cgspace-notes/images/favicon.ico" />
<link rel="apple-touch-icon" href="/cgspace-notes/images/apple-touch-icon.png" />
</head>
<body>
<header class="global-header" style="background-image:url(../images/bg.jpg )">
<section class="header-text">
<h1><a href="/cgspace-notes/">CGSpace Notes</a></h1>
<div class="sns-links hidden-print">
</div>
<a href="/cgspace-notes/" class="btn-header btn-back hidden-xs">
<i class="fa fa-angle-left" aria-hidden="true"></i>
&nbsp;Home
</a>
</section>
</header>
<main class="container">
<article>
<header>
<h1 class="text-primary">April, 2016</h1>
<div class="post-meta clearfix">
<div class="post-date pull-left">
Posted on
<time datetime="2016-04-04T11:06:00&#43;03:00">
Apr 4, 2016
</time>
</div>
<div class="pull-right">
<span class="post-tag small"><a href="/cgspace-notes//tags/notes">#notes</a></span>
</div>
</div>
</header>
<section>
<h2 id="2016-04-04:c88be15f5b2f07c85f7742556a955e47">2016-04-04</h2>
<ul>
<li>Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit</li>
<li>We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc</li>
<li>After running DSpace for over five years I&rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year!</li>
<li>This will save us a few gigs of backup space we&rsquo;re paying for on S3</li>
<li>Also, I noticed the <code>checker</code> log has some errors we should pay attention to:</li>
</ul>
<pre><code>Run start time: 03/06/2016 04:00:22
Error retrieving bitstream ID 71274 from asset store.
java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290601546459645925328536011917633626 (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.&lt;init&gt;(FileInputStream.java:146)
at edu.sdsc.grid.io.local.LocalFileInputStream.open(LocalFileInputStream.java:171)
at edu.sdsc.grid.io.GeneralFileInputStream.&lt;init&gt;(GeneralFileInputStream.java:145)
at edu.sdsc.grid.io.local.LocalFileInputStream.&lt;init&gt;(LocalFileInputStream.java:139)
at edu.sdsc.grid.io.FileFactory.newFileInputStream(FileFactory.java:630)
at org.dspace.storage.bitstore.BitstreamStorageManager.retrieve(BitstreamStorageManager.java:525)
at org.dspace.checker.BitstreamDAO.getBitstream(BitstreamDAO.java:60)
at org.dspace.checker.CheckerCommand.processBitstream(CheckerCommand.java:303)
at org.dspace.checker.CheckerCommand.checkBitstream(CheckerCommand.java:171)
at org.dspace.checker.CheckerCommand.process(CheckerCommand.java:120)
at org.dspace.app.checker.ChecksumChecker.main(ChecksumChecker.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
******************************************************
</code></pre>
<ul>
<li>So this would be the <code>tomcat7</code> Unix user, who seems to have a default limit of 1024 files in its shell</li>
<li>For what it&rsquo;s worth, we have been setting the actual Tomcat 7 process&rsquo; limit to 16384 for a few years (in <code>/etc/default/tomcat7</code>)</li>
<li>Looks like cron will read limits from <code>/etc/security/limits.*</code> so we can do something for the tomcat7 user there</li>
<li>Submit pull request for Tomcat 7 limits in Ansible dspace role (<a href="https://github.com/ilri/rmg-ansible-public/pull/30">#30</a>)</li>
</ul>
</section>
<footer>
<section class="author-info row">
<div class="author-avatar col-md-2">
</div>
<div class="author-meta col-md-6">
<h1 class="author-name text-primary">Alan Orth</h1>
</div>
</section>
<ul class="pager">
<li class="previous"><a href="/cgspace-notes/2016-03/"><span aria-hidden="true">&larr;</span> Older</a></li>
<li class="next disabled"><a href="#">Newer <span aria-hidden="true">&rarr;</span></a></li>
</ul>
</footer>
</article>
</main>
<footer class="container global-footer">
<div class="copyright-note pull-left">
</div>
<div class="sns-links hidden-print">
</div>
</footer>
<script src="/cgspace-notes/js/highlight.pack.js"></script>
<script>
hljs.initHighlightingOnLoad();
</script>
</body>
</html>

View File

@ -58,6 +58,34 @@
<div class="article-list">
<article>
<header>
<h2><a href="/cgspace-notes/2016-04/">April, 2016</a></h2>
<div class="post-meta clearfix">
<div class="post-date pull-left">
Posted on
<time datetime="2016-04-04T11:06:00&#43;03:00">
Apr 4, 2016
</time>
</div>
</div>
</header>
<div>
2016-04-04 Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc After running DSpace for over five years I&rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year! This will save us a few gigs of backup space we&rsquo;re paying for on S3 Also, I noticed the checker log has some errors we should pay attention to: Run start time: 03/06/2016 04:00:22 Error retrieving bitstream ID 71274 from asset store.
</div>
<footer>
<ul class="pager">
<li class="next"><a href="/cgspace-notes/2016-04/">Read more <span aria-hidden="true">&raquo;</span></a></li>
</ul>
</footer>
</article>
<hr/>
<article>
<header>
<h2><a href="/cgspace-notes/2016-03/">March, 2016</a></h2>

View File

@ -6,9 +6,60 @@
<description>Recent content on CGSpace Notes</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<lastBuildDate>Wed, 02 Mar 2016 16:50:00 +0300</lastBuildDate>
<lastBuildDate>Mon, 04 Apr 2016 11:06:00 +0300</lastBuildDate>
<atom:link href="/cgspace-notes/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>April, 2016</title>
<link>/cgspace-notes/2016-04/</link>
<pubDate>Mon, 04 Apr 2016 11:06:00 +0300</pubDate>
<guid>/cgspace-notes/2016-04/</guid>
<description>
&lt;h2 id=&#34;2016-04-04:c88be15f5b2f07c85f7742556a955e47&#34;&gt;2016-04-04&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit&lt;/li&gt;
&lt;li&gt;We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc&lt;/li&gt;
&lt;li&gt;After running DSpace for over five years I&amp;rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year!&lt;/li&gt;
&lt;li&gt;This will save us a few gigs of backup space we&amp;rsquo;re paying for on S3&lt;/li&gt;
&lt;li&gt;Also, I noticed the &lt;code&gt;checker&lt;/code&gt; log has some errors we should pay attention to:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;Run start time: 03/06/2016 04:00:22
Error retrieving bitstream ID 71274 from asset store.
java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290601546459645925328536011917633626 (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.&amp;lt;init&amp;gt;(FileInputStream.java:146)
at edu.sdsc.grid.io.local.LocalFileInputStream.open(LocalFileInputStream.java:171)
at edu.sdsc.grid.io.GeneralFileInputStream.&amp;lt;init&amp;gt;(GeneralFileInputStream.java:145)
at edu.sdsc.grid.io.local.LocalFileInputStream.&amp;lt;init&amp;gt;(LocalFileInputStream.java:139)
at edu.sdsc.grid.io.FileFactory.newFileInputStream(FileFactory.java:630)
at org.dspace.storage.bitstore.BitstreamStorageManager.retrieve(BitstreamStorageManager.java:525)
at org.dspace.checker.BitstreamDAO.getBitstream(BitstreamDAO.java:60)
at org.dspace.checker.CheckerCommand.processBitstream(CheckerCommand.java:303)
at org.dspace.checker.CheckerCommand.checkBitstream(CheckerCommand.java:171)
at org.dspace.checker.CheckerCommand.process(CheckerCommand.java:120)
at org.dspace.app.checker.ChecksumChecker.main(ChecksumChecker.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
******************************************************
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;So this would be the &lt;code&gt;tomcat7&lt;/code&gt; Unix user, who seems to have a default limit of 1024 files in its shell&lt;/li&gt;
&lt;li&gt;For what it&amp;rsquo;s worth, we have been setting the actual Tomcat 7 process&amp;rsquo; limit to 16384 for a few years (in &lt;code&gt;/etc/default/tomcat7&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Looks like cron will read limits from &lt;code&gt;/etc/security/limits.*&lt;/code&gt; so we can do something for the tomcat7 user there&lt;/li&gt;
&lt;li&gt;Submit pull request for Tomcat 7 limits in Ansible dspace role (&lt;a href=&#34;https://github.com/ilri/rmg-ansible-public/pull/30&#34;&gt;#30&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
</description>
</item>
<item>
<title>March, 2016</title>
<link>/cgspace-notes/2016-03/</link>

View File

@ -3,10 +3,15 @@
<url>
<loc>/cgspace-notes/</loc>
<lastmod>2016-03-02T16:50:00+03:00</lastmod>
<lastmod>2016-04-04T11:06:00+03:00</lastmod>
<priority>0</priority>
</url>
<url>
<loc>/cgspace-notes/2016-04/</loc>
<lastmod>2016-04-04T11:06:00+03:00</lastmod>
</url>
<url>
<loc>/cgspace-notes/2016-03/</loc>
<lastmod>2016-03-02T16:50:00+03:00</lastmod>

View File

@ -64,7 +64,7 @@
<ul>
<li><a href="/cgspace-notes//tags/notes">notes</a> 5</li>
<li><a href="/cgspace-notes//tags/notes">notes</a> 6</li>
</ul>
</section>

View File

@ -61,6 +61,32 @@
<section class="article-list">
<h1>Notes</h1>
<hr/>
<article>
<header>
<h2><a href="/cgspace-notes/2016-04/">April, 2016</a></h2>
<div class="post-meta clearfix">
<div class="post-date pull-left">
Posted on
<time datetime="2016-04-04T11:06:00&#43;03:00">
Apr 4, 2016
</time>
</div>
</div>
</header>
<div>
2016-04-04 Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc After running DSpace for over five years I&rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year! This will save us a few gigs of backup space we&rsquo;re paying for on S3 Also, I noticed the checker log has some errors we should pay attention to: Run start time: 03/06/2016 04:00:22 Error retrieving bitstream ID 71274 from asset store.
</div>
<footer>
<ul class="pager">
<li class="next"><a href="/cgspace-notes/2016-04/">Read more <span aria-hidden="true">&raquo;</span></a></li>
</ul>
</footer>
</article>
<hr/>
<article>
<header>

View File

@ -6,9 +6,60 @@
<description>Recent content in Notes on CGSpace Notes</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language>
<lastBuildDate>Wed, 02 Mar 2016 16:50:00 +0300</lastBuildDate>
<lastBuildDate>Mon, 04 Apr 2016 11:06:00 +0300</lastBuildDate>
<atom:link href="/cgspace-notes/tags/notes/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>April, 2016</title>
<link>/cgspace-notes/2016-04/</link>
<pubDate>Mon, 04 Apr 2016 11:06:00 +0300</pubDate>
<guid>/cgspace-notes/2016-04/</guid>
<description>
&lt;h2 id=&#34;2016-04-04:c88be15f5b2f07c85f7742556a955e47&#34;&gt;2016-04-04&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit&lt;/li&gt;
&lt;li&gt;We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc&lt;/li&gt;
&lt;li&gt;After running DSpace for over five years I&amp;rsquo;ve never needed to look in any other log file than dspace.log, leave alone one from last year!&lt;/li&gt;
&lt;li&gt;This will save us a few gigs of backup space we&amp;rsquo;re paying for on S3&lt;/li&gt;
&lt;li&gt;Also, I noticed the &lt;code&gt;checker&lt;/code&gt; log has some errors we should pay attention to:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;Run start time: 03/06/2016 04:00:22
Error retrieving bitstream ID 71274 from asset store.
java.io.FileNotFoundException: /home/cgspace.cgiar.org/assetstore/64/29/06/64290601546459645925328536011917633626 (Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.&amp;lt;init&amp;gt;(FileInputStream.java:146)
at edu.sdsc.grid.io.local.LocalFileInputStream.open(LocalFileInputStream.java:171)
at edu.sdsc.grid.io.GeneralFileInputStream.&amp;lt;init&amp;gt;(GeneralFileInputStream.java:145)
at edu.sdsc.grid.io.local.LocalFileInputStream.&amp;lt;init&amp;gt;(LocalFileInputStream.java:139)
at edu.sdsc.grid.io.FileFactory.newFileInputStream(FileFactory.java:630)
at org.dspace.storage.bitstore.BitstreamStorageManager.retrieve(BitstreamStorageManager.java:525)
at org.dspace.checker.BitstreamDAO.getBitstream(BitstreamDAO.java:60)
at org.dspace.checker.CheckerCommand.processBitstream(CheckerCommand.java:303)
at org.dspace.checker.CheckerCommand.checkBitstream(CheckerCommand.java:171)
at org.dspace.checker.CheckerCommand.process(CheckerCommand.java:120)
at org.dspace.app.checker.ChecksumChecker.main(ChecksumChecker.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:225)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:77)
******************************************************
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;So this would be the &lt;code&gt;tomcat7&lt;/code&gt; Unix user, who seems to have a default limit of 1024 files in its shell&lt;/li&gt;
&lt;li&gt;For what it&amp;rsquo;s worth, we have been setting the actual Tomcat 7 process&amp;rsquo; limit to 16384 for a few years (in &lt;code&gt;/etc/default/tomcat7&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Looks like cron will read limits from &lt;code&gt;/etc/security/limits.*&lt;/code&gt; so we can do something for the tomcat7 user there&lt;/li&gt;
&lt;li&gt;Submit pull request for Tomcat 7 limits in Ansible dspace role (&lt;a href=&#34;https://github.com/ilri/rmg-ansible-public/pull/30&#34;&gt;#30&lt;/a&gt;)&lt;/li&gt;
&lt;/ul&gt;
</description>
</item>
<item>
<title>March, 2016</title>
<link>/cgspace-notes/2016-03/</link>