mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-26 00:18:21 +01:00
Update notes for 2019-03-19
This commit is contained in:
parent
fa99c86dd2
commit
6c0563f5ce
GPG Key ID:
0FB860CC9C45B1B9
@ -499,5 +499,56 @@ dspace=# SELECT count(text_value) FROM metadatavalue WHERE resource_type_id=2 AN
|
||||
```
|
||||
|
||||
- Perhaps my `agrovoc-lookup.py` script could notify if it finds these because they potentially give false negatives
|
||||
- CGSpace (linode18) is having problems with Solr again, I'm seeing "Error opening new searcher" in the Solr logs and there are no stats for previous years
|
||||
- Apparently the Solr statistics shards didn't load properly when we restarted Tomcat *yesterday*:
|
||||
|
||||
```
|
||||
2019-03-18 12:32:39,799 ERROR org.apache.solr.core.CoreContainer @ Error creating core [statistics-2018]: Error opening new searcher
|
||||
...
|
||||
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
|
||||
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1565)
|
||||
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1677)
|
||||
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845)
|
||||
... 31 more
|
||||
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/home/cgspace.cgiar.org/solr/statistics-2018/data/index/write.lock
|
||||
```
|
||||
|
||||
- For reference, I don't see the `ulimit -v unlimited` in the `catalina.sh` script, though the `tomcat7` systemd service has `LimitAS=infinity`
|
||||
- The limits of the current Tomcat java process are:
|
||||
|
||||
```
|
||||
# cat /proc/27182/limits
|
||||
Limit Soft Limit Hard Limit Units
|
||||
Max cpu time unlimited unlimited seconds
|
||||
Max file size unlimited unlimited bytes
|
||||
Max data size unlimited unlimited bytes
|
||||
Max stack size 8388608 unlimited bytes
|
||||
Max core file size 0 unlimited bytes
|
||||
Max resident set unlimited unlimited bytes
|
||||
Max processes 128589 128589 processes
|
||||
Max open files 16384 16384 files
|
||||
Max locked memory 65536 65536 bytes
|
||||
Max address space unlimited unlimited bytes
|
||||
Max file locks unlimited unlimited locks
|
||||
Max pending signals 128589 128589 signals
|
||||
Max msgqueue size 819200 819200 bytes
|
||||
Max nice priority 0 0
|
||||
Max realtime priority 0 0
|
||||
Max realtime timeout unlimited unlimited us
|
||||
```
|
||||
|
||||
- I will try to add `ulimit -v unlimited` to the Catalina startup script and check the output of the limits to see if it's different in practice, as some wisdom on Stack Overflow says this solves the Solr core issues and I've superstitiously tried it various times in the past
|
||||
- The result is the same before and after, so *adding the ulimit directly is unneccessary* (whether or not unlimited address space is useful or not is another question)
|
||||
- For now I will just stop Tomcat, delete Solr locks, then start Tomcat again:
|
||||
|
||||
```
|
||||
# systemctl stop tomcat7
|
||||
# find /home/cgspace.cgiar.org/solr/ -iname "*.lock" -delete
|
||||
# systemctl start tomcat7
|
||||
```
|
||||
|
||||
- After restarting I confirmed that all Solr statistics cores were loaded successfully...
|
||||
- Another avenue might be to look at point releases in Solr 4.10.x, as we're running 4.10.2 and they released 4.10.3 and 4.10.4 back in 2014 or 2015
|
||||
- I see several issues regarding locks and IndexWriter that were fixed in Solr and Lucene 4.10.3 and 4.10.4...
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -25,7 +25,7 @@ I think I will need to ask Udana to re-copy and paste the abstracts with more ca
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-03/" />
|
||||
<meta property="article:published_time" content="2019-03-01T12:16:30+01:00"/>
|
||||
<meta property="article:modified_time" content="2019-03-18T21:55:08+02:00"/>
|
||||
<meta property="article:modified_time" content="2019-03-19T11:33:51+02:00"/>
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="March, 2019"/>
|
||||
@ -55,9 +55,9 @@ I think I will need to ask Udana to re-copy and paste the abstracts with more ca
|
||||
"@type": "BlogPosting",
|
||||
"headline": "March, 2019",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2019-03/",
|
||||
"wordCount": "3049",
|
||||
"wordCount": "3421",
|
||||
"datePublished": "2019-03-01T12:16:30+01:00",
|
||||
"dateModified": "2019-03-18T21:55:08+02:00",
|
||||
"dateModified": "2019-03-19T11:33:51+02:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -702,6 +702,66 @@ $ psql -c 'select * from pg_stat_activity' | grep -o -E '(dspaceWeb|dspaceApi|ds
|
||||
|
||||
<ul>
|
||||
<li>Perhaps my <code>agrovoc-lookup.py</code> script could notify if it finds these because they potentially give false negatives</li>
|
||||
<li>CGSpace (linode18) is having problems with Solr again, I’m seeing “Error opening new searcher” in the Solr logs and there are no stats for previous years</li>
|
||||
<li>Apparently the Solr statistics shards didn’t load properly when we restarted Tomcat <em>yesterday</em>:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>2019-03-18 12:32:39,799 ERROR org.apache.solr.core.CoreContainer @ Error creating core [statistics-2018]: Error opening new searcher
|
||||
...
|
||||
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
|
||||
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1565)
|
||||
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1677)
|
||||
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:845)
|
||||
... 31 more
|
||||
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@/home/cgspace.cgiar.org/solr/statistics-2018/data/index/write.lock
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>For reference, I don’t see the <code>ulimit -v unlimited</code> in the <code>catalina.sh</code> script, though the <code>tomcat7</code> systemd service has <code>LimitAS=infinity</code></li>
|
||||
<li>The limits of the current Tomcat java process are:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code># cat /proc/27182/limits
|
||||
Limit Soft Limit Hard Limit Units
|
||||
Max cpu time unlimited unlimited seconds
|
||||
Max file size unlimited unlimited bytes
|
||||
Max data size unlimited unlimited bytes
|
||||
Max stack size 8388608 unlimited bytes
|
||||
Max core file size 0 unlimited bytes
|
||||
Max resident set unlimited unlimited bytes
|
||||
Max processes 128589 128589 processes
|
||||
Max open files 16384 16384 files
|
||||
Max locked memory 65536 65536 bytes
|
||||
Max address space unlimited unlimited bytes
|
||||
Max file locks unlimited unlimited locks
|
||||
Max pending signals 128589 128589 signals
|
||||
Max msgqueue size 819200 819200 bytes
|
||||
Max nice priority 0 0
|
||||
Max realtime priority 0 0
|
||||
Max realtime timeout unlimited unlimited us
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>I will try to add <code>ulimit -v unlimited</code> to the Catalina startup script and check the output of the limits to see if it’s different in practice, as some wisdom on Stack Overflow says this solves the Solr core issues and I’ve superstitiously tried it various times in the past
|
||||
|
||||
<ul>
|
||||
<li>The result is the same before and after, so <em>adding the ulimit directly is unneccessary</em> (whether or not unlimited address space is useful or not is another question)</li>
|
||||
</ul></li>
|
||||
<li>For now I will just stop Tomcat, delete Solr locks, then start Tomcat again:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code># systemctl stop tomcat7
|
||||
# find /home/cgspace.cgiar.org/solr/ -iname "*.lock" -delete
|
||||
# systemctl start tomcat7
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>After restarting I confirmed that all Solr statistics cores were loaded successfully…</li>
|
||||
<li>Another avenue might be to look at point releases in Solr 4.10.x, as we’re running 4.10.2 and they released 4.10.3 and 4.10.4 back in 2014 or 2015
|
||||
|
||||
<ul>
|
||||
<li>I see several issues regarding locks and IndexWriter that were fixed in Solr and Lucene 4.10.3 and 4.10.4…</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -45,7 +45,7 @@ Disallow: /cgspace-notes/2015-12/
|
||||
Disallow: /cgspace-notes/2015-11/
|
||||
Disallow: /cgspace-notes/
|
||||
Disallow: /cgspace-notes/categories/
|
||||
Disallow: /cgspace-notes/categories/notes/
|
||||
Disallow: /cgspace-notes/tags/notes/
|
||||
Disallow: /cgspace-notes/categories/notes/
|
||||
Disallow: /cgspace-notes/posts/
|
||||
Disallow: /cgspace-notes/tags/
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2019-03/</loc>
|
||||
<lastmod>2019-03-18T21:55:08+02:00</lastmod>
|
||||
<lastmod>2019-03-19T11:33:51+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
@ -214,7 +214,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2019-03-18T21:55:08+02:00</lastmod>
|
||||
<lastmod>2019-03-19T11:33:51+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -223,27 +223,27 @@
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2019-03-19T11:33:51+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
|
||||
<lastmod>2018-03-09T22:10:33+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2019-03-18T21:55:08+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||
<lastmod>2019-03-18T21:55:08+02:00</lastmod>
|
||||
<lastmod>2019-03-19T11:33:51+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||
<lastmod>2019-03-18T21:55:08+02:00</lastmod>
|
||||
<lastmod>2019-03-19T11:33:51+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user