Add notes for 2017-04-24

content/post/2017-04.md

@@ -333,3 +333,70 @@ $ time schedtool -D -e ionice -c2 -n7 nice -n19 [dspace]/bin/dspace filter-media
 ```
 dspace=# update bundle set primary_bitstream_id=NULL where primary_bitstream_id in (435, 1136, 1132, 1220, 1236, 3002, 3255, 5322);
 ```
+
+## 2017-04-24
+
+- Two users mentioned some items they recently approved not showing up in the search / XMLUI
+- I looked at the logs from yesterday and it seems the Discovery indexing has been crashing:
+
+```
+2017-04-24 00:00:15,578 INFO  com.atmire.dspace.discovery.AtmireSolrService @ Processing (55 of 58853): 70590
+2017-04-24 00:00:15,586 INFO  com.atmire.dspace.discovery.AtmireSolrService @ Processing (56 of 58853): 74507
+2017-04-24 00:00:15,614 ERROR com.atmire.dspace.discovery.AtmireSolrService @ this IndexWriter is closed
+org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: this IndexWriter is closed
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
+        at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
+        at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
+        at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
+        at org.dspace.discovery.SolrServiceImpl.unIndexContent(SolrServiceImpl.java:331)
+        at org.dspace.discovery.SolrServiceImpl.unIndexContent(SolrServiceImpl.java:315)
+        at com.atmire.dspace.discovery.AtmireSolrService.indexContent(AtmireSolrService.java:803)
+        at com.atmire.dspace.discovery.AtmireSolrService.updateIndex(AtmireSolrService.java:876)
+        at org.dspace.discovery.IndexClient.main(IndexClient.java:127)
+        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
+        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
+        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
+        at java.lang.reflect.Method.invoke(Method.java:498)
+        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:226)
+        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:78)
+```
+
+- Looking at the past few days of logs, it looks like the indexing process started crashing on 2017-04-20:
+
+```
+# grep -c 'IndexWriter is closed' [dspace]/log/dspace.log.2017-04-*
+[dspace]/log/dspace.log.2017-04-01:0
+[dspace]/log/dspace.log.2017-04-02:0
+[dspace]/log/dspace.log.2017-04-03:0
+[dspace]/log/dspace.log.2017-04-04:0
+[dspace]/log/dspace.log.2017-04-05:0
+[dspace]/log/dspace.log.2017-04-06:0
+[dspace]/log/dspace.log.2017-04-07:0
+[dspace]/log/dspace.log.2017-04-08:0
+[dspace]/log/dspace.log.2017-04-09:0
+[dspace]/log/dspace.log.2017-04-10:0
+[dspace]/log/dspace.log.2017-04-11:0
+[dspace]/log/dspace.log.2017-04-12:0
+[dspace]/log/dspace.log.2017-04-13:0
+[dspace]/log/dspace.log.2017-04-14:0
+[dspace]/log/dspace.log.2017-04-15:0
+[dspace]/log/dspace.log.2017-04-16:0
+[dspace]/log/dspace.log.2017-04-17:0
+[dspace]/log/dspace.log.2017-04-18:0
+[dspace]/log/dspace.log.2017-04-19:0
+[dspace]/log/dspace.log.2017-04-20:2293
+[dspace]/log/dspace.log.2017-04-21:5992
+[dspace]/log/dspace.log.2017-04-22:13278
+[dspace]/log/dspace.log.2017-04-23:22720
+[dspace]/log/dspace.log.2017-04-24:21422
+```
+
+- I restarted Tomcat and re-ran the discovery process manually:
+
+```
+[dspace]/bin/dspace index-discovery
+```
+
+- Now everything is ok

public/2017-04/index.html

@@ -30,7 +30,7 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Th
 
 
 <meta property="article:published_time" content="2017-04-02T17:08:52&#43;02:00"/>
-<meta property="article:modified_time" content="2017-04-20T18:40:42&#43;03:00"/>
+<meta property="article:modified_time" content="2017-04-22T19:34:59&#43;03:00"/>
 
 
 
@@ -79,9 +79,9 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p &quot;ImageMagick PDF Th
   "@type": "BlogPosting",
   "headline": "April, 2017",
   "url": "https://alanorth.github.io/cgspace-notes/2017-04/",
-  "wordCount": "2267",
+  "wordCount": "2433",
   "datePublished": "2017-04-02T17:08:52&#43;02:00",
-  "dateModified": "2017-04-20T18:40:42&#43;03:00",
+  "dateModified": "2017-04-22T19:34:59&#43;03:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -543,6 +543,78 @@ $ time schedtool -D -e ionice -c2 -n7 nice -n19 [dspace]/bin/dspace filter-media
 <pre><code>dspace=# update bundle set primary_bitstream_id=NULL where primary_bitstream_id in (435, 1136, 1132, 1220, 1236, 3002, 3255, 5322);
 </code></pre>
 
+<h2 id="2017-04-24">2017-04-24</h2>
+
+<ul>
+<li>Two users mentioned some items they recently approved not showing up in the search / XMLUI</li>
+<li>I looked at the logs from yesterday and it seems the Discovery indexing has been crashing:</li>
+</ul>
+
+<pre><code>2017-04-24 00:00:15,578 INFO  com.atmire.dspace.discovery.AtmireSolrService @ Processing (55 of 58853): 70590
+2017-04-24 00:00:15,586 INFO  com.atmire.dspace.discovery.AtmireSolrService @ Processing (56 of 58853): 74507
+2017-04-24 00:00:15,614 ERROR com.atmire.dspace.discovery.AtmireSolrService @ this IndexWriter is closed
+org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: this IndexWriter is closed
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
+        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
+        at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
+        at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:285)
+        at org.apache.solr.client.solrj.SolrServer.deleteByQuery(SolrServer.java:271)
+        at org.dspace.discovery.SolrServiceImpl.unIndexContent(SolrServiceImpl.java:331)
+        at org.dspace.discovery.SolrServiceImpl.unIndexContent(SolrServiceImpl.java:315)
+        at com.atmire.dspace.discovery.AtmireSolrService.indexContent(AtmireSolrService.java:803)
+        at com.atmire.dspace.discovery.AtmireSolrService.updateIndex(AtmireSolrService.java:876)
+        at org.dspace.discovery.IndexClient.main(IndexClient.java:127)
+        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
+        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
+        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
+        at java.lang.reflect.Method.invoke(Method.java:498)
+        at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:226)
+        at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:78)
+</code></pre>
+
+<ul>
+<li>Looking at the past few days of logs, it looks like the indexing process started crashing on 2017-04-20:</li>
+</ul>
+
+<pre><code># grep -c 'IndexWriter is closed' [dspace]/log/dspace.log.2017-04-*
+[dspace]/log/dspace.log.2017-04-01:0
+[dspace]/log/dspace.log.2017-04-02:0
+[dspace]/log/dspace.log.2017-04-03:0
+[dspace]/log/dspace.log.2017-04-04:0
+[dspace]/log/dspace.log.2017-04-05:0
+[dspace]/log/dspace.log.2017-04-06:0
+[dspace]/log/dspace.log.2017-04-07:0
+[dspace]/log/dspace.log.2017-04-08:0
+[dspace]/log/dspace.log.2017-04-09:0
+[dspace]/log/dspace.log.2017-04-10:0
+[dspace]/log/dspace.log.2017-04-11:0
+[dspace]/log/dspace.log.2017-04-12:0
+[dspace]/log/dspace.log.2017-04-13:0
+[dspace]/log/dspace.log.2017-04-14:0
+[dspace]/log/dspace.log.2017-04-15:0
+[dspace]/log/dspace.log.2017-04-16:0
+[dspace]/log/dspace.log.2017-04-17:0
+[dspace]/log/dspace.log.2017-04-18:0
+[dspace]/log/dspace.log.2017-04-19:0
+[dspace]/log/dspace.log.2017-04-20:2293
+[dspace]/log/dspace.log.2017-04-21:5992
+[dspace]/log/dspace.log.2017-04-22:13278
+[dspace]/log/dspace.log.2017-04-23:22720
+[dspace]/log/dspace.log.2017-04-24:21422
+</code></pre>
+
+<ul>
+<li>I restarted Tomcat and re-ran the discovery process manually:</li>
+</ul>
+
+<pre><code>[dspace]/bin/dspace index-discovery
+</code></pre>
+
+<ul>
+<li>Now everything is ok</li>
+</ul>
+
   
 
   

public/sitemap.xml

@@ -3,7 +3,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2017-04/</loc>
-    <lastmod>2017-04-20T18:40:42+03:00</lastmod>
+    <lastmod>2017-04-22T19:34:59+03:00</lastmod>
   </url>
   
   <url>
@@ -93,7 +93,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2017-04-20T18:40:42+03:00</lastmod>
+    <lastmod>2017-04-22T19:34:59+03:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -104,19 +104,19 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2017-04-20T18:40:42+03:00</lastmod>
+    <lastmod>2017-04-22T19:34:59+03:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/post/</loc>
-    <lastmod>2017-04-20T18:40:42+03:00</lastmod>
+    <lastmod>2017-04-22T19:34:59+03:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2017-04-20T18:40:42+03:00</lastmod>
+    <lastmod>2017-04-22T19:34:59+03:00</lastmod>
     <priority>0</priority>
   </url>