Update notes for 2019-07-01

This commit is contained in:
Alan Orth 2019-07-01 18:54:35 +03:00
parent 467a3eda5c
commit f6f6e767fe
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
13 changed files with 205 additions and 8 deletions

View File

@ -8,7 +8,63 @@ tags: ["Notes"]
## 2019-07-01 ## 2019-07-01
- Create an "AfricaRice books and book chapters" collection on CGSpace for AfricaRice - Create an "AfricaRice books and book chapters" collection on CGSpace for AfricaRice
- Last month Sisay asked why the following "most popular" statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
- [DSpace Test](https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&time_filter_end_date=01%2F12%2F2018)
- [CGSpace](https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&time_filter_end_date=01%2F12%2F2018)
- Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community
<!--more--> <!--more-->
- If I change the parameters to 2019 I see stats, so I'm really thinking it has something to do with the sharded yearly Solr statistics cores
- I checked the Solr admin UI and I see all Solr cores loaded, so I don't know what it could be
- When I check the Atmire content and usage module it seems obvious that there is a problem with the old cores because I dont have anything before 2019-01
![Atmire CUA 2018 stats missing](/cgspace-notes/2019/07/atmire-cua-2018-missing.png)
- I don't see anyone logged in right now so I'm going to try to restart Tomcat and see if the stats are accessible after Solr comes back up
- I decided to run all system updates on the server (linode18) and reboot it
- After rebooting Tomcat came back up, but the the Solr statistics cores were not all loaded
- The error is always (with a different core):
```
org.apache.solr.common.SolrException: Error CREATEing SolrCore 'statistics-2010': Unable to create core [statistics-2010] Caused by: Lock obtain timed out: NativeFSLock@/home/cgspace.cgiar.org/solr/statistics-2010/data/index/write.lock
```
- I restarted Tomcat *ten times* and it never worked...
- I tried to stop Tomcat and delete the write locks:
```
# systemctl stop tomcat7
# find /dspace/solr/statistics* -iname "*.lock" -print -delete
/dspace/solr/statistics/data/index/write.lock
/dspace/solr/statistics-2010/data/index/write.lock
/dspace/solr/statistics-2011/data/index/write.lock
/dspace/solr/statistics-2012/data/index/write.lock
/dspace/solr/statistics-2013/data/index/write.lock
/dspace/solr/statistics-2014/data/index/write.lock
/dspace/solr/statistics-2015/data/index/write.lock
/dspace/solr/statistics-2016/data/index/write.lock
/dspace/solr/statistics-2017/data/index/write.lock
/dspace/solr/statistics-2018/data/index/write.lock
# find /dspace/solr/statistics* -iname "*.lock" -print -delete
# systemctl start tomcat7
```
- But it still didn't work!
- I stopped Tomcat, deleted the old locks, and will try to use the "simple" lock file type in `solr/statistics/conf/solrconfig.xml`:
```
<lockType>${solr.lock.type:simple}</lockType>
```
- And after restarting Tomcat it still doesn't work
- Now I'll try going back to "native" locking with `unlockAtStartup`:
```
<unlockOnStartup>true</unlockOnStartup>
```
- Now the cores seem to load, but I still see an error in the Solr Admin UI and I still can't access any stats before 2018
- I filed an [issue with Atmire](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=685), so let's see if they can help
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -10,11 +10,18 @@
Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice
Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
DSpace Test
CGSpace
Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community
" /> " />
<meta property="og:type" content="article" /> <meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-07/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-07/" />
<meta property="article:published_time" content="2019-07-01T12:13:51&#43;03:00"/> <meta property="article:published_time" content="2019-07-01T12:13:51&#43;03:00"/>
<meta property="article:modified_time" content="2019-07-01T12:13:51&#43;03:00"/> <meta property="article:modified_time" content="2019-07-01T12:22:43&#43;03:00"/>
<meta name="twitter:card" content="summary"/> <meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="July, 2019"/> <meta name="twitter:title" content="July, 2019"/>
@ -22,6 +29,13 @@ Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace
Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice
Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
DSpace Test
CGSpace
Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community
"/> "/>
<meta name="generator" content="Hugo 0.55.6" /> <meta name="generator" content="Hugo 0.55.6" />
@ -33,9 +47,9 @@ Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "July, 2019", "headline": "July, 2019",
"url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-07\/", "url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-07\/",
"wordCount": "13", "wordCount": "359",
"datePublished": "2019-07-01T12:13:51\x2b03:00", "datePublished": "2019-07-01T12:13:51\x2b03:00",
"dateModified": "2019-07-01T12:13:51\x2b03:00", "dateModified": "2019-07-01T12:22:43\x2b03:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -106,6 +120,77 @@ Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul>
<ul>
<li>If I change the parameters to 2019 I see stats, so I&rsquo;m really thinking it has something to do with the sharded yearly Solr statistics cores
<ul>
<li>I checked the Solr admin UI and I see all Solr cores loaded, so I don&rsquo;t know what it could be</li>
<li>When I check the Atmire content and usage module it seems obvious that there is a problem with the old cores because I dont have anything before 2019-01</li>
</ul></li>
</ul>
<p><img src="/cgspace-notes/2019/07/atmire-cua-2018-missing.png" alt="Atmire CUA 2018 stats missing" /></p>
<ul>
<li>I don&rsquo;t see anyone logged in right now so I&rsquo;m going to try to restart Tomcat and see if the stats are accessible after Solr comes back up</li>
<li><p>I decided to run all system updates on the server (linode18) and reboot it</p>
<ul>
<li>After rebooting Tomcat came back up, but the the Solr statistics cores were not all loaded</li>
<li><p>The error is always (with a different core):</p>
<pre><code>org.apache.solr.common.SolrException: Error CREATEing SolrCore 'statistics-2010': Unable to create core [statistics-2010] Caused by: Lock obtain timed out: NativeFSLock@/home/cgspace.cgiar.org/solr/statistics-2010/data/index/write.lock
</code></pre></li>
</ul></li>
<li><p>I restarted Tomcat <em>ten times</em> and it never worked&hellip;</p></li>
<li><p>I tried to stop Tomcat and delete the write locks:</p>
<pre><code># systemctl stop tomcat7
# find /dspace/solr/statistics* -iname &quot;*.lock&quot; -print -delete
/dspace/solr/statistics/data/index/write.lock
/dspace/solr/statistics-2010/data/index/write.lock
/dspace/solr/statistics-2011/data/index/write.lock
/dspace/solr/statistics-2012/data/index/write.lock
/dspace/solr/statistics-2013/data/index/write.lock
/dspace/solr/statistics-2014/data/index/write.lock
/dspace/solr/statistics-2015/data/index/write.lock
/dspace/solr/statistics-2016/data/index/write.lock
/dspace/solr/statistics-2017/data/index/write.lock
/dspace/solr/statistics-2018/data/index/write.lock
# find /dspace/solr/statistics* -iname &quot;*.lock&quot; -print -delete
# systemctl start tomcat7
</code></pre></li>
<li><p>But it still didn&rsquo;t work!</p></li>
<li><p>I stopped Tomcat, deleted the old locks, and will try to use the &ldquo;simple&rdquo; lock file type in <code>solr/statistics/conf/solrconfig.xml</code>:</p>
<pre><code>&lt;lockType&gt;${solr.lock.type:simple}&lt;/lockType&gt;
</code></pre></li>
<li><p>And after restarting Tomcat it still doesn&rsquo;t work</p></li>
<li><p>Now I&rsquo;ll try going back to &ldquo;native&rdquo; locking with <code>unlockAtStartup</code>:</p>
<pre><code>&lt;unlockOnStartup&gt;true&lt;/unlockOnStartup&gt;
</code></pre></li>
<li><p>Now the cores seem to load, but I still see an error in the Solr Admin UI and I still can&rsquo;t access any stats before 2018</p></li>
<li><p>I filed an <a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=685">issue with Atmire</a>, so let&rsquo;s see if they can help</p></li>
</ul> </ul>
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB

View File

@ -102,6 +102,13 @@
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul> </ul>
<a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a> <a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a>
</article> </article>

View File

@ -102,6 +102,13 @@
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul> </ul>
<a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a> <a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a>
</article> </article>

View File

@ -21,6 +21,13 @@
&lt;ul&gt; &lt;ul&gt;
&lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt; &lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt;
&lt;li&gt;Last month Sisay asked why the following &amp;ldquo;most popular&amp;rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;DSpace Test&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;CGSpace&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;&lt;/li&gt;
&lt;li&gt;Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community&lt;/li&gt;
&lt;/ul&gt;</description> &lt;/ul&gt;</description>
</item> </item>

View File

@ -102,6 +102,13 @@
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul> </ul>
<a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a> <a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a>
</article> </article>

View File

@ -21,6 +21,13 @@
&lt;ul&gt; &lt;ul&gt;
&lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt; &lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt;
&lt;li&gt;Last month Sisay asked why the following &amp;ldquo;most popular&amp;rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;DSpace Test&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;CGSpace&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;&lt;/li&gt;
&lt;li&gt;Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community&lt;/li&gt;
&lt;/ul&gt;</description> &lt;/ul&gt;</description>
</item> </item>

View File

@ -4,30 +4,30 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2019-07-01T12:14:35+03:00</lastmod> <lastmod>2019-07-01T12:22:43+03:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/2019-07/</loc> <loc>https://alanorth.github.io/cgspace-notes/2019-07/</loc>
<lastmod>2019-07-01T12:13:51+03:00</lastmod> <lastmod>2019-07-01T12:22:43+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2019-07-01T12:14:35+03:00</lastmod> <lastmod>2019-07-01T12:22:43+03:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc> <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2019-07-01T12:14:35+03:00</lastmod> <lastmod>2019-07-01T12:22:43+03:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc> <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2019-07-01T12:14:35+03:00</lastmod> <lastmod>2019-07-01T12:22:43+03:00</lastmod>
<priority>0</priority> <priority>0</priority>
</url> </url>

View File

@ -102,6 +102,13 @@
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul> </ul>
<a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a> <a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a>
</article> </article>

View File

@ -87,6 +87,13 @@
<ul> <ul>
<li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li> <li>Create an &ldquo;AfricaRice books and book chapters&rdquo; collection on CGSpace for AfricaRice</li>
<li>Last month Sisay asked why the following &ldquo;most popular&rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
<ul>
<li><a href="https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">DSpace Test</a></li>
<li><a href="https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;time_filter_end_date=01%2F12%2F2018">CGSpace</a></li>
</ul></li>
<li>Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community</li>
</ul> </ul>
<a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a> <a href='https://alanorth.github.io/cgspace-notes/2019-07/'>Read more →</a>
</article> </article>

View File

@ -21,6 +21,13 @@
&lt;ul&gt; &lt;ul&gt;
&lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt; &lt;li&gt;Create an &amp;ldquo;AfricaRice books and book chapters&amp;rdquo; collection on CGSpace for AfricaRice&lt;/li&gt;
&lt;li&gt;Last month Sisay asked why the following &amp;ldquo;most popular&amp;rdquo; statistics link for a range of months in 2018 works for the CIAT community on DSpace Test, but not on CGSpace:
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;https://dspacetest.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;DSpace Test&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;https://cgspace.cgiar.org/handle/10568/35697/most-popular/item#simplefilter=custom&amp;amp;time_filter_end_date=01%2F12%2F2018&#34;&gt;CGSpace&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;&lt;/li&gt;
&lt;li&gt;Abenet had another similar issue a few days ago when trying to find the stats for 2018 in the RTB community&lt;/li&gt;
&lt;/ul&gt;</description> &lt;/ul&gt;</description>
</item> </item>

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.4 KiB