mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-12-23 21:44:30 +01:00
Add notes for 2018-09-21
This commit is contained in:
parent
c6ab9193aa
commit
81f3ee523b
GPG Key ID:
0FB860CC9C45B1B9
@ -368,5 +368,30 @@ dspace=# select item_id from item where in_archive is True and withdrawn is Fals
|
||||
## 2018-09-20
|
||||
|
||||
- Contact Atmire to ask how we can buy more credits for future development
|
||||
- I researched the Solr `filterCache` size and I found out that the formula for calculating the potential memory use of **each entry** in the cache is:
|
||||
|
||||
```
|
||||
((maxDoc/8) + 128) * (size_defined_in_solrconfig.xml)
|
||||
```
|
||||
|
||||
- Which means that, for our statistics core with *149 million* documents, each entry in our `filterCache` would use 8.9 GB!
|
||||
|
||||
```
|
||||
((149374568/8) + 128) * 512 = 9560037888 bytes (8.9 GB)
|
||||
```
|
||||
|
||||
- So I think we can forget about tuning this for now!
|
||||
- [Discussion on the mailing list about `filterCache` size](http://lucene.472066.n3.nabble.com/Calculating-filterCache-size-td4142526.html)
|
||||
- [Article discussing testing methodology for different `filterCache` sizes](https://docs.google.com/document/d/1vl-nmlprSULvNZKQNrqp65eLnLhG9s_ydXQtg9iML10/edit)
|
||||
- Discuss Handle links on Twitter with IWMI
|
||||
|
||||
## 2018-09-21
|
||||
|
||||
- I see that there was a nice optimization to the ImageMagick PDF CMYK detection in the upstream `dspace-5_x` branch: [DS-3664](https://github.com/DSpace/DSpace/pull/2204)
|
||||
- The fix will go into DSpace 5.10, and we are currently on DSpace 5.8 but I think I'll cherry-pick that fix into our `5_x-prod` branch:
|
||||
- 4e8c7b578bdbe26ead07e36055de6896bbf02f83: ImageMagick: Only execute "identify" on first page
|
||||
- I think it would also be nice to cherry-pick the fixes for [DS-3883](https://github.com/DSpace/DSpace/pull/2020), which is related to optimizing the XMLUI item display of items with many bitstreams
|
||||
- a0ea20bd1821720b111e2873b08e03ce2bf93307: DS-3883: Don't loop through original bitstreams if only displaying thumbnails
|
||||
- 8d81e825dee62c2aa9d403a505e4a4d798964e8d: DS-3883: If only including thumbnails, only load the main item thumbnail.
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -18,7 +18,7 @@ I’m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I
|
||||
" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2018-09/" /><meta property="article:published_time" content="2018-09-02T09:55:54+03:00"/>
|
||||
<meta property="article:modified_time" content="2018-09-19T20:40:18+03:00"/>
|
||||
<meta property="article:modified_time" content="2018-09-20T13:11:48+03:00"/>
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="September, 2018"/>
|
||||
<meta name="twitter:description" content="2018-09-02
|
||||
@ -41,9 +41,9 @@ I’m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I
|
||||
"@type": "BlogPosting",
|
||||
"headline": "September, 2018",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2018-09/",
|
||||
"wordCount": "2617",
|
||||
"wordCount": "2816",
|
||||
"datePublished": "2018-09-02T09:55:54+03:00",
|
||||
"dateModified": "2018-09-19T20:40:18+03:00",
|
||||
"dateModified": "2018-09-20T13:11:48+03:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -530,6 +530,41 @@ $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=50.116.102.77' dspace.log.2018-09-
|
||||
|
||||
<ul>
|
||||
<li>Contact Atmire to ask how we can buy more credits for future development</li>
|
||||
<li>I researched the Solr <code>filterCache</code> size and I found out that the formula for calculating the potential memory use of <strong>each entry</strong> in the cache is:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>((maxDoc/8) + 128) * (size_defined_in_solrconfig.xml)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Which means that, for our statistics core with <em>149 million</em> documents, each entry in our <code>filterCache</code> would use 8.9 GB!</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>((149374568/8) + 128) * 512 = 9560037888 bytes (8.9 GB)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>So I think we can forget about tuning this for now!</li>
|
||||
<li><a href="http://lucene.472066.n3.nabble.com/Calculating-filterCache-size-td4142526.html">Discussion on the mailing list about <code>filterCache</code> size</a></li>
|
||||
<li><a href="https://docs.google.com/document/d/1vl-nmlprSULvNZKQNrqp65eLnLhG9s_ydXQtg9iML10/edit">Article discussing testing methodology for different <code>filterCache</code> sizes</a></li>
|
||||
<li>Discuss Handle links on Twitter with IWMI</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-09-21">2018-09-21</h2>
|
||||
|
||||
<ul>
|
||||
<li>I see that there was a nice optimization to the ImageMagick PDF CMYK detection in the upstream <code>dspace-5_x</code> branch: <a href="https://github.com/DSpace/DSpace/pull/2204">DS-3664</a></li>
|
||||
<li>The fix will go into DSpace 5.10, and we are currently on DSpace 5.8 but I think I’ll cherry-pick that fix into our <code>5_x-prod</code> branch:
|
||||
|
||||
<ul>
|
||||
<li>4e8c7b578bdbe26ead07e36055de6896bbf02f83: ImageMagick: Only execute “identify” on first page</li>
|
||||
</ul></li>
|
||||
<li>I think it would also be nice to cherry-pick the fixes for <a href="https://github.com/DSpace/DSpace/pull/2020">DS-3883</a>, which is related to optimizing the XMLUI item display of items with many bitstreams
|
||||
|
||||
<ul>
|
||||
<li>a0ea20bd1821720b111e2873b08e03ce2bf93307: DS-3883: Don’t loop through original bitstreams if only displaying thumbnails</li>
|
||||
<li>8d81e825dee62c2aa9d403a505e4a4d798964e8d: DS-3883: If only including thumbnails, only load the main item thumbnail.</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2018-09/</loc>
|
||||
<lastmod>2018-09-19T20:40:18+03:00</lastmod>
|
||||
<lastmod>2018-09-20T13:11:48+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
@ -184,7 +184,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2018-09-19T20:40:18+03:00</lastmod>
|
||||
<lastmod>2018-09-20T13:11:48+03:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -195,7 +195,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2018-09-19T20:40:18+03:00</lastmod>
|
||||
<lastmod>2018-09-20T13:11:48+03:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -207,13 +207,13 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||
<lastmod>2018-09-19T20:40:18+03:00</lastmod>
|
||||
<lastmod>2018-09-20T13:11:48+03:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||
<lastmod>2018-09-19T20:40:18+03:00</lastmod>
|
||||
<lastmod>2018-09-20T13:11:48+03:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user