Update notes for 2020-04-29

This commit is contained in:
Alan Orth 2020-04-29 14:20:28 +03:00
parent acf1c6a818
commit 546ac80881
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
3 changed files with 82 additions and 8 deletions

View File

@ -324,5 +324,40 @@ atmire-cua.version.number=${cua.version.number}
- I manually editied the CUA version variable and was then able to run the `com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdateCLI` script - I manually editied the CUA version variable and was then able to run the `com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdateCLI` script
- On the first run it took one hour to process 100,000 records on my local test instance... - On the first run it took one hour to process 100,000 records on my local test instance...
- On the second run it took one hour to process 140,000 records - On the second run it took one hour to process 140,000 records
- On the third run it took one hour to process 150,000 records
## 2020-04-29
- I found out that running the Atmire CUA script with more memory and a larger number of records (-r 20000) makes it run faster
- Now the process finishes, but there are errors on some records:
```
Record uid: ee085cc0-0110-42c5-80b9-0fad4015ed9f couldn't be processed
com.atmire.statistics.util.update.atomic.ProcessingException: something went wrong while processing record uid: ee085cc0-0110-42c5-80b9-0fad4015ed9f, an error occured in the com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.applyProcessors(AtomicStatisticsUpdater.java:304)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.processRecords(AtomicStatisticsUpdater.java:176)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.performRun(AtomicStatisticsUpdater.java:161)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.update(AtomicStatisticsUpdater.java:128)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdateCLI.main(AtomicStatisticsUpdateCLI.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
Caused by: java.lang.NullPointerException
at com.atmire.dspace.cua.CUADSOServiceImpl.findByLegacyID(CUADSOServiceImpl.java:40)
at com.atmire.statistics.util.update.atomic.processor.AtomicUpdateProcessor.getDSpaceObject(AtomicUpdateProcessor.java:49)
at com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor.process(ContainerOwnerDBProcessor.java:45)
at com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor.visit(ContainerOwnerDBProcessor.java:38)
at com.atmire.statistics.util.update.atomic.record.UsageRecord.accept(UsageRecord.java:23)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.applyProcessors(AtomicStatisticsUpdater.java:301)
... 10 more
```
- I've sent a message to Atmire to ask for advice
- Also, now I can actually see the CUA statlets and usage statistics
- Unfortunately it seems they are using Font Awesome 4 in their CUA module and this means that some icons are broken because the names have changed, but also some of their code is using Unicode characters instead of classes in spans!
- I've reverted my sweet SVG work from yesterday and adjusted the Font Awesome 5 SCSS to add a few more icons that they are using
<!-- vim: set sw=2 ts=2: --> <!-- vim: set sw=2 ts=2: -->

View File

@ -25,7 +25,7 @@ On the same note, the one item Abenet pointed out last week now has a donut with
<meta property="og:type" content="article" /> <meta property="og:type" content="article" />
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-04/" /> <meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-04/" />
<meta property="article:published_time" content="2020-04-02T10:53:24+03:00" /> <meta property="article:published_time" content="2020-04-02T10:53:24+03:00" />
<meta property="article:modified_time" content="2020-04-27T16:57:32+03:00" /> <meta property="article:modified_time" content="2020-04-28T18:18:34+03:00" />
<meta name="twitter:card" content="summary"/> <meta name="twitter:card" content="summary"/>
<meta name="twitter:title" content="April, 2020"/> <meta name="twitter:title" content="April, 2020"/>
@ -55,9 +55,9 @@ On the same note, the one item Abenet pointed out last week now has a donut with
"@type": "BlogPosting", "@type": "BlogPosting",
"headline": "April, 2020", "headline": "April, 2020",
"url": "https://alanorth.github.io/cgspace-notes/2020-04/", "url": "https://alanorth.github.io/cgspace-notes/2020-04/",
"wordCount": "2394", "wordCount": "2592",
"datePublished": "2020-04-02T10:53:24+03:00", "datePublished": "2020-04-02T10:53:24+03:00",
"dateModified": "2020-04-27T16:57:32+03:00", "dateModified": "2020-04-28T18:18:34+03:00",
"author": { "author": {
"@type": "Person", "@type": "Person",
"name": "Alan Orth" "name": "Alan Orth"
@ -458,6 +458,45 @@ atmire-cua.version.number=${cua.version.number}
<ul> <ul>
<li>On the first run it took one hour to process 100,000 records on my local test instance&hellip;</li> <li>On the first run it took one hour to process 100,000 records on my local test instance&hellip;</li>
<li>On the second run it took one hour to process 140,000 records</li> <li>On the second run it took one hour to process 140,000 records</li>
<li>On the third run it took one hour to process 150,000 records</li>
</ul>
</li>
</ul>
<h2 id="2020-04-29">2020-04-29</h2>
<ul>
<li>I found out that running the Atmire CUA script with more memory and a larger number of records (-r 20000) makes it run faster
<ul>
<li>Now the process finishes, but there are errors on some records:</li>
</ul>
</li>
</ul>
<pre><code>Record uid: ee085cc0-0110-42c5-80b9-0fad4015ed9f couldn't be processed
com.atmire.statistics.util.update.atomic.ProcessingException: something went wrong while processing record uid: ee085cc0-0110-42c5-80b9-0fad4015ed9f, an error occured in the com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.applyProcessors(AtomicStatisticsUpdater.java:304)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.processRecords(AtomicStatisticsUpdater.java:176)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.performRun(AtomicStatisticsUpdater.java:161)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.update(AtomicStatisticsUpdater.java:128)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdateCLI.main(AtomicStatisticsUpdateCLI.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.dspace.app.launcher.ScriptLauncher.runOneCommand(ScriptLauncher.java:229)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:81)
Caused by: java.lang.NullPointerException
at com.atmire.dspace.cua.CUADSOServiceImpl.findByLegacyID(CUADSOServiceImpl.java:40)
at com.atmire.statistics.util.update.atomic.processor.AtomicUpdateProcessor.getDSpaceObject(AtomicUpdateProcessor.java:49)
at com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor.process(ContainerOwnerDBProcessor.java:45)
at com.atmire.statistics.util.update.atomic.processor.ContainerOwnerDBProcessor.visit(ContainerOwnerDBProcessor.java:38)
at com.atmire.statistics.util.update.atomic.record.UsageRecord.accept(UsageRecord.java:23)
at com.atmire.statistics.util.update.atomic.AtomicStatisticsUpdater.applyProcessors(AtomicStatisticsUpdater.java:301)
... 10 more
</code></pre><ul>
<li>I&rsquo;ve sent a message to Atmire to ask for advice</li>
<li>Also, now I can actually see the CUA statlets and usage statistics
<ul>
<li>Unfortunately it seems they are using Font Awesome 4 in their CUA module and this means that some icons are broken because the names have changed, but also some of their code is using Unicode characters instead of classes in spans!</li>
<li>I&rsquo;ve reverted my sweet SVG work from yesterday and adjusted the Font Awesome 5 SCSS to add a few more icons that they are using</li>
</ul> </ul>
</li> </li>
</ul> </ul>

View File

@ -4,27 +4,27 @@
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/2020-04/</loc> <loc>https://alanorth.github.io/cgspace-notes/2020-04/</loc>
<lastmod>2020-04-27T16:57:32+03:00</lastmod> <lastmod>2020-04-28T18:18:34+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc> <loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
<lastmod>2020-04-27T16:57:32+03:00</lastmod> <lastmod>2020-04-28T18:18:34+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2020-04-27T16:57:32+03:00</lastmod> <lastmod>2020-04-28T18:18:34+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc> <loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
<lastmod>2020-04-27T16:57:32+03:00</lastmod> <lastmod>2020-04-28T18:18:34+03:00</lastmod>
</url> </url>
<url> <url>
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc> <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
<lastmod>2020-04-27T16:57:32+03:00</lastmod> <lastmod>2020-04-28T18:18:34+03:00</lastmod>
</url> </url>
<url> <url>