October, 2022

Sat Oct 01, 2022 in Notes

2022-10-01

Start a harvest on AReS last night
Yesterday I realized how to use GraphicsMagick with im4java and I want to re-visit some of my thumbnail tests
- I’m also interested in libvips support via jVips, though last time I checked it was only for Java 8
- I filed an issue to ask about Java 11+ support

2022-10-03

Make two pull requests for DSpace 7.x
- Update PDFBox dependency to version 2.0.27
- Update Apache commons-dbcp2 and commons-pool2 dependencies
Udana had asked me about their RSS feed and it not showing the latest publications in his email inbox
- He is using this feed from FeedBurner: https://feeds.feedburner.com/iwmi-cgspace
- I don’t have access to the FeedBurner configuration, but I looked at the raw feed and see it’s just getting all the items in the IWMI community
- This OpenSearch query should do the same: https://cgspace.cgiar.org/open-search/discover?scope=10568/16814&query=*&sort_by=3&order=DESC
- The sort_by=3 corresponds to webui.itemlist.sort-option.3 = dateaccessioned:dc.date.accessioned:date in dspace.cfg
Peter sent me a CSV file a few days ago that he was unable to upload to CGSpace
- The stacktrace from the error he was getting was:

Java stacktrace: java.lang.ClassCastException: org.apache.cocoon.servlet.multipart.PartInMemory cannot be cast to org.dspace.app.xmlui.cocoon.servlet.multipart.DSpacePartOnDisk
    at org.dspace.app.xmlui.aspect.administrative.FlowMetadataImportUtils.processUploadCSV(FlowMetadataImportUtils.java:116)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:155)
    at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:243)
    at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:3237)
    at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:2394)
    at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:162)
    at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:393)
    at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:2834)
    at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:160)
    at org.mozilla.javascript.Context.call(Context.java:538)
    at org.mozilla.javascript.ScriptableObject.callMethod(ScriptableObject.java:1833)
    at org.mozilla.javascript.ScriptableObject.callMethod(ScriptableObject.java:1803)
    at org.apache.cocoon.components.flow.javascript.fom.FOM_JavaScriptInterpreter.handleContinuation(FOM_JavaScriptInterpreter.java:698)
    at org.apache.cocoon.components.treeprocessor.sitemap.CallFunctionNode.invoke(CallFunctionNode.java:94)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.SelectNode.invoke(SelectNode.java:82)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.buildPipeline(ConcreteTreeProcessor.java:186)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.buildPipeline(TreeProcessor.java:260)
    at org.apache.cocoon.components.treeprocessor.sitemap.MountNode.invoke(MountNode.java:107)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.SelectNode.invoke(SelectNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.buildPipeline(ConcreteTreeProcessor.java:186)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.buildPipeline(TreeProcessor.java:260)
    at org.apache.cocoon.components.treeprocessor.sitemap.MountNode.invoke(MountNode.java:107)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.buildPipeline(ConcreteTreeProcessor.java:186)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.buildPipeline(TreeProcessor.java:260)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:277)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at org.dspace.app.xmlui.cocoon.AspectGenerator.setup(AspectGenerator.java:81)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.prepareInternal(AbstractProcessingPipeline.java:480)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.prepareInternal(Unknown Source)
    at org.apache.cocoon.components.source.impl.SitemapSource.init(SitemapSource.java:292)
    at org.apache.cocoon.components.source.impl.SitemapSource.<init>(SitemapSource.java:148)
    at org.apache.cocoon.components.source.impl.SitemapSourceFactory.getSource(SitemapSourceFactory.java:62)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:153)
    at org.apache.cocoon.components.source.CocoonSourceResolver.resolveURI(CocoonSourceResolver.java:183)
    at org.apache.cocoon.generation.FileGenerator.setup(FileGenerator.java:99)
    at sun.reflect.GeneratedMethodAccessor255.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy190.setup(Unknown Source)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.setupPipeline(AbstractProcessingPipeline.java:343)
    at org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.setupPipeline(AbstractCachingProcessingPipeline.java:710)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.preparePipeline(AbstractProcessingPipeline.java:466)
    at org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:411)
    at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.cocoon.core.container.spring.avalon.PoolableProxyHandler.invoke(PoolableProxyHandler.java:71)
    at com.sun.proxy.$Proxy189.process(Unknown Source)
    at org.apache.cocoon.components.treeprocessor.sitemap.SerializeNode.invoke(SerializeNode.java:147)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:171)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.process(TreeProcessor.java:247)
    at org.apache.cocoon.components.treeprocessor.sitemap.MountNode.invoke(MountNode.java:117)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:55)
    at org.apache.cocoon.components.treeprocessor.sitemap.MatchNode.invoke(MatchNode.java:87)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:171)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.process(TreeProcessor.java:247)
    at org.apache.cocoon.components.treeprocessor.sitemap.MountNode.invoke(MountNode.java:117)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelineNode.invoke(PipelineNode.java:143)
    at org.apache.cocoon.components.treeprocessor.AbstractParentProcessingNode.invokeNodes(AbstractParentProcessingNode.java:78)
    at org.apache.cocoon.components.treeprocessor.sitemap.PipelinesNode.invoke(PipelinesNode.java:81)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:239)
    at org.apache.cocoon.components.treeprocessor.ConcreteTreeProcessor.process(ConcreteTreeProcessor.java:171)
    at org.apache.cocoon.components.treeprocessor.TreeProcessor.process(TreeProcessor.java:247)
    at org.apache.cocoon.servlet.RequestProcessor.process(RequestProcessor.java:351)
    at org.apache.cocoon.servlet.RequestProcessor.service(RequestProcessor.java:169)
    at org.apache.cocoon.sitemap.SitemapServlet.service(SitemapServlet.java:84)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
    at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:468)
    at org.apache.cocoon.servletservice.ServletServiceContext$PathDispatcher.forward(ServletServiceContext.java:443)
    at org.apache.cocoon.servletservice.spring.ServletFactoryBean$ServiceInterceptor.invoke(ServletFactoryBean.java:264)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
    at com.sun.proxy.$Proxy186.service(Unknown Source)
    at org.dspace.springmvc.CocoonView.render(CocoonView.java:113)
    at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1216)
    at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1001)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:945)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:853)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.dspace.app.xmlui.cocoon.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:113)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.dspace.app.xmlui.cocoon.DSpaceCocoonServletFilter.doFilter(DSpaceCocoonServletFilter.java:160)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.dspace.app.xmlui.cocoon.servlet.multipart.DSpaceMultipartFilter.doFilter(DSpaceMultipartFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.dspace.utils.servlet.DSpaceWebappServletFilter.doFilter(DSpaceWebappServletFilter.java:78)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
    at org.apache.catalina.valves.CrawlerSessionManagerValve.invoke(CrawlerSessionManagerValve.java:235)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:451)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1201)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:317)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:750)

So this is a broken side effect from the org.apache.cocoon.uploads.autosave=false change I made a few weeks ago
- Importing the CSV via the command line works fine

2022-10-04

I stumbled across more low-quality thumbnails on CGSpace
- Some have the description “Generated Thumbnail”, and others are manually uploaded “.jpg.jpg” ones…
- I want to develop some more thumbnail fixer scripts to the cgspace-java-helpers suite:
  - If an item has an IM Thumbnail and a Generated Thumbnail in the THUMBNAIL bundle, remove the Generated Thumbnail
  - If an item has a PDF bitstream and a JPG bitstream with description /thumbnail/ in the ORIGINAL bundle, remove the /thumbnail/ bitstream in the ORIGINAL bundle and try to remove the /thumbnail/.jpg bitstream in the THUMBNAIL bundle

2022-10-05

I updated the cgspace-java-helpers to include a new FixLowQualityThumbnails script to detect the low-quality thumbnails I found above
Add missing ORCID identifier for an Alliance author
I’ve been running the dspace cleanup -v script every few weeks or months on CGSpace and assuming it finished successfully because I didn’t get a error on the stdout/stderr, but today I noticed that the script keeps saying it is deleting the same bitstreams
- I looked in dspace.log and found the error I used to see a lot:

Caused by: org.postgresql.util.PSQLException: ERROR: update or delete on table "bitstream" violates foreign key constraint "bundle_primary_bitstream_id_fkey" on table "bundle"
  Detail: Key (uuid)=(99b76ee4-15c6-458c-a940-866148bc7dee) is still referenced from table "bundle".

If I mark the primary bitstream as null manually the cleanup script continues until it finds a few more
- I ended up with a long list of UUIDs to fix before the script would complete:

$ psql -d dspace  -c "update bundle set primary_bitstream_id=NULL where primary_bitstream_id in ('b76d41c0-0a02-4f53-bfde-a840ccfff903','1981efaa-eadb-46cd-9d7b-12d7a8cff4c4','97a8b1fa-3c12-4122-9c7b-fc2a3eaf570d','99b76ee4-15c6-458c-a940-866148bc7dee','f330fc22-a787-46e2-b8d0-64cc3e166124','592f4a0d-1ed5-4663-be0e-958c0d3e653b','e73b3178-8f29-42bc-bfd1-1a454903343c','e3a5f592-ac23-4934-a2b2-26735fac0c4f','73f4ff6c-6679-44e8-8cbd-9f28a1df6927','11c9a75c-17a6-4966-a4e8-a473010eb34c','155faf93-92c5-4c17-866e-1db50b1f9687','8e073e9e-ab54-4d99-971a-66de073d51e3','76ddd62c-6499-4a8c-beea-3fc8c60200d8','2850fcc9-f450-430a-9317-c42def74e813','8fef3198-2aea-4bd8-aeab-bf5fccb46e42','9e3c3528-e20f-4da3-a0bd-ae9b8515b770')"

2022-10-06

I finished running the cleanup script on CGSpace and the before and after on the number of bitstreams is interesting:

$ find /home/cgspace.cgiar.org/assetstore -type f | wc -l
181094
$ find /home/cgspace.cgiar.org/assetstore -type f | wc -l
178329

So that cleaned up ~2,700 bitstreams!
Interesting, someone on the DSpace Slack mentioned this as being a known issue with discussion, reproducers, and a pull request: https://github.com/DSpace/DSpace/issues/7348
I am having an issue with the new FixLowQualityThumbnails script on some communities like 10568/117865 and 10568/97114
- For some reason it doesn’t descend into the collections
- Also, my old FixJpgJpgThumbnails doesn’t either… weird
- I might have to resort to getting a list of collections and doing it that way:

$ psql -h localhost -U postgres -d dspacetest -c 'SELECT ds6_collection2collectionhandle(uuid) FROM collection WHERE uuid in (SELECT uuid FROM collection);' |
    sed 1,2d |
    tac |
    sed 1,3d > /tmp/collections

Strange, I don’t think doing it by collections is actually working because it says it’s replacing the bitstreams, but it doesn’t actually do it
- I don’t have time to figure out what’s happening, because I see “update_item” in dspace.log when the script says it’s doing it, but it doesn’t do it
- I might just extract a list of items that have .jpg.jpg thumbnails from the database and run the script through item mode
- There might be a problem with the context commit logic…?
I exported a list of items that have .jpg.jpg thumbnails on CGSpace:

$ psql -h localhost -p 5432 -U postgres -d dspacetest -c "SELECT ds6_bitstream2itemhandle(dspace_object_id) FROM metadatavalue WHERE text_value ~ '.*\.(jpg|jpeg|JPG|JPEG)\.(jpg|jpeg|JPG|JPEG)' AND dspace_object_id IS NOT NULL;" |
  sed 1,2d |
  tac |
  sed 1,3d |
  grep -v '␀' |
  sort -u |
  sed 's/ //' > /tmp/jpgjpg-handles.txt

I restarted DSpace Test because it had high load since yesterday and I don’t know why
Run check-duplicates.py on the 1642 MARLO Innovations to try to include matches from the OICRs we uploaded last month
- Then I processed those matches like I did with the OICRs themselves last month, and then cleaned them one last time with csv-metadata-quality, created a SAF bundle, and uploaded them to CGSpace
- BTW this bumps CGSpace over 100,000 items…
- Then I did the same for the 749 MARLO MELIAs and imported them to CGSpace
Meeting about CG Core types with Abenet, Marie-Angelique, Sara, Margarita, and Valentina
I made some minor logic changes to the FixJpgJpgThumbnails script in cgspace-java-helpers
- Now it checks to make sure the bitstream description is not empty or null, and also excludes Maps (in addition to Infographics) since those are likely to be JPEG files in the ORIGINAL bundle on purpose

2022-10-07

I did the matching and cleaning on the 512 MARLO Policies and uploaded them to CGSpace
I sent a list of the IDs and Handles for all four groups of MARLO items to Jose so he can do the redirects on their server:

$ wc -l /tmp/*mappings.csv
  1643 /tmp/crp-innovation-mappings.csv
   750 /tmp/crp-melia-mappings.csv
   683 /tmp/crp-oicr-mappings.csv
   513 /tmp/crp-policy-mappings.csv
  3589 total

I fixed the mysterious issue with my cgspace-java-helpers scripts not working on communities and collections
- It was because the code wasn’t committing the context!
- I ran both FixJpgJpgThumbnails and FixLowQualityThumbnails on a dozen or so large collections on CGSpace and processed about 1,200 low-quality thumbnails
I did a complete re-sync of CGSpace to DSpace Test

2022-10-08

Start a harvest on AReS
Experiment with PDF thumbnails in ImageMagick again, I found an interesting reference on their legacy website saying we can use -unsharp after -thumbnail to make them less blurry
- There are a few examples for unsharp values (starting from a DSpace default of a flattened JPEG from the PDF, then the thumbnail in a second operation:

$ convert '10568-103447.pdf[0]' -flatten 10568-103447-dspace-step1.pdf.jpg 
$ convert 10568-103447-dspace-step1.pdf.jpg -thumbnail 600x600 -unsharp 0x.5 10568-103447-dspace-step2-600-unsharp.pdf.jpg
$ convert 10568-103447-dspace-step1.pdf.jpg -thumbnail 600x600 -unsharp 2x0.5+0.7+0 10568-103447-dspace-step2-600-unsharp2.pdf.jpg
$ convert 10568-103447-dspace-step1.pdf.jpg -thumbnail 600x600 -unsharp 0x0.75+0.75+0.008 10568-103447-dspace-step2-600-unsharp3.pdf.jpg
$ convert 10568-103447-dspace-step1.pdf.jpg -thumbnail 600x600 -unsharp 1.5x1+0.7+0.02 10568-103447-dspace-step2-600-unsharp4.pdf.jpg

I merged all the changes from 6_x-dev to 6_x-prod after having run them on DSpace Test for the last ten days

2022-10-11

I put together the microsite for improving DSpace PDF thumbnails: https://github.com/alanorth/improved-dspace-thumbnails/
- I need to make the pull request to DSpace
I also discussed the thumbnails with Dani in Addis

2022-10-12

I submitted a pull request to DSpace 7 for the -unsharp 0x0.5 change: https://github.com/DSpace/DSpace/pull/8515
I did some tests on CGSpace and verified that MEL will indeed need admin permissions on every collection that they want to map to
I had a call with Salem and he asked me about redirecting from some CRP duplicates that exist in both MELSpace and CGSpace
- We decided that the only way is to use an HTTP 301 redirect in the nginx web server, but I said that I’d check with CNRI to see if there was a way to do this within the Handle system

2022-10-13

Disable the REST API cache on CGSpace temporarily to see if that fixes a strange problem we are seeing with listing publications on ilri.org
Meeting with MEL, MARLO, and CG Core people to continue discussing dcterms.type
I added the new MEL account to all the appropriate authorizations for Initiatives that ICARDA is involved in on CGSpace
- I still have to add the few that WorldFish is involved in

2022-10-14

Abenet finalized adding the MEL user to all initiative collections on CGSpace
Re-sync CGSpace to DSpace Test to get the new MEL user and authorizations
I checked ilri.org and I see more publications for 2021 and earlier
- The results are still strange though because I only see a few for each year

2022-10-15

I’m going to turn the REST API cache on CGSpace back on to see if the ilri.org publications thing gets broken again
Start a harvest on AReS

2022-10-16

The harvest on AReS finished but somehow there are 10,000 less items than the previous indexing… hmmm
- I don’t see any hits from MELSpace there so I will start another harvest…
- After starting the harvesting the load on the server went up to 20 and UptimeRobot said CGSpace was down for three hours, sigh
- I stopped the harvesting and the load went down immediately
- I am trying to find a pattern with the load on Sundays
I see this in the AReS backend logs:

[Nest] 1   - 10/16/2022, 6:42:04 PM   [HarvesterService] Starting Harvest =>0
[Nest] 1   - 10/16/2022, 6:42:07 PM   [HarvesterService] Starting Harvest =>101555
[Nest] 1   - 10/16/2022, 6:42:10 PM   [HarvesterService] Starting Harvest =>4936

Which means MELSpace is having some issue
I’m not sure what was going on on CGSpace yesterday, but the load was indeed very high according to Munin:

CGSpace CPU load day

The pattern is clear on Sundays if you look at the past month:

CGSpace CPU load month

I have yet to find an increased nginx request pattern correlating with the increased load, but looking back on the last year it seems something started happening around March, 2022, and also I start seeing CPU steal in July (red coming from the top of the graph):

CGSpace CPU load year

The amount of CPU steal is very low if I look at it now, around 1 or 2 percent, but what’s happening now reminds me of the mysterious load problems I had in 2019-03 that were due to CPU steal
Salem said there was an issue with the sitemaps on MELSpace so that’s why it wasn’t working in AReS
- Load on CGSpace is low in the evening so I’ll start a new AReS harvest

2022-10-18

Start mapping the Initiative names on CGSpace to tne new short names from Enrico’s spreadsheet
Then I will update them for existing CGSpace items:

$ ./ilri/fix-metadata-values.py -i 2022-10-18-update-initiatives.csv -db dspace -u dspace -p 'fuuu' -f cg.contributor.initiative -m 258 -t correct -d -n

And later in the controlled vocabulary
Apply some corrections to a few hundred items on CGSpace for Peter
Meeting with Abenet, Sara, and Valentina about CG Core types
- We finished going over our list and agreed to send a message to concerned parties in our organizations for feedback by November 4th
- Next week we will continue doing the definitions
Re-sync CGSpace to DSpace Test to get the latest Initiatives changes
- I also need to re-create the CIAT/Alliance TIP accounts so they can continue testing
- I re-created the tip-submit@cgiar.org and tip-approve@cgiar.org account on DSpace Test
- According to my notes:
  - A user must be in the collection admin group in order to deposit via the REST API (not in the collection’s “Submit” group, which is for normal submission)
  - A user must be in the collection’s “Accept/Reject/Edit Metadata” step in order to see and approve the item in the DSpace workflow
- I created a new “TIP test” collection under Alliance’s community and added the users accordingly
- I think I’ll be able to just add these two submit/approve users to the Alliance Admins and Alliance Editors groups once we’re ready