alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-21 22:25:02 +01:00
Code Issues Projects Releases Wiki Activity

Update DSpace wiki URL

Browse Source 
They moved to Lyrasis in 2019.
This commit is contained in:
Alan Orth 2020-04-13 15:30:24 +03:00
parent 0aed9d896b
commit 246538db59
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
23 changed files with 36 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
content/posts/2016-03.md
View File

@ -179,7 +179,7 @@ Can't find method org.dspace.app.xmlui.aspect.administrative.FlowGroupUtils.proc
## 2016-03-28 ## 2016-03-28
- Look into enabling the embargo during item submission, see: https://wiki.duraspace.org/display/DSDOC5x/Embargo#Embargo-SubmissionProcess - Look into enabling the embargo during item submission, see: https://wiki.lyrasis.org/display/DSDOC5x/Embargo#Embargo-SubmissionProcess
- Seems we only want `AccessStep` because `UploadWithEmbargoStep` disables the ability to edit embargos at the item level - Seems we only want `AccessStep` because `UploadWithEmbargoStep` disables the ability to edit embargos at the item level
- This pull request enables the ability to set an item-level embargo during submission: https://github.com/ilri/DSpace/pull/203 - This pull request enables the ability to set an item-level embargo during submission: https://github.com/ilri/DSpace/pull/203
- I figured out that the problem with Listings and Reports was because I disabled the `search.index.*` last week, and they are still used by JSPUI apparently - I figured out that the problem with Listings and Reports was because I disabled the `search.index.*` last week, and they are still used by JSPUI apparently

2
content/posts/2016-05.md
View File

@ -77,7 +77,7 @@ fi
- dcterms.type - dcterms.type
- dcterms.spatial - dcterms.spatial
- Not sure what `dcterms` is... - Not sure what `dcterms` is...
- Looks like these were [added in DSpace 4](https://wiki.duraspace.org/display/DSDOC5x/Metadata+and+Bitstream+Format+Registries#MetadataandBitstreamFormatRegistries-DublinCoreTermsRegistry(DCTERMS)) to allow for future work to make DSpace more flexible - Looks like these were [added in DSpace 4](https://wiki.lyrasis.org/display/DSDOC5x/Metadata+and+Bitstream+Format+Registries#MetadataandBitstreamFormatRegistries-DublinCoreTermsRegistry(DCTERMS)) to allow for future work to make DSpace more flexible
- CGSpace's `dc` registry has 96 items, and the default DSpace one has 73. - CGSpace's `dc` registry has 96 items, and the default DSpace one has 73.
## 2016-05-11 ## 2016-05-11

2
content/posts/2016-06.md
View File

@ -137,7 +137,7 @@ dspacetest=# \copy (select text_value, count(*) from metadatavalue where resourc
## 2016-06-08 ## 2016-06-08
- Discuss controlled vocabularies for ~28 fields - Discuss controlled vocabularies for ~28 fields
- Looks like this is all we need: https://wiki.duraspace.org/display/DSDOC5x/Submission+User+Interface#SubmissionUserInterface-ConfiguringControlledVocabularies - Looks like this is all we need: https://wiki.lyrasis.org/display/DSDOC5x/Submission+User+Interface#SubmissionUserInterface-ConfiguringControlledVocabularies
- I wrote an XPath expression to extract the ILRI subjects from `input-forms.xml` (uses xmlstartlet): - I wrote an XPath expression to extract the ILRI subjects from `input-forms.xml` (uses xmlstartlet):
``` ```

2
content/posts/2016-10.md
View File

@ -7,7 +7,7 @@ tags = ["Notes"]
+++ +++
## 2016-10-03 ## 2016-10-03
- Testing adding [ORCIDs to a CSV](https://wiki.duraspace.org/display/DSDOC5x/ORCID+Integration#ORCIDIntegration-EditingexistingitemsusingBatchCSVEditing) file for a single item to see if the author orders get messed up - Testing adding [ORCIDs to a CSV](https://wiki.lyrasis.org/display/DSDOC5x/ORCID+Integration#ORCIDIntegration-EditingexistingitemsusingBatchCSVEditing) file for a single item to see if the author orders get messed up
- Need to test the following scenarios to see how author order is affected: - Need to test the following scenarios to see how author order is affected:
- ORCIDs only - ORCIDs only
- ORCIDs plus normal authors - ORCIDs plus normal authors

4
content/posts/2016-11.md
View File

@ -419,7 +419,7 @@ INFO: FrameworkServlet 'oai': initialization completed in 2600 ms
``` ```
- It says OAI, and seems to start at 3:00 AM, but I only see the `filter-media` cron job set to start then - It says OAI, and seems to start at 3:00 AM, but I only see the `filter-media` cron job set to start then
- Double checking the [DSpace 5.x upgrade notes](https://wiki.duraspace.org/display/DSDOC5x/Upgrading+DSpace) for anything I missed, or troubleshooting tips - Double checking the [DSpace 5.x upgrade notes](https://wiki.lyrasis.org/display/DSDOC5x/Upgrading+DSpace) for anything I missed, or troubleshooting tips
- Running some manual processes just in case: - Running some manual processes just in case:
``` ```
@ -430,7 +430,7 @@ $ /home/dspacetest.cgiar.org/bin/dspace registry-loader -metadata /home/dspacete
``` ```
- Start working on paper for KM4Dev journal - Start working on paper for KM4Dev journal
- Wow, Bram from Atmire pointed out this solution for using multiple handles with one DSpace instance: https://wiki.duraspace.org/display/DSDOC5x/Installing+DSpace?focusedCommentId=78163296#comment-78163296 - Wow, Bram from Atmire pointed out this solution for using multiple handles with one DSpace instance: https://wiki.lyrasis.org/display/DSDOC5x/Installing+DSpace?focusedCommentId=78163296#comment-78163296
- We might be able to migrate the [CGIAR Library](http://library.cgiar.org/) now, as they had wanted to keep their handles - We might be able to migrate the [CGIAR Library](http://library.cgiar.org/) now, as they had wanted to keep their handles
## 2016-11-29 ## 2016-11-29

2
content/posts/2017-02.md
View File

@ -161,7 +161,7 @@ dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'http:
``` ```
- Run all DOI corrections on CGSpace - Run all DOI corrections on CGSpace
- Something to think about here is to write a [Curation Task](https://wiki.duraspace.org/display/DSDOC5x/Curation+System#CurationSystem-ScriptedTasks) in Java to do these sanity checks / corrections every night - Something to think about here is to write a [Curation Task](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System#CurationSystem-ScriptedTasks) in Java to do these sanity checks / corrections every night
- Then we could add a cron job for them and run them from the command line like: - Then we could add a cron job for them and run them from the command line like:
``` ```

2
content/posts/2017-03.md
View File

@ -78,7 +78,7 @@ $ curl -s -H "accept: application/json" -H "Content-Type: application/json" -X P
- But there are hundreds of combinations of fields and values (like `dc.subject` and all the center subjects), and we can't use wildcards in REST! - But there are hundreds of combinations of fields and values (like `dc.subject` and all the center subjects), and we can't use wildcards in REST!
- Reading about enabling multiple handle prefixes in DSpace - Reading about enabling multiple handle prefixes in DSpace
- There is a mailing list thread from 2011 about it: http://dspace.2283337.n4.nabble.com/Multiple-handle-prefixes-merged-DSpace-instances-td3427192.html - There is a mailing list thread from 2011 about it: http://dspace.2283337.n4.nabble.com/Multiple-handle-prefixes-merged-DSpace-instances-td3427192.html
- And a comment from Atmire's Bram about it on the DSpace wiki: https://wiki.duraspace.org/display/DSDOC5x/Installing+DSpace?focusedCommentId=78163296#comment-78163296 - And a comment from Atmire's Bram about it on the DSpace wiki: https://wiki.lyrasis.org/display/DSDOC5x/Installing+DSpace?focusedCommentId=78163296#comment-78163296
- Bram mentions an undocumented configuration option `handle.plugin.checknameauthority`, but I noticed another one in `dspace.cfg`: - Bram mentions an undocumented configuration option `handle.plugin.checknameauthority`, but I noticed another one in `dspace.cfg`:
``` ```

6
content/posts/2017-04.md
View File

@ -95,7 +95,7 @@ $ grep -c profile /tmp/filter-media-cmyk.txt
## 2017-04-06 ## 2017-04-06
- After reading the [notes for DCAT April 2017](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+April+2017) I am testing some new settings for PostgreSQL on DSpace Test: - After reading the [notes for DCAT April 2017](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+April+2017) I am testing some new settings for PostgreSQL on DSpace Test:
- `db.maxconnections` 30→70 (the default PostgreSQL config allows 100 connections, so DSpace's default of 30 is quite low) - `db.maxconnections` 30→70 (the default PostgreSQL config allows 100 connections, so DSpace's default of 30 is quite low)
- `db.maxwait` 5000→10000 - `db.maxwait` 5000→10000
- `db.maxidle` 8→20 (DSpace default is -1, unlimited, but we had set it to 8 earlier) - `db.maxidle` 8→20 (DSpace default is -1, unlimited, but we had set it to 8 earlier)
@ -103,7 +103,7 @@ $ grep -c profile /tmp/filter-media-cmyk.txt
- Run system updates on DSpace Test and reboot the server - Run system updates on DSpace Test and reboot the server
- Discussing harvesting CIFOR's DSpace via OAI - Discussing harvesting CIFOR's DSpace via OAI
- Sisay added their OAI as a source to a new collection, but using the Simple Dublin Core method, so many fields are unqualified and duplicated - Sisay added their OAI as a source to a new collection, but using the Simple Dublin Core method, so many fields are unqualified and duplicated
- Looking at the [documentation](https://wiki.duraspace.org/display/DSDOC5x/XMLUI+Configuration+and+Customization) it seems that we probably want to be using DSpace Intermediate Metadata - Looking at the [documentation](https://wiki.lyrasis.org/display/DSDOC5x/XMLUI+Configuration+and+Customization) it seems that we probably want to be using DSpace Intermediate Metadata
## 2017-04-10 ## 2017-04-10
@ -198,7 +198,7 @@ sys 1m29.310s
- DSpace committers reviewed my patch for DS-3516 and proposed a simpler idea involving incorrect use of `SelfRegisteredInputFormats` - DSpace committers reviewed my patch for DS-3516 and proposed a simpler idea involving incorrect use of `SelfRegisteredInputFormats`
- I tested the idea and it works, so I made a new patch: https://github.com/DSpace/DSpace/pull/1709 - I tested the idea and it works, so I made a new patch: https://github.com/DSpace/DSpace/pull/1709
- I discovered that we can override metadata formats in OAI by creating a new "context": https://wiki.duraspace.org/display/DSDOC5x/OAI+2.0+Server - I discovered that we can override metadata formats in OAI by creating a new "context": https://wiki.lyrasis.org/display/DSDOC5x/OAI+2.0+Server
- This allows us to have, say a default "request" context and a "cgiar" context, both of which implement the DSpace Intermediate Metadata formats, but have the later use a overridden version that exposes CG metadata - This allows us to have, say a default "request" context and a "cgiar" context, both of which implement the DSpace Intermediate Metadata formats, but have the later use a overridden version that exposes CG metadata
- Compare the following results: - Compare the following results:
- https://dspacetest.cgiar.org/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai:dspacetest.cgiar.org:10568/6 - https://dspacetest.cgiar.org/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai:dspacetest.cgiar.org:10568/6

4
content/posts/2017-05.md
View File

@ -39,7 +39,7 @@ $ [dspace]/bin/dspace curate -t requiredmetadata -i 10568/1 -r - > /tmp/curation
- Add "Blog Post" to `dc.type` - Add "Blog Post" to `dc.type`
- Create ticket on Atmire tracker to ask about commissioning them to develop the feature to expose ORCID via REST/OAI: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=510 - Create ticket on Atmire tracker to ask about commissioning them to develop the feature to expose ORCID via REST/OAI: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=510
- According to the [DSpace curation docs](https://wiki.duraspace.org/display/DSDOC5x/Curation+System) the fact that the `requiredmetadata` curation task stops when it finds a missing metadata field is by design - According to the [DSpace curation docs](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System) the fact that the `requiredmetadata` curation task stops when it finds a missing metadata field is by design
## 2017-05-07 ## 2017-05-07
@ -260,7 +260,7 @@ COPY 111
- File an issue on GitHub to explore/track migration to proper country/region codes (ISO 2/3 and UN M.49): [#326](https://github.com/ilri/DSpace/issues/326) - File an issue on GitHub to explore/track migration to proper country/region codes (ISO 2/3 and UN M.49): [#326](https://github.com/ilri/DSpace/issues/326)
- Ask Peter how the Landportal.info people should acknowledge us as the source of data on their website - Ask Peter how the Landportal.info people should acknowledge us as the source of data on their website
- Communicate with MARLO people about progress on exposing ORCIDs via the REST API, as it is set to be discussed in the [June, 2017 DCAT meeting](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+June+2017) - Communicate with MARLO people about progress on exposing ORCIDs via the REST API, as it is set to be discussed in the [June, 2017 DCAT meeting](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+June+2017)
- Find all of Amos Omore's author name variations so I can link them to his authority entry that has an ORCID: - Find all of Amos Omore's author name variations so I can link them to his authority entry that has an ORCID:
``` ```

2
content/posts/2017-06.md
View File

@ -133,7 +133,7 @@ Java stacktrace: java.util.NoSuchElementException: Timeout waiting for idle obje
``` ```
- After looking at the Tomcat logs, Munin graphs, and PostgreSQL connection stats, it seems there is just a high load - After looking at the Tomcat logs, Munin graphs, and PostgreSQL connection stats, it seems there is just a high load
- Might be a good time to adjust DSpace's database connection settings, like I first mentioned in April, 2017 after reading the [2017-04 DCAT comments](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+April+2017) - Might be a good time to adjust DSpace's database connection settings, like I first mentioned in April, 2017 after reading the [2017-04 DCAT comments](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+April+2017)
- I've adjusted the following in CGSpace's config: - I've adjusted the following in CGSpace's config:
- `db.maxconnections` 30→70 (the default PostgreSQL config allows 100 connections, so DSpace's default of 30 is quite low) - `db.maxconnections` 30→70 (the default PostgreSQL config allows 100 connections, so DSpace's default of 30 is quite low)
- `db.maxwait` 5000→10000 - `db.maxwait` 5000→10000

2
content/posts/2017-07.md
View File

@ -31,7 +31,7 @@ $ psql dspacenew -x -c 'select element, qualifier, scope_note from metadatafield
- We also need to check to see which ORCID authority core this uses, because it seems to be using an entirely new one rather than the one for `dc.contributor.author` (which makes sense of course, but fuck, all the author problems aren't bad enough?!) - We also need to check to see which ORCID authority core this uses, because it seems to be using an entirely new one rather than the one for `dc.contributor.author` (which makes sense of course, but fuck, all the author problems aren't bad enough?!)
- Also would need to edit XMLUI item displays to incorporate this into authors list - Also would need to edit XMLUI item displays to incorporate this into authors list
- And fuck, then anyone consuming our data via REST / OAI will not notice that we have an author outside of `dc.contributor.authors`... ugh - And fuck, then anyone consuming our data via REST / OAI will not notice that we have an author outside of `dc.contributor.authors`... ugh
- What if we modify the item submission form to use [`type-bind` fields to show/hide certain fields depending on the type](https://wiki.duraspace.org/display/DSDOC5x/Submission+User+Interface#SubmissionUserInterface-ItemtypeBasedMetadataCollection)? - What if we modify the item submission form to use [`type-bind` fields to show/hide certain fields depending on the type](https://wiki.lyrasis.org/display/DSDOC5x/Submission+User+Interface#SubmissionUserInterface-ItemtypeBasedMetadataCollection)?
## 2017-07-05 ## 2017-07-05

6
content/posts/2017-08.md
View File

@ -77,7 +77,7 @@ dspace#= \copy (select distinct text_value, count(*) from metadatavalue where me
- Meeting with Peter and CGSpace team - Meeting with Peter and CGSpace team
- Alan to follow up with ICARDA about depositing in CGSpace, we want ICARD and Drylands legacy content but not duplicates - Alan to follow up with ICARDA about depositing in CGSpace, we want ICARD and Drylands legacy content but not duplicates
- Alan to follow up on dc.rights, where are we? - Alan to follow up on dc.rights, where are we?
- Alan to follow up with Atmire about a dedicated field for ORCIDs, based on the discussion in the [June, 2017 DCAT meeting](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+June+2017) - Alan to follow up with Atmire about a dedicated field for ORCIDs, based on the discussion in the [June, 2017 DCAT meeting](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+June+2017)
- Alan to ask about how to query external services like AGROVOC in the DSpace submission form - Alan to ask about how to query external services like AGROVOC in the DSpace submission form
- Follow up with Atmire on the [ticket about ORCID metadata in DSpace](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=510) - Follow up with Atmire on the [ticket about ORCID metadata in DSpace](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=510)
- Follow up with Lili and Andrea about the pending CCAFS metadata and flagship updates - Follow up with Lili and Andrea about the pending CCAFS metadata and flagship updates
@ -138,7 +138,7 @@ dspace=# delete from metadatavalue where resource_type_id=2 and metadata_field_i
``` ```
- Generate a new list of authors from the CGIAR Library community for Peter to look through now that the initial corrections have been done - Generate a new list of authors from the CGIAR Library community for Peter to look through now that the initial corrections have been done
- Thinking about resource limits for PostgreSQL again after last week's CGSpace crash and related to a recently discussion I had in the comments of the [April, 2017 DCAT meeting notes](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+April+2017) - Thinking about resource limits for PostgreSQL again after last week's CGSpace crash and related to a recently discussion I had in the comments of the [April, 2017 DCAT meeting notes](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+April+2017)
- In that thread Chris Wilper suggests a new default of 35 max connections for `db.maxconnections` (from the current default of 30), knowing that _each DSpace web application_ gets to use up to this many on its own - In that thread Chris Wilper suggests a new default of 35 max connections for `db.maxconnections` (from the current default of 30), knowing that _each DSpace web application_ gets to use up to this many on its own
- It would be good to approximate what the theoretical maximum number of connections on a busy server would be, perhaps by looking to see which apps use SQL: - It would be good to approximate what the theoretical maximum number of connections on a busy server would be, perhaps by looking to see which apps use SQL:
@ -289,7 +289,7 @@ dspace.log.2017-08-17:584
- He linked to some examples from DSpace-CRIS that use this functionality: [VIAFAuthority](https://github.com/4Science/DSpace/blob/dspace-5_x_x-cris/dspace-api/src/main/java/org/dspace/content/authority/VIAFAuthority.java) - He linked to some examples from DSpace-CRIS that use this functionality: [VIAFAuthority](https://github.com/4Science/DSpace/blob/dspace-5_x_x-cris/dspace-api/src/main/java/org/dspace/content/authority/VIAFAuthority.java)
- I wired it up to the `dc.subject` field of the submission interface using the "lookup" type and it works! - I wired it up to the `dc.subject` field of the submission interface using the "lookup" type and it works!
- I think we can use this example to get a working AGROVOC query - I think we can use this example to get a working AGROVOC query
- More information about authority framework: https://wiki.duraspace.org/display/DSPACE/Authority+Control+of+Metadata+Values - More information about authority framework: https://wiki.lyrasis.org/display/DSPACE/Authority+Control+of+Metadata+Values
- Wow, I'm playing with the AGROVOC SPARQL endpoint using the [sparql-query tool](https://github.com/tialaramex/sparql-query): - Wow, I'm playing with the AGROVOC SPARQL endpoint using the [sparql-query tool](https://github.com/tialaramex/sparql-query):
``` ```

6
content/posts/2017-09.md
View File

@ -63,7 +63,7 @@ dspace.log.2017-09-10:0
## 2017-09-12 ## 2017-09-12
- I was testing the [METS XSD caching during AIP ingest](https://wiki.duraspace.org/display/DSDOC5x/AIP+Backup+and+Restore#AIPBackupandRestore-AIPConfigurationsToImproveIngestionSpeedwhileValidating) but it doesn't seem to help actually - I was testing the [METS XSD caching during AIP ingest](https://wiki.lyrasis.org/display/DSDOC5x/AIP+Backup+and+Restore#AIPBackupandRestore-AIPConfigurationsToImproveIngestionSpeedwhileValidating) but it doesn't seem to help actually
- The import process takes the same amount of time with and without the caching - The import process takes the same amount of time with and without the caching
- Also, I captured TCP packets destined for port 80 and both imports only captured ONE packet (an update check from some component in Java): - Also, I captured TCP packets destined for port 80 and both imports only captured ONE packet (an update check from some component in Java):
@ -378,7 +378,7 @@ DELETE 207
- More work on the CGIAR Library migration test run locally, as I was having problem with importing the last fourteen items from the CGIAR System Management Office community - More work on the CGIAR Library migration test run locally, as I was having problem with importing the last fourteen items from the CGIAR System Management Office community
- The problem was that we remapped the items to new collections after the initial import, so the items were using the 10947 prefix but the community and collection was using 10568 - The problem was that we remapped the items to new collections after the initial import, so the items were using the 10947 prefix but the community and collection was using 10568
- I ended up having to read the [AIP Backup and Restore](https://wiki.duraspace.org/display/DSDOC5x/AIP+Backup+and+Restore#AIPBackupandRestore-ForceReplaceMode) closely a few times and then explicitly preserve handles and ignore parents: - I ended up having to read the [AIP Backup and Restore](https://wiki.lyrasis.org/display/DSDOC5x/AIP+Backup+and+Restore#AIPBackupandRestore-ForceReplaceMode) closely a few times and then explicitly preserve handles and ignore parents:
``` ```
$ for item in 10568-93759/ITEM@10947-46*; do ~/dspace/bin/dspace packager -r -t AIP -o ignoreHandle=false -o ignoreParent=true -e aorth@mjanja.ch -p 10568/87738 $item; done $ for item in 10568-93759/ITEM@10947-46*; do ~/dspace/bin/dspace packager -r -t AIP -o ignoreHandle=false -o ignoreParent=true -e aorth@mjanja.ch -p 10568/87738 $item; done
@ -435,7 +435,7 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 dspace filter-media -f -i 10947/1 -p
## 2017-09-22 ## 2017-09-22
- Experimenting with setting up a global JNDI database resource that can be pooled among all the DSpace webapps (reference the [April, 2017 DCAT meeting](https://wiki.duraspace.org/display/cmtygp/DCAT+Meeting+April+2017) comments) - Experimenting with setting up a global JNDI database resource that can be pooled among all the DSpace webapps (reference the [April, 2017 DCAT meeting](https://wiki.lyrasis.org/display/cmtygp/DCAT+Meeting+April+2017) comments)
- See: https://www.journaldev.com/2513/tomcat-datasource-jndi-example-java - See: https://www.journaldev.com/2513/tomcat-datasource-jndi-example-java
- See: http://memorynotfound.com/configure-jndi-datasource-tomcat/ - See: http://memorynotfound.com/configure-jndi-datasource-tomcat/

2
content/posts/2017-12.md
View File

@ -418,7 +418,7 @@ $ schedtool -D -e ionice -c2 -n7 nice -n19 dspace index-discovery
- I don't have time now to look into this but the Solr sharding has long been an issue! - I don't have time now to look into this but the Solr sharding has long been an issue!
- Looking into using JDBC / JNDI to provide a database pool to DSpace - Looking into using JDBC / JNDI to provide a database pool to DSpace
- The [DSpace 6.x configuration docs](https://wiki.duraspace.org/display/DSDOC6x/Configuration+Reference) have more notes about setting up the database pool than the 5.x ones (which actually have none!) - The [DSpace 6.x configuration docs](https://wiki.lyrasis.org/display/DSDOC6x/Configuration+Reference) have more notes about setting up the database pool than the 5.x ones (which actually have none!)
- First, I uncomment `db.jndi` in _dspace/config/dspace.cfg_ - First, I uncomment `db.jndi` in _dspace/config/dspace.cfg_
- Then I create a global `Resource` in the main Tomcat _server.xml_ (inside `GlobalNamingResources`): - Then I create a global `Resource` in the main Tomcat _server.xml_ (inside `GlobalNamingResources`):

8
content/posts/2018-01.md
View File

@ -358,7 +358,7 @@ Caused by: org.apache.http.client.ClientProtocolException
... 10 more ... 10 more
``` ```
- There is interesting documentation about this on the DSpace Wiki: https://wiki.duraspace.org/display/DSDOC5x/SOLR+Statistics+Maintenance#SOLRStatisticsMaintenance-SolrShardingByYear - There is interesting documentation about this on the DSpace Wiki: https://wiki.lyrasis.org/display/DSDOC5x/SOLR+Statistics+Maintenance#SOLRStatisticsMaintenance-SolrShardingByYear
- I'm looking to see maybe if we're hitting the issues mentioned in [DS-2212](https://jira.duraspace.org/browse/DS-2212) that were apparently fixed in DSpace 5.2 - I'm looking to see maybe if we're hitting the issues mentioned in [DS-2212](https://jira.duraspace.org/browse/DS-2212) that were apparently fixed in DSpace 5.2
- I can apparently search for records in the Solr stats core that have an empty `owningColl` field using this in the Solr admin query: `-owningColl:*` - I can apparently search for records in the Solr stats core that have an empty `owningColl` field using this in the Solr admin query: `-owningColl:*`
- On CGSpace I see 48,000,000 records that have an `owningColl` field and 34,000,000 that don't: - On CGSpace I see 48,000,000 records that have an `owningColl` field and 34,000,000 that don't:
@ -378,7 +378,7 @@ $ http 'http://localhost:3000/solr/statistics/select?q=-owningColl%3A*&wt=json&i
2018-01-10 10:51:19,301 INFO org.dspace.statistics.SolrLogger @ Moving: 3821 records into core statistics-2016 2018-01-10 10:51:19,301 INFO org.dspace.statistics.SolrLogger @ Moving: 3821 records into core statistics-2016
``` ```
- Terry Brady has written some notes on the DSpace Wiki about Solr sharing issues: https://wiki.duraspace.org/display/%7Eterrywbrady/Statistics+Import+Export+Issues - Terry Brady has written some notes on the DSpace Wiki about Solr sharing issues: https://wiki.lyrasis.org/display/%7Eterrywbrady/Statistics+Import+Export+Issues
- Uptime Robot said that CGSpace went down at around 9:43 AM - Uptime Robot said that CGSpace went down at around 9:43 AM
- I looked at PostgreSQL's `pg_stat_activity` table and saw 161 active connections, but no pool errors in the DSpace logs: - I looked at PostgreSQL's `pg_stat_activity` table and saw 161 active connections, but no pool errors in the DSpace logs:
@ -519,7 +519,7 @@ db.url = jdbc:postgresql://localhost:5432/dspacetest?ApplicationName=dspaceDefau
## 2018-01-12 ## 2018-01-12
- I'm looking at the [DSpace 6.0 Install docs](https://wiki.duraspace.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-ServletEngine(ApacheTomcat7orlater,Jetty,CauchoResinorequivalent)) and notice they tweak the number of threads in their Tomcat connector: - I'm looking at the [DSpace 6.0 Install docs](https://wiki.lyrasis.org/display/DSDOC6x/Installing+DSpace#InstallingDSpace-ServletEngine(ApacheTomcat7orlater,Jetty,CauchoResinorequivalent)) and notice they tweak the number of threads in their Tomcat connector:
``` ```
<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 --> <!-- Define a non-SSL HTTP/1.1 Connector on port 8080 -->
@ -643,7 +643,7 @@ dspace=# select handle from item, handle where handle.resource_id = item.item_id
``` ```
- Even searching in the DSpace advanced search for author equals "Tarawali" produces nothing... - Even searching in the DSpace advanced search for author equals "Tarawali" produces nothing...
- Otherwise, the [DSpace 5 SQL Helper Functions](https://wiki.duraspace.org/display/DSPACE/Helper+SQL+functions+for+DSpace+5) provide `ds5_item2itemhandle()`, which is much easier than my long query above that I always have to go search for - Otherwise, the [DSpace 5 SQL Helper Functions](https://wiki.lyrasis.org/display/DSPACE/Helper+SQL+functions+for+DSpace+5) provide `ds5_item2itemhandle()`, which is much easier than my long query above that I always have to go search for
- For example, to find the Handle for an item that has the author "Erni": - For example, to find the Handle for an item that has the author "Erni":
``` ```

2
content/posts/2018-05.md
View File

@ -306,7 +306,7 @@ ga('send', 'pageview', {
## 2018-05-21 ## 2018-05-21
- Geoffrey from IITA got back with more questions about depositing items programatically into the CGSpace workflow - Geoffrey from IITA got back with more questions about depositing items programatically into the CGSpace workflow
- I pointed out that [SWORD](http://swordapp.org/) might be an option, as [DSpace supports the SWORDv2 protocol](https://wiki.duraspace.org/display/DSDOC5x/SWORDv2+Server) (although we have never tested it) - I pointed out that [SWORD](http://swordapp.org/) might be an option, as [DSpace supports the SWORDv2 protocol](https://wiki.lyrasis.org/display/DSDOC5x/SWORDv2+Server) (although we have never tested it)
- Work on implementing [cookie consent](https://cookieconsent.insites.com) popup for all XMLUI themes (SASS theme with primary / secondary branding from Bootstrap) - Work on implementing [cookie consent](https://cookieconsent.insites.com) popup for all XMLUI themes (SASS theme with primary / secondary branding from Bootstrap)
## 2018-05-22 ## 2018-05-22

4
content/posts/2018-09.md
View File

@ -296,7 +296,7 @@ https://cgspace.cgiar.org/rest/statlets?handle=10568/97103
- Put a link and or picture on the CGSpace homepage saying "Visualized CGSpace research" or something, and post a message on Yammer - Put a link and or picture on the CGSpace homepage saying "Visualized CGSpace research" or something, and post a message on Yammer
- I want to explore creating a thin API to make the item view and download stats available from Solr so CodeObia can use them in the AReS explorer - I want to explore creating a thin API to make the item view and download stats available from Solr so CodeObia can use them in the AReS explorer
- Currently CodeObia is exploring using the Atmire statlets internal API, but I don't really like that... - Currently CodeObia is exploring using the Atmire statlets internal API, but I don't really like that...
- There are some example queries on the [DSpace Solr wiki](https://wiki.duraspace.org/display/DSPACE/Solr) - There are some example queries on the [DSpace Solr wiki](https://wiki.lyrasis.org/display/DSPACE/Solr)
- For example, this query returns 1655 rows for item [10568/10630](https://cgspace.cgiar.org/handle/10568/10630): - For example, this query returns 1655 rows for item [10568/10630](https://cgspace.cgiar.org/handle/10568/10630):
``` ```
@ -477,7 +477,7 @@ dspacestatistics-> (id INT PRIMARY KEY, views INT DEFAULT 0, downloads INT DEFAU
- I want to purge the bot hits from the Solr statistics core, as I am now realizing that I don't give a shit about tens of millions of hits by Google and Bing indexing my shit every day (at least not in Solr!) - I want to purge the bot hits from the Solr statistics core, as I am now realizing that I don't give a shit about tens of millions of hits by Google and Bing indexing my shit every day (at least not in Solr!)
- CGSpace's Solr core has 150,000,000 documents in it... and it's still pretty fast to query, but it's really a maintenance and backup burden - CGSpace's Solr core has 150,000,000 documents in it... and it's still pretty fast to query, but it's really a maintenance and backup burden
- DSpace Test currently has about 2,000,000 documents with `isBot:true` in its Solr statistics core, and the size on disk is 2GB (it's not much, but I have to test this somewhere!) - DSpace Test currently has about 2,000,000 documents with `isBot:true` in its Solr statistics core, and the size on disk is 2GB (it's not much, but I have to test this somewhere!)
- According to the [DSpace 5.x Solr documentation](https://wiki.duraspace.org/display/DSDOC5x/SOLR+Statistics+Maintenance) I can use `dspace stats-util -f`, so let's try it: - According to the [DSpace 5.x Solr documentation](https://wiki.lyrasis.org/display/DSDOC5x/SOLR+Statistics+Maintenance) I can use `dspace stats-util -f`, so let's try it:
``` ```
$ dspace stats-util -f $ dspace stats-util -f

2
content/posts/2018-10.md
View File

@ -584,7 +584,7 @@ $ curl -X GET -H "Content-Type: application/json" -H "Accept: application/json"
## 2018-10-30 ## 2018-10-30
- Meet with the COPO guys to walk them through the CGSpace submission workflow and discuss CG core, REST API, etc - Meet with the COPO guys to walk them through the CGSpace submission workflow and discuss CG core, REST API, etc
- I suggested that they look into submitting via the [SWORDv2](https://wiki.duraspace.org/display/DSDOC5x/SWORDv2+Server) protocol because it respects the workflows - I suggested that they look into submitting via the [SWORDv2](https://wiki.lyrasis.org/display/DSDOC5x/SWORDv2+Server) protocol because it respects the workflows
- They said that they're not too worried about the hierarchical CG core schema, that they would just flatten metadata like affiliations when depositing to a DSpace repository - They said that they're not too worried about the hierarchical CG core schema, that they would just flatten metadata like affiliations when depositing to a DSpace repository
- I said that it might be time to engage the DSpace community to add support for more advanced schemas in DSpace 7+ (perhaps partnership with Atmire?) - I said that it might be time to engage the DSpace community to add support for more advanced schemas in DSpace 7+ (perhaps partnership with Atmire?)

2
content/posts/2019-03.md
View File

@ -155,7 +155,7 @@ if(isBlank(value), 'PLANT PRODUCTION & HEALTH', value + '||PLANT PRODUCTION & HE
- Create a pull request to change Swaziland to Eswatini and Macedonia to North Macedonia ([#414](https://github.com/ilri/DSpace/pull/414)) - Create a pull request to change Swaziland to Eswatini and Macedonia to North Macedonia ([#414](https://github.com/ilri/DSpace/pull/414))
- I see thirty-six items using Swaziland country metadata, and Peter says we should change only those from 2018 and 2019 - I see thirty-six items using Swaziland country metadata, and Peter says we should change only those from 2018 and 2019
- I think that I could get the resource IDs from SQL and then export them using `dspace metadata-export`... - I think that I could get the resource IDs from SQL and then export them using `dspace metadata-export`...
- This is a bit ugly, but it works (using the [DSpace 5 SQL helper function](https://wiki.duraspace.org/display/DSPACE/Helper+SQL+functions+for+DSpace+5) to resolve ID to handle): - This is a bit ugly, but it works (using the [DSpace 5 SQL helper function](https://wiki.lyrasis.org/display/DSPACE/Helper+SQL+functions+for+DSpace+5) to resolve ID to handle):
``` ```
for id in $(psql -U postgres -d dspacetest -h localhost -c "SELECT resource_id FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=228 AND text_value LIKE '%SWAZILAND%'" | grep -oE '[0-9]{3,}'); do for id in $(psql -U postgres -d dspacetest -h localhost -c "SELECT resource_id FROM metadatavalue WHERE resource_type_id=2 AND metadata_field_id=228 AND text_value LIKE '%SWAZILAND%'" | grep -oE '[0-9]{3,}'); do

4
content/posts/2019-04.md
View File

@ -310,7 +310,7 @@ X-XSS-Protection: 1; mode=block
- Ugh, even after optimizing there are no Solr results for requests from my IP, and actually I only see 18 results from 2019-04 so far and none of them are `statistics_type:view`... very weird - Ugh, even after optimizing there are no Solr results for requests from my IP, and actually I only see 18 results from 2019-04 so far and none of them are `statistics_type:view`... very weird
- I don't even see many hits for days after 2019-03-17, when I migrated the server to Ubuntu 18.04 and copied the statistics core from CGSpace (linode18) - I don't even see many hits for days after 2019-03-17, when I migrated the server to Ubuntu 18.04 and copied the statistics core from CGSpace (linode18)
- I will try to re-deploy the `5_x-dev` branch and test again - I will try to re-deploy the `5_x-dev` branch and test again
- According to the [DSpace 5.x Solr documentation](https://wiki.duraspace.org/display/DSDOC5x/SOLR+Statistics) the default commit time is after 15 minutes or 10,000 documents (see `solrconfig.xml`) - According to the [DSpace 5.x Solr documentation](https://wiki.lyrasis.org/display/DSDOC5x/SOLR+Statistics) the default commit time is after 15 minutes or 10,000 documents (see `solrconfig.xml`)
- I looped some GET and HEAD requests to a bitstream on my local instance and after some time I see that they *do* register as downloads (even though they are internal): - I looped some GET and HEAD requests to a bitstream on my local instance and after some time I see that they *do* register as downloads (even though they are internal):
``` ```
@ -1014,7 +1014,7 @@ dspace=# SELECT * FROM item WHERE item_id=74648;
2019-04-28 08:53:07,170 ERROR org.dspace.rest.ItemsResource @ User(anonymous) has not permission to read item(id=74648)! 2019-04-28 08:53:07,170 ERROR org.dspace.rest.ItemsResource @ User(anonymous) has not permission to read item(id=74648)!
``` ```
- I even tried to "expunge" the item using an [action in CSV](https://wiki.duraspace.org/display/DSDOC5x/Batch+Metadata+Editing#BatchMetadataEditing-Performing'actions'onitems), and it said "EXPUNGED!" but the item is still there... - I even tried to "expunge" the item using an [action in CSV](https://wiki.lyrasis.org/display/DSDOC5x/Batch+Metadata+Editing#BatchMetadataEditing-Performing'actions'onitems), and it said "EXPUNGED!" but the item is still there...
## 2019-04-30 ## 2019-04-30

2
content/posts/2019-05.md
View File

@ -55,7 +55,7 @@ curl: (22) The requested URL returned error: 401 Unauthorized
- I think the solution is to recommend people don't use the `/items/find-by-metadata-value` endpoint - I think the solution is to recommend people don't use the `/items/find-by-metadata-value` endpoint
- CIP is asking about embedding PDF thumbnail images in their RSS feeds again - CIP is asking about embedding PDF thumbnail images in their RSS feeds again
- They asked in 2018-09 as well and I told them it wasn't possible - They asked in 2018-09 as well and I told them it wasn't possible
- To make sure, I looked at [the documentation for RSS media feeds](https://wiki.duraspace.org/display/DSPACE/Enable+Media+RSS+Feeds) and tried it, but couldn't get it to work - To make sure, I looked at [the documentation for RSS media feeds](https://wiki.lyrasis.org/display/DSPACE/Enable+Media+RSS+Feeds) and tried it, but couldn't get it to work
- It seems to be geared towards iTunes and Podcasts... I dunno - It seems to be geared towards iTunes and Podcasts... I dunno
- CIP also asked for a way to get an XML file of all their RTB journal articles on CGSpace - CIP also asked for a way to get an XML file of all their RTB journal articles on CGSpace
- I told them to use the REST API like (where `1179` is the id of the RTB journal articles collection): - I told them to use the REST API like (where `1179` is the id of the RTB journal articles collection):

2
content/posts/2019-09.md
View File

@ -80,7 +80,7 @@ Mozilla/5.0 ((Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3
- Now I just checked a few manually and they are correct in the original input file, so something must have happened when Sisay was processing them for upload - Now I just checked a few manually and they are correct in the original input file, so something must have happened when Sisay was processing them for upload
- I have asked Sisay to fix them... - I have asked Sisay to fix them...
- Continue working on CG Core v2 migration, focusing on the crosswalk mappings - Continue working on CG Core v2 migration, focusing on the crosswalk mappings
- I think we can skip the MODS crosswalk for now because it is only used in [AIP exports that are meant for non-DSpace systems](https://wiki.duraspace.org/display/DSDOC5x/DSpace+AIP+Format#DSpaceAIPFormat-MODSSchema) - I think we can skip the MODS crosswalk for now because it is only used in [AIP exports that are meant for non-DSpace systems](https://wiki.lyrasis.org/display/DSDOC5x/DSpace+AIP+Format#DSpaceAIPFormat-MODSSchema)
- We should probably do the QDC crosswalk as well as those in `xhtml-head-item.properties`... - We should probably do the QDC crosswalk as well as those in `xhtml-head-item.properties`...
- Ouch, there is potentially a lot of work in the OAI metadata formats like DIM, METS, and QDC (see `dspace/config/crosswalks/oai/*.xsl`) - Ouch, there is potentially a lot of work in the OAI metadata formats like DIM, METS, and QDC (see `dspace/config/crosswalks/oai/*.xsl`)
- In general I think I should only modify the left side of the crosswalk mappings (ie, where metadata is coming from) so we maintain the same exact output for search engines, etc - In general I think I should only modify the left side of the crosswalk mappings (ie, where metadata is coming from) so we maintain the same exact output for search engines, etc

2
content/posts/cgspace-cgcorev2-migration.md
View File

@ -112,7 +112,7 @@ dcterms.audience | ✓ | ✓ | - | - | - | ✓ | |
There are a few things that I need to check once I get a deployment of this code up and running: There are a few things that I need to check once I get a deployment of this code up and running:
- Assess the XSL changes to see if things like `not(@qualifier)]` still make sense after we move fields from DC to DCTERMS, as some fields will no longer have qualifiers - Assess the XSL changes to see if things like `not(@qualifier)]` still make sense after we move fields from DC to DCTERMS, as some fields will no longer have qualifiers
- Do I need to edit crosswalks that we are not using, like [MODS](https://wiki.duraspace.org/display/DSDOC5x/DSpace+AIP+Format#DSpaceAIPFormat-MODSSchema)? - Do I need to edit crosswalks that we are not using, like [MODS](https://wiki.lyrasis.org/display/DSDOC5x/DSpace+AIP+Format#DSpaceAIPFormat-MODSSchema)?
- There is potentially a lot of work in the OAI metadata formats like DIM, METS, and QDC (see `dspace/config/crosswalks/oai/*.xsl`) - There is potentially a lot of work in the OAI metadata formats like DIM, METS, and QDC (see `dspace/config/crosswalks/oai/*.xsl`)
------ ------