CGSpace Notes

2017-03-01

• Run the 279 CIAT author corrections on CGSpace

2017-03-02

• Skype with Michael and Peter, discussing moving the CGIAR Library to CGSpace
• CGIAR people possibly open to moving content, redirecting library.cgiar.org to CGSpace and letting CGSpace resolve their handles
• They might come in at the top level in one “CGIAR System” community, or with several communities
• I need to spend a bit of time looking at the multiple handle support in DSpace and see if new content can be minted in both handles, or just one?
• Need to send Peter and Michael some notes about this in a few days
• Also, need to consider talking to Atmire about hiring them to bring ORCiD metadata to REST / OAI
• Filed an issue on DSpace issue tracker for the filter-media bug that causes it to process JPGs even when limiting to the PDF thumbnail plugin: DS-3516
• Discovered that the ImageMagic filter-media plugin creates JPG thumbnails with the CMYK colorspace when the source PDF is using CMYK

• Interestingly, it seems DSpace 4.x’s thumbnails were sRGB, but forcing regeneration using DSpace 5.x’s ImageMagick plugin creates CMYK JPGs if the source PDF was CMYK (see ¹⁰⁵⁶⁸⁄₅₁₉₉₉):

  $ identify ~/Desktop/alc_contrastes_desafios.jpg
  /Users/aorth/Desktop/alc_contrastes_desafios.jpg JPEG 464x600 464x600+0+0 8-bit CMYK 168KB 0.000u 0:00.000
  

2017-02-07

• An item was mapped twice erroneously again, so I had to remove one of the mappings manually:

  dspace=# select * from collection2item where item_id = '80278';
  id   | collection_id | item_id
  -------+---------------+---------
  92551 |           313 |   80278
  92550 |           313 |   80278
  90774 |          1051 |   80278
  (3 rows)
  dspace=# delete from collection2item where id = 92551 and item_id = 80278;
  DELETE 1
  

• Create issue on GitHub to track the addition of CCAFS Phase II project tags (#301)

• Looks like we’ll be using cg.identifier.ccafsprojectpii as the field name

2017-01-02

• I checked to see if the Solr sharding task that is supposed to run on January 1st had run and saw there was an error
• I tested on DSpace Test as well and it doesn’t work there either
• I asked on the dspace-tech mailing list because it seems to be broken, and actually now I’m not sure if we’ve ever had the sharding task run successfully over all these years

2016-12-02

• CGSpace was down for five hours in the morning while I was sleeping

• While looking in the logs for errors, I see tons of warnings about Atmire MQM:

  2016-12-02 03:00:32,352 WARN  com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=CREATE, SubjectType=BUNDLE, SubjectID=70316, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632305, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607")
  2016-12-02 03:00:32,353 WARN  com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=MODIFY_METADATA, SubjectType=BUNDLE, SubjectID =70316, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632309, dispatcher=1544803905, detail="dc.title", transactionID="TX157907838689377964651674089851855413607")
  2016-12-02 03:00:32,353 WARN  com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=ADD, SubjectType=ITEM, SubjectID=80044, Object Type=BUNDLE, ObjectID=70316, TimeStamp=1480647632311, dispatcher=1544803905, detail="THUMBNAIL", transactionID="TX157907838689377964651674089851855413607")
  2016-12-02 03:00:32,353 WARN  com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=ADD, SubjectType=BUNDLE, SubjectID=70316, Obje ctType=BITSTREAM, ObjectID=86715, TimeStamp=1480647632318, dispatcher=1544803905, detail="-1", transactionID="TX157907838689377964651674089851855413607")
  2016-12-02 03:00:32,353 WARN  com.atmire.metadataquality.batchedit.BatchEditConsumer @ BatchEditConsumer should not have been given this kind of Subject in an event, skipping: org.dspace.event.Event(eventType=MODIFY, SubjectType=ITEM, SubjectID=80044, ObjectType=(Unknown), ObjectID=-1, TimeStamp=1480647632351, dispatcher=1544803905, detail=[null], transactionID="TX157907838689377964651674089851855413607")
  

• I see thousands of them in the logs for the last few months, so it’s not related to the DSpace 5.5 upgrade

• I’ve raised a ticket with Atmire to ask

• Another worrying error from dspace.log is:

2016-11-01

• Add dc.type to the output options for Atmire’s Listings and Reports module (#286)

2016-10-03

• Testing adding ORCIDs to a CSV file for a single item to see if the author orders get messed up
• Need to test the following scenarios to see how author order is affected:
  • ORCIDs only
  • ORCIDs plus normal authors

• I exported a random item’s metadata as CSV, deleted all columns except id and collection, and made a new coloum called ORCID:dc.contributor.author with the following random ORCIDs from the ORCID registry:

  0000-0002-6115-0956||0000-0002-3812-8793||0000-0001-7462-405X
  

2016-09-01

• Discuss helping CCAFS with some batch tagging of ORCID IDs for their authors
• Discuss how the migration of CGIAR’s Active Directory to a flat structure will break our LDAP groups in DSpace
• We had been using DC=ILRI to determine whether a user was ILRI or not

• It looks like we might be able to use OUs now, instead of DCs:

  $ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b "dc=cgiarad,dc=org" -D "admigration1@cgiarad.org" -W "(sAMAccountName=admigration1)"
  

2016-08-01

• Add updated distribution license from Sisay (#259)
• Play with upgrading Mirage 2 dependencies in bower.json because most are several versions of out date
• Bootstrap is at 3.3.0 but upstream is at 3.3.7, and upgrading to anything beyond 3.3.1 breaks glyphicons and probably more
• bower stuff is a dead end, waste of time, too many issues
• Anything after Bootstrap 3.3.1 makes glyphicons disappear (HTTP 404 trying to access from incorrect path of fonts)

• Start working on DSpace 5.1 → 5.5 port:

  $ git checkout -b 55new 5_x-prod
  $ git reset --hard ilri/5_x-prod
  $ git rebase -i dspace-5.5
  

2016-07-01

• Add dc.description.sponsorship to Discovery sidebar facets and make investors clickable in item view (#232)

• I think this query should find and replace all authors that have “,” at the end of their names:

  dspacetest=# update metadatavalue set text_value = regexp_replace(text_value, '(^.+?),$', '\1') where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
  UPDATE 95
  dspacetest=# select text_value from  metadatavalue where metadata_field_id=3 and resource_type_id=2 and text_value ~ '^.+?,$';
  text_value
  ------------
  (0 rows)
  

• In this case the select query was showing 95 results before the update

2016-06-01

• Experimenting with IFPRI OAI (we want to harvest their publications)
• After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php
• After reading the OAI documentation and testing with an OAI validator I found out how to get their publications
• This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc
• You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets
• Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship