+++ date = "2017-04-02T17:08:52+02:00" author = "Alan Orth" title = "April, 2017" tags = ["Notes"]

+++

2017-04-02

• Merge one change to CCAFS flagships that I had forgotten to remove last month ("MANAGING CLIMATE RISK"): https://github.com/ilri/DSpace/pull/317
• Quick proof-of-concept hack to add dc.rights to the input form, including some inline instructions/hints:

• Remove redundant/duplicate text in the DSpace submission license
• Testing the CMYK patch on a collection with 650 items:

$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt

2017-04-03

• Continue testing the CMYK patch on more communities:

$ [dspace]/bin/dspace filter-media -f -i 10568/1 -p "ImageMagick PDF Thumbnail" -v >> /tmp/filter-media-cmyk.txt 2>&1

• So far there are almost 500:

$ grep -c profile /tmp/filter-media-cmyk.txt
484

• Looking at the CG Core document again, I'll send some feedback to Peter and Abenet:
  • We use cg.contributor.crp to indicate the CRP(s) affiliated with the item
  • DSpace has dc.date.available, but this field isn't particularly meaningful other than as an automatic timestamp at the time of item accession (and is identical to dc.date.accessioned)
  • dc.relation exists in CGSpace, but isn't used—rather dc.relation.ispartofseries, which is used ~5,000 times to Series name and number within that series
• Also, I'm noticing some weird outliers in cg.coverage.region, need to remember to go correct these later:

dspace=# select text_value from metadatavalue where resource_type_id=2 and metadata_field_id=227;

2017-04-04

• The filter-media script has been running on more large communities and now there are many more CMYK PDFs that have been fixed:

$ grep -c profile /tmp/filter-media-cmyk.txt
1584

• Trying to find a way to get the number of items submitted by a certain user in 2016
• It's not possible in the DSpace search / module interfaces, but might be able to be derived from dc.description.provenance, as that field contains the name and email of the submitter/approver, ie:

Submitted by Francesca Giampieri (fgiampieri) on 2016-01-19T13:56:43Z^M
No. of bitstreams: 1^M
ILAC_Brief21_PMCA.pdf: 113462 bytes, checksum: 249fef468f401c066a119f5db687add0 (MD5)

• This SQL query returns fields that were submitted or approved by giampieri in 2016 and contain a "checksum" (ie, there was a bitstream in the submission):

dspace=# select * from metadatavalue where resource_type_id=2 and metadata_field_id=28 and text_value ~ '^(Submitted|Approved).*giampieri.*2016-.*checksum.*';

• Then this one does the same, but for fields that don't contain checksums (ie, there was no bitstream in the submission):

dspace=# select * from metadatavalue where resource_type_id=2 and metadata_field_id=28 and text_value ~ '^(Submitted|Approved).*giampieri.*2016-.*' and text_value !~ '^(Submitted|Approved).*giampieri.*2016-.*checksum.*';

• For some reason there seem to be way too many fields, for example there are 498 + 13 here, which is 511 items for just this one user.
• It looks like there can be a scenario where the user submitted AND approved it, so some records might be doubled...
• In that case it might just be better to see how many the user submitted (both with and without bitstreams):

dspace=# select * from metadatavalue where resource_type_id=2 and metadata_field_id=28 and text_value ~ '^Submitted.*giampieri.*2016-.*';

2017-04-05

• After doing a few more large communities it seems this is the final count of CMYK PDFs:

$ grep -c profile /tmp/filter-media-cmyk.txt
2505

2017-04-06

• After reading the notes for DCAT April 2017 I am testing some new settings for PostgreSQL on DSpace Test:
  • db.maxconnections 30→70 (the default PostgreSQL config allows 100 connections, so DSpace's default of 30 is quite low)
  • db.maxwait 5000→10000
  • db.maxidle 8→20 (DSpace default is -1, unlimited, but we had set it to 8 earlier)
• I need to look at the Munin graphs after a few days to see if the load has changed
• Run system updates on DSpace Test and reboot the server