# CGSpace Notes
Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository.

## June, 2016

Wed Jun 01, 2016 by Alan Orth in Notes

## 2016-06-01

- Experimenting with IFPRI OAI (we want to harvest their publications)
- After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php
- After reading the OAI documentation and testing with an OAI validator I found out how to get their publications
- This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc
- You can see the others by using the OAI `ListSets` verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets
- Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in `dc.identifier.fund` to `cg.identifier.cpwfproject` and then the rest to `dc.description.sponsorship`

Read more →

## May, 2016

Sun May 01, 2016 by Alan Orth in Notes

## 2016-05-01

- Since yesterday there have been 10,000 REST errors and the site has been unstable again
- I have blocked access to the API now
- There are 3,000 IPs accessing the REST API in a 24-hour period!

    # awk '{print $1}' /var/log/nginx/rest.log  | uniq | wc -l
    3168

Read more →

## April, 2016

Mon Apr 04, 2016 by Alan Orth in Notes

## 2016-04-04

- Looking at log file use on CGSpace and notice that we need to work on our cron setup a bit
- We are backing up all logs in the log folder, including useless stuff like solr, cocoon, handle-plugin, etc
- After running DSpace for over five years I’ve never needed to look in any other log file than dspace.log, leave alone one from last year!
- This will save us a few gigs of backup space we’re paying for on S3
- Also, I noticed the `checker` log has some errors we should pay attention to:

Read more →

## March, 2016

Wed Mar 02, 2016 by Alan Orth in Notes

## 2016-03-02

- Looking at issues with author authorities on CGSpace
- For some reason we still have the `index-lucene-update` cron job active on CGSpace, but I’m pretty sure we don’t need it as of the latest few versions of Atmire’s Listings and Reports module
- Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Java JDK 1.7 to match environment on CGSpace server

Read more →

## February, 2016

Fri Feb 05, 2016 by Alan Orth in Notes

## 2016-02-05

- Looking at some DAGRIS data for Abenet Yabowork
- Lots of issues with spaces, newlines, etc causing the import to fail
- I noticed we have a very interesting list of countries on CGSpace:

- Not only are there 49,000 countries, we have some blanks (25)…
- Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE”

Read more →

## January, 2016

Wed Jan 13, 2016 by Alan Orth in Notes

## 2016-01-13

- Move ILRI collection `10568/12503` from `10568/27869` to `10568/27629` using the move_collections.sh script I wrote last year.
- I realized it is only necessary to clear the Cocoon cache after moving collections—rather than reindexing—as no metadata has changed, and therefore no search or browse indexes need to be updated.
- Update GitHub wiki for documentation of maintenance tasks.

Read more →

## December, 2015

Wed Dec 02, 2015 by Alan Orth in Notes

## 2015-12-02

- Replace `lzop` with `xz` in log compression cron jobs on DSpace Test—it uses less space:

    # cd /home/dspacetest.cgiar.org/log
    # ls -lh dspace.log.2015-11-18*
    -rw-rw-r-- 1 tomcat7 tomcat7 2.0M Nov 18 23:59 dspace.log.2015-11-18
    -rw-rw-r-- 1 tomcat7 tomcat7 387K Nov 18 23:59 dspace.log.2015-11-18.lzo
    -rw-rw-r-- 1 tomcat7 tomcat7 169K Nov 18 23:59 dspace.log.2015-11-18.xz

Read more →

## November, 2015

Mon Nov 23, 2015 by Alan Orth in Notes

## 2015-11-22

- CGSpace went down
- Looks like DSpace exhausted its PostgreSQL connection pool
- Last week I had increased the limit from 30 to 60, which seemed to help, but now there are many more idle connections:

    $ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspace
    78

Read more →