diff --git a/content/cgiar-library-migration.md b/content/cgiar-library-migration.md index a5042a3cc..969aeb150 100644 --- a/content/cgiar-library-migration.md +++ b/content/cgiar-library-migration.md @@ -13,7 +13,7 @@ Rough notes for importing the CGIAR Library content. It was decided that this co ## Pre-migration Technical TODOs Things that need to happen before the migration: -- [x] Create top-level community on CGSpace to hold the CGIAR Library content: 10568/83389 +- [x] Create top-level community on CGSpace to hold the CGIAR Library content: `10568/83389` - [x] Update nginx redirects in ansible templates - [x] Update handle in DSpace XMLUI config - Set up nginx redirects for URLs like: @@ -23,6 +23,16 @@ Things that need to happen before the migration: - [x] Increase `max_connections` in `/etc/postgresql/9.5/main/postgresql.conf` by ~10 - `SELECT * FROM pg_stat_activity;` seems to show ~6 extra connections used by the command line tools during import - [x] Temporarily disable nightly `index-discovery` cron job because the import process will be taking place during some of this time and I don't want them to be competing to update the Solr index +- [x] Copy HTTPS certificate key pair from CGIAR Library server's Tomcat keystore: + +``` +$ keytool -list -keystore tomcat.keystore +$ keytool -importkeystore -srckeystore tomcat.keystore -destkeystore library.cgiar.org.p12 -deststoretype PKCS12 -srcalias tomcat +$ openssl pkcs12 -in library.cgiar.org.p12 -nokeys -out library.cgiar.org.crt.pem +$ openssl pkcs12 -in library.cgiar.org.p12 -nodes -nocerts -out library.cgiar.org.key.pem +$ wget https://certs.godaddy.com/repository/gdroot-g2.crt https://certs.godaddy.com/repository/gdig2.crt.pem +$ cat library.cgiar.org.crt.pem gdig2.crt.pem > library.cgiar.org-chained.pem +``` ## Migration Process @@ -79,8 +89,8 @@ This submits AIP hierarchies recursively (-r) and suppresses errors when an item **Create new subcommunities and collections for content we reorganized into new hierarchies from the original:** -- [x] Create _CGIAR System Management Board_ sub-community: 10568/83536 - - [x] Content from _CGIAR System Management Board documents_ collection (10947/4561) goes here +- [x] Create _CGIAR System Management Board_ sub-community: `10568/83536` + - [x] Content from _CGIAR System Management Board documents_ collection (`10947/4561`) goes here - Import collection hierarchy first and then the items: ``` @@ -88,8 +98,8 @@ $ dspace packager -r -t AIP -o ignoreHandle=false -e aorth@mjanja.ch -p 10568/83 $ for item in 10568-93760/ITEM@10947-465*; do dspace packager -r -f -u -t AIP -e aorth@mjanja.ch $item; done ``` -- [x] Create _CGIAR System Management Office_ sub-community: 10568/83537 - - [x] Create _CGIAR System Management Office documents_ collection: 10568/83538 +- [x] Create _CGIAR System Management Office_ sub-community: `10568/83537` + - [x] Create _CGIAR System Management Office documents_ collection: `10568/83538` - Import items to collection individually in replace mode (-r) while explicitly preserving handles and ignoring parents: ``` @@ -116,7 +126,7 @@ $ for item in 10947-latest/*.zip; do dspace packager -r -u -t AIP -e aorth@mjanj ## Post Migration -- [ ] Shut down Tomcat and run `update-sequences.sql` as the system's `postgres` user +- [x] Shut down Tomcat and run `update-sequences.sql` as the system's `postgres` user - [x] Remove ingestion overrides from `dspace.cfg` - [x] Reset PostgreSQL `max_connections` to 183 - [x] Enable nightly `index-discovery` cron job @@ -153,24 +163,11 @@ $ sudo su - - Now I'm wondering how we'll do this when we move servers in the future, because the `make-handle-config` basically assumes you only have one handle - Also, there is `dspace make-handle-config` and `bin/make-handle-config` and both behave differently (the first is interactive, the second reads your `dspace.cfg` and generates your handle config and `sitebndl.zip` accordingly) - I'm really not sure on the proper order of events actually - -- HTTPS certificates: - - [x] Install current certificates from their Tomcat keystore - -``` -$ keytool -list -keystore tomcat.keystore -$ keytool -importkeystore -srckeystore tomcat.keystore -destkeystore library.cgiar.org.p12 -deststoretype PKCS12 -srcalias tomcat -$ openssl pkcs12 -in library.cgiar.org.p12 -nokeys -out library.cgiar.org.crt.pem -$ openssl pkcs12 -in library.cgiar.org.p12 -nodes -nocerts -out library.cgiar.org.key.pem -$ wget https://certs.godaddy.com/repository/gdroot-g2.crt https://certs.godaddy.com/repository/gdig2.crt.pem -$ cat library.cgiar.org.crt.pem gdig2.crt.pem > library.cgiar.org-chained.pem -``` - - [ ] Update DNS records: - CNAME: cgspace.cgiar.org -- [ ] Re-deploy DSpace from freshly built `5_x-prod` branch -- [ ] Merge `cgiar-library` branch to `master` and re-run ansible nginx templates -- [ ] Run system updates and reboot server +- [x] Re-deploy DSpace from freshly built `5_x-prod` branch +- [x] Merge `cgiar-library` branch to `master` and re-run ansible nginx templates +- [x] Run system updates and reboot server - [ ] Switch to Let's Encrypt HTTPS certificates (after DNS is updated and server isn't busy): ``` diff --git a/content/post/2017-09.md b/content/post/2017-09.md index ee43d5668..158dfe6d1 100644 --- a/content/post/2017-09.md +++ b/content/post/2017-09.md @@ -413,4 +413,5 @@ $ for item in 10568-93759/ITEM@10947-46*; do ~/dspace/bin/dspace packager -r -t - I had a look at the collection and noticed a bunch of issues with item types and donors, so I asked him to fix those and import it to DSpace Test again first - Abenet wants to be able to filter by ISI Journal in advanced search on queries like this: https://cgspace.cgiar.org/discover?filtertype_0=dateIssued&filtertype_1=dateIssued&filter_relational_operator_1=equals&filter_relational_operator_0=equals&filter_1=%5B2010+TO+2017%5D&filter_0=2017&filtertype=type&filter_relational_operator=equals&filter=Journal+Article - I opened an issue to track this ([#340](https://github.com/ilri/DSpace/issues/340)) and will test it on DSpace Test soon - +- Marianne Gadeberg from WLE asked if I would add an account for Adam Hunt on CGSpace and give him permissions to approve all WLE publications +- I told him to register first, as he's a CGIAR user and needs an account to be created before I can add him to the groups diff --git a/public/2016-12/index.html b/public/2016-12/index.html index 3017cd33e..a8e2343ef 100644 --- a/public/2016-12/index.html +++ b/public/2016-12/index.html @@ -33,7 +33,7 @@ Another worrying error from dspace.log is: - + @@ -79,7 +79,7 @@ Another worrying error from dspace.log is: "url": "https://alanorth.github.io/cgspace-notes/2016-12/", "wordCount": "4078", "datePublished": "2016-12-02T10:43:00+03:00", - "dateModified": "2017-01-10T16:21:47+02:00", + "dateModified": "2017-09-19T16:07:20+03:00", "author": { "@type": "Person", "name": "Alan Orth" diff --git a/public/2017-09/index.html b/public/2017-09/index.html index ca1361789..0b70390ce 100644 --- a/public/2017-09/index.html +++ b/public/2017-09/index.html @@ -61,7 +61,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne’s user account "@type": "BlogPosting", "headline": "September, 2017", "url": "https://alanorth.github.io/cgspace-notes/2017-09/", - "wordCount": "2886", + "wordCount": "2937", "datePublished": "2017-09-07T16:54:52+07:00", "dateModified": "2017-09-19T12:53:00+03:00", "author": { @@ -585,6 +585,8 @@ DELETE 207
  • I had a look at the collection and noticed a bunch of issues with item types and donors, so I asked him to fix those and import it to DSpace Test again first
  • Abenet wants to be able to filter by ISI Journal in advanced search on queries like this: https://cgspace.cgiar.org/discover?filtertype_0=dateIssued&filtertype_1=dateIssued&filter_relational_operator_1=equals&filter_relational_operator_0=equals&filter_1=%5B2010+TO+2017%5D&filter_0=2017&filtertype=type&filter_relational_operator=equals&filter=Journal+Article
  • I opened an issue to track this (#340) and will test it on DSpace Test soon
  • +
  • Marianne Gadeberg from WLE asked if I would add an account for Adam Hunt on CGSpace and give him permissions to approve all WLE publications
  • +
  • I told him to register first, as he’s a CGIAR user and needs an account to be created before I can add him to the groups
  • diff --git a/public/categories/notes/index.xml b/public/categories/notes/index.xml index 099a592fb..f6a06dd07 100644 --- a/public/categories/notes/index.xml +++ b/public/categories/notes/index.xml @@ -20,7 +20,7 @@ Note: I’m temporarily making this a page because it seems Hugo (currently 0.27.1) cannot use a custom slug for a post when there is a permalink defined in config.toml Rough notes for importing the CGIAR Library content. It was decided that this content would go to a new top-level community called CGIAR System Organization. Pre-migration Technical TODOs Things that need to happen before the migration: -Create top-level community on CGSpace to hold the CGIAR Library content: 10568⁄83389 Update nginx redirects in ansible templates Update handle in DSpace XMLUI config Set up nginx redirects for URLs like: https://library. +Create top-level community on CGSpace to hold the CGIAR Library content: 10568/83389 Update nginx redirects in ansible templates Update handle in DSpace XMLUI config Set up nginx redirects for URLs like: https://library. diff --git a/public/cgiar-library-migration/index.html b/public/cgiar-library-migration/index.html index 1673625fb..392a2a300 100644 --- a/public/cgiar-library-migration/index.html +++ b/public/cgiar-library-migration/index.html @@ -13,7 +13,7 @@ - + @@ -37,9 +37,9 @@ "@type": "BlogPosting", "headline": "CGIAR Library Migration", "url": "https://alanorth.github.io/cgspace-notes/cgiar-library-migration/", - "wordCount": "1319", + "wordCount": "1321", "datePublished": "2017-09-18T16:38:35+03:00", - "dateModified": "2017-09-19T12:53:00+03:00", + "dateModified": "2017-09-19T16:07:20+03:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -117,7 +117,7 @@

    Things that need to happen before the migration:

    +
    $ keytool -list -keystore tomcat.keystore
    +$ keytool -importkeystore -srckeystore tomcat.keystore -destkeystore library.cgiar.org.p12 -deststoretype PKCS12 -srcalias tomcat
    +$ openssl pkcs12 -in library.cgiar.org.p12 -nokeys -out library.cgiar.org.crt.pem
    +$ openssl pkcs12 -in library.cgiar.org.p12 -nodes -nocerts -out library.cgiar.org.key.pem
    +$ wget https://certs.godaddy.com/repository/gdroot-g2.crt https://certs.godaddy.com/repository/gdig2.crt.pem
    +$ cat library.cgiar.org.crt.pem gdig2.crt.pem > library.cgiar.org-chained.pem
    +
    +

    Migration Process

    Export all top-level communities and collections from DSpace Test:

    @@ -195,10 +204,10 @@ $ for item in 10947-1/ITEM@10947-*; do dspace packager -r -f -u -t AIP -e aorth@

    Create new subcommunities and collections for content we reorganized into new hierarchies from the original:

    @@ -208,10 +217,10 @@ $ for item in 10568-93760/ITEM@10947-465*; do dspace packager -r -f -u -t AIP -e @@ -241,7 +250,7 @@ $ for item in 10568-93760/ITEM@10947-465*; do dspace packager -r -f -u -t AIP -e

    Post Migration

    - -
    $ keytool -list -keystore tomcat.keystore
    -$ keytool -importkeystore -srckeystore tomcat.keystore -destkeystore library.cgiar.org.p12 -deststoretype PKCS12 -srcalias tomcat
    -$ openssl pkcs12 -in library.cgiar.org.p12 -nokeys -out library.cgiar.org.crt.pem
    -$ openssl pkcs12 -in library.cgiar.org.p12 -nodes -nocerts -out library.cgiar.org.key.pem
    -$ wget https://certs.godaddy.com/repository/gdroot-g2.crt https://certs.godaddy.com/repository/gdig2.crt.pem
    -$ cat library.cgiar.org.crt.pem gdig2.crt.pem > library.cgiar.org-chained.pem
    -
    - - diff --git a/public/index.xml b/public/index.xml index 1142a58fd..429cb2ed6 100644 --- a/public/index.xml +++ b/public/index.xml @@ -20,7 +20,7 @@ Note: I’m temporarily making this a page because it seems Hugo (currently 0.27.1) cannot use a custom slug for a post when there is a permalink defined in config.toml Rough notes for importing the CGIAR Library content. It was decided that this content would go to a new top-level community called CGIAR System Organization. Pre-migration Technical TODOs Things that need to happen before the migration: -Create top-level community on CGSpace to hold the CGIAR Library content: 10568⁄83389 Update nginx redirects in ansible templates Update handle in DSpace XMLUI config Set up nginx redirects for URLs like: https://library. +Create top-level community on CGSpace to hold the CGIAR Library content: 10568/83389 Update nginx redirects in ansible templates Update handle in DSpace XMLUI config Set up nginx redirects for URLs like: https://library. diff --git a/public/robots.txt b/public/robots.txt index defa6419a..bb802e51c 100644 --- a/public/robots.txt +++ b/public/robots.txt @@ -27,7 +27,7 @@ Disallow: /cgspace-notes/2015-12/ Disallow: /cgspace-notes/2015-11/ Disallow: /cgspace-notes/ Disallow: /cgspace-notes/categories/ -Disallow: /cgspace-notes/categories/notes/ Disallow: /cgspace-notes/tags/notes/ +Disallow: /cgspace-notes/categories/notes/ Disallow: /cgspace-notes/post/ Disallow: /cgspace-notes/tags/ diff --git a/public/sitemap.xml b/public/sitemap.xml index 804cf99bb..cd2555652 100644 --- a/public/sitemap.xml +++ b/public/sitemap.xml @@ -4,7 +4,7 @@ https://alanorth.github.io/cgspace-notes/cgiar-library-migration/ - 2017-09-19T12:53:00+03:00 + 2017-09-19T16:07:20+03:00 @@ -54,7 +54,7 @@ https://alanorth.github.io/cgspace-notes/2016-12/ - 2017-01-10T16:21:47+02:00 + 2017-09-19T16:07:20+03:00 @@ -124,7 +124,7 @@ https://alanorth.github.io/cgspace-notes/ - 2017-09-19T12:53:00+03:00 + 2017-09-19T16:07:20+03:00 0 @@ -134,14 +134,14 @@ - https://alanorth.github.io/cgspace-notes/categories/notes/ + https://alanorth.github.io/cgspace-notes/tags/notes/ 2017-09-19T12:53:00+03:00 0 - https://alanorth.github.io/cgspace-notes/tags/notes/ - 2017-09-19T12:53:00+03:00 + https://alanorth.github.io/cgspace-notes/categories/notes/ + 2017-09-19T16:07:20+03:00 0