diff --git a/content/posts/2020-05.md b/content/posts/2020-05.md index a22c6d172..38529d37e 100644 --- a/content/posts/2020-05.md +++ b/content/posts/2020-05.md @@ -175,4 +175,54 @@ $ ./add-orcid-identifiers-csv.py -i 2020-05-25-add-orcids.csv -db dspace -u dspa - Atmire got back to me about the [Solr CUA issue in the DSpace 6 upgrade](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=706) and they cannot reproduce the error - The next step is for me to migrate DSpace Test (linode26) to DSpace 6 and try to reproduce the error there +## 2020-05-31 + +- Start preparing to migrate DSpace Test (linode26) to the `6_x-dev-atmire-modules` branch + - Run all system updates and reboot + - For now I will disable all yearly Solr statistics cores except the current `statistics` one + - Prepare PostgreSQL with a clean snapshot of CGSpace's DSpace 5.8 database: + +``` +$ sudo su - postgres +$ dropdb dspacetest +$ createdb -O dspacetest --encoding=UNICODE dspacetest +$ psql dspacetest -c 'alter user dspacetest superuser;' +$ pg_restore -d dspacetest -O --role=dspacetest /tmp/cgspace_2020-05-31.backup +$ psql dspacetest -c 'alter user dspacetest nosuperuser;' +# run DSpace 5 version of update-sequences.sql!!! +$ psql -f /home/dspace/src/git/DSpace/dspace/etc/postgres/update-sequences.sql dspacetest +$ psql dspacetest -c "DELETE FROM schema_version WHERE version IN ('5.8.2015.12.03.3');" +$ psql dspacetest -c 'CREATE EXTENSION pgcrypto;' +$ exit +``` + +- Now switch to the DSpace 6.x branch and start a build: + +``` +$ chrt -i 0 ionice -c2 -n7 nice -n19 mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false package +... +[ERROR] Failed to execute goal on project additions: Could not resolve dependencies for project org.dspace.modules:additions:jar:6.3: Failed to collect dependencies at com.atmire:atmire-listings-and-reports-api:jar:6.x-2.10.8-0-SNAPSHOT: Failed to read artifact descriptor for com.atmire:atmire-listings-and-reports-api:jar:6.x-2.10.8-0-SNAPSHOT: Could not transfer artifact com.atmire:atmire-listings-and-reports-api:pom:6.x-2.10.8-0-SNAPSHOT from/to atmire.com-snapshots (https://atmire.com/artifactory/atmire.com-snapshots): Not authorized , ReasonPhrase:Unauthorized. -> [Help 1] +``` + +- Great! I will have to send Atmire a note about this... but for now I can sync over my local `~/.m2` directory and the build completes +- After the Maven build completed successfully I installed the updated code with Ant (make sure to delete the old spring directory): + +``` +$ cd dspace/target/dspace-installer +$ rm -rf /blah/dspacetest/config/spring +$ ant update +``` + +- Database migrations take 10:18.287s during the first startup... + - perhaps when we do the production CGSpace migration I can do this in advance and tell users not to make any submissions? +- I had a mistake in my Solr internal URL parameter so DSpace couldn't find it, but once I fixed that DSpace starts up OK! +- Once the initial Discovery reindexing is completed I started the Solr statistics UUID migration: + +``` +$ export JAVA_OPTS="-Xmx1024m -Dfile.encoding=UTF-8" +``` + +- Experiment a bit with the Python [country-converter](https://pypi.org/project/country-converter/) library as it can convert between different formats (like ISO 3166 and UN m49) + - We need to eventually find a format we can use for all CGIAR DSpaces... + diff --git a/docs/2020-05/index.html b/docs/2020-05/index.html index 49806a0fa..ab31f2feb 100644 --- a/docs/2020-05/index.html +++ b/docs/2020-05/index.html @@ -18,7 +18,7 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2 - + @@ -41,9 +41,9 @@ I see that CGSpace (linode18) is still using PostgreSQL JDBC driver version 42.2 "@type": "BlogPosting", "headline": "May, 2020", "url": "https://alanorth.github.io/cgspace-notes/2020-05/", - "wordCount": "1515", + "wordCount": "1861", "datePublished": "2020-05-02T09:52:04+03:00", - "dateModified": "2020-05-29T10:25:41+03:00", + "dateModified": "2020-05-30T18:38:16+03:00", "author": { "@type": "Person", "name": "Alan Orth" @@ -345,6 +345,57 @@ $ ./add-orcid-identifiers-csv.py -i 2020-05-25-add-orcids.csv -db dspace -u dspa +

2020-05-31

+ +
$ sudo su - postgres
+$ dropdb dspacetest
+$ createdb -O dspacetest --encoding=UNICODE dspacetest
+$ psql dspacetest -c 'alter user dspacetest superuser;'
+$ pg_restore -d dspacetest -O --role=dspacetest /tmp/cgspace_2020-05-31.backup
+$ psql dspacetest -c 'alter user dspacetest nosuperuser;'
+# run DSpace 5 version of update-sequences.sql!!!
+$ psql -f /home/dspace/src/git/DSpace/dspace/etc/postgres/update-sequences.sql dspacetest
+$ psql dspacetest -c "DELETE FROM schema_version WHERE version IN ('5.8.2015.12.03.3');"
+$ psql dspacetest -c 'CREATE EXTENSION pgcrypto;'
+$ exit
+
+
$ chrt -i 0 ionice -c2 -n7 nice -n19 mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false package
+...
+[ERROR] Failed to execute goal on project additions: Could not resolve dependencies for project org.dspace.modules:additions:jar:6.3: Failed to collect dependencies at com.atmire:atmire-listings-and-reports-api:jar:6.x-2.10.8-0-SNAPSHOT: Failed to read artifact descriptor for com.atmire:atmire-listings-and-reports-api:jar:6.x-2.10.8-0-SNAPSHOT: Could not transfer artifact com.atmire:atmire-listings-and-reports-api:pom:6.x-2.10.8-0-SNAPSHOT from/to atmire.com-snapshots (https://atmire.com/artifactory/atmire.com-snapshots): Not authorized , ReasonPhrase:Unauthorized. -> [Help 1]
+
+
$ cd dspace/target/dspace-installer
+$ rm -rf /blah/dspacetest/config/spring
+$ ant update
+
+
$ export JAVA_OPTS="-Xmx1024m -Dfile.encoding=UTF-8"
+
diff --git a/docs/categories/index.html b/docs/categories/index.html index e57174879..3e6cc03c4 100644 --- a/docs/categories/index.html +++ b/docs/categories/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/notes/index.html b/docs/categories/notes/index.html index b844bcb9b..f28c8f802 100644 --- a/docs/categories/notes/index.html +++ b/docs/categories/notes/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/notes/page/2/index.html b/docs/categories/notes/page/2/index.html index b0a63ff01..3f276bf93 100644 --- a/docs/categories/notes/page/2/index.html +++ b/docs/categories/notes/page/2/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/notes/page/3/index.html b/docs/categories/notes/page/3/index.html index 37f84f5b5..d0a2ea765 100644 --- a/docs/categories/notes/page/3/index.html +++ b/docs/categories/notes/page/3/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/notes/page/4/index.html b/docs/categories/notes/page/4/index.html index 2ac87497c..4c072da04 100644 --- a/docs/categories/notes/page/4/index.html +++ b/docs/categories/notes/page/4/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/page/2/index.html b/docs/categories/page/2/index.html index 66afd475c..5e30db1b5 100644 --- a/docs/categories/page/2/index.html +++ b/docs/categories/page/2/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/page/3/index.html b/docs/categories/page/3/index.html index dea17385d..98cdc21ac 100644 --- a/docs/categories/page/3/index.html +++ b/docs/categories/page/3/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/page/4/index.html b/docs/categories/page/4/index.html index f9deb00c9..8deca296d 100644 --- a/docs/categories/page/4/index.html +++ b/docs/categories/page/4/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/page/5/index.html b/docs/categories/page/5/index.html index 048cfa8b2..603c85821 100644 --- a/docs/categories/page/5/index.html +++ b/docs/categories/page/5/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/categories/page/6/index.html b/docs/categories/page/6/index.html index aaed3ea37..8a96d11a6 100644 --- a/docs/categories/page/6/index.html +++ b/docs/categories/page/6/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/index.html b/docs/index.html index bb94ec5c3..c27c3989f 100644 --- a/docs/index.html +++ b/docs/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/page/2/index.html b/docs/page/2/index.html index 30b0e4972..842e99e56 100644 --- a/docs/page/2/index.html +++ b/docs/page/2/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/page/3/index.html b/docs/page/3/index.html index b457f7d85..9454522da 100644 --- a/docs/page/3/index.html +++ b/docs/page/3/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/page/4/index.html b/docs/page/4/index.html index c9ea028b0..378ffe6d0 100644 --- a/docs/page/4/index.html +++ b/docs/page/4/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/page/5/index.html b/docs/page/5/index.html index 7efbd7982..205bef9ad 100644 --- a/docs/page/5/index.html +++ b/docs/page/5/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/page/6/index.html b/docs/page/6/index.html index e5d4521e7..1c8905698 100644 --- a/docs/page/6/index.html +++ b/docs/page/6/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/index.html b/docs/posts/index.html index c8a1b7215..e7216d1f1 100644 --- a/docs/posts/index.html +++ b/docs/posts/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/page/2/index.html b/docs/posts/page/2/index.html index 755b51626..de6b875c1 100644 --- a/docs/posts/page/2/index.html +++ b/docs/posts/page/2/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/page/3/index.html b/docs/posts/page/3/index.html index 52203cc34..cf33be82a 100644 --- a/docs/posts/page/3/index.html +++ b/docs/posts/page/3/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/page/4/index.html b/docs/posts/page/4/index.html index dadead430..9fd294df2 100644 --- a/docs/posts/page/4/index.html +++ b/docs/posts/page/4/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/page/5/index.html b/docs/posts/page/5/index.html index f7aa91c65..1d269d755 100644 --- a/docs/posts/page/5/index.html +++ b/docs/posts/page/5/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/posts/page/6/index.html b/docs/posts/page/6/index.html index a3b347f8b..e06bcbf01 100644 --- a/docs/posts/page/6/index.html +++ b/docs/posts/page/6/index.html @@ -9,7 +9,7 @@ - + diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 3ed6d49eb..448c663bd 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -4,27 +4,27 @@ https://alanorth.github.io/cgspace-notes/categories/ - 2020-05-29T10:25:41+03:00 + 2020-05-30T18:38:16+03:00 https://alanorth.github.io/cgspace-notes/ - 2020-05-29T10:25:41+03:00 + 2020-05-30T18:38:16+03:00 https://alanorth.github.io/cgspace-notes/2020-05/ - 2020-05-29T10:25:41+03:00 + 2020-05-30T18:38:16+03:00 https://alanorth.github.io/cgspace-notes/categories/notes/ - 2020-05-29T10:25:41+03:00 + 2020-05-30T18:38:16+03:00 https://alanorth.github.io/cgspace-notes/posts/ - 2020-05-29T10:25:41+03:00 + 2020-05-30T18:38:16+03:00