mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-22 06:35:03 +01:00
Add notes for 2020-11-15
This commit is contained in:
parent
61474b6663
commit
ad60e0525e
GPG Key ID:
0FB860CC9C45B1B9
@ -127,6 +127,8 @@ dspace=# COMMIT;
|
||||
- I did some tests to add a usage statistics chart to the item views on DSpace Test
|
||||
- It is inspired by Salem's work on WorldFish's repository, and it hits the dspace-statistics-api for the current item and displays a graph
|
||||
- I got it working very easily for all-time statistics with Chart.js, but I think I will need to use Highcharts or something else because Chart.js is HTML5 canvas and doesn't allow theming via CSS (so our Bootstrap brand colors for each theme won't work)
|
||||
- Hmm, Highcharts is not licensed under and open source license so I will not use it
|
||||
- Perhaps I'll use Chartist with the popover plugin...
|
||||
- I think I'll pursue this after the DSpace 6 upgrade...
|
||||
|
||||
## 2020-11-12
|
||||
@ -155,4 +157,53 @@ facet=true&facet.field=owningComm&facet.mincount=1&facet.limit=100&facet.offset=
|
||||
- I hadn't seen monit before, but the others are already in DSpace's spider agents lists for some time so probably only appear in older stats cores
|
||||
- The issue with purging these using `check-spider-hits.sh` is that it can't do case-insensitive regexes and some metacharacters like `\s` don't work so I added case-sensitive patterns to a local agents file and purged them with the script
|
||||
|
||||
## 2020-11-15
|
||||
|
||||
- Upgrade CGSpace to DSpace 6.3
|
||||
- First build, update, and migrate the database:
|
||||
|
||||
```
|
||||
$ dspace cleanup -v
|
||||
$ git checkout origin/6_x-dev-atmire-modules
|
||||
$ npm install -g yarn
|
||||
$ chrt -b 0 mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false -P \!dspace-lni,\!dspace-rdf,\!dspace-sword,\!dspace-swordv2,\!dspace-jspui clean package
|
||||
$ sudo su - postgres
|
||||
$ psql dspace -c 'CREATE EXTENSION pgcrypto;'
|
||||
$ psql dspace -c "DELETE FROM schema_version WHERE version IN ('5.8.2015.12.03.3');"
|
||||
$ exit
|
||||
$ rm -rf /home/cgspace/config/spring
|
||||
$ ant update
|
||||
$ dspace database info
|
||||
$ dspace database migrate
|
||||
$ sudo systemctl start tomcat7
|
||||
```
|
||||
|
||||
- After starting Tomcat DSpace should start up OK and begin Discovery indexing, but I want to also upgrade from PostgreSQL 9.6 to 10
|
||||
- I installed and configured PostgreSQL 10 using the Ansible playbooks, then migrated the database manually:
|
||||
|
||||
```
|
||||
# systemctl stop tomcat7
|
||||
# pg_ctlcluster 9.6 main stop
|
||||
# tar -cvzpf var-lib-postgresql-9.6.tar.gz /var/lib/postgresql/9.6
|
||||
# tar -cvzpf etc-postgresql-9.6.tar.gz /etc/postgresql/9.6
|
||||
# pg_ctlcluster 10 main stop
|
||||
# pg_dropcluster 10 main
|
||||
# pg_upgradecluster 9.6 main
|
||||
# pg_dropcluster 9.6 main
|
||||
# systemctl start postgresql
|
||||
# dpkg -l | grep postgresql | grep 9.6 | awk '{print $2}' | xargs dpkg -r
|
||||
```
|
||||
|
||||
- Then I ran all system updates and rebooted the server...
|
||||
- After the server came back up I re-ran the Ansible playbook to make sure all configs and services were updated
|
||||
- I disabled the dspace-statistsics-api for now because it won't work until I migrate all the Solr statistics anyways
|
||||
- Start a full Discovery re-indexing:
|
||||
|
||||
```
|
||||
$ time chrt -b 0 ionice -c2 -n7 nice -n19 dspace index-discovery -b
|
||||
```
|
||||
|
||||
- I updated all the Ansible infrastructure and DSpace branches to be the DSpace 6 ones
|
||||
- I will wait until the Discovery indexing is finished to start doing the Solr statistics migration
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -17,7 +17,7 @@ So far we’ve spent at least fifty hours to process the statistics and stat
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-11/" />
|
||||
<meta property="article:published_time" content="2020-11-01T13:11:54+02:00" />
|
||||
<meta property="article:modified_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="article:modified_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="November, 2020"/>
|
||||
@ -39,9 +39,9 @@ So far we’ve spent at least fifty hours to process the statistics and stat
|
||||
"@type": "BlogPosting",
|
||||
"headline": "November, 2020",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2020-11/",
|
||||
"wordCount": "1266",
|
||||
"wordCount": "1565",
|
||||
"datePublished": "2020-11-01T13:11:54+02:00",
|
||||
"dateModified": "2020-11-10T17:00:02+02:00",
|
||||
"dateModified": "2020-11-13T09:22:18+02:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -257,7 +257,12 @@ dspace=# COMMIT;
|
||||
<li>I did some tests to add a usage statistics chart to the item views on DSpace Test
|
||||
<ul>
|
||||
<li>It is inspired by Salem’s work on WorldFish’s repository, and it hits the dspace-statistics-api for the current item and displays a graph</li>
|
||||
<li>I got it working very easily for all-time statistics with Chart.js, but I think I will need to use Highcharts or something else because Chart.js is HTML5 canvas and doesn’t allow theming via CSS (so our Bootstrap brand colors for each theme won’t work)</li>
|
||||
<li>I got it working very easily for all-time statistics with Chart.js, but I think I will need to use Highcharts or something else because Chart.js is HTML5 canvas and doesn’t allow theming via CSS (so our Bootstrap brand colors for each theme won’t work)
|
||||
<ul>
|
||||
<li>Hmm, Highcharts is not licensed under and open source license so I will not use it</li>
|
||||
<li>Perhaps I’ll use Chartist with the popover plugin…</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>I think I’ll pursue this after the DSpace 6 upgrade…</li>
|
||||
</ul>
|
||||
</li>
|
||||
@ -293,6 +298,55 @@ dspace=# COMMIT;
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<h2 id="2020-11-15">2020-11-15</h2>
|
||||
<ul>
|
||||
<li>Upgrade CGSpace to DSpace 6.3
|
||||
<ul>
|
||||
<li>First build, update, and migrate the database:</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<pre><code>$ dspace cleanup -v
|
||||
$ git checkout origin/6_x-dev-atmire-modules
|
||||
$ npm install -g yarn
|
||||
$ chrt -b 0 mvn -U -Dmirage2.on=true -Dmirage2.deps.included=false -P \!dspace-lni,\!dspace-rdf,\!dspace-sword,\!dspace-swordv2,\!dspace-jspui clean package
|
||||
$ sudo su - postgres
|
||||
$ psql dspace -c 'CREATE EXTENSION pgcrypto;'
|
||||
$ psql dspace -c "DELETE FROM schema_version WHERE version IN ('5.8.2015.12.03.3');"
|
||||
$ exit
|
||||
$ rm -rf /home/cgspace/config/spring
|
||||
$ ant update
|
||||
$ dspace database info
|
||||
$ dspace database migrate
|
||||
$ sudo systemctl start tomcat7
|
||||
</code></pre><ul>
|
||||
<li>After starting Tomcat DSpace should start up OK and begin Discovery indexing, but I want to also upgrade from PostgreSQL 9.6 to 10
|
||||
<ul>
|
||||
<li>I installed and configured PostgreSQL 10 using the Ansible playbooks, then migrated the database manually:</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<pre><code># systemctl stop tomcat7
|
||||
# pg_ctlcluster 9.6 main stop
|
||||
# tar -cvzpf var-lib-postgresql-9.6.tar.gz /var/lib/postgresql/9.6
|
||||
# tar -cvzpf etc-postgresql-9.6.tar.gz /etc/postgresql/9.6
|
||||
# pg_ctlcluster 10 main stop
|
||||
# pg_dropcluster 10 main
|
||||
# pg_upgradecluster 9.6 main
|
||||
# pg_dropcluster 9.6 main
|
||||
# systemctl start postgresql
|
||||
# dpkg -l | grep postgresql | grep 9.6 | awk '{print $2}' | xargs dpkg -r
|
||||
</code></pre><ul>
|
||||
<li>Then I ran all system updates and rebooted the server…</li>
|
||||
<li>After the server came back up I re-ran the Ansible playbook to make sure all configs and services were updated</li>
|
||||
<li>I disabled the dspace-statistsics-api for now because it won’t work until I migrate all the Solr statistics anyways</li>
|
||||
<li>Start a full Discovery re-indexing:</li>
|
||||
</ul>
|
||||
<pre><code>$ time chrt -b 0 ionice -c2 -n7 nice -n19 dspace index-discovery -b
|
||||
</code></pre><ul>
|
||||
<li>I updated all the Ansible infrastructure and DSpace branches to be the DSpace 6 ones</li>
|
||||
<li>I will wait until the Discovery indexing is finished to start doing the Solr statistics migration</li>
|
||||
</ul>
|
||||
<!-- raw HTML omitted -->
|
||||
|
||||
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Categories"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-11-10T17:00:02+02:00" />
|
||||
<meta property="og:updated_time" content="2020-11-13T09:22:18+02:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -4,27 +4,27 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
|
||||
<lastmod>2020-11-10T17:00:02+02:00</lastmod>
|
||||
<lastmod>2020-11-13T09:22:18+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2020-11-10T17:00:02+02:00</lastmod>
|
||||
<lastmod>2020-11-13T09:22:18+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
|
||||
<lastmod>2020-11-10T17:00:02+02:00</lastmod>
|
||||
<lastmod>2020-11-13T09:22:18+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2020-11/</loc>
|
||||
<lastmod>2020-11-10T17:00:02+02:00</lastmod>
|
||||
<lastmod>2020-11-13T09:22:18+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||
<lastmod>2020-11-10T17:00:02+02:00</lastmod>
|
||||
<lastmod>2020-11-13T09:22:18+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user