alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2024-11-22 14:45:03 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2017-09-14

Browse Source
This commit is contained in:
Alan Orth 2017-09-14 17:59:44 +03:00
parent 3eae0388dd
commit 7f6bf50d5d
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
34 changed files with 155 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
content/post/2017-09.md
View File

@ -330,3 +330,10 @@ dspace=# select distinct text_value, authority, confidence from metadatavalue wh
```
- It added *another* authority... surely this is not the desired behavior, or maybe we are not using this as intented?
## 2017-09-14
- Communicate with Handle.net admins to try to get some guidance about the 10947 prefix
- CGSpace was very slow and Uptime Robot even said it was down at one time
- I didn't see any abnormally high usage in the REST or OAI logs, but looking at Munin I see the average JVM usage was at 4.9GB and the heap is only 5GB (5120M), so I think it's just normal growing pains
- Every few months I generally try to increase the JVM heap to be 512M higher than the average usage reported by Munin, so now I adjusted it to 5632M

2
public/2015-11/index.html
View File

@ -51,7 +51,7 @@ $ psql -c 'SELECT * from pg_stat_activity;' | grep idle | grep -c cgspac
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2015-12/index.html
View File

@ -53,7 +53,7 @@ Replace lzop with xz in log compression cron jobs on DSpace Test—it uses less
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-01/index.html
View File

@ -43,7 +43,7 @@ Update GitHub wiki for documentation of maintenance tasks.
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-02/index.html
View File

@ -57,7 +57,7 @@ Also, lots of things like &ldquo;COTE D`LVOIRE&rdquo; and &ldquo;COTE D IVOIRE&r
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-03/index.html
View File

@ -43,7 +43,7 @@ Reinstall my local (Mac OS X) DSpace stack with Tomcat 7, PostgreSQL 9.3, and Ja
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-04/index.html
View File

@ -47,7 +47,7 @@ Also, I noticed the checker log has some errors we should pay attention to:
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-05/index.html
View File

@ -51,7 +51,7 @@ There are 3,000 IPs accessing the REST API in a 24-hour period!
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-06/index.html
View File

@ -49,7 +49,7 @@ Working on second phase of metadata migration, looks like this will work for mov
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-07/index.html
View File

@ -65,7 +65,7 @@ In this case the select query was showing 95 results before the update
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-08/index.html
View File

@ -59,7 +59,7 @@ $ git rebase -i dspace-5.5
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-09/index.html
View File

@ -51,7 +51,7 @@ $ ldapsearch -x -H ldaps://svcgroot2.cgiarad.org:3269/ -b &quot;dc=cgiarad,dc=or
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-10/index.html
View File

@ -59,7 +59,7 @@ I exported a random item&rsquo;s metadata as CSV, deleted all columns except id
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-11/index.html
View File

@ -43,7 +43,7 @@ Add dc.type to the output options for Atmire&rsquo;s Listings and Reports module
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2016-12/index.html
View File

@ -67,7 +67,7 @@ Another worrying error from dspace.log is:
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-01/index.html
View File

@ -43,7 +43,7 @@ I asked on the dspace-tech mailing list because it seems to be broken, and actua
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-02/index.html
View File

@ -71,7 +71,7 @@ Looks like we&rsquo;ll be using cg.identifier.ccafsprojectpii as the field name
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-03/index.html
View File

@ -75,7 +75,7 @@ $ identify ~/Desktop/alc_contrastes_desafios.jpg
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-04/index.html
View File

@ -61,7 +61,7 @@ $ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p &quot;ImageMagick PDF Th
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-05/index.html
View File

@ -27,7 +27,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="May, 2017"/>
<meta name="twitter:description" content="2017-05-01 ICARDA apparently started working on CG Core on their MEL repository They have done a few cg.* fields, but not very consistent and even copy some of CGSpace items: https://mel.cgiar.org/xmlui/handle/20.500.11766/6911?show=full https://cgspace.cgiar.org/handle/10568/73683 2017-05-02 Atmire got back about the Workflow Statistics issue, and apparently it&rsquo;s a bug in the CUA module so they will send us a pull request 2017-05-04 Sync DSpace Test with database and assetstore from CGSpace Re-deploy DSpace Test with Atmire&rsquo;s CUA patch for workflow statistics, run system updates, and restart the server Now I can see the workflow statistics and am able to select users, but everything returns 0 items Megan says there are still some mapped items are not appearing since last week, so I forced a full index-discovery -b Need to remember to check if the collection has more items (currently 39 on CGSpace, but 118 on the freshly reindexed DSPace Test) tomorrow: https://cgspace."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-06/index.html
View File

@ -27,7 +27,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="June, 2017"/>
<meta name="twitter:description" content="2017-06-01 After discussion with WLE and CGSpace content people, we decided to just add one metadata field for the WLE Research Themes The cg.identifier.wletheme field will be used for both Phase I and Phase II Research Themes Then we&rsquo;ll create a new sub-community for Phase II and create collections for the research themes there The current &ldquo;Research Themes&rdquo; community will be renamed to &ldquo;WLE Phase I Research Themes&rdquo; Tagged all items in the current Phase I collections with their appropriate themes Create pull request to add Phase II research themes to the submission form: #328 Add cg."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-07/index.html
View File

@ -55,7 +55,7 @@ We can use PostgreSQL&rsquo;s extended output format (-x) plus sed to format the
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/2017-08/index.html
View File

@ -75,7 +75,7 @@ Then I cleaned up the author authorities and HTML characters in OpenRefine and s
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

116
public/2017-09/index.html
View File

@ -25,7 +25,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne&rsquo;s user account
<meta property="article:published_time" content="2017-09-07T16:54:52&#43;07:00"/>
<meta property="article:modified_time" content="2017-09-13T16:42:17&#43;03:00"/>
<meta property="article:modified_time" content="2017-09-13T17:34:55&#43;03:00"/>
@ -51,7 +51,7 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne&rsquo;s user account
"/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />
@ -61,9 +61,9 @@ Ask Sisay to clean up the WLE approvers a bit, as Marianne&rsquo;s user account
"@type": "BlogPosting",
"headline": "September, 2017",
"url": "https://alanorth.github.io/cgspace-notes/2017-09/",
"wordCount": "1691",
"wordCount": "2332",
"datePublished": "2017-09-07T16:54:52&#43;07:00",
"dateModified": "2017-09-13T16:42:17&#43;03:00",
"dateModified": "2017-09-13T17:34:55&#43;03:00",
"author": {
"@type": "Person",
"name": "Alan Orth"
@ -386,6 +386,114 @@ delete from metadatavalue where resource_type_id=2 and metadata_field_id in (134
<li>I merged the CGIAR Library theme changes (<a href="https://github.com/ilri/DSpace/pull/338">#338</a>) to the <code>5_x-prod</code> branch in preparation for next week&rsquo;s migration</li>
<li>I emailed the Handle administrators (hdladmin@cnri.reston.va.us) to ask them what the process for changing their prefix to be resolved by our resolver</li>
<li>They responded and said that they need email confirmation from the contact of record of the other prefix, so I should have the CGIAR System Organization people email them before I send the new <code>sitebndl.zip</code></li>
<li>Testing to see how we end up with all these new authorities after we keep cleaning and merging them in the database</li>
<li>Here are all my distinct authority combinations in the database before:</li>
</ul>
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | -1
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 600
Orth, A. | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | -1
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 0
Orth, Alan | 0d575fa3-8ac4-4763-a90a-1248d4791793 | -1
Orth, Alan | 67a9588f-d86a-4155-81a2-af457e9d13f9 | 600
(8 rows)
</code></pre>
<ul>
<li>And then after adding a new item and selecting an existing &ldquo;Orth, Alan&rdquo; with an ORCID in the author lookup:</li>
</ul>
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | -1
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 600
Orth, A. | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | -1
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 0
Orth, Alan | cb3aa5ae-906f-4902-97b1-2667cf148dde | 600
Orth, Alan | 0d575fa3-8ac4-4763-a90a-1248d4791793 | -1
Orth, Alan | 67a9588f-d86a-4155-81a2-af457e9d13f9 | 600
(9 rows)
</code></pre>
<ul>
<li>It created a new authority&hellip; let&rsquo;s try to add another item and select the same existing author and see what happens in the database:</li>
</ul>
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | -1
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 600
Orth, A. | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | -1
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 0
Orth, Alan | cb3aa5ae-906f-4902-97b1-2667cf148dde | 600
Orth, Alan | 0d575fa3-8ac4-4763-a90a-1248d4791793 | -1
Orth, Alan | 67a9588f-d86a-4155-81a2-af457e9d13f9 | 600
(9 rows)
</code></pre>
<ul>
<li>No new one&hellip; so now let me try to add another item and select the italicized result from the ORCID lookup and see what happens in the database:</li>
</ul>
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | -1
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | d85a8a5b-9b82-4aaf-8033-d7e0c7d9cb8f | 600
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 600
Orth, A. | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | -1
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 0
Orth, Alan | cb3aa5ae-906f-4902-97b1-2667cf148dde | 600
Orth, Alan | 0d575fa3-8ac4-4763-a90a-1248d4791793 | -1
Orth, Alan | 67a9588f-d86a-4155-81a2-af457e9d13f9 | 600
(10 rows)
</code></pre>
<ul>
<li>Shit, it created another authority! Let&rsquo;s try it again!</li>
</ul>
<pre><code>dspace=# select distinct text_value, authority, confidence from metadatavalue where resource_type_id=2 and metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | -1
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | d85a8a5b-9b82-4aaf-8033-d7e0c7d9cb8f | 600
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 600
Orth, Alan | 9aed566a-a248-4878-9577-0caedada43db | 600
Orth, A. | 1a1943a0-3f87-402f-9afe-e52fb46a513e | 600
Orth, Alan | 1a1943a0-3f87-402f-9afe-e52fb46a513e | -1
Orth, Alan | 7c2bffb8-58c9-4bc8-b102-ebe8aec200ad | 0
Orth, Alan | cb3aa5ae-906f-4902-97b1-2667cf148dde | 600
Orth, Alan | 0d575fa3-8ac4-4763-a90a-1248d4791793 | -1
Orth, Alan | 67a9588f-d86a-4155-81a2-af457e9d13f9 | 600
(11 rows)
</code></pre>
<ul>
<li>It added <em>another</em> authority&hellip; surely this is not the desired behavior, or maybe we are not using this as intented?</li>
</ul>
<h2 id="2017-09-14">2017-09-14</h2>
<ul>
<li>Communicate with Handle.net admins to try to get some guidance about the 10947 prefix</li>
<li>CGSpace was very slow and Uptime Robot even said it was down at one time</li>
<li>I didn&rsquo;t see any abnormally high usage in the REST or OAI logs, but looking at Munin I see the average JVM usage was at 4.9GB and the heap is only 5GB (5120M), so I think it&rsquo;s just normal growing pains</li>
<li>Every few months I generally try to increase the JVM heap to be 512M higher than the average usage reported by Munin, so now I adjusted it to 5632M</li>
</ul>

2
public/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="CGSpace Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/page/2/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="CGSpace Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/page/3/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="CGSpace Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/post/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Posts"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/post/page/2/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Posts"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/post/page/3/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Posts"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

10
public/sitemap.xml
View File

@ -4,7 +4,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/2017-09/</loc>
<lastmod>2017-09-13T16:42:17+03:00</lastmod>
<lastmod>2017-09-13T17:34:55+03:00</lastmod>
</url>
<url>
@ -119,7 +119,7 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/</loc>
<lastmod>2017-09-13T16:42:17+03:00</lastmod>
<lastmod>2017-09-13T17:34:55+03:00</lastmod>
<priority>0</priority>
</url>
@ -130,19 +130,19 @@
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
<lastmod>2017-09-13T16:42:17+03:00</lastmod>
<lastmod>2017-09-13T17:34:55+03:00</lastmod>
<priority>0</priority>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/post/</loc>
<lastmod>2017-09-13T16:42:17+03:00</lastmod>
<lastmod>2017-09-13T17:34:55+03:00</lastmod>
<priority>0</priority>
</url>
<url>
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
<lastmod>2017-09-13T16:42:17+03:00</lastmod>
<lastmod>2017-09-13T17:34:55+03:00</lastmod>
<priority>0</priority>
</url>

2
public/tags/notes/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/tags/notes/page/2/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />

2
public/tags/notes/page/3/index.html
View File

@ -25,7 +25,7 @@
<meta name="twitter:card" content="summary"/><meta name="twitter:title" content="Notes"/>
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
<meta name="generator" content="Hugo 0.27" />
<meta name="generator" content="Hugo 0.27.1" />