Update notes for 2016-06-03

This commit is contained in:
Alan Orth 2016-06-04 11:28:02 +03:00
parent 338415b9d9
commit fc6e55937f
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
6 changed files with 287 additions and 2 deletions

View File

@ -38,3 +38,69 @@ webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text
- A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue - A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue
- I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 - I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740
- The patch applies successfully on DSpace 5.1 so I will try it later - The patch applies successfully on DSpace 5.1 so I will try it later
## 2016-06-03
- Investigating the CCAFS authority issue, I exported the metadata for the Videos collection
- The top two authors are:
```
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
```
- So the only difference is the "confidence"
- Ok, well THAT is interesting:
```
dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
(13 rows)
```
- And now an actually relevent example:
```
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
count
-------
707
(1 row)
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
count
-------
253
(1 row)
```
- Trying something experimental:
```
dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
UPDATE 960
```
- And then re-indexing authority and Discovery...?
- After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet
- The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:
```
webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
```
- That would only be for the "Browse by" function... so we'll have to see what effect that has later

View File

@ -116,6 +116,79 @@ UPDATE 14
<li>A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue</li> <li>A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue</li>
<li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li> <li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
<li>The patch applies successfully on DSpace 5.1 so I will try it later</li> <li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
</ul>
<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
<ul>
<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
<li>The top two authors are:</li>
</ul>
<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
</code></pre>
<ul>
<li>So the only difference is the &ldquo;confidence&rdquo;</li>
<li>Ok, well THAT is interesting:</li>
</ul>
<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
(13 rows)
</code></pre>
<ul>
<li>And now an actually relevent example:</li>
</ul>
<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
count
-------
707
(1 row)
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
count
-------
253
(1 row)
</code></pre>
<ul>
<li>Trying something experimental:</li>
</ul>
<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
UPDATE 960
</code></pre>
<ul>
<li>And then re-indexing authority and Discovery&hellip;?</li>
<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
</ul>
<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
</code></pre>
<ul>
<li>That would only be for the &ldquo;Browse by&rdquo; function&hellip; so we&rsquo;ll have to see what effect that has later</li>
</ul> </ul>
</section> </section>

View File

@ -99,7 +99,7 @@
</div> </div>
</header> </header>
<div> <div>
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will 2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can&rsquo;t handle the &lsquo;-&rsquo; in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I&rsquo;ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the &ldquo;confidence&rdquo; Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
</div> </div>
<footer> <footer>

View File

@ -362,6 +362,79 @@ UPDATE 14
&lt;li&gt;I found a thread on the mailing list talking about it and there is bug report and a patch: &lt;a href=&#34;https://jira.duraspace.org/browse/DS-2740&#34;&gt;https://jira.duraspace.org/browse/DS-2740&lt;/a&gt;&lt;/li&gt; &lt;li&gt;I found a thread on the mailing list talking about it and there is bug report and a patch: &lt;a href=&#34;https://jira.duraspace.org/browse/DS-2740&#34;&gt;https://jira.duraspace.org/browse/DS-2740&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;The patch applies successfully on DSpace 5.1 so I will try it later&lt;/li&gt; &lt;li&gt;The patch applies successfully on DSpace 5.1 so I will try it later&lt;/li&gt;
&lt;/ul&gt; &lt;/ul&gt;
&lt;h2 id=&#34;2016-06-03:6783872e82b68b1517e00f494e6b6504&#34;&gt;2016-06-03&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Investigating the CCAFS authority issue, I exported the metadata for the Videos collection&lt;/li&gt;
&lt;li&gt;The top two authors are:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;So the only difference is the &amp;ldquo;confidence&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Ok, well THAT is interesting:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like &#39;%Orth, %&#39;;
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
(13 rows)
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;And now an actually relevent example:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39; and confidence = 500;
count
-------
707
(1 row)
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39; and confidence != 500;
count
-------
253
(1 row)
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;Trying something experimental:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39;;
UPDATE 960
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;And then re-indexing authority and Discovery&amp;hellip;?&lt;/li&gt;
&lt;li&gt;After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet&lt;/li&gt;
&lt;li&gt;The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;That would only be for the &amp;ldquo;Browse by&amp;rdquo; function&amp;hellip; so we&amp;rsquo;ll have to see what effect that has later&lt;/li&gt;
&lt;/ul&gt;
</description> </description>
</item> </item>

View File

@ -101,7 +101,7 @@
</div> </div>
</header> </header>
<div> <div>
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will 2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can&rsquo;t handle the &lsquo;-&rsquo; in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I&rsquo;ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the &ldquo;confidence&rdquo; Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
</div> </div>
<footer> <footer>

View File

@ -362,6 +362,79 @@ UPDATE 14
&lt;li&gt;I found a thread on the mailing list talking about it and there is bug report and a patch: &lt;a href=&#34;https://jira.duraspace.org/browse/DS-2740&#34;&gt;https://jira.duraspace.org/browse/DS-2740&lt;/a&gt;&lt;/li&gt; &lt;li&gt;I found a thread on the mailing list talking about it and there is bug report and a patch: &lt;a href=&#34;https://jira.duraspace.org/browse/DS-2740&#34;&gt;https://jira.duraspace.org/browse/DS-2740&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;The patch applies successfully on DSpace 5.1 so I will try it later&lt;/li&gt; &lt;li&gt;The patch applies successfully on DSpace 5.1 so I will try it later&lt;/li&gt;
&lt;/ul&gt; &lt;/ul&gt;
&lt;h2 id=&#34;2016-06-03:6783872e82b68b1517e00f494e6b6504&#34;&gt;2016-06-03&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Investigating the CCAFS authority issue, I exported the metadata for the Videos collection&lt;/li&gt;
&lt;li&gt;The top two authors are:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;So the only difference is the &amp;ldquo;confidence&amp;rdquo;&lt;/li&gt;
&lt;li&gt;Ok, well THAT is interesting:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like &#39;%Orth, %&#39;;
text_value | authority | confidence
------------+--------------------------------------+------------
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, Alan | | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
(13 rows)
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;And now an actually relevent example:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39; and confidence = 500;
count
-------
707
(1 row)
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39; and confidence != 500;
count
-------
253
(1 row)
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;Trying something experimental:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like &#39;CGIAR Research Program on Climate Change, Agriculture and Food Security&#39;;
UPDATE 960
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;And then re-indexing authority and Discovery&amp;hellip;?&lt;/li&gt;
&lt;li&gt;After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet&lt;/li&gt;
&lt;li&gt;The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:&lt;/li&gt;
&lt;/ul&gt;
&lt;pre&gt;&lt;code&gt;webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;That would only be for the &amp;ldquo;Browse by&amp;rdquo; function&amp;hellip; so we&amp;rsquo;ll have to see what effect that has later&lt;/li&gt;
&lt;/ul&gt;
</description> </description>
</item> </item>