Update notes for 2016-06-03

content/2016-06.md

@@ -38,3 +38,69 @@ webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text
 - A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue
 - I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740
 - The patch applies successfully on DSpace 5.1 so I will try it later
+
+## 2016-06-03
+
+- Investigating the CCAFS authority issue, I exported the metadata for the Videos collection
+- The top two authors are:
+
+```
+CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
+CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
+```
+
+- So the only difference is the "confidence"
+- Ok, well THAT is interesting:
+
+```
+dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
+ text_value |              authority               | confidence
+------------+--------------------------------------+------------
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+(13 rows)
+```
+
+- And now an actually relevent example:
+
+```
+dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
+ count
+-------
+   707
+(1 row)
+
+dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
+ count
+-------
+   253
+(1 row)
+```
+
+- Trying something experimental:
+
+```
+dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
+UPDATE 960
+```
+
+- And then re-indexing authority and Discovery...?
+- After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet
+- The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:
+
+```
+webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
+```
+
+- That would only be for the "Browse by" function... so we'll have to see what effect that has later

public/2016-06/index.html

@@ -116,6 +116,79 @@ UPDATE 14
 <li>A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue</li>
 <li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
 <li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
+</ul>
+
+<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
+
+<ul>
+<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
+<li>The top two authors are:</li>
+</ul>
+
+<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
+CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
+</code></pre>
+
+<ul>
+<li>So the only difference is the &ldquo;confidence&rdquo;</li>
+<li>Ok, well THAT is interesting:</li>
+</ul>
+
+<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
+ text_value |              authority               | confidence
+------------+--------------------------------------+------------
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+(13 rows)
+</code></pre>
+
+<ul>
+<li>And now an actually relevent example:</li>
+</ul>
+
+<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
+ count
+-------
+   707
+(1 row)
+
+dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
+ count
+-------
+   253
+(1 row)
+</code></pre>
+
+<ul>
+<li>Trying something experimental:</li>
+</ul>
+
+<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
+UPDATE 960
+</code></pre>
+
+<ul>
+<li>And then re-indexing authority and Discovery&hellip;?</li>
+<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
+<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
+</ul>
+
+<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
+</code></pre>
+
+<ul>
+<li>That would only be for the &ldquo;Browse by&rdquo; function&hellip; so we&rsquo;ll have to see what effect that has later</li>
 </ul>
 
   </section>

public/index.html

@@ -99,7 +99,7 @@
     </div>
   </header>
   <div>
-    2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will
+    2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can&rsquo;t handle the &lsquo;-&rsquo; in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I&rsquo;ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the &ldquo;confidence&rdquo; Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
   </div>
   
   <footer>

public/index.xml

@@ -362,6 +362,79 @@ UPDATE 14
 <li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
 <li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
 </ul>
+
+<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
+
+<ul>
+<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
+<li>The top two authors are:</li>
+</ul>
+
+<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
+CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
+</code></pre>
+
+<ul>
+<li>So the only difference is the “confidence”</li>
+<li>Ok, well THAT is interesting:</li>
+</ul>
+
+<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
+ text_value |              authority               | confidence
+------------+--------------------------------------+------------
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+(13 rows)
+</code></pre>
+
+<ul>
+<li>And now an actually relevent example:</li>
+</ul>
+
+<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
+ count
+-------
+   707
+(1 row)
+
+dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
+ count
+-------
+   253
+(1 row)
+</code></pre>
+
+<ul>
+<li>Trying something experimental:</li>
+</ul>
+
+<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
+UPDATE 960
+</code></pre>
+
+<ul>
+<li>And then re-indexing authority and Discovery…?</li>
+<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
+<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
+</ul>
+
+<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
+</code></pre>
+
+<ul>
+<li>That would only be for the “Browse by” function… so we’ll have to see what effect that has later</li>
+</ul>
 </description>
     </item>
     

public/tags/notes/index.html

@@ -101,7 +101,7 @@
     </div>
   </header>
   <div>
-    2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will
+    2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI&rsquo;s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&amp;from=2016-01-01&amp;set=p15738coll2&amp;metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can&rsquo;t handle the &lsquo;-&rsquo; in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I&rsquo;ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the &ldquo;confidence&rdquo; Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
   </div>
   
   <footer>

public/tags/notes/index.xml

@@ -362,6 +362,79 @@ UPDATE 14
 <li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
 <li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
 </ul>
+
+<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
+
+<ul>
+<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
+<li>The top two authors are:</li>
+</ul>
+
+<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
+CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
+</code></pre>
+
+<ul>
+<li>So the only difference is the “confidence”</li>
+<li>Ok, well THAT is interesting:</li>
+</ul>
+
+<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
+ text_value |              authority               | confidence
+------------+--------------------------------------+------------
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, Alan |                                      |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | 05c2c622-d252-4efb-b9ed-95a07d3adf11 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, A.   | ab606e3a-2b04-4c7d-9423-14beccf54257 |         -1
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+ Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 |        600
+(13 rows)
+</code></pre>
+
+<ul>
+<li>And now an actually relevent example:</li>
+</ul>
+
+<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
+ count
+-------
+   707
+(1 row)
+
+dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
+ count
+-------
+   253
+(1 row)
+</code></pre>
+
+<ul>
+<li>Trying something experimental:</li>
+</ul>
+
+<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
+UPDATE 960
+</code></pre>
+
+<ul>
+<li>And then re-indexing authority and Discovery…?</li>
+<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
+<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
+</ul>
+
+<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
+</code></pre>
+
+<ul>
+<li>That would only be for the “Browse by” function… so we’ll have to see what effect that has later</li>
+</ul>
 </description>
     </item>