mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-12-26 15:04:30 +01:00
Update notes for 2016-06-03
This commit is contained in:
parent
338415b9d9
commit
fc6e55937f
GPG Key ID:
0FB860CC9C45B1B9
@ -38,3 +38,69 @@ webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text
|
||||
- A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue
|
||||
- I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740
|
||||
- The patch applies successfully on DSpace 5.1 so I will try it later
|
||||
|
||||
## 2016-06-03
|
||||
|
||||
- Investigating the CCAFS authority issue, I exported the metadata for the Videos collection
|
||||
- The top two authors are:
|
||||
|
||||
```
|
||||
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
|
||||
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
|
||||
```
|
||||
|
||||
- So the only difference is the "confidence"
|
||||
- Ok, well THAT is interesting:
|
||||
|
||||
```
|
||||
dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
|
||||
text_value | authority | confidence
|
||||
------------+--------------------------------------+------------
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
(13 rows)
|
||||
```
|
||||
|
||||
- And now an actually relevent example:
|
||||
|
||||
```
|
||||
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
|
||||
count
|
||||
-------
|
||||
707
|
||||
(1 row)
|
||||
|
||||
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
|
||||
count
|
||||
-------
|
||||
253
|
||||
(1 row)
|
||||
```
|
||||
|
||||
- Trying something experimental:
|
||||
|
||||
```
|
||||
dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
|
||||
UPDATE 960
|
||||
```
|
||||
|
||||
- And then re-indexing authority and Discovery...?
|
||||
- After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet
|
||||
- The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:
|
||||
|
||||
```
|
||||
webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
|
||||
```
|
||||
|
||||
- That would only be for the "Browse by" function... so we'll have to see what effect that has later
|
||||
|
||||
@ -116,6 +116,79 @@ UPDATE 14
|
||||
<li>A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue</li>
|
||||
<li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
|
||||
<li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
|
||||
<li>The top two authors are:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
|
||||
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>So the only difference is the “confidence”</li>
|
||||
<li>Ok, well THAT is interesting:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
|
||||
text_value | authority | confidence
|
||||
------------+--------------------------------------+------------
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
(13 rows)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And now an actually relevent example:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
|
||||
count
|
||||
-------
|
||||
707
|
||||
(1 row)
|
||||
|
||||
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
|
||||
count
|
||||
-------
|
||||
253
|
||||
(1 row)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Trying something experimental:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
|
||||
UPDATE 960
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And then re-indexing authority and Discovery…?</li>
|
||||
<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
|
||||
<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>That would only be for the “Browse by” function… so we’ll have to see what effect that has later</li>
|
||||
</ul>
|
||||
|
||||
</section>
|
||||
|
||||
@ -99,7 +99,7 @@
|
||||
</div>
|
||||
</header>
|
||||
<div>
|
||||
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will
|
||||
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can’t handle the ‘-’ in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I’ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the “confidence” Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
|
||||
</div>
|
||||
|
||||
<footer>
|
||||
|
||||
@ -362,6 +362,79 @@ UPDATE 14
|
||||
<li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
|
||||
<li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
|
||||
<li>The top two authors are:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
|
||||
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>So the only difference is the &ldquo;confidence&rdquo;</li>
|
||||
<li>Ok, well THAT is interesting:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
|
||||
text_value | authority | confidence
|
||||
------------+--------------------------------------+------------
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
(13 rows)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And now an actually relevent example:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
|
||||
count
|
||||
-------
|
||||
707
|
||||
(1 row)
|
||||
|
||||
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
|
||||
count
|
||||
-------
|
||||
253
|
||||
(1 row)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Trying something experimental:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
|
||||
UPDATE 960
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And then re-indexing authority and Discovery&hellip;?</li>
|
||||
<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
|
||||
<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>That would only be for the &ldquo;Browse by&rdquo; function&hellip; so we&rsquo;ll have to see what effect that has later</li>
|
||||
</ul>
|
||||
</description>
|
||||
</item>
|
||||
|
||||
|
||||
@ -101,7 +101,7 @@
|
||||
</div>
|
||||
</header>
|
||||
<div>
|
||||
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will
|
||||
2016-06-01 Experimenting with IFPRI OAI (we want to harvest their publications) After reading the ContentDM documentation I found IFPRI’s OAI endpoint: http://ebrary.ifpri.org/oai/oai.php After reading the OAI documentation and testing with an OAI validator I found out how to get their publications This is their publications set: http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc You can see the others by using the OAI ListSets verb: http://ebrary.ifpri.org/oai/oai.php?verb=ListSets Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in dc.identifier.fund to cg.identifier.cpwfproject and then the rest to dc.description.sponsorship dspacetest=# update metadatavalue set metadata_field_id=130 where metadata_field_id=75 and (text_value like 'PN%' or text_value like 'PHASE%' or text_value = 'CBA' or text_value = 'IA'); UPDATE 497 dspacetest=# update metadatavalue set metadata_field_id=29 where metadata_field_id=75; UPDATE 14 Fix a few minor miscellaneous issues in dspace.cfg (#227) 2016-06-02 Testing the configuration and theme changes for the upcoming metadata migration and I found some issues with cg.coverage.admin-unit Seems that the Browse configuration in dspace.cfg can’t handle the ‘-’ in the field name: webui.browse.index.12 = subregion:metadata:cg.coverage.admin-unit:text But actually, I think since DSpace 4 or 5 (we are 5.1) the Browse indexes come from Discovery (defined in discovery.xml) so this is really just a parsing error I’ve sent a message to the DSpace mailing list to ask about the Browse index definition A user was having problems with submission and from the stacktrace it looks like a Sherpa/Romeo issue I found a thread on the mailing list talking about it and there is bug report and a patch: https://jira.duraspace.org/browse/DS-2740 The patch applies successfully on DSpace 5.1 so I will try it later 2016-06-03 Investigating the CCAFS authority issue, I exported the metadata for the Videos collection The top two authors are: CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500 CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600 So the only difference is the “confidence” Ok, well THAT is interesting: dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %'; text_value | authority | confidence ------------+--------------------------------------+------------ Orth, A.
|
||||
</div>
|
||||
|
||||
<footer>
|
||||
|
||||
@ -362,6 +362,79 @@ UPDATE 14
|
||||
<li>I found a thread on the mailing list talking about it and there is bug report and a patch: <a href="https://jira.duraspace.org/browse/DS-2740">https://jira.duraspace.org/browse/DS-2740</a></li>
|
||||
<li>The patch applies successfully on DSpace 5.1 so I will try it later</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2016-06-03:6783872e82b68b1517e00f494e6b6504">2016-06-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Investigating the CCAFS authority issue, I exported the metadata for the Videos collection</li>
|
||||
<li>The top two authors are:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::500
|
||||
CGIAR Research Program on Climate Change, Agriculture and Food Security::acd00765-02f1-4b5b-92fa-bfa3877229ce::600
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>So the only difference is the &ldquo;confidence&rdquo;</li>
|
||||
<li>Ok, well THAT is interesting:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select text_value, authority, confidence from metadatavalue where metadata_field_id=3 and text_value like '%Orth, %';
|
||||
text_value | authority | confidence
|
||||
------------+--------------------------------------+------------
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, Alan | | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | 05c2c622-d252-4efb-b9ed-95a07d3adf11 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, A. | ab606e3a-2b04-4c7d-9423-14beccf54257 | -1
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
Orth, Alan | ad281dbf-ef81-4007-96c3-a7f5d2eaa6d9 | 600
|
||||
(13 rows)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And now an actually relevent example:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence = 500;
|
||||
count
|
||||
-------
|
||||
707
|
||||
(1 row)
|
||||
|
||||
dspacetest=# select count(*) from metadatavalue where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security' and confidence != 500;
|
||||
count
|
||||
-------
|
||||
253
|
||||
(1 row)
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Trying something experimental:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspacetest=# update metadatavalue set confidence=500 where metadata_field_id=3 and text_value like 'CGIAR Research Program on Climate Change, Agriculture and Food Security';
|
||||
UPDATE 960
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>And then re-indexing authority and Discovery&hellip;?</li>
|
||||
<li>After Discovery reindex the CCAFS authors are all together in the Authors sidebar facet</li>
|
||||
<li>The docs for the ORCiD and Authority stuff for DSpace 5 mention changing the browse indexes to use the Authority as well:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>webui.browse.index.2 = author:metadataAuthority:dc.contributor.author:authority
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>That would only be for the &ldquo;Browse by&rdquo; function&hellip; so we&rsquo;ll have to see what effect that has later</li>
|
||||
</ul>
|
||||
</description>
|
||||
</item>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user