mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2025-01-27 05:49:12 +01:00
Add notes for 2022-03-04
This commit is contained in:
@ -48,7 +48,7 @@ I filed a bug on OpenRXV: https://github.com/ilri/OpenRXV/issues/39
|
||||
|
||||
I filed an issue on OpenRXV to make some minor edits to the admin UI: https://github.com/ilri/OpenRXV/issues/40
|
||||
"/>
|
||||
<meta name="generator" content="Hugo 0.92.2" />
|
||||
<meta name="generator" content="Hugo 0.93.1" />
|
||||
|
||||
|
||||
|
||||
@ -173,7 +173,7 @@ $ grep -c added /tmp/2020-09-02-countrycodetagger.log
|
||||
</code></pre><ul>
|
||||
<li>I tried to query LDAP directly using the application credentials with ldapsearch and it works:</li>
|
||||
</ul>
|
||||
<pre tabindex="0"><code>$ ldapsearch -x -H ldaps://AZCGNEROOT2.CGIARAD.ORG:636/ -b "dc=cgiarad,dc=org" -D "applicationaccount@cgiarad.org" -W "(sAMAccountName=me)"
|
||||
<pre tabindex="0"><code>$ ldapsearch -x -H ldaps://AZCGNEROOT2.CGIARAD.ORG:636/ -b "dc=cgiarad,dc=org" -D "applicationaccount@cgiarad.org" -W "(sAMAccountName=me)"
|
||||
</code></pre><ul>
|
||||
<li>According to the <a href="https://wiki.lyrasis.org/display/DSDOC6x/Authentication+Plugins#AuthenticationPlugins-LDAPAuthentication">DSpace 6 docs</a> we need to escape commas in our LDAP parameters due to the new configuration system
|
||||
<ul>
|
||||
@ -206,8 +206,8 @@ Report
|
||||
Formally Published
|
||||
Poster
|
||||
Unrefereed reprint
|
||||
$ ./delete-metadata-values.py -i 2020-09-03-delete-review-status.csv -db dspace -u dspace -p 'fuuu' -f dc.description.version -m 68
|
||||
$ ./fix-metadata-values.py -i 2020-09-03-fix-review-status.csv -db dspace -u dspace -p 'fuuu' -f dc.description.version -t 'correct' -m 68
|
||||
$ ./delete-metadata-values.py -i 2020-09-03-delete-review-status.csv -db dspace -u dspace -p 'fuuu' -f dc.description.version -m 68
|
||||
$ ./fix-metadata-values.py -i 2020-09-03-fix-review-status.csv -db dspace -u dspace -p 'fuuu' -f dc.description.version -t 'correct' -m 68
|
||||
</code></pre><ul>
|
||||
<li>Start reviewing 95 items for IITA (20201stbatch)
|
||||
<ul>
|
||||
@ -259,9 +259,9 @@ java.lang.NullPointerException
|
||||
</li>
|
||||
<li>I will update our nearly 6,000 metadata values for CIFOR in the database accordingly:</li>
|
||||
</ul>
|
||||
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^(http://)?www\.cifor\.org/(nc/)?online-library/browse/view-publication/publication/([[:digit:]]+)\.html$', 'https://www.cifor.org/knowledge/publication/\3') WHERE metadata_field_id=219 AND text_value ~ 'www\.cifor\.org/(nc/)?online-library/browse/view-publication/publication/[[:digit:]]+';
|
||||
dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^https?://www\.cifor\.org/library/([[:digit:]]+)/?$', 'https://www.cifor.org/knowledge/publication/\1') WHERE metadata_field_id=219 AND text_value ~ 'https?://www\.cifor\.org/library/[[:digit:]]+/?';
|
||||
dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^https?://www\.cifor\.org/pid/([[:digit:]]+)/?$', 'https://www.cifor.org/knowledge/publication/\1') WHERE metadata_field_id=219 AND text_value ~ 'https?://www\.cifor\.org/pid/[[:digit:]]+';
|
||||
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^(http://)?www\.cifor\.org/(nc/)?online-library/browse/view-publication/publication/([[:digit:]]+)\.html$', 'https://www.cifor.org/knowledge/publication/\3') WHERE metadata_field_id=219 AND text_value ~ 'www\.cifor\.org/(nc/)?online-library/browse/view-publication/publication/[[:digit:]]+';
|
||||
dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^https?://www\.cifor\.org/library/([[:digit:]]+)/?$', 'https://www.cifor.org/knowledge/publication/\1') WHERE metadata_field_id=219 AND text_value ~ 'https?://www\.cifor\.org/library/[[:digit:]]+/?';
|
||||
dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, '^https?://www\.cifor\.org/pid/([[:digit:]]+)/?$', 'https://www.cifor.org/knowledge/publication/\1') WHERE metadata_field_id=219 AND text_value ~ 'https?://www\.cifor\.org/pid/[[:digit:]]+';
|
||||
</code></pre><ul>
|
||||
<li>I did some cleanup on the author affiliations of the IITA data our 2019-04 list using reconcile-csv and OpenRefine:
|
||||
<ul>
|
||||
@ -328,7 +328,7 @@ AFRICA SOUTH OF SAHARA,SUB-SAHARAN AFRICA
|
||||
NORTH AFRICA,NORTHERN AFRICA
|
||||
WEST ASIA,WESTERN ASIA
|
||||
SOUTHWEST ASIA,SOUTHWESTERN ASIA
|
||||
$ ./fix-metadata-values.py -i 2020-09-10-fix-cgspace-regions.csv -db dspace -u dspace -p 'fuuu' -f cg.coverage.region -t 'correct' -m 227 -d -n
|
||||
$ ./fix-metadata-values.py -i 2020-09-10-fix-cgspace-regions.csv -db dspace -u dspace -p 'fuuu' -f cg.coverage.region -t 'correct' -m 227 -d -n
|
||||
Connected to database.
|
||||
Would fix 12227 occurences of: EAST AFRICA
|
||||
Would fix 7996 occurences of: WEST AFRICA
|
||||
@ -417,7 +417,7 @@ Would fix 3 occurences of: SOUTHWEST ASIA
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<pre tabindex="0"><code>value + "__description:" + cells["dc.type"].value
|
||||
<pre tabindex="0"><code>value + "__description:" + cells["dc.type"].value
|
||||
</code></pre><ul>
|
||||
<li>Then I created a SAF bundle with SAFBuilder:</li>
|
||||
</ul>
|
||||
@ -477,9 +477,9 @@ Would fix 3 occurences of: SOUTHWEST ASIA
|
||||
</ul>
|
||||
<pre tabindex="0"><code>$ cat 2020-09-17-add-bioversity-orcids.csv
|
||||
dc.contributor.author,cg.creator.id
|
||||
"Etten, Jacob van","Jacob van Etten: 0000-0001-7554-2558"
|
||||
"van Etten, Jacob","Jacob van Etten: 0000-0001-7554-2558"
|
||||
$ ./add-orcid-identifiers-csv.py -i 2020-09-17-add-bioversity-orcids.csv -db dspace -u dspace -p 'dom@in34sniper'
|
||||
"Etten, Jacob van","Jacob van Etten: 0000-0001-7554-2558"
|
||||
"van Etten, Jacob","Jacob van Etten: 0000-0001-7554-2558"
|
||||
$ ./add-orcid-identifiers-csv.py -i 2020-09-17-add-bioversity-orcids.csv -db dspace -u dspace -p 'dom@in34sniper'
|
||||
</code></pre><ul>
|
||||
<li>I sent a follow-up message to Atmire to look into the two remaining issues with the DSpace 6 upgrade
|
||||
<ul>
|
||||
@ -496,7 +496,7 @@ $ ./add-orcid-identifiers-csv.py -i 2020-09-17-add-bioversity-orcids.csv -db dsp
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<pre tabindex="0"><code>https://cgspace.cgiar.org/open-search/discover?query=type:"Journal Article" AND status:"Open Access" AND crpsubject:"Water, Land and Ecosystems" AND "tradeoffs"&rpp=100
|
||||
<pre tabindex="0"><code>https://cgspace.cgiar.org/open-search/discover?query=type:"Journal Article" AND status:"Open Access" AND crpsubject:"Water, Land and Ecosystems" AND "tradeoffs"&rpp=100
|
||||
</code></pre><ul>
|
||||
<li>I noticed that my <code>move-collections.sh</code> script didn’t work on DSpace 6 because of the change from IDs to UUIDs, so I modified it to quote the collection <code>resource_id</code> parameters in the PostgreSQL query</li>
|
||||
</ul>
|
||||
@ -538,7 +538,7 @@ dspacestatistics=# SELECT SUM(downloads) FROM items;
|
||||
</ul>
|
||||
<pre tabindex="0"><code>dspace=# BEGIN;
|
||||
BEGIN
|
||||
dspace=# DELETE FROM metadatavalue WHERE text_value='Report' AND resource_type_id=2 AND metadata_field_id=68;
|
||||
dspace=# DELETE FROM metadatavalue WHERE text_value='Report' AND resource_type_id=2 AND metadata_field_id=68;
|
||||
DELETE 12
|
||||
dspace=# COMMIT;
|
||||
</code></pre><ul>
|
||||
@ -573,23 +573,23 @@ dspace=# COMMIT;
|
||||
</li>
|
||||
</ul>
|
||||
<pre tabindex="0"><code>...
|
||||
item_ids = ['0079470a-87a1-4373-beb1-b16e3f0c4d81', '007a9df1-0871-4612-8b28-5335982198cb']
|
||||
item_ids_str = ' OR '.join(item_ids).replace('-', '\-')
|
||||
item_ids = ['0079470a-87a1-4373-beb1-b16e3f0c4d81', '007a9df1-0871-4612-8b28-5335982198cb']
|
||||
item_ids_str = ' OR '.join(item_ids).replace('-', '\-')
|
||||
...
|
||||
solr_query_params = {
|
||||
"q": f"id:({item_ids_str})",
|
||||
"fq": "type:2 AND isBot:false AND statistics_type:view AND time:[2020-01-01T00:00:00Z TO 2020-09-02T00:00:00Z]",
|
||||
"facet": "true",
|
||||
"facet.field": "id",
|
||||
"facet.mincount": 1,
|
||||
"facet.limit": 1,
|
||||
"facet.offset": 0,
|
||||
"stats": "true",
|
||||
"stats.field": "id",
|
||||
"stats.calcdistinct": "true",
|
||||
"shards": shards,
|
||||
"rows": 0,
|
||||
"wt": "json",
|
||||
"q": f"id:({item_ids_str})",
|
||||
"fq": "type:2 AND isBot:false AND statistics_type:view AND time:[2020-01-01T00:00:00Z TO 2020-09-02T00:00:00Z]",
|
||||
"facet": "true",
|
||||
"facet.field": "id",
|
||||
"facet.mincount": 1,
|
||||
"facet.limit": 1,
|
||||
"facet.offset": 0,
|
||||
"stats": "true",
|
||||
"stats.field": "id",
|
||||
"stats.calcdistinct": "true",
|
||||
"shards": shards,
|
||||
"rows": 0,
|
||||
"wt": "json",
|
||||
}
|
||||
</code></pre><ul>
|
||||
<li>The date range format for Solr is important, but it seems we only need to add <code>T00:00:00Z</code> to the normal ISO 8601 YYYY-MM-DD strings</li>
|
||||
@ -600,61 +600,61 @@ solr_query_params = {
|
||||
</ul>
|
||||
<pre tabindex="0"><code>$ curl -s -d @request.json https://dspacetest.cgiar.org/rest/statistics/items | json_pp
|
||||
{
|
||||
"currentPage" : 0,
|
||||
"limit" : 10,
|
||||
"statistics" : [
|
||||
"currentPage" : 0,
|
||||
"limit" : 10,
|
||||
"statistics" : [
|
||||
{
|
||||
"downloads" : 3329,
|
||||
"id" : "b2c1bbfd-65b0-438c-9e49-d271c49b2696",
|
||||
"views" : 1565
|
||||
"downloads" : 3329,
|
||||
"id" : "b2c1bbfd-65b0-438c-9e49-d271c49b2696",
|
||||
"views" : 1565
|
||||
},
|
||||
{
|
||||
"downloads" : 3797,
|
||||
"id" : "f44cf173-2344-4eb2-8f00-ee55df32c76f",
|
||||
"views" : 48
|
||||
"downloads" : 3797,
|
||||
"id" : "f44cf173-2344-4eb2-8f00-ee55df32c76f",
|
||||
"views" : 48
|
||||
},
|
||||
{
|
||||
"downloads" : 11064,
|
||||
"id" : "8542f9da-9ce1-4614-abf4-f2e3fdb4b305",
|
||||
"views" : 26
|
||||
"downloads" : 11064,
|
||||
"id" : "8542f9da-9ce1-4614-abf4-f2e3fdb4b305",
|
||||
"views" : 26
|
||||
},
|
||||
{
|
||||
"downloads" : 6782,
|
||||
"id" : "2324aa41-e9de-4a2b-bc36-16241464683e",
|
||||
"views" : 19
|
||||
"downloads" : 6782,
|
||||
"id" : "2324aa41-e9de-4a2b-bc36-16241464683e",
|
||||
"views" : 19
|
||||
},
|
||||
{
|
||||
"downloads" : 48,
|
||||
"id" : "0fe573e7-042a-4240-a4d9-753b61233908",
|
||||
"views" : 12
|
||||
"downloads" : 48,
|
||||
"id" : "0fe573e7-042a-4240-a4d9-753b61233908",
|
||||
"views" : 12
|
||||
},
|
||||
{
|
||||
"downloads" : 0,
|
||||
"id" : "000e61ca-695d-43e5-9ab8-1f3fd7a67a32",
|
||||
"views" : 4
|
||||
"downloads" : 0,
|
||||
"id" : "000e61ca-695d-43e5-9ab8-1f3fd7a67a32",
|
||||
"views" : 4
|
||||
},
|
||||
{
|
||||
"downloads" : 0,
|
||||
"id" : "000dc7cd-9485-424b-8ecf-78002613cc87",
|
||||
"views" : 1
|
||||
"downloads" : 0,
|
||||
"id" : "000dc7cd-9485-424b-8ecf-78002613cc87",
|
||||
"views" : 1
|
||||
},
|
||||
{
|
||||
"downloads" : 0,
|
||||
"id" : "000e1616-3901-4431-80b1-c6bc67312d8c",
|
||||
"views" : 1
|
||||
"downloads" : 0,
|
||||
"id" : "000e1616-3901-4431-80b1-c6bc67312d8c",
|
||||
"views" : 1
|
||||
},
|
||||
{
|
||||
"downloads" : 0,
|
||||
"id" : "000ea897-5557-49c7-9f54-9fa192c0f83b",
|
||||
"views" : 1
|
||||
"downloads" : 0,
|
||||
"id" : "000ea897-5557-49c7-9f54-9fa192c0f83b",
|
||||
"views" : 1
|
||||
},
|
||||
{
|
||||
"downloads" : 0,
|
||||
"id" : "000ec427-97e5-4766-85a5-e8dd62199ab5",
|
||||
"views" : 1
|
||||
"downloads" : 0,
|
||||
"id" : "000ec427-97e5-4766-85a5-e8dd62199ab5",
|
||||
"views" : 1
|
||||
}
|
||||
],
|
||||
"totalPages" : 13
|
||||
"totalPages" : 13
|
||||
}
|
||||
</code></pre><ul>
|
||||
<li>I deployed it on DSpace Test and sent a note to Salem so he can test it</li>
|
||||
|
||||
Reference in New Issue
Block a user