alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2022-03-04

Browse Source
This commit is contained in:
Alan Orth
2022-03-04 15:30:06 +03:00
parent 7453499827
commit 27acbac859
115 changed files with 6550 additions and 6444 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
docs/2020-06/index.html
View File

@ -36,7 +36,7 @@ I sent Atmire the dspace.log from today and told them to log into the server to
In other news, I checked the statistics API on DSpace 6 and it’s working
I tried to build the OAI registry on the freshly migrated DSpace 6 on DSpace Test and I get an error:
"/>
<meta name="generator" content="Hugo 0.92.2" />
<meta name="generator" content="Hugo 0.93.1" />
@ -161,8 +161,8 @@ java.lang.NullPointerException
</ul>
</li>
</ul>
<pre tabindex="0"><code>$ curl http://localhost:8080/solr/oai/update -H &quot;Content-type: text/xml&quot; --data-binary '&lt;delete&gt;&lt;query&gt;*:*&lt;/query&gt;&lt;/delete&gt;'
$ curl http://localhost:8080/solr/oai/update -H &quot;Content-type: text/xml&quot; --data-binary '&lt;commit /&gt;'
<pre tabindex="0"><code>$ curl http://localhost:8080/solr/oai/update -H &#34;Content-type: text/xml&#34; --data-binary &#39;&lt;delete&gt;&lt;query&gt;*:*&lt;/query&gt;&lt;/delete&gt;&#39;
$ curl http://localhost:8080/solr/oai/update -H &#34;Content-type: text/xml&#34; --data-binary &#39;&lt;commit /&gt;&#39;
$ ~/dspace63/bin/dspace oai import
OAI 2.0 manager action started
...
@ -279,7 +279,7 @@ sys 3m13.929s
<li>In theory we can have different languages for metadata fields but in practice we don&rsquo;t do that, so we might as well normalize everything to &ldquo;en_US&rdquo; (and perhaps I should make a curation task to do this)</li>
<li>For now I will do it manually on CGSpace and DSpace Test:</li>
</ul>
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_lang='en_US' WHERE resource_type_id=2;
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_lang=&#39;en_US&#39; WHERE resource_type_id=2;
UPDATE 2414738
</code></pre><ul>
<li>Note: DSpace Test doesn&rsquo;t have the <code>resource_type_id</code> column because it&rsquo;s running DSpace 6 and <a href="https://wiki.lyrasis.org/display/DSPACE/DSpace+Service+based+api">the schema changed to use an object model there</a>
@ -288,7 +288,7 @@ UPDATE 2414738
</ul>
</li>
</ul>
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_lang='en_US' WHERE dspace_object_id IN (SELECT uuid FROM item);
<pre tabindex="0"><code>dspace=# UPDATE metadatavalue SET text_lang=&#39;en_US&#39; WHERE dspace_object_id IN (SELECT uuid FROM item);
</code></pre><ul>
<li>Peter asked if it was possible to find all ILRI items that have &ldquo;zoonoses&rdquo; or &ldquo;zoonotic&rdquo; in their titles and check if they have the ILRI subject &ldquo;ZOONOTIC DISEASES&rdquo; (and add it if not)
<ul>
@ -320,7 +320,7 @@ UPDATE 2414738
</li>
</ul>
<pre tabindex="0"><code>$ dspace metadata-export -i 10568/1 -f /tmp/2020-06-08-ILRI.csv
$ csvcut -c 'id,cg.subject.ilri[en_US],dc.title[en_US]' ~/Downloads/2020-06-08-ILRI.csv &gt; /tmp/ilri.csv
$ csvcut -c &#39;id,cg.subject.ilri[en_US],dc.title[en_US]&#39; ~/Downloads/2020-06-08-ILRI.csv &gt; /tmp/ilri.csv
</code></pre><ul>
<li>Moayad asked why he&rsquo;s getting HTTP 500 errors on CGSpace
<ul>
@ -329,7 +329,7 @@ $ csvcut -c 'id,cg.subject.ilri[en_US],dc.title[en_US]' ~/Downloads/2020-06-08-I
</ul>
</li>
</ul>
<pre tabindex="0"><code># journalctl --since=today -u tomcat7 | grep -c 'Internal Server Error'
<pre tabindex="0"><code># journalctl --since=today -u tomcat7 | grep -c &#39;Internal Server Error&#39;
482
</code></pre><ul>
<li>They are all related to the REST API, like:</li>
@ -366,12 +366,12 @@ Jun 06 08:19:54 linode18 tomcat7[6286]: at java.lang.reflect.Method.invo
</code></pre><ul>
<li>Looking back, I see ~800 of these errors since I changed the database configuration last week:</li>
</ul>
<pre tabindex="0"><code># journalctl --since=2020-06-04 --until=today -u tomcat7 | grep -c 'javax.ws.rs.WebApplicationException'
<pre tabindex="0"><code># journalctl --since=2020-06-04 --until=today -u tomcat7 | grep -c &#39;javax.ws.rs.WebApplicationException&#39;
795
</code></pre><ul>
<li>And only ~280 in the entire month before that&hellip;</li>
</ul>
<pre tabindex="0"><code># journalctl --since=2020-05-01 --until=2020-06-04 -u tomcat7 | grep -c 'javax.ws.rs.WebApplicationException'
<pre tabindex="0"><code># journalctl --since=2020-05-01 --until=2020-06-04 -u tomcat7 | grep -c &#39;javax.ws.rs.WebApplicationException&#39;
286
</code></pre><ul>
<li>So it seems to be related to the database, perhaps that there are less connections in the pool?
@ -394,7 +394,7 @@ Jun 06 08:19:54 linode18 tomcat7[6286]: at java.lang.reflect.Method.invo
</code></pre><ul>
<li>Looking at the nginx access logs I see that, other than something that seems like Google Feedburner, all hosts using this user agent are all in Sweden!</li>
</ul>
<pre tabindex="0"><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/access.log.*.gz | grep 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36' | grep -v '/feed' | awk '{print $1}' | sort | uniq -c | sort -n
<pre tabindex="0"><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 /var/log/nginx/access.log.*.gz | grep &#39;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36&#39; | grep -v &#39;/feed&#39; | awk &#39;{print $1}&#39; | sort | uniq -c | sort -n
1624 192.36.136.246
1627 192.36.241.95
1629 192.165.45.204
@ -480,7 +480,7 @@ Total number of bot hits purged: 29025
</ul>
</li>
</ul>
<pre tabindex="0"><code>172.104.229.92 - - [13/Jun/2020:02:00:00 +0200] &quot;GET /rest/items?expand=metadata,bitstreams,parentCommunityList&amp;limit=50&amp;offset=0 HTTP/1.1&quot; 403 260 &quot;-&quot; &quot;-&quot;
<pre tabindex="0"><code>172.104.229.92 - - [13/Jun/2020:02:00:00 +0200] &#34;GET /rest/items?expand=metadata,bitstreams,parentCommunityList&amp;limit=50&amp;offset=0 HTTP/1.1&#34; 403 260 &#34;-&#34; &#34;-&#34;
</code></pre><ul>
<li>I created an nginx map based on the host&rsquo;s IP address that sets a temporary user agent (ua) and then changed the conditional in the REST API location block so that it checks this mapped ua instead of the default one
<ul>
@ -497,11 +497,11 @@ Total number of bot hits purged: 29025
</ul>
</li>
</ul>
<pre tabindex="0"><code>$ curl -s 'https://cgspace.cgiar.org/rest/handle/10568/51671?expand=collections' 'https://cgspace.cgiar.org/rest/handle/10568/89346?expand=collections' | grep -oE '10568/[0-9]+' | sort | uniq &gt; /tmp/cip-collections.txt
<pre tabindex="0"><code>$ curl -s &#39;https://cgspace.cgiar.org/rest/handle/10568/51671?expand=collections&#39; &#39;https://cgspace.cgiar.org/rest/handle/10568/89346?expand=collections&#39; | grep -oE &#39;10568/[0-9]+&#39; | sort | uniq &gt; /tmp/cip-collections.txt
</code></pre><ul>
<li>Then I formatted it into a SQL query and exported a CSV:</li>
</ul>
<pre tabindex="0"><code>dspace=# \COPY (SELECT DISTINCT text_value AS author, COUNT(*) FROM metadatavalue WHERE metadata_field_id = (SELECT metadata_field_id FROM metadatafieldregistry WHERE element = 'contributor' AND qualifier = 'author') AND resource_type_id = 2 AND resource_id IN (SELECT item_id FROM collection2item WHERE collection_id IN (SELECT resource_id FROM hANDle WHERE hANDle IN ('10568/100533', '10568/100653', '10568/101955', '10568/106580', '10568/108469', '10568/51671', '10568/53085', '10568/53086', '10568/53087', '10568/53088', '10568/53089', '10568/53090', '10568/53091', '10568/53092', '10568/53093', '10568/53094', '10568/64874', '10568/69069', '10568/70150', '10568/88229', '10568/89346', '10568/89347', '10568/99301', '10568/99302', '10568/99303', '10568/99304', '10568/99428'))) GROUP BY text_value ORDER BY count DESC) TO /tmp/cip-authors.csv WITH CSV;
<pre tabindex="0"><code>dspace=# \COPY (SELECT DISTINCT text_value AS author, COUNT(*) FROM metadatavalue WHERE metadata_field_id = (SELECT metadata_field_id FROM metadatafieldregistry WHERE element = &#39;contributor&#39; AND qualifier = &#39;author&#39;) AND resource_type_id = 2 AND resource_id IN (SELECT item_id FROM collection2item WHERE collection_id IN (SELECT resource_id FROM hANDle WHERE hANDle IN (&#39;10568/100533&#39;, &#39;10568/100653&#39;, &#39;10568/101955&#39;, &#39;10568/106580&#39;, &#39;10568/108469&#39;, &#39;10568/51671&#39;, &#39;10568/53085&#39;, &#39;10568/53086&#39;, &#39;10568/53087&#39;, &#39;10568/53088&#39;, &#39;10568/53089&#39;, &#39;10568/53090&#39;, &#39;10568/53091&#39;, &#39;10568/53092&#39;, &#39;10568/53093&#39;, &#39;10568/53094&#39;, &#39;10568/64874&#39;, &#39;10568/69069&#39;, &#39;10568/70150&#39;, &#39;10568/88229&#39;, &#39;10568/89346&#39;, &#39;10568/89347&#39;, &#39;10568/99301&#39;, &#39;10568/99302&#39;, &#39;10568/99303&#39;, &#39;10568/99304&#39;, &#39;10568/99428&#39;))) GROUP BY text_value ORDER BY count DESC) TO /tmp/cip-authors.csv WITH CSV;
COPY 3917
</code></pre><h2 id="2020-06-15">2020-06-15</h2>
<ul>
@ -632,7 +632,7 @@ COPY 3917
</li>
<li>I also notice that there is a <a href="https://www.crossref.org/services/funder-registry/">CrossRef funders registry</a> with 23,000+ funders that you can <a href="https://gitlab.com/crossref/open_funder_registry">download as RDF</a> or <a href="https://www.crossref.org/education/funder-registry/accessing-the-funder-registry/">access via an API</a></li>
</ul>
<pre tabindex="0"><code>$ http 'https://api.crossref.org/funders?query=Bill+and+Melinda+Gates&amp;mailto=a.orth@cgiar.org'
<pre tabindex="0"><code>$ http &#39;https://api.crossref.org/funders?query=Bill+and+Melinda+Gates&amp;mailto=a.orth@cgiar.org&#39;
</code></pre><ul>
<li>Searching for &ldquo;Bill and Melinda Gates&rdquo; we can see the <code>name</code> literal and a list of <code>alt-names</code> literals
<ul>
@ -697,14 +697,14 @@ SUSTAIN
AGRICULTURAL INNOVATIONS
NATIVE VARIETIES
PHYTOPHTHORA INFESTANS
$ ./delete-metadata-values.py -i /tmp/2020-06-30-remove-cip-subjects.csv -db dspace -u dspace -p 'fuuu' -f cg.subject.cip -m 127 -d
$ ./delete-metadata-values.py -i /tmp/2020-06-30-remove-cip-subjects.csv -db dspace -u dspace -p &#39;fuuu&#39; -f cg.subject.cip -m 127 -d
</code></pre><ul>
<li>She also wants to change their <code>SWEET POTATOES</code> term to <code>SWEETPOTATOES</code>, both in the CIP subject list and existing items so I updated those too:</li>
</ul>
<pre tabindex="0"><code>$ cat /tmp/2020-06-30-fix-cip-subjects.csv
cg.subject.cip,correct
SWEET POTATOES,SWEETPOTATOES
$ ./fix-metadata-values.py -i /tmp/2020-06-30-fix-cip-subjects.csv -db dspace -u dspace -p 'fuuu' -f cg.subject.cip -t correct -m 127 -d
$ ./fix-metadata-values.py -i /tmp/2020-06-30-fix-cip-subjects.csv -db dspace -u dspace -p &#39;fuuu&#39; -f cg.subject.cip -t correct -m 127 -d
</code></pre><ul>
<li>She also finished doing all the corrections to authors that I had sent her last week, but many of the changes are removing Spanish accents from authors names so I asked if she&rsquo;s really should she wants to do that</li>
<li>I ran the fixes and deletes on CGSpace, but not on DSpace Test yet because those scripts need updating for DSpace 6 UUIDs</li>
@ -712,63 +712,63 @@ $ ./fix-metadata-values.py -i /tmp/2020-06-30-fix-cip-subjects.csv -db dspace -u
</ul>
<pre tabindex="0"><code>$ cat 2020-06-29-fix-sponsors.csv
dc.description.sponsorship,correct
&quot;Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazil&quot;,&quot;Conselho Nacional de Desenvolvimento Científico e Tecnológico&quot;
&quot;Claussen Simon Stiftung&quot;,&quot;Claussen-Simon-Stiftung&quot;
&quot;Fonds pour la formation á la Recherche dans l'Industrie et dans l'Agriculture, Belgium&quot;,&quot;Fonds pour la Formation à la Recherche dans l’Industrie et dans l’Agriculture&quot;
&quot;Fundação de Amparo à Pesquisa do Estado de São Paulo, Brazil&quot;,&quot;Fundação de Amparo à Pesquisa do Estado de São Paulo&quot;
&quot;Schlumberger Foundation Faculty for the Future&quot;,&quot;Schlumberger Foundation&quot;
&quot;Wildlife Conservation Society, United States&quot;,&quot;Wildlife Conservation Society&quot;
&quot;Portuguese Foundation for Science and Technology&quot;,&quot;Portuguese Science and Technology Foundation&quot;
&quot;Wageningen University and Research&quot;,&quot;Wageningen University and Research Centre&quot;
&quot;Leverhulme Centre for Integrative Research in Agriculture and Health&quot;,&quot;Leverhulme Centre for Integrative Research on Agriculture and Health&quot;
&quot;Natural Science and Engineering Research Council of Canada&quot;,&quot;Natural Sciences and Engineering Research Council of Canada&quot;
&quot;Biotechnology and Biological Sciences Research Council, United Kingdom&quot;,&quot;Biotechnology and Biological Sciences Research Council&quot;
&quot;Home Grown Ceraels Authority United Kingdom&quot;,&quot;Home-Grown Cereals Authority&quot;
&quot;Fiat Panis Foundation&quot;,&quot;Foundation fiat panis&quot;
&quot;Defence Science and Technology Laboratory, United Kingdom&quot;,&quot;Defence Science and Technology Laboratory&quot;
&quot;African Development Bank&quot;,&quot;African Development Bank Group&quot;
&quot;Ministry of Health, Labour, and Welfare, Japan&quot;,&quot;Ministry of Health, Labour and Welfare&quot;
&quot;World Academy of Sciences&quot;,&quot;The World Academy of Sciences&quot;
&quot;Agricultural Research Council, South Africa&quot;,&quot;Agricultural Research Council&quot;
&quot;Department of Homeland Security, USA&quot;,&quot;U.S. Department of Homeland Security&quot;
&quot;Quadram Institute&quot;,&quot;Quadram Institute Bioscience&quot;
&quot;Google.org&quot;,&quot;Google&quot;
&quot;Department for Environment, Food and Rural Affairs, United Kingdom&quot;,&quot;Department for Environment, Food and Rural Affairs, UK Government&quot;
&quot;National Commission for Science, Technology and Innovation, Kenya&quot;,&quot;National Commission for Science, Technology and Innovation&quot;
&quot;Hainan Province Natural Science Foundation of China&quot;,&quot;Natural Science Foundation of Hainan Province&quot;
&quot;German Society for International Cooperation (GIZ)&quot;,&quot;GIZ&quot;
&quot;German Federal Ministry of Food and Agriculture&quot;,&quot;Federal Ministry of Food and Agriculture&quot;
&quot;State Key Laboratory of Environmental Geochemistry, China&quot;,&quot;State Key Laboratory of Environmental Geochemistry&quot;
&quot;QUT student scholarship&quot;,&quot;Queensland University of Technology&quot;
&quot;Australia Centre for International Agricultural Research&quot;,&quot;Australian Centre for International Agricultural Research&quot;
&quot;Belgian Science Policy&quot;,&quot;Belgian Federal Science Policy Office&quot;
&quot;U.S. Department of Agriculture USDA&quot;,&quot;U.S. Department of Agriculture&quot;
&quot;U.S.. Department of Agriculture (USDA)&quot;,&quot;U.S. Department of Agriculture&quot;
&quot;Fundação de Amparo à Pesquisa do Estado de São Paulo ( FAPESP)&quot;,&quot;Fundação de Amparo à Pesquisa do Estado de São Paulo&quot;
&quot;Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul, Brazil&quot;,&quot;Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul&quot;
&quot;Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro, Brazil&quot;,&quot;Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro&quot;
&quot;Swedish University of Agricultural Sciences (SLU)&quot;,&quot;Swedish University of Agricultural Sciences&quot;
&quot;U.S. Department of Agriculture (USDA)&quot;,&quot;U.S. Department of Agriculture&quot;
&quot;Swedish International Development Cooperation Agency (Sida)&quot;,&quot;Sida&quot;
&quot;Swedish International Development Agency&quot;,&quot;Sida&quot;
&quot;Federal Ministry for Economic Cooperation and Development, Germany&quot;,&quot;Federal Ministry for Economic Cooperation and Development&quot;
&quot;Natural Environment Research Council, United Kingdom&quot;,&quot;Natural Environment Research Council&quot;
&quot;Economic and Social Research Council, United Kingdom&quot;,&quot;Economic and Social Research Council&quot;
&quot;Medical Research Council, United Kingdom&quot;,&quot;Medical Research Council&quot;
&quot;Federal Ministry for Education and Research, Germany&quot;,&quot;Federal Ministry for Education, Science, Research and Technology&quot;
&quot;UK Government’s Department for International Development&quot;,&quot;Department for International Development, UK Government&quot;
&quot;Department for International Development, United Kingdom&quot;,&quot;Department for International Development, UK Government&quot;
&quot;United Nations Children's Fund&quot;,&quot;United Nations Children's Emergency Fund&quot;
&quot;Swedish Research Council for Environment, Agricultural Science and Spatial Planning&quot;,&quot;Swedish Research Council for Environment, Agricultural Sciences and Spatial Planning&quot;
&quot;Agence Nationale de la Recherche, France&quot;,&quot;French National Research Agency&quot;
&quot;Fondation pour la recherche sur la biodiversité&quot;,&quot;Foundation for Research on Biodiversity&quot;
&quot;Programa Nacional de Innovacion Agraria, Peru&quot;,&quot;Programa Nacional de Innovación Agraria, Peru&quot;
&quot;United States Agency for International Development (USAID)&quot;,&quot;United States Agency for International Development&quot;
&quot;West Africa Agricultural Productivity Programme&quot;,&quot;West Africa Agricultural Productivity Program&quot;
&quot;West African Agricultural Productivity Project&quot;,&quot;West Africa Agricultural Productivity Program&quot;
&quot;Rural Development Administration, Republic of Korea&quot;,&quot;Rural Development Administration&quot;
&quot;UK’s Biotechnology and Biological Sciences Research Council (BBSRC)&quot;,&quot;Biotechnology and Biological Sciences Research Council&quot;
$ ./fix-metadata-values.py -i /tmp/2020-06-29-fix-sponsors.csv -db dspace -u dspace -p 'fuuu' -f dc.description.sponsorship -t correct -m 29
&#34;Conselho Nacional de Desenvolvimento Científico e Tecnológico, Brazil&#34;,&#34;Conselho Nacional de Desenvolvimento Científico e Tecnológico&#34;
&#34;Claussen Simon Stiftung&#34;,&#34;Claussen-Simon-Stiftung&#34;
&#34;Fonds pour la formation á la Recherche dans l&#39;Industrie et dans l&#39;Agriculture, Belgium&#34;,&#34;Fonds pour la Formation à la Recherche dans l’Industrie et dans l’Agriculture&#34;
&#34;Fundação de Amparo à Pesquisa do Estado de São Paulo, Brazil&#34;,&#34;Fundação de Amparo à Pesquisa do Estado de São Paulo&#34;
&#34;Schlumberger Foundation Faculty for the Future&#34;,&#34;Schlumberger Foundation&#34;
&#34;Wildlife Conservation Society, United States&#34;,&#34;Wildlife Conservation Society&#34;
&#34;Portuguese Foundation for Science and Technology&#34;,&#34;Portuguese Science and Technology Foundation&#34;
&#34;Wageningen University and Research&#34;,&#34;Wageningen University and Research Centre&#34;
&#34;Leverhulme Centre for Integrative Research in Agriculture and Health&#34;,&#34;Leverhulme Centre for Integrative Research on Agriculture and Health&#34;
&#34;Natural Science and Engineering Research Council of Canada&#34;,&#34;Natural Sciences and Engineering Research Council of Canada&#34;
&#34;Biotechnology and Biological Sciences Research Council, United Kingdom&#34;,&#34;Biotechnology and Biological Sciences Research Council&#34;
&#34;Home Grown Ceraels Authority United Kingdom&#34;,&#34;Home-Grown Cereals Authority&#34;
&#34;Fiat Panis Foundation&#34;,&#34;Foundation fiat panis&#34;
&#34;Defence Science and Technology Laboratory, United Kingdom&#34;,&#34;Defence Science and Technology Laboratory&#34;
&#34;African Development Bank&#34;,&#34;African Development Bank Group&#34;
&#34;Ministry of Health, Labour, and Welfare, Japan&#34;,&#34;Ministry of Health, Labour and Welfare&#34;
&#34;World Academy of Sciences&#34;,&#34;The World Academy of Sciences&#34;
&#34;Agricultural Research Council, South Africa&#34;,&#34;Agricultural Research Council&#34;
&#34;Department of Homeland Security, USA&#34;,&#34;U.S. Department of Homeland Security&#34;
&#34;Quadram Institute&#34;,&#34;Quadram Institute Bioscience&#34;
&#34;Google.org&#34;,&#34;Google&#34;
&#34;Department for Environment, Food and Rural Affairs, United Kingdom&#34;,&#34;Department for Environment, Food and Rural Affairs, UK Government&#34;
&#34;National Commission for Science, Technology and Innovation, Kenya&#34;,&#34;National Commission for Science, Technology and Innovation&#34;
&#34;Hainan Province Natural Science Foundation of China&#34;,&#34;Natural Science Foundation of Hainan Province&#34;
&#34;German Society for International Cooperation (GIZ)&#34;,&#34;GIZ&#34;
&#34;German Federal Ministry of Food and Agriculture&#34;,&#34;Federal Ministry of Food and Agriculture&#34;
&#34;State Key Laboratory of Environmental Geochemistry, China&#34;,&#34;State Key Laboratory of Environmental Geochemistry&#34;
&#34;QUT student scholarship&#34;,&#34;Queensland University of Technology&#34;
&#34;Australia Centre for International Agricultural Research&#34;,&#34;Australian Centre for International Agricultural Research&#34;
&#34;Belgian Science Policy&#34;,&#34;Belgian Federal Science Policy Office&#34;
&#34;U.S. Department of Agriculture USDA&#34;,&#34;U.S. Department of Agriculture&#34;
&#34;U.S.. Department of Agriculture (USDA)&#34;,&#34;U.S. Department of Agriculture&#34;
&#34;Fundação de Amparo à Pesquisa do Estado de São Paulo ( FAPESP)&#34;,&#34;Fundação de Amparo à Pesquisa do Estado de São Paulo&#34;
&#34;Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul, Brazil&#34;,&#34;Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul&#34;
&#34;Fundação de Amparo à Pesquisa do Estado do Rio de Janeiro, Brazil&#34;,&#34;Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro&#34;
&#34;Swedish University of Agricultural Sciences (SLU)&#34;,&#34;Swedish University of Agricultural Sciences&#34;
&#34;U.S. Department of Agriculture (USDA)&#34;,&#34;U.S. Department of Agriculture&#34;
&#34;Swedish International Development Cooperation Agency (Sida)&#34;,&#34;Sida&#34;
&#34;Swedish International Development Agency&#34;,&#34;Sida&#34;
&#34;Federal Ministry for Economic Cooperation and Development, Germany&#34;,&#34;Federal Ministry for Economic Cooperation and Development&#34;
&#34;Natural Environment Research Council, United Kingdom&#34;,&#34;Natural Environment Research Council&#34;
&#34;Economic and Social Research Council, United Kingdom&#34;,&#34;Economic and Social Research Council&#34;
&#34;Medical Research Council, United Kingdom&#34;,&#34;Medical Research Council&#34;
&#34;Federal Ministry for Education and Research, Germany&#34;,&#34;Federal Ministry for Education, Science, Research and Technology&#34;
&#34;UK Government’s Department for International Development&#34;,&#34;Department for International Development, UK Government&#34;
&#34;Department for International Development, United Kingdom&#34;,&#34;Department for International Development, UK Government&#34;
&#34;United Nations Children&#39;s Fund&#34;,&#34;United Nations Children&#39;s Emergency Fund&#34;
&#34;Swedish Research Council for Environment, Agricultural Science and Spatial Planning&#34;,&#34;Swedish Research Council for Environment, Agricultural Sciences and Spatial Planning&#34;
&#34;Agence Nationale de la Recherche, France&#34;,&#34;French National Research Agency&#34;
&#34;Fondation pour la recherche sur la biodiversité&#34;,&#34;Foundation for Research on Biodiversity&#34;
&#34;Programa Nacional de Innovacion Agraria, Peru&#34;,&#34;Programa Nacional de Innovación Agraria, Peru&#34;
&#34;United States Agency for International Development (USAID)&#34;,&#34;United States Agency for International Development&#34;
&#34;West Africa Agricultural Productivity Programme&#34;,&#34;West Africa Agricultural Productivity Program&#34;
&#34;West African Agricultural Productivity Project&#34;,&#34;West Africa Agricultural Productivity Program&#34;
&#34;Rural Development Administration, Republic of Korea&#34;,&#34;Rural Development Administration&#34;
&#34;UK’s Biotechnology and Biological Sciences Research Council (BBSRC)&#34;,&#34;Biotechnology and Biological Sciences Research Council&#34;
$ ./fix-metadata-values.py -i /tmp/2020-06-29-fix-sponsors.csv -db dspace -u dspace -p &#39;fuuu&#39; -f dc.description.sponsorship -t correct -m 29
</code></pre><ul>
<li>Then I started a full re-index at batch CPU priority:</li>
</ul>
@ -784,9 +784,9 @@ sys 2m56.635s
</ul>
</li>
</ul>
<pre tabindex="0"><code>$ export JAVA_OPTS=&quot;-Xmx512m -Dfile.encoding=UTF-8&quot;
<pre tabindex="0"><code>$ export JAVA_OPTS=&#34;-Xmx512m -Dfile.encoding=UTF-8&#34;
$ dspace metadata-export -i 10568/1 -f /tmp/ilri.cs
$ csvcut -c 'id,cg.subject.ilri[],cg.subject.ilri[en_US],dc.subject[en_US]' /tmp/ilri.csv &gt; /tmp/ilri-covid19.csv
$ csvcut -c &#39;id,cg.subject.ilri[],cg.subject.ilri[en_US],dc.subject[en_US]&#39; /tmp/ilri.csv &gt; /tmp/ilri-covid19.csv
</code></pre><ul>
<li>I see that all items with &ldquo;COVID19&rdquo; already have &ldquo;CORONAVIRUS DISEASE&rdquo; so I don&rsquo;t need to do anything</li>
</ul>