Add notes for 2021-08-17

content/posts/2021-08.md

@@ -294,5 +294,54 @@ $ dspace community-filiator --set --parent=10568/114644 --child=10568/76451
 
 - I made an initial attempt on the policy statements page on DSpace Test
   - It is modeled on Sherpa Romeo's OpenDOAR policy statements advice
+- Sit with Moayad and discuss the future of AReS
+  - We specifically discussed formalizing the API and documenting its use to allow as an alternative to harvesting directly from CGSpace
+  - We also discussed allowing linking to search results to enable something like "Explore this collection" links on CGSpace collection pages
+- Lower case all AGROVOC metadata, as I had noticed a few in sentence case:
+
+```console
+dspace=# UPDATE metadatavalue SET text_value=LOWER(text_value) WHERE dspace_object_id IN (SELECT uuid FROM item) AND metadata_field_id=187 AND text_value ~ '[[:upper:]]';
+UPDATE 484
+```
+
+- Also update some DOIs using the `dx.doi.org` format, just to keep things uniform:
+
+```console
+dspace=# UPDATE metadatavalue SET text_value = regexp_replace(text_value, 'https://dx.doi.org', 'https://doi.org') WHERE dspace_object_id IN (SELECT uuid FROM item) AND metadata_field_id = 220 AND text_value LIKE 'https://dx.doi.org%';
+UPDATE 469
+```
+
+- Then start a full Discovery re-indexing to update the Feed the Future community item counts that have been stuck at 0 since we moved the three projects to be a subcommunity a few days ago:
+
+```console
+$ time chrt -b 0 ionice -c2 -n7 nice -n19 dspace index-discovery -b
+
+real    322m16.917s
+user    226m43.121s
+sys     3m17.469s
+```
+
+- I learned how to use the OpenRXV API, which is just a thin wrapper around Elasticsearch:
+
+```console
+$ curl -X POST 'https://cgspace.cgiar.org/explorer/api/search?scroll=1d' \
+    -H 'Content-Type: application/json' \
+    -d '{
+    "size": 10,
+    "query": {
+        "bool": {
+            "filter": {
+                "term": {
+                    "repo.keyword": "CGSpace"
+                }
+            }
+        }
+    }
+}'
+$ curl -X POST 'https://cgspace.cgiar.org/explorer/api/search/scroll/DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAASekWMTRwZ3lEMkVRYUtKZjgyMno4dV9CUQ==
+```
+
+- This uses the Elasticsearch scroll ID to page through results
+  - The second query doesn't need the request body because it is saved for 1 day as part of the first request
 
 <!-- v[im: set sw=2 ts=2: -->