Add notes for 2023-07-08

content/posts/2023-07.md

@@ -35,4 +35,59 @@ categories: ["Notes"]
   - It displays the bistream description instead of the file name, just like we did in DSpace 6 XMLUI
 - I added a custom component to show share icons
 
+## 2023-07-05
+
+- I spent some time trying to update OpenRXV from Angular 9 to 10 to 11 to 12 to 13
+  - Most things work but there are some minor bugs it seems
+- Mishell from CIP emailed me to say she was having problems approving an item on CGSpace
+  - Looking at PostgreSQL I saw there were a dozen or so locks that were several hours and even over one day old so I killed those processes and told her to try again
+
+## 2023-07-06
+
+- Types meeting
+- I wrote a Python script to check Unpaywall for some information about DOIs
+
+## 2023-07-7
+
+- Continue exploring Unpaywall data for some of our DOIs
+  - In the past I've found their _licensing_ information to not be very reliable (preferring Crossref), but I think their _open access status_ is more reliable, especially when the provider is listed as being the publisher
+  - Even so, sometimes the version can be "acceptedVersion", which is presumably the author's version, as opposed to the "publishedVersion", which means it's available as open access on the publisher's website
+  - I did some quality assurance and found ~100 that were marked as Limited Access, but should have been Open Access, and fixed a handful of licenses
+- Delete duplicate metadata as describe in my DSpace issue from last year: https://github.com/DSpace/DSpace/issues/8253
+- Start working on some statistics on AGROVOC usage for my presenation next week
+  - I used the following SQL query to dump values from all subject fields and lower case them:
+
+```console
+localhost/dspacetest= ☘ \COPY (SELECT DISTINCT(lower(text_value)) AS "subject" FROM metadatavalue WHERE dspace_object_id in (SELECT dspace_object_id FROM item) AND metadata_field_id IN (187, 120, 210, 122, 215, 127, 208, 124, 128, 123, 125, 135, 203, 236, 238, 119)) to /tmp/2023-07-07-cgspace-subjects.csv WITH CSV HEADER;
+COPY 26443
+Time: 2564.851 ms (00:02.565)
+```
+
+- Then I extracted the subjects and looked them up against AGROVOC:
+
+```console
+$ csvcut -c subject /tmp/2023-07-07-cgspace-subjects.csv | sed '1d' > /tmp/2023-07-07-cgspace-subjects.txt
+$ ./ilri/agrovoc_lookup.py -i /tmp/2023-07-07-cgspace-subjects.txt -o /tmp/2023-07-07-cgspace-subjects-results.csv
+```
+
+- I did some more tests with Angular 13 on OpenRXV and found out why the repository type dropdown wasn't working
+  - It was because of a missing 1-line JSON file in the data directory, which is runtime data, not code
+  - I copied the data directory from the production serve and rebuild and the site is working well now
+  - I did a full harvest with plugins and it worked!
+  - So it seems Angular 13.4.0 will work, yay
+
+## 2023-07-08
+
+- Export CGSpace to check for missing Initiative collection mappings
+  - Start a harvest on AReS
+- The AGROVOC lookup finished, so I checked the number of matches:
+
+```console
+$ csvgrep -c 'match type' -r '^.+$' ~/Downloads/2023-07-07-cgspace-subjects-resolved.csv | sed 1d | wc -l
+12528
+```
+
+- So that's 12,528 out of 26,443 unique terms (47.3%)
+- I did a LOT of work on the OpenRXV frontend build dependencies to bring more in line with Angular 13
+
 <!-- vim: set sw=2 ts=2: -->