alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2021-03-08

Browse Source
This commit is contained in:
Alan Orth
2021-03-08 20:13:40 +02:00
parent 75596fd524
commit 551adc7053
29 changed files with 123 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
content/posts/2021-03.md
View File

@ -234,4 +234,46 @@ $ curl -X PUT "localhost:9200/openrxv-items/_settings" -H 'Content-Type: applica
- [Migrated away from links to use networks](https://github.com/ilri/OpenRXV/issues/89)
- [Converted the backend container to use a custom image that includes `unoconv`](https://github.com/ilri/OpenRXV/issues/68) so we don't have to manually install it anymore
## 2021-03-08
- I approved the WLE item that I edited last week, and all the metadata is there: https://hdl.handle.net/10568/111810
- So I'm not sure what Niroshini's issue with metadata is...
- Peter sent a message yesterday saying that his item finally got committed
- I looked at the Munin graphs and there was a MASSIVE spike in database activity two days ago, and now database locks are back down to normal levels (from 1000+):
```console
$ psql -c 'SELECT * FROM pg_locks pl LEFT JOIN pg_stat_activity psa ON pl.pid = psa.pid;' | wc -l
13
```
- On 2021-03-03 the PostgreSQL transactions started rising:
![PostgreSQL query length week](/cgspace-notes/2021/03/postgres_querylength_ALL-week.png)
- After that the connections and locks started going up, peaking on 2021-03-06:
![PostgreSQL locks week](/cgspace-notes/2021/03/postgres_locks_ALL-week.png)
![PostgreSQL connections week](/cgspace-notes/2021/03/postgres_connections_ALL-week.png)
- I sent another message to Atmire to ask if they have time to look into this
- CIFOR is pressuring me to upload the batch items from last week
- Vika sent me a final file with some duplicates that Peter identified removed
- I extracted and re-applied my basic corrections from last week in OpenRefine, then ran the items through `csv-metadata-quality` checker and uploaded them to CGSpace
- In total there are 1,088 items
- Udana from IWMI emailed to ask about CGSpace thumbnails
- Udana from IWMI emailed to ask about an item uploaded recently that does not appear in AReS
- [The item](https://hdl.handle.net/10568/111794) was added to the archive on 2021-03-05, and I last harvested on 2021-03-06, so this might be an issue of a missing item
- Abenet got a quote from Atmire to buy 125 credits for 3750€
- Maria at Bioversity sent some feedback about duplicate items on AReS
- I'm wondering if the issue of the `openrxv-items-final` index not getting cleared after a successful harvest (which results in having 200,000, then 300,000, etc items) has to do with the alias issue I fixed yesterday
- I will start a fresh harvest on AReS without now to check, but first back up the current index just in case:
```console
$ curl -X PUT "localhost:9200/openrxv-items-final/_settings" -H 'Content-Type: application/json' -d'{"settings": {"index.blocks.write": true}}'
$ curl -s -X POST http://localhost:9200/openrxv-items-final/_clone/openrxv-items-final-2021-03-08
# start harvesting on AReS
```
- As I saw on my local test instance, even when you cancel a harvesting, it replaces the `openrxv-items-final` index with whatever is in `openrxv-items-temp` automatically, so I assume it will do the same now
<!-- vim: set sw=2 ts=2: -->