mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-12-19 03:32:19 +01:00
419 lines
16 KiB
HTML
419 lines
16 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en" >
|
|
|
|
<head>
|
|
<meta charset="utf-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
|
|
|
|
|
<meta property="og:title" content="Notes" />
|
|
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
|
<meta property="og:type" content="website" />
|
|
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
|
<meta property="og:updated_time" content="2024-01-10T08:34:16+03:00" />
|
|
|
|
|
|
|
|
<meta name="twitter:card" content="summary"/>
|
|
<meta name="twitter:title" content="Notes"/>
|
|
<meta name="twitter:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository."/>
|
|
<meta name="generator" content="Hugo 0.121.2">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<link rel="canonical" href="https://alanorth.github.io/cgspace-notes/categories/notes/">
|
|
|
|
<title>CGSpace Notes</title>
|
|
|
|
|
|
<!-- combined, minified CSS -->
|
|
|
|
<link href="https://alanorth.github.io/cgspace-notes/css/style.c6ba80bc50669557645abe05f86b73cc5af84408ed20f1551a267bc19ece8228.css" rel="stylesheet" integrity="sha256-xrqAvFBmlVdkWr4F+GtzzFr4RAjtIPFVGiZ7wZ7Ogig=" crossorigin="anonymous">
|
|
|
|
|
|
<!-- minified Font Awesome for SVG icons -->
|
|
|
|
<script defer src="https://alanorth.github.io/cgspace-notes/js/fontawesome.min.f5072c55a0721857184db93a50561d7dc13975b4de2e19db7f81eb5f3fa57270.js" integrity="sha256-9QcsVaByGFcYTbk6UFYdfcE5dbTeLhnbf4HrXz+lcnA=" crossorigin="anonymous"></script>
|
|
|
|
<!-- RSS 2.0 feed -->
|
|
<link rel="alternate" type="application/rss+xml" href="https://alanorth.github.io/cgspace-notes/categories/notes/index.xml" title="CGSpace Notes" />
|
|
|
|
|
|
|
|
|
|
</head>
|
|
|
|
<body>
|
|
|
|
|
|
<div class="blog-masthead">
|
|
<div class="container">
|
|
<nav class="nav blog-nav">
|
|
<a class="nav-link " href="https://alanorth.github.io/cgspace-notes/">Home</a>
|
|
</nav>
|
|
</div>
|
|
</div>
|
|
|
|
|
|
|
|
|
|
<header class="blog-header">
|
|
<div class="container">
|
|
<h1 class="blog-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/" rel="home">CGSpace Notes</a></h1>
|
|
<p class="lead blog-description" dir="auto">Documenting day-to-day work on the <a href="https://cgspace.cgiar.org">CGSpace</a> repository.</p>
|
|
</div>
|
|
</header>
|
|
|
|
|
|
|
|
|
|
<div class="container">
|
|
<div class="row">
|
|
<div class="col-sm-8 blog-main">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2024-01/">January, 2024</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2024-01-02T10:08:00+03:00">Tue Jan 02, 2024</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2024-01-02">2024-01-02</h2>
|
|
<ul>
|
|
<li>Work on preparation of new server for DSpace 7 migration
|
|
<ul>
|
|
<li>I’m not quite sure what we need to do for the Handle server</li>
|
|
<li>For now I just ran the <code>dspace make-handle-config</code> script and diffed it with the one from DSpace 6</li>
|
|
<li>I sent the bundle to the Handle admins to make sure it’s OK before we do the migration</li>
|
|
</ul>
|
|
</li>
|
|
<li>Continue testing and debugging the cgspace-java-helpers on DSpace 7</li>
|
|
<li>Work on IFPRI ISNAR archive cleanup</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2024-01/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-12/">December, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-12-01T08:48:36+03:00">Fri Dec 01, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
2023-12-01 There is still high load on CGSpace and I don’t know why I don’t see a high number of sessions compared to previous days in the last few weeks $ for file in dspace.log.2023-11-[23]*; do echo "$file"; grep -a -oE 'session_id=[A-Z0-9]{32}' "$file" | sort | uniq | wc -l; done dspace.log.2023-11-20 22865 dspace.log.2023-11-21 20296 dspace.log.2023-11-22 19688 dspace.log.2023-11-23 17906 dspace.log.2023-11-24 18453 dspace.log.2023-11-25 17513 dspace.log.2023-11-26 19037 dspace.log.2023-11-27 21103 dspace.log.2023-11-28 23023 dspace.log.2023-11-29 23545 dspace.
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-12/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-11/">November, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-11-02T12:59:36+03:00">Thu Nov 02, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-11-01">2023-11-01</h2>
|
|
<ul>
|
|
<li>Work a bit on the ETL pipeline for the CGIAR Climate Change Synthesis
|
|
<ul>
|
|
<li>I improved the filtering and wrote some Python using pandas to merge my sources more reliably</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<h2 id="2023-11-02">2023-11-02</h2>
|
|
<ul>
|
|
<li>Export CGSpace to check missing Initiative collection mappings</li>
|
|
<li>Start a harvest on AReS</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-11/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-10/">October, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-10-02T09:05:36+03:00">Mon Oct 02, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-10-02">2023-10-02</h2>
|
|
<ul>
|
|
<li>Export CGSpace to check DOIs against Crossref
|
|
<ul>
|
|
<li>I found that <a href="https://www.crossref.org/documentation/retrieve-metadata/rest-api/rest-api-metadata-license-information/">Crossref’s metadata is in the public domain under the CC0 license</a></li>
|
|
<li>One interesting thing is the abstracts, which are copyrighted by the copyright owner, meaning Crossref cannot waive the copyright under the terms of the CC0 license, because it is not theirs to waive</li>
|
|
<li>We can be on the safe side by using only abstracts for items that are licensed under Creative Commons</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-10/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-09/">September, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-09-02T17:29:36+03:00">Sat Sep 02, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-09-02">2023-09-02</h2>
|
|
<ul>
|
|
<li>Export CGSpace to check for missing Initiative collection mappings</li>
|
|
<li>Start a harvest on AReS</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-09/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-08/">August, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-08-03T11:18:36+03:00">Thu Aug 03, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-08-03">2023-08-03</h2>
|
|
<ul>
|
|
<li>I finally got around to working on Peter’s cleanups for affiliations, authors, and donors from last week
|
|
<ul>
|
|
<li>I did some minor cleanups myself and applied them to CGSpace</li>
|
|
</ul>
|
|
</li>
|
|
<li>Start working on some batch uploads for IFPRI</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-08/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-07/">July, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-07-01T17:14:36+03:00">Sat Jul 01, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
2023-07-01 Export CGSpace to check for missing Initiative collection mappings Start harvesting on AReS 2023-07-02 Minor edits to the crossref_doi_lookup.py script while running some checks from 22,000 CGSpace DOIs 2023-07-03 I analyzed the licenses declared by Crossref and found with high confidence that ~400 of ours were incorrect I took the more accurate ones from Crossref and updated the items on CGSpace I took a few hundred ISBNs as well for where we were missing them I also tagged ~4,700 items with missing licenses as “Copyrighted; all rights reserved” based on their Crossref license status being TDM, mostly from Elsevier, Wiley, and Springer Checking a dozen or so manually, I confirmed that if Crossref only has a TDM license then it’s usually copyrighted (could still be open access, but we can’t tell via Crossref) I would be curious to write a script to check the Unpaywall API for open access status… In the past I found that their license status was not very accurate, but the open access status might be more reliable More minor work on the DSpace 7 item views I learned some new Angular template syntax I created a custom component to show Creative Commons licenses on the simple item page I also decided that I don’t like the Impact Area icons as a component because they don’t have any visual meaning 2023-07-04 Focus group meeting with CGSpace partners about DSpace 7 I added a themed file selection component to the CGSpace theme It displays the bistream description instead of the file name, just like we did in DSpace 6 XMLUI I added a custom component to show share icons 2023-07-05 I spent some time trying to update OpenRXV from Angular 9 to 10 to 11 to 12 to 13 Most things work but there are some minor bugs it seems Mishell from CIP emailed me to say she was having problems approving an item on CGSpace Looking at PostgreSQL I saw there were a dozen or so locks that were several hours and even over one day old so I killed those processes and told her to try again 2023-07-06 Types meeting I wrote a Python script to check Unpaywall for some information about DOIs 2023-07-7 Continue exploring Unpaywall data for some of our DOIs In the past I’ve found their licensing information to not be very reliable (preferring Crossref), but I think their open access status is more reliable, especially when the provider is listed as being the publisher Even so, sometimes the version can be “acceptedVersion”, which is presumably the author’s version, as opposed to the “publishedVersion”, which means it’s available as open access on the publisher’s website I did some quality assurance and found ~100 that were marked as Limited Access, but should have been Open Access, and fixed a handful of licenses Delete duplicate metadata as described in my DSpace issue from last year: https://github.
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-07/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-06/">June, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-06-02T10:29:36+03:00">Fri Jun 02, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-06-02">2023-06-02</h2>
|
|
<ul>
|
|
<li>Spend some time testing my <code>post_bitstreams.py</code> script to update thumbnails for items on CGSpace
|
|
<ul>
|
|
<li>Interestingly I found an item with a JFIF thumbnail and another with a WebP thumbnail…</li>
|
|
</ul>
|
|
</li>
|
|
<li>Meeting with Valentina, Stefano, and Sara about MODS metadata in CGSpace
|
|
<ul>
|
|
<li>They have experience with improving the MODS interface in MELSpace’s OAI-PMH for use with AGRIS and were curious if we could do the same in CGSpace</li>
|
|
<li>From what I can see we need to upgrade the MODS schema from 3.1 to 3.7 and then just add a bunch of our fields to the crosswalk</li>
|
|
</ul>
|
|
</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-06/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-05/">May, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-05-03T08:53:36+03:00">Wed May 03, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-05-03">2023-05-03</h2>
|
|
<ul>
|
|
<li>Alliance’s TIP team emailed me to ask about issues authenticating on CGSpace
|
|
<ul>
|
|
<li>It seems their password expired, which is annoying</li>
|
|
</ul>
|
|
</li>
|
|
<li>I continued looking at the CGSpace subjects for the FAO / AGROVOC exercise that I started last week
|
|
<ul>
|
|
<li>There are many of our subjects that would match if they added a “-” like “high yielding varieties” or used singular…</li>
|
|
<li>Also I found at least two spelling mistakes, for example “decison support systems”, which would match if it was spelled correctly</li>
|
|
</ul>
|
|
</li>
|
|
<li>Work on cleaning, proofing, and uploading twenty-seven records for IFPRI to CGSpace</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-05/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<article class="blog-post">
|
|
<header>
|
|
<h2 class="blog-post-title" dir="auto"><a href="https://alanorth.github.io/cgspace-notes/2023-04/">April, 2023</a></h2>
|
|
<p class="blog-post-meta"><time datetime="2023-04-02T08:19:36+03:00">Sun Apr 02, 2023</time> by Alan Orth in
|
|
<span class="fas fa-folder" aria-hidden="true"></span> <a href="/categories/notes/" rel="category tag">Notes</a>
|
|
|
|
|
|
</p>
|
|
</header>
|
|
<h2 id="2023-04-02">2023-04-02</h2>
|
|
<ul>
|
|
<li>Run all system updates on CGSpace and reboot it</li>
|
|
<li>I exported CGSpace to CSV to check for any missing Initiative collection mappings
|
|
<ul>
|
|
<li>I also did a check for missing country/region mappings with csv-metadata-quality</li>
|
|
</ul>
|
|
</li>
|
|
<li>Start a harvest on AReS</li>
|
|
</ul>
|
|
<a href='https://alanorth.github.io/cgspace-notes/2023-04/'>Read more →</a>
|
|
</article>
|
|
|
|
|
|
|
|
|
|
|
|
<nav class="blog-pagination">
|
|
|
|
|
|
|
|
<a class="btn btn-outline-primary disabled" href="#" role="button" aria-disabled="true">Previous page</a>
|
|
<a class="btn btn-outline-primary" href="/cgspace-notes/categories/notes/page/2/" rel="next" role="button">Next page</a>
|
|
|
|
</nav>
|
|
|
|
|
|
|
|
|
|
|
|
</div> <!-- /.blog-main -->
|
|
|
|
<aside class="col-sm-3 ml-auto blog-sidebar">
|
|
|
|
|
|
|
|
<section class="sidebar-module">
|
|
<h4>Recent Posts</h4>
|
|
<ol class="list-unstyled">
|
|
|
|
|
|
<li><a href="/cgspace-notes/2024-01/">January, 2024</a></li>
|
|
|
|
<li><a href="/cgspace-notes/2023-12/">December, 2023</a></li>
|
|
|
|
<li><a href="/cgspace-notes/2023-11/">November, 2023</a></li>
|
|
|
|
<li><a href="/cgspace-notes/2023-10/">October, 2023</a></li>
|
|
|
|
<li><a href="/cgspace-notes/2023-09/">September, 2023</a></li>
|
|
|
|
</ol>
|
|
</section>
|
|
|
|
|
|
|
|
|
|
<section class="sidebar-module">
|
|
<h4>Links</h4>
|
|
<ol class="list-unstyled">
|
|
|
|
<li><a href="https://cgspace.cgiar.org">CGSpace</a></li>
|
|
|
|
<li><a href="https://dspacetest.cgiar.org">DSpace Test</a></li>
|
|
|
|
<li><a href="https://github.com/ilri/DSpace">CGSpace @ GitHub</a></li>
|
|
|
|
</ol>
|
|
</section>
|
|
|
|
</aside>
|
|
|
|
|
|
</div> <!-- /.row -->
|
|
</div> <!-- /.container -->
|
|
|
|
|
|
|
|
<footer class="blog-footer">
|
|
<p dir="auto">
|
|
|
|
Blog template created by <a href="https://twitter.com/mdo">@mdo</a>, ported to Hugo by <a href='https://twitter.com/mralanorth'>@mralanorth</a>.
|
|
|
|
</p>
|
|
<p>
|
|
<a href="#">Back to top</a>
|
|
</p>
|
|
</footer>
|
|
|
|
|
|
</body>
|
|
|
|
</html>
|