mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-22 06:35:03 +01:00
Add notes for 2020-10-30
This commit is contained in:
parent
272cc0cf58
commit
1e04a7da72
GPG Key ID:
0FB860CC9C45B1B9
@ -935,4 +935,36 @@ $ dspace community-filiator --set --parent 10568/83389 --child 10568/1208
|
||||
$ dspace community-filiator --set --parent 10568/83389 --child 10568/56924
|
||||
```
|
||||
|
||||
## 2020-10-30
|
||||
|
||||
- The `AtomicStatisticsUpdateCLI` process finished on the current DSpace Test statistics core after about 32 hours
|
||||
- I started it on the statistics-2019 core
|
||||
- Atmire responded about the duplicate values in Solr that I had asked about a few days ago
|
||||
- They said it could be due to the schema and asked if I see it only on old records or even on new ones created in the new CUA with DSpace 6
|
||||
- I did a test and found that I got duplicate data after browsing for a minute on DSpace Test (version 6) and sent them a screenshot
|
||||
- Looking over Peter's corrections to journal titles (dc.source) and publishers (dc.publisher)
|
||||
- I had to check the corrections for strange Unicode errors and replacements with "|" and ";" in OpenRefine using this GREL:
|
||||
|
||||
```
|
||||
or(
|
||||
isNotNull(value.match(/.*\uFFFD.*/)),
|
||||
isNotNull(value.match(/.*\u00A0.*/)),
|
||||
isNotNull(value.match(/.*\u200A.*/)),
|
||||
isNotNull(value.match(/.*\u2019.*/)),
|
||||
isNotNull(value.match(/.*\u00b4.*/)),
|
||||
isNotNull(value.match(/.*\u007e.*/))
|
||||
).toString()
|
||||
```
|
||||
|
||||
- Then I did a test to apply the corrections and deletions on my local DSpace:
|
||||
|
||||
```
|
||||
$ ./fix-metadata-values.py -i 2020-10-30-fix-854-journals.csv -db dspace -u dspace -p 'fuuu' -f dc.source -t 'correct' -m 55
|
||||
$ ./delete-metadata-values.py -i 2020-10-30-delete-90-journals.csv -db dspace -u dspace -p 'fuuu' -f dc.source -m 55
|
||||
$ ./fix-metadata-values.py -i 2020-10-30-fix-386-publishers.csv -db dspace -u dspace -p 'fuuu' -f dc.publisher -t correct -m 39
|
||||
$ ./delete-metadata-values.py -i 2020-10-30-delete-10-publishers.csv -db dspace -u dspace -p 'fuuu' -f dc.publisher -m 39
|
||||
```
|
||||
|
||||
- I will wait to apply them on CGSpace when I have all the other corrections from Peter processed
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
@ -23,7 +23,7 @@ During the FlywayDB migration I got an error:
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2020-10/" />
|
||||
<meta property="article:published_time" content="2020-10-06T16:55:54+03:00" />
|
||||
<meta property="article:modified_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="article:modified_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="October, 2020"/>
|
||||
@ -51,9 +51,9 @@ During the FlywayDB migration I got an error:
|
||||
"@type": "BlogPosting",
|
||||
"headline": "October, 2020",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2020-10/",
|
||||
"wordCount": "6354",
|
||||
"wordCount": "6585",
|
||||
"datePublished": "2020-10-06T16:55:54+03:00",
|
||||
"dateModified": "2020-10-28T17:53:19+03:00",
|
||||
"dateModified": "2020-10-29T00:05:25+03:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -1165,7 +1165,44 @@ dspace=# COMMIT;
|
||||
</ul>
|
||||
<pre><code>$ dspace community-filiator --set --parent 10568/83389 --child 10568/1208
|
||||
$ dspace community-filiator --set --parent 10568/83389 --child 10568/56924
|
||||
</code></pre><!-- raw HTML omitted -->
|
||||
</code></pre><h2 id="2020-10-30">2020-10-30</h2>
|
||||
<ul>
|
||||
<li>The <code>AtomicStatisticsUpdateCLI</code> process finished on the current DSpace Test statistics core after about 32 hours
|
||||
<ul>
|
||||
<li>I started it on the statistics-2019 core</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Atmire responded about the duplicate values in Solr that I had asked about a few days ago
|
||||
<ul>
|
||||
<li>They said it could be due to the schema and asked if I see it only on old records or even on new ones created in the new CUA with DSpace 6</li>
|
||||
<li>I did a test and found that I got duplicate data after browsing for a minute on DSpace Test (version 6) and sent them a screenshot</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Looking over Peter’s corrections to journal titles (dc.source) and publishers (dc.publisher)
|
||||
<ul>
|
||||
<li>I had to check the corrections for strange Unicode errors and replacements with “|” and “;” in OpenRefine using this GREL:</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
<pre><code>or(
|
||||
isNotNull(value.match(/.*\uFFFD.*/)),
|
||||
isNotNull(value.match(/.*\u00A0.*/)),
|
||||
isNotNull(value.match(/.*\u200A.*/)),
|
||||
isNotNull(value.match(/.*\u2019.*/)),
|
||||
isNotNull(value.match(/.*\u00b4.*/)),
|
||||
isNotNull(value.match(/.*\u007e.*/))
|
||||
).toString()
|
||||
</code></pre><ul>
|
||||
<li>Then I did a test to apply the corrections and deletions on my local DSpace:</li>
|
||||
</ul>
|
||||
<pre><code>$ ./fix-metadata-values.py -i 2020-10-30-fix-854-journals.csv -db dspace -u dspace -p 'fuuu' -f dc.source -t 'correct' -m 55
|
||||
$ ./delete-metadata-values.py -i 2020-10-30-delete-90-journals.csv -db dspace -u dspace -p 'fuuu' -f dc.source -m 55
|
||||
$ ./fix-metadata-values.py -i 2020-10-30-fix-386-publishers.csv -db dspace -u dspace -p 'fuuu' -f dc.publisher -t correct -m 39
|
||||
$ ./delete-metadata-values.py -i 2020-10-30-delete-10-publishers.csv -db dspace -u dspace -p 'fuuu' -f dc.publisher -m 39
|
||||
</code></pre><ul>
|
||||
<li>I will wait to apply them on CGSpace when I have all the other corrections from Peter processed</li>
|
||||
</ul>
|
||||
<!-- raw HTML omitted -->
|
||||
|
||||
|
||||
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Categories"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/categories/notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
<meta property="og:description" content="Documenting day-to-day work on the [CGSpace](https://cgspace.cgiar.org) repository." />
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
<meta property="og:updated_time" content="2020-10-28T17:53:19+03:00" />
|
||||
<meta property="og:updated_time" content="2020-10-29T00:05:25+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
|
||||
@ -4,27 +4,27 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/</loc>
|
||||
<lastmod>2020-10-28T17:53:19+03:00</lastmod>
|
||||
<lastmod>2020-10-29T00:05:25+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2020-10-28T17:53:19+03:00</lastmod>
|
||||
<lastmod>2020-10-29T00:05:25+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
|
||||
<lastmod>2020-10-28T17:53:19+03:00</lastmod>
|
||||
<lastmod>2020-10-29T00:05:25+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2020-10/</loc>
|
||||
<lastmod>2020-10-28T17:53:19+03:00</lastmod>
|
||||
<lastmod>2020-10-29T00:05:25+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||
<lastmod>2020-10-28T17:53:19+03:00</lastmod>
|
||||
<lastmod>2020-10-29T00:05:25+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
|
||||
Loading…
Reference in New Issue
Block a user