mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-12-23 21:44:30 +01:00
Update notes
This commit is contained in:
parent
b8302741d7
commit
e9b8b7af44
GPG Key ID:
0FB860CC9C45B1B9
@ -30,3 +30,34 @@ v_oai.value 0
|
||||
|
||||
- Bram from Atmire responded about the high load caused by the Solr updater script and said it will be fixed with the updates to DSpace 5.8 compatibility: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=566
|
||||
- We will close that ticket for now and wait for the 5.8 stuff: https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560
|
||||
- I finally took a look at the second round of cleanups Peter had sent me for author affiliations in mid January
|
||||
- After trimming whitespace and quickly scanning for encoding errors I applied them on CGSpace:
|
||||
|
||||
```
|
||||
$ ./delete-metadata-values.py -i /tmp/2018-02-03-Affiliations-12-deletions.csv -f cg.contributor.affiliation -m 211 -d dspace -u dspace -p 'fuuu'
|
||||
$ ./fix-metadata-values.py -i /tmp/2018-02-03-Affiliations-1116-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p 'fuuu'
|
||||
```
|
||||
|
||||
- Then I started a full Discovery reindex:
|
||||
|
||||
```
|
||||
$ time schedtool -D -e ionice -c2 -n7 nice -n19 [dspace]/bin/dspace index-discovery -b
|
||||
```
|
||||
|
||||
- Generate a new list of affiliations for Peter to sort through:
|
||||
|
||||
```
|
||||
dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element = 'contributor' and qualifier = 'affiliation') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
|
||||
COPY 3723
|
||||
```
|
||||
|
||||
- Oh, and it looks like we processed over 3.1 million requests in January, up from 2.9 million in [December]({{< relref "2017-12.md" >}}):
|
||||
|
||||
```
|
||||
# time zcat --force /var/log/nginx/* | grep -cE "[0-9]{1,2}/Jan/2018"
|
||||
3126109
|
||||
|
||||
real 0m23.839s
|
||||
user 0m27.225s
|
||||
sys 0m1.905s
|
||||
```
|
||||
|
||||
@ -23,7 +23,7 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl
|
||||
|
||||
<meta property="article:published_time" content="2018-02-01T16:28:54+02:00"/>
|
||||
|
||||
<meta property="article:modified_time" content="2018-02-01T19:04:07+02:00"/>
|
||||
<meta property="article:modified_time" content="2018-02-03T09:44:31+02:00"/>
|
||||
|
||||
|
||||
|
||||
@ -57,9 +57,9 @@ I copied the logic in the jmx_tomcat_dbpools provided by Ubuntu’s munin-pl
|
||||
"@type": "BlogPosting",
|
||||
"headline": "February, 2018",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2018-02/",
|
||||
"wordCount": "149",
|
||||
"wordCount": "322",
|
||||
"datePublished": "2018-02-01T16:28:54+02:00",
|
||||
"dateModified": "2018-02-01T19:04:07+02:00",
|
||||
"dateModified": "2018-02-03T09:44:31+02:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -156,8 +156,41 @@ v_oai.value 0
|
||||
<ul>
|
||||
<li>Bram from Atmire responded about the high load caused by the Solr updater script and said it will be fixed with the updates to DSpace 5.8 compatibility: <a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=566">https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=566</a></li>
|
||||
<li>We will close that ticket for now and wait for the 5.8 stuff: <a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560">https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560</a></li>
|
||||
<li>I finally took a look at the second round of cleanups Peter had sent me for author affiliations in mid January</li>
|
||||
<li>After trimming whitespace and quickly scanning for encoding errors I applied them on CGSpace:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>$ ./delete-metadata-values.py -i /tmp/2018-02-03-Affiliations-12-deletions.csv -f cg.contributor.affiliation -m 211 -d dspace -u dspace -p 'fuuu'
|
||||
$ ./fix-metadata-values.py -i /tmp/2018-02-03-Affiliations-1116-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p 'fuuu'
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Then I started a full Discovery reindex:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>$ time schedtool -D -e ionice -c2 -n7 nice -n19 [dspace]/bin/dspace index-discovery -b
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Generate a new list of affiliations for Peter to sort through:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element = 'contributor' and qualifier = 'affiliation') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
|
||||
COPY 3723
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Oh, and it looks like we processed over 3.1 million requests in January, up from 2.9 million in <a href="/cgspace-notes/2017-12/">December</a>:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code># time zcat --force /var/log/nginx/* | grep -cE "[0-9]{1,2}/Jan/2018"
|
||||
3126109
|
||||
|
||||
real 0m23.839s
|
||||
user 0m27.225s
|
||||
sys 0m1.905s
|
||||
</code></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@ -32,7 +32,7 @@ Disallow: /cgspace-notes/2015-12/
|
||||
Disallow: /cgspace-notes/2015-11/
|
||||
Disallow: /cgspace-notes/
|
||||
Disallow: /cgspace-notes/categories/
|
||||
Disallow: /cgspace-notes/tags/notes/
|
||||
Disallow: /cgspace-notes/categories/notes/
|
||||
Disallow: /cgspace-notes/tags/notes/
|
||||
Disallow: /cgspace-notes/post/
|
||||
Disallow: /cgspace-notes/tags/
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2018-02/</loc>
|
||||
<lastmod>2018-02-01T19:04:07+02:00</lastmod>
|
||||
<lastmod>2018-02-03T09:44:31+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
@ -149,7 +149,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2018-02-01T19:04:07+02:00</lastmod>
|
||||
<lastmod>2018-02-03T09:44:31+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -158,27 +158,27 @@
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2018-02-01T19:04:07+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/categories/notes/</loc>
|
||||
<lastmod>2017-09-28T12:00:49+03:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2018-02-03T09:44:31+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/post/</loc>
|
||||
<lastmod>2018-02-01T19:04:07+02:00</lastmod>
|
||||
<lastmod>2018-02-03T09:44:31+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||
<lastmod>2018-02-01T19:04:07+02:00</lastmod>
|
||||
<lastmod>2018-02-03T09:44:31+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user