mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-22 06:35:03 +01:00
Add notes for 2019-09-01
This commit is contained in:
parent
7203ff6798
commit
cd8f3f26de
GPG Key ID:
0FB860CC9C45B1B9
66
content/posts/2019-09.md
Normal file
66
content/posts/2019-09.md
Normal file
@ -0,0 +1,66 @@
|
||||
---
|
||||
title: "September, 2019"
|
||||
date: 2019-09-01T10:17:51+03:00
|
||||
author: "Alan Orth"
|
||||
tags: ["Notes"]
|
||||
---
|
||||
|
||||
## 2019-09-01
|
||||
|
||||
- Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning
|
||||
- Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:
|
||||
|
||||
```
|
||||
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
```
|
||||
|
||||
<!--more-->
|
||||
|
||||
- `3.94.211.189` is MauiBot, and most of its requests are to Discovery and get rate limited with HTTP 503
|
||||
- `163.172.71.23` is some IP on Online SAS in France and its user agent is:
|
||||
|
||||
```
|
||||
Mozilla/5.0 ((Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6)
|
||||
```
|
||||
|
||||
- It actually got mostly HTTP 200 responses:
|
||||
|
||||
```
|
||||
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | grep 163.172.71.23 | awk '{print $9}' | sort | uniq -c
|
||||
1775 200
|
||||
703 499
|
||||
72 503
|
||||
```
|
||||
|
||||
- And it was mostly requesting Discover pages:
|
||||
|
||||
```
|
||||
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | grep 163.172.71.23 | grep -o -E "(bitstream|discover|handle)" | sort | uniq -c
|
||||
2350 discover
|
||||
71 handle
|
||||
```
|
||||
|
||||
- I'm not sure why the outbound traffic rate was so high...
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
@ -278,16 +278,16 @@ db.statementpool = true
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -294,16 +294,16 @@ $ curl -o /dev/null -s -w %{time_total}\\n https://cgspace.cgiar.org/rest/handle
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -210,16 +210,16 @@ $ find SimpleArchiveForBio/ -iname “*.pdf” -exec basename {} \; | so
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -455,16 +455,16 @@ Bitstream: tést señora alimentación.pdf
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -361,16 +361,16 @@ metadata_value_id | resource_id | metadata_field_id | text_value | text_lang | p
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -577,16 +577,16 @@ dspace.log.2016-04-27:7271
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -435,16 +435,16 @@ sys 0m20.540s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -470,16 +470,16 @@ $ ./delete-metadata-values.py -f dc.contributor.corporate -i Corporate-Authors-D
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -381,16 +381,16 @@ discovery.index.authority.ignore-variants=true
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -460,16 +460,16 @@ $ JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx512m" /home/cgspace.cgiar.org/b
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -734,16 +734,16 @@ $ ./delete-metadata-values.py -i ilrisubjects-delete-13.csv -f cg.subject.ilri -
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -439,16 +439,16 @@ dspace=# update metadatavalue set text_value = regexp_replace(text_value, 'http:
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -640,16 +640,16 @@ org.dspace.discovery.SearchServiceException: Error executing query
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -907,16 +907,16 @@ $ exit
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -432,16 +432,16 @@ $ gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook -dNOPAUSE -
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -497,16 +497,16 @@ COPY 1968
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -434,16 +434,16 @@ $ ./delete-metadata-values.py -i Investors-Delete-121.csv -f dc.description.spon
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -702,16 +702,16 @@ $ gem install compass -v 1.0.3
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -462,16 +462,16 @@ UPDATE 187
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -295,16 +295,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -315,16 +315,16 @@ delete from metadatavalue where resource_type_id=2 and metadata_field_id=235 and
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -611,16 +611,16 @@ org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -781,16 +781,16 @@ Cert Status: good
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -547,16 +547,16 @@ session_id=6C30F10B4351A4ED83EC6ED50AFD6B6A
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1157,16 +1157,16 @@ $ cat dspace.log.2017-11-28 | grep -o -E 'session_id=[A-Z0-9]{32}' | sort -n | u
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -938,16 +938,16 @@ DELETE 20
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1732,16 +1732,16 @@ Catalina:type=Manager,context=/,host=localhost activeSessions 8
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1286,16 +1286,16 @@ UPDATE 3
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -712,16 +712,16 @@ Fixed 5 occurences of: GENEBANKS
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -720,16 +720,16 @@ java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -615,16 +615,16 @@ $ psql -h localhost -U postgres dspacetest
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -583,16 +583,16 @@ $ sed '/^id/d' 10568-*.csv | csvcut -c 1,2 > map-to-cifor-archive.csv
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -674,16 +674,16 @@ count
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -547,16 +547,16 @@ $ dspace database migrate ignored
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -909,16 +909,16 @@ UPDATE metadatavalue SET text_value='ja' WHERE resource_type_id=2 AND metadata_f
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -796,16 +796,16 @@ $ curl -X GET -H "Content-Type: application/json" -H "Accept: app
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -651,16 +651,16 @@ $ dspace dsrun org.dspace.eperson.Groomer -a -b 11/27/2016 -d
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -684,16 +684,16 @@ UPDATE 1
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1397,16 +1397,16 @@ identify: CorruptImageProfile `xmp' @ warning/profile.c/SetImageProfileInternal/
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1588,16 +1588,16 @@ Please see the DSpace documentation for assistance.
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1380,16 +1380,16 @@ sys 0m2.551s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1439,16 +1439,16 @@ UPDATE 14
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -728,16 +728,16 @@ COPY 64871
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -341,16 +341,16 @@ UPDATE 2
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -624,16 +624,16 @@ issn.validate('1020-3362')
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -27,7 +27,7 @@ Run system updates on DSpace Test (linode19) and reboot it
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-08/" />
|
||||
<meta property="article:published_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="article:modified_time" content="2019-08-29T19:25:06+03:00" />
|
||||
<meta property="article:modified_time" content="2019-09-01T01:54:55+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="August, 2019"/>
|
||||
@ -61,7 +61,7 @@ Run system updates on DSpace Test (linode19) and reboot it
|
||||
"url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-08\/",
|
||||
"wordCount": "2703",
|
||||
"datePublished": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-08-29T19:25:06\x2b03:00",
|
||||
"dateModified": "2019-09-01T01:54:55\x2b03:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -626,16 +626,16 @@ sys 2m27.496s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
287
docs/2019-09/index.html
Normal file
287
docs/2019-09/index.html
Normal file
@ -0,0 +1,287 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
|
||||
|
||||
<meta property="og:title" content="September, 2019" />
|
||||
<meta property="og:description" content="2019-09-01
|
||||
|
||||
|
||||
Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning
|
||||
|
||||
Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:
|
||||
|
||||
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
|
||||
" />
|
||||
<meta property="og:type" content="article" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2019-09/" />
|
||||
<meta property="article:published_time" content="2019-09-01T10:17:51+03:00" />
|
||||
<meta property="article:modified_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="September, 2019"/>
|
||||
<meta name="twitter:description" content="2019-09-01
|
||||
|
||||
|
||||
Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning
|
||||
|
||||
Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:
|
||||
|
||||
# zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
|
||||
"/>
|
||||
<meta name="generator" content="Hugo 0.57.2" />
|
||||
|
||||
|
||||
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "http://schema.org",
|
||||
"@type": "BlogPosting",
|
||||
"headline": "September, 2019",
|
||||
"url": "https:\/\/alanorth.github.io\/cgspace-notes\/2019-09\/",
|
||||
"wordCount": "247",
|
||||
"datePublished": "2019-09-01T10:17:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"keywords": "Notes"
|
||||
}
|
||||
</script>
|
||||
|
||||
|
||||
|
||||
<link rel="canonical" href="https://alanorth.github.io/cgspace-notes/2019-09/">
|
||||
|
||||
<title>September, 2019 | CGSpace Notes</title>
|
||||
|
||||
<!-- combined, minified CSS -->
|
||||
<link href="https://alanorth.github.io/cgspace-notes/css/style.css" rel="stylesheet" integrity="sha384-G5B34w7DFTumWTswxYzTX7NWfbvQEg1HbFFEg6ItN03uTAAoS2qkPS/fu3LhuuSA" crossorigin="anonymous">
|
||||
|
||||
<!-- RSS 2.0 feed -->
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body>
|
||||
|
||||
|
||||
<div class="blog-masthead">
|
||||
<div class="container">
|
||||
<nav class="nav blog-nav">
|
||||
<a class="nav-link " href="https://alanorth.github.io/cgspace-notes/">Home</a>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
<header class="blog-header">
|
||||
<div class="container">
|
||||
<h1 class="blog-title"><a href="https://alanorth.github.io/cgspace-notes/" rel="home">CGSpace Notes</a></h1>
|
||||
<p class="lead blog-description">Documenting day-to-day work on the <a href="https://cgspace.cgiar.org">CGSpace</a> repository.</p>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
|
||||
|
||||
|
||||
<div class="container">
|
||||
<div class="row">
|
||||
<div class="col-sm-8 blog-main">
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
|
||||
<ul>
|
||||
<li><code>3.94.211.189</code> is MauiBot, and most of its requests are to Discovery and get rate limited with HTTP 503</li>
|
||||
|
||||
<li><p><code>163.172.71.23</code> is some IP on Online SAS in France and its user agent is:</p>
|
||||
|
||||
<pre><code>Mozilla/5.0 ((Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6)
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>It actually got mostly HTTP 200 responses:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | grep 163.172.71.23 | awk '{print $9}' | sort | uniq -c
|
||||
1775 200
|
||||
703 499
|
||||
72 503
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And it was mostly requesting Discover pages:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | grep 163.172.71.23 | grep -o -E "(bitstream|discover|handle)" | sort | uniq -c
|
||||
2350 discover
|
||||
71 handle
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>I’m not sure why the outbound traffic rate was so high…</p></li>
|
||||
</ul>
|
||||
|
||||
<!-- vim: set sw=2 ts=2: -->
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
</div> <!-- /.blog-main -->
|
||||
|
||||
<aside class="col-sm-3 ml-auto blog-sidebar">
|
||||
|
||||
|
||||
|
||||
<section class="sidebar-module">
|
||||
<h4>Recent Posts</h4>
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
|
||||
|
||||
<section class="sidebar-module">
|
||||
<h4>Links</h4>
|
||||
<ol class="list-unstyled">
|
||||
|
||||
<li><a href="https://cgspace.cgiar.org">CGSpace</a></li>
|
||||
|
||||
<li><a href="https://dspacetest.cgiar.org">DSpace Test</a></li>
|
||||
|
||||
<li><a href="https://github.com/ilri/DSpace">CGSpace @ GitHub</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
</aside>
|
||||
|
||||
|
||||
</div> <!-- /.row -->
|
||||
</div> <!-- /.container -->
|
||||
|
||||
|
||||
|
||||
<footer class="blog-footer">
|
||||
<p>
|
||||
|
||||
Blog template created by <a href="https://twitter.com/mdo">@mdo</a>, ported to Hugo by <a href='https://twitter.com/mralanorth'>@mralanorth</a>.
|
||||
|
||||
</p>
|
||||
<p>
|
||||
<a href="#">Back to top</a>
|
||||
</p>
|
||||
</footer>
|
||||
|
||||
|
||||
</body>
|
||||
|
||||
</html>
|
||||
@ -87,16 +87,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -89,6 +89,54 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2019-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-08/">August, 2019</a></h2>
|
||||
@ -414,36 +462,6 @@ sys 0m1.979s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
|
||||
@ -468,16 +486,16 @@ sys 0m1.979s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -105,16 +105,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -89,6 +89,36 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
@ -347,98 +377,6 @@ sys 2m7.289s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/categories/" rel="prev" role="button">Previous page</a>
|
||||
@ -463,16 +401,16 @@ dspace.log.2018-01-02:34
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -89,6 +89,98 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-12/">December, 2017</a></h2>
|
||||
@ -332,40 +424,6 @@ COPY 54701
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/categories/page/2/" rel="prev" role="button">Previous page</a>
|
||||
@ -390,16 +448,16 @@ COPY 54701
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -89,6 +89,40 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
@ -380,33 +414,6 @@ text_value
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/categories/page/3/" rel="prev" role="button">Previous page</a>
|
||||
@ -431,16 +438,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -89,6 +89,33 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
@ -304,16 +331,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -325,16 +325,16 @@ dspace=# select setval('handle_seq',86873);
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,54 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2019-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-08/">August, 2019</a></h2>
|
||||
@ -414,36 +462,6 @@ sys 0m1.979s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
|
||||
@ -468,16 +486,16 @@ sys 0m1.979s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -6,11 +6,50 @@
|
||||
<description>Recent content on CGSpace Notes</description>
|
||||
<generator>Hugo -- gohugo.io</generator>
|
||||
<language>en-us</language>
|
||||
<lastBuildDate>Sat, 03 Aug 2019 12:39:51 +0300</lastBuildDate>
|
||||
<lastBuildDate>Sun, 01 Sep 2019 10:17:51 +0300</lastBuildDate>
|
||||
|
||||
<atom:link href="https://alanorth.github.io/cgspace-notes/index.xml" rel="self" type="application/rss+xml" />
|
||||
|
||||
|
||||
<item>
|
||||
<title>September, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-09/</link>
|
||||
<pubDate>Sun, 01 Sep 2019 10:17:51 +0300</pubDate>
|
||||
|
||||
<guid>https://alanorth.github.io/cgspace-notes/2019-09/</guid>
|
||||
<description><h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
<title>August, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-08/</link>
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,36 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
@ -347,98 +377,6 @@ sys 2m7.289s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/" rel="prev" role="button">Previous page</a>
|
||||
@ -463,16 +401,16 @@ dspace.log.2018-01-02:34
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,98 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-12/">December, 2017</a></h2>
|
||||
@ -332,40 +424,6 @@ COPY 54701
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/page/2/" rel="prev" role="button">Previous page</a>
|
||||
@ -390,16 +448,16 @@ COPY 54701
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,40 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
@ -380,33 +414,6 @@ text_value
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/page/3/" rel="prev" role="button">Previous page</a>
|
||||
@ -431,16 +438,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="CGSpace Notes"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,33 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
@ -304,16 +331,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,54 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2019-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-08/">August, 2019</a></h2>
|
||||
@ -414,36 +462,6 @@ sys 0m1.979s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
|
||||
@ -468,16 +486,16 @@ sys 0m1.979s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -6,11 +6,50 @@
|
||||
<description>Recent content in Posts on CGSpace Notes</description>
|
||||
<generator>Hugo -- gohugo.io</generator>
|
||||
<language>en-us</language>
|
||||
<lastBuildDate>Sat, 03 Aug 2019 12:39:51 +0300</lastBuildDate>
|
||||
<lastBuildDate>Sun, 01 Sep 2019 10:17:51 +0300</lastBuildDate>
|
||||
|
||||
<atom:link href="https://alanorth.github.io/cgspace-notes/posts/index.xml" rel="self" type="application/rss+xml" />
|
||||
|
||||
|
||||
<item>
|
||||
<title>September, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-09/</link>
|
||||
<pubDate>Sun, 01 Sep 2019 10:17:51 +0300</pubDate>
|
||||
|
||||
<guid>https://alanorth.github.io/cgspace-notes/2019-09/</guid>
|
||||
<description><h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
<title>August, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-08/</link>
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,36 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
@ -347,98 +377,6 @@ sys 2m7.289s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/posts/" rel="prev" role="button">Previous page</a>
|
||||
@ -463,16 +401,16 @@ dspace.log.2018-01-02:34
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,98 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-12/">December, 2017</a></h2>
|
||||
@ -332,40 +424,6 @@ COPY 54701
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/posts/page/2/" rel="prev" role="button">Previous page</a>
|
||||
@ -390,16 +448,16 @@ COPY 54701
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,40 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
@ -380,33 +414,6 @@ text_value
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/posts/page/3/" rel="prev" role="button">Previous page</a>
|
||||
@ -431,16 +438,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/posts/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Posts"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,33 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
@ -304,16 +331,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -1,11 +1,12 @@
|
||||
User-agent: *
|
||||
|
||||
|
||||
Disallow: /cgspace-notes/2019-08/
|
||||
Disallow: /cgspace-notes/
|
||||
Disallow: /cgspace-notes/tags/notes/
|
||||
Disallow: /cgspace-notes/posts/
|
||||
Disallow: /cgspace-notes/2019-09/
|
||||
Disallow: /cgspace-notes/tags/
|
||||
Disallow: /cgspace-notes/2019-08/
|
||||
Disallow: /cgspace-notes/2019-07/
|
||||
Disallow: /cgspace-notes/2019-06/
|
||||
Disallow: /cgspace-notes/2019-05/
|
||||
|
||||
@ -2,29 +2,34 @@
|
||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
|
||||
xmlns:xhtml="http://www.w3.org/1999/xhtml">
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2019-08/</loc>
|
||||
<lastmod>2019-08-29T19:25:06+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2019-08-29T19:25:06+03:00</lastmod>
|
||||
<lastmod>2019-09-01T10:17:51+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2019-08-29T19:25:06+03:00</lastmod>
|
||||
<lastmod>2019-09-01T10:17:51+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||
<lastmod>2019-08-29T19:25:06+03:00</lastmod>
|
||||
<lastmod>2019-09-01T10:17:51+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2019-09/</loc>
|
||||
<lastmod>2019-09-01T10:17:51+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||
<lastmod>2019-08-29T19:25:06+03:00</lastmod>
|
||||
<lastmod>2019-09-01T10:17:51+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2019-08/</loc>
|
||||
<lastmod>2019-09-01T01:54:55+03:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Tags"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,54 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2019-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-08/">August, 2019</a></h2>
|
||||
@ -414,36 +462,6 @@ sys 0m1.979s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
|
||||
@ -468,16 +486,16 @@ sys 0m1.979s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
<description>Recent content in Tags on CGSpace Notes</description>
|
||||
<generator>Hugo -- gohugo.io</generator>
|
||||
<language>en-us</language>
|
||||
<lastBuildDate>Sat, 03 Aug 2019 12:39:51 +0300</lastBuildDate>
|
||||
<lastBuildDate>Sun, 01 Sep 2019 10:17:51 +0300</lastBuildDate>
|
||||
|
||||
<atom:link href="https://alanorth.github.io/cgspace-notes/tags/index.xml" rel="self" type="application/rss+xml" />
|
||||
|
||||
@ -14,7 +14,7 @@
|
||||
<item>
|
||||
<title>Notes</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/tags/notes/</link>
|
||||
<pubDate>Sat, 03 Aug 2019 12:39:51 +0300</pubDate>
|
||||
<pubDate>Sun, 01 Sep 2019 10:17:51 +0300</pubDate>
|
||||
|
||||
<guid>https://alanorth.github.io/cgspace-notes/tags/notes/</guid>
|
||||
<description></description>
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
@ -74,6 +74,54 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-09/">September, 2019</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2019-09-01T10:17:51+03:00">Sun Sep 01, 2019</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E "01/Sep/2019:0" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2019-09/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2019-08/">August, 2019</a></h2>
|
||||
@ -399,36 +447,6 @@ sys 0m1.979s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
|
||||
@ -453,16 +471,16 @@ sys 0m1.979s
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -6,11 +6,50 @@
|
||||
<description>Recent content in Notes on CGSpace Notes</description>
|
||||
<generator>Hugo -- gohugo.io</generator>
|
||||
<language>en-us</language>
|
||||
<lastBuildDate>Sat, 03 Aug 2019 12:39:51 +0300</lastBuildDate>
|
||||
<lastBuildDate>Sun, 01 Sep 2019 10:17:51 +0300</lastBuildDate>
|
||||
|
||||
<atom:link href="https://alanorth.github.io/cgspace-notes/tags/notes/index.xml" rel="self" type="application/rss+xml" />
|
||||
|
||||
|
||||
<item>
|
||||
<title>September, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-09/</link>
|
||||
<pubDate>Sun, 01 Sep 2019 10:17:51 +0300</pubDate>
|
||||
|
||||
<guid>https://alanorth.github.io/cgspace-notes/2019-09/</guid>
|
||||
<description><h2 id="2019-09-01">2019-09-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode emailed to say that CGSpace (linode18) had a high rate of outbound traffic for several hours this morning</li>
|
||||
|
||||
<li><p>Here are the top ten IPs in the nginx XMLUI and REST/OAI logs this morning:</p>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log /var/log/nginx/access.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
440 17.58.101.255
|
||||
441 157.55.39.101
|
||||
485 207.46.13.43
|
||||
728 169.60.128.125
|
||||
730 207.46.13.108
|
||||
758 157.55.39.9
|
||||
808 66.160.140.179
|
||||
814 207.46.13.212
|
||||
2472 163.172.71.23
|
||||
6092 3.94.211.189
|
||||
# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 /var/log/nginx/oai.log /var/log/nginx/oai.log.1 | grep -E &quot;01/Sep/2019:0&quot; | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10
|
||||
33 2a01:7e00::f03c:91ff:fe16:fcb
|
||||
57 3.83.192.124
|
||||
57 3.87.77.25
|
||||
57 54.82.1.8
|
||||
822 2a01:9cc0:47:1:1a:4:0:2
|
||||
1223 45.5.184.72
|
||||
1633 172.104.229.92
|
||||
5112 205.186.128.185
|
||||
7249 2a01:7e00::f03c:91ff:fe18:7396
|
||||
9124 45.5.186.2
|
||||
</code></pre></li>
|
||||
</ul></description>
|
||||
</item>
|
||||
|
||||
<item>
|
||||
<title>August, 2019</title>
|
||||
<link>https://alanorth.github.io/cgspace-notes/2019-08/</link>
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
@ -74,6 +74,36 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
@ -332,98 +362,6 @@ sys 2m7.289s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/notes/" rel="prev" role="button">Previous page</a>
|
||||
@ -448,16 +386,16 @@ dspace.log.2018-01-02:34
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
@ -74,6 +74,98 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-12/">December, 2017</a></h2>
|
||||
@ -333,47 +425,6 @@ COPY 54701
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-03-01T17:08:52+02:00">Wed Mar 01, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-03-01">2017-03-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Run the 279 CIAT author corrections on CGSpace</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2017-03-02">2017-03-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Skype with Michael and Peter, discussing moving the CGIAR Library to CGSpace</li>
|
||||
<li>CGIAR people possibly open to moving content, redirecting library.cgiar.org to CGSpace and letting CGSpace resolve their handles</li>
|
||||
<li>They might come in at the top level in one “CGIAR System” community, or with several communities</li>
|
||||
<li>I need to spend a bit of time looking at the multiple handle support in DSpace and see if new content can be minted in both handles, or just one?</li>
|
||||
<li>Need to send Peter and Michael some notes about this in a few days</li>
|
||||
<li>Also, need to consider talking to Atmire about hiring them to bring ORCiD metadata to REST / OAI</li>
|
||||
<li>Filed an issue on DSpace issue tracker for the <code>filter-media</code> bug that causes it to process JPGs even when limiting to the PDF thumbnail plugin: <a href="https://jira.duraspace.org/browse/DS-3516">DS-3516</a></li>
|
||||
<li>Discovered that the ImageMagic <code>filter-media</code> plugin creates JPG thumbnails with the CMYK colorspace when the source PDF is using CMYK</li>
|
||||
|
||||
<li><p>Interestingly, it seems DSpace 4.x’s thumbnails were sRGB, but forcing regeneration using DSpace 5.x’s ImageMagick plugin creates CMYK JPGs if the source PDF was CMYK (see <a href="https://cgspace.cgiar.org/handle/10568/51999"><sup>10568</sup>⁄<sub>51999</sub></a>):</p>
|
||||
|
||||
<pre><code>$ identify ~/Desktop/alc_contrastes_desafios.jpg
|
||||
/Users/aorth/Desktop/alc_contrastes_desafios.jpg JPEG 464x600 464x600+0+0 8-bit CMYK 168KB 0.000u 0:00.000
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-03/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/notes/page/2/" rel="prev" role="button">Previous page</a>
|
||||
@ -398,16 +449,16 @@ COPY 54701
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
@ -74,6 +74,47 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-03-01T17:08:52+02:00">Wed Mar 01, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-03-01">2017-03-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Run the 279 CIAT author corrections on CGSpace</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2017-03-02">2017-03-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Skype with Michael and Peter, discussing moving the CGIAR Library to CGSpace</li>
|
||||
<li>CGIAR people possibly open to moving content, redirecting library.cgiar.org to CGSpace and letting CGSpace resolve their handles</li>
|
||||
<li>They might come in at the top level in one “CGIAR System” community, or with several communities</li>
|
||||
<li>I need to spend a bit of time looking at the multiple handle support in DSpace and see if new content can be minted in both handles, or just one?</li>
|
||||
<li>Need to send Peter and Michael some notes about this in a few days</li>
|
||||
<li>Also, need to consider talking to Atmire about hiring them to bring ORCiD metadata to REST / OAI</li>
|
||||
<li>Filed an issue on DSpace issue tracker for the <code>filter-media</code> bug that causes it to process JPGs even when limiting to the PDF thumbnail plugin: <a href="https://jira.duraspace.org/browse/DS-3516">DS-3516</a></li>
|
||||
<li>Discovered that the ImageMagic <code>filter-media</code> plugin creates JPG thumbnails with the CMYK colorspace when the source PDF is using CMYK</li>
|
||||
|
||||
<li><p>Interestingly, it seems DSpace 4.x’s thumbnails were sRGB, but forcing regeneration using DSpace 5.x’s ImageMagick plugin creates CMYK JPGs if the source PDF was CMYK (see <a href="https://cgspace.cgiar.org/handle/10568/51999"><sup>10568</sup>⁄<sub>51999</sub></a>):</p>
|
||||
|
||||
<pre><code>$ identify ~/Desktop/alc_contrastes_desafios.jpg
|
||||
/Users/aorth/Desktop/alc_contrastes_desafios.jpg JPEG 464x600 464x600+0+0 8-bit CMYK 168KB 0.000u 0:00.000
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-03/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-02/">February, 2017</a></h2>
|
||||
@ -351,35 +392,6 @@ text_value
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-05-01T23:06:00+03:00">Sun May 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-05-01">2016-05-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Since yesterday there have been 10,000 REST errors and the site has been unstable again</li>
|
||||
<li>I have blocked access to the API now</li>
|
||||
|
||||
<li><p>There are 3,000 IPs accessing the REST API in a 24-hour period!</p>
|
||||
|
||||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||||
3168
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-05/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/notes/page/3/" rel="prev" role="button">Previous page</a>
|
||||
@ -404,16 +416,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/notes/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Notes"/>
|
||||
@ -74,6 +74,35 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-05-01T23:06:00+03:00">Sun May 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-05-01">2016-05-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Since yesterday there have been 10,000 REST errors and the site has been unstable again</li>
|
||||
<li>I have blocked access to the API now</li>
|
||||
|
||||
<li><p>There are 3,000 IPs accessing the REST API in a 24-hour period!</p>
|
||||
|
||||
<pre><code># awk '{print $1}' /var/log/nginx/rest.log | uniq | wc -l
|
||||
3168
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-05/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-04/">April, 2016</a></h2>
|
||||
@ -260,16 +289,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Tags"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,36 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-11/">November, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-11-01T16:41:30+02:00">Thu Nov 01, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-11-01">2018-11-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Finalize AReS Phase I and Phase II ToRs</li>
|
||||
<li>Send a note about my <a href="https://github.com/ilri/dspace-statistics-api">dspace-statistics-api</a> to the dspace-tech mailing list</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-11-03">2018-11-03</h2>
|
||||
|
||||
<ul>
|
||||
<li>Linode has been sending mails a few times a day recently that CGSpace (linode18) has had high CPU usage</li>
|
||||
<li>Today these are the top 10 IPs:</li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-11/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-10/">October, 2018</a></h2>
|
||||
@ -347,98 +377,6 @@ sys 2m7.289s
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/" rel="prev" role="button">Previous page</a>
|
||||
@ -463,16 +401,16 @@ dspace.log.2018-01-02:34
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Tags"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,98 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2018-01/">January, 2018</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2018-01-02T08:35:54-08:00">Tue Jan 02, 2018</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2018-01-02">2018-01-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Uptime Robot noticed that CGSpace went down and up a few times last night, for a few minutes each time</li>
|
||||
<li>I didn’t get any load alerts from Linode and the REST and XMLUI logs don’t show anything out of the ordinary</li>
|
||||
<li>The nginx logs show HTTP 200s until <code>02/Jan/2018:11:27:17 +0000</code> when Uptime Robot got an HTTP 500</li>
|
||||
<li>In dspace.log around that time I see many errors like “Client closed the connection before file download was complete”</li>
|
||||
|
||||
<li><p>And just before that I see this:</p>
|
||||
|
||||
<pre><code>Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException: [http-bio-127.0.0.1-8443-exec-980] Timeout: Pool empty. Unable to fetch a connection in 5 seconds, none available[size:50; busy:50; idle:0; lastwait:5000].
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Ah hah! So the pool was actually empty!</p></li>
|
||||
|
||||
<li><p>I need to increase that, let’s try to bump it up from 50 to 75</p></li>
|
||||
|
||||
<li><p>After that one client got an HTTP 499 but then the rest were HTTP 200, so I don’t know what the hell Uptime Robot saw</p></li>
|
||||
|
||||
<li><p>I notice this error quite a few times in dspace.log:</p>
|
||||
|
||||
<pre><code>2018-01-02 01:21:19,137 ERROR org.dspace.app.xmlui.aspect.discovery.SidebarFacetsTransformer @ Error while searching for sidebar facets
|
||||
org.dspace.discovery.SearchServiceException: org.apache.solr.search.SyntaxError: Cannot parse 'dateIssued_keyword:[1976+TO+1979]': Encountered " "]" "] "" at line 1, column 32.
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>And there are many of these errors every day for the past month:</p>
|
||||
|
||||
<pre><code>$ grep -c "Error while searching for sidebar facets" dspace.log.*
|
||||
dspace.log.2017-11-21:4
|
||||
dspace.log.2017-11-22:1
|
||||
dspace.log.2017-11-23:4
|
||||
dspace.log.2017-11-24:11
|
||||
dspace.log.2017-11-25:0
|
||||
dspace.log.2017-11-26:1
|
||||
dspace.log.2017-11-27:7
|
||||
dspace.log.2017-11-28:21
|
||||
dspace.log.2017-11-29:31
|
||||
dspace.log.2017-11-30:15
|
||||
dspace.log.2017-12-01:15
|
||||
dspace.log.2017-12-02:20
|
||||
dspace.log.2017-12-03:38
|
||||
dspace.log.2017-12-04:65
|
||||
dspace.log.2017-12-05:43
|
||||
dspace.log.2017-12-06:72
|
||||
dspace.log.2017-12-07:27
|
||||
dspace.log.2017-12-08:15
|
||||
dspace.log.2017-12-09:29
|
||||
dspace.log.2017-12-10:35
|
||||
dspace.log.2017-12-11:20
|
||||
dspace.log.2017-12-12:44
|
||||
dspace.log.2017-12-13:36
|
||||
dspace.log.2017-12-14:59
|
||||
dspace.log.2017-12-15:104
|
||||
dspace.log.2017-12-16:53
|
||||
dspace.log.2017-12-17:66
|
||||
dspace.log.2017-12-18:83
|
||||
dspace.log.2017-12-19:101
|
||||
dspace.log.2017-12-20:74
|
||||
dspace.log.2017-12-21:55
|
||||
dspace.log.2017-12-22:66
|
||||
dspace.log.2017-12-23:50
|
||||
dspace.log.2017-12-24:85
|
||||
dspace.log.2017-12-25:62
|
||||
dspace.log.2017-12-26:49
|
||||
dspace.log.2017-12-27:30
|
||||
dspace.log.2017-12-28:54
|
||||
dspace.log.2017-12-29:68
|
||||
dspace.log.2017-12-30:89
|
||||
dspace.log.2017-12-31:53
|
||||
dspace.log.2018-01-01:45
|
||||
dspace.log.2018-01-02:34
|
||||
</code></pre></li>
|
||||
|
||||
<li><p>Danny wrote to ask for help renewing the wildcard ilri.org certificate and I advised that we should probably use Let’s Encrypt if it’s just a handful of domains</p></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2018-01/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-12/">December, 2017</a></h2>
|
||||
@ -332,40 +424,6 @@ COPY 54701
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/page/2/" rel="prev" role="button">Previous page</a>
|
||||
@ -390,16 +448,16 @@ COPY 54701
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Tags"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,40 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-04/">April, 2017</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2017-04-02T17:08:52+02:00">Sun Apr 02, 2017</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2017-04-02">2017-04-02</h2>
|
||||
|
||||
<ul>
|
||||
<li>Merge one change to CCAFS flagships that I had forgotten to remove last month (“MANAGING CLIMATE RISK”): <a href="https://github.com/ilri/DSpace/pull/317">https://github.com/ilri/DSpace/pull/317</a></li>
|
||||
<li>Quick proof-of-concept hack to add <code>dc.rights</code> to the input form, including some inline instructions/hints:</li>
|
||||
</ul>
|
||||
|
||||
<p><img src="/cgspace-notes/2017/04/dc-rights.png" alt="dc.rights in the submission form" /></p>
|
||||
|
||||
<ul>
|
||||
<li>Remove redundant/duplicate text in the DSpace submission license</li>
|
||||
|
||||
<li><p>Testing the CMYK patch on a collection with 650 items:</p>
|
||||
|
||||
<pre><code>$ [dspace]/bin/dspace filter-media -f -i 10568/16498 -p "ImageMagick PDF Thumbnail" -v >& /tmp/filter-media-cmyk.txt
|
||||
</code></pre></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2017-04/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2017-03/">March, 2017</a></h2>
|
||||
@ -380,33 +414,6 @@ text_value
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<nav class="blog-pagination">
|
||||
|
||||
<a class="btn btn-outline-primary" href="/cgspace-notes/tags/page/3/" rel="prev" role="button">Previous page</a>
|
||||
@ -431,16 +438,16 @@ text_value
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
@ -10,7 +10,7 @@
|
||||
<meta property="og:type" content="website" />
|
||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/tags/" />
|
||||
|
||||
<meta property="og:updated_time" content="2019-08-03T12:39:51+03:00" />
|
||||
<meta property="og:updated_time" content="2019-09-01T10:17:51+03:00" />
|
||||
|
||||
<meta name="twitter:card" content="summary"/>
|
||||
<meta name="twitter:title" content="Tags"/>
|
||||
@ -29,7 +29,7 @@
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
},
|
||||
"dateModified": "2019-08-03T12:39:51\x2b03:00",
|
||||
"dateModified": "2019-09-01T10:17:51\x2b03:00",
|
||||
"keywords": "notes,notes,",
|
||||
"description": "Documenting day-to-day work on the [CGSpace](https:\/\/cgspace.cgiar.org) repository."
|
||||
}
|
||||
@ -89,6 +89,33 @@
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-06/">June, 2016</a></h2>
|
||||
<p class="blog-post-meta"><time datetime="2016-06-01T10:53:00+03:00">Wed Jun 01, 2016</time> by Alan Orth in
|
||||
|
||||
<i class="fa fa-tag" aria-hidden="true"></i> <a href="/cgspace-notes/tags/notes" rel="tag">Notes</a>
|
||||
|
||||
</p>
|
||||
</header>
|
||||
<h2 id="2016-06-01">2016-06-01</h2>
|
||||
|
||||
<ul>
|
||||
<li>Experimenting with IFPRI OAI (we want to harvest their publications)</li>
|
||||
<li>After reading the <a href="https://www.oclc.org/support/services/contentdm/help/server-admin-help/oai-support.en.html">ContentDM documentation</a> I found IFPRI’s OAI endpoint: <a href="http://ebrary.ifpri.org/oai/oai.php">http://ebrary.ifpri.org/oai/oai.php</a></li>
|
||||
<li>After reading the <a href="https://www.openarchives.org/OAI/openarchivesprotocol.html">OAI documentation</a> and testing with an <a href="http://validator.oaipmh.com/">OAI validator</a> I found out how to get their publications</li>
|
||||
<li>This is their publications set: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc">http://ebrary.ifpri.org/oai/oai.php?verb=ListRecords&from=2016-01-01&set=p15738coll2&metadataPrefix=oai_dc</a></li>
|
||||
<li>You can see the others by using the OAI <code>ListSets</code> verb: <a href="http://ebrary.ifpri.org/oai/oai.php?verb=ListSets">http://ebrary.ifpri.org/oai/oai.php?verb=ListSets</a></li>
|
||||
<li>Working on second phase of metadata migration, looks like this will work for moving CPWF-specific data in <code>dc.identifier.fund</code> to <code>cg.identifier.cpwfproject</code> and then the rest to <code>dc.description.sponsorship</code></li>
|
||||
</ul>
|
||||
<a href='https://alanorth.github.io/cgspace-notes/2016-06/'>Read more →</a>
|
||||
</article>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<article class="blog-post">
|
||||
<header>
|
||||
<h2 class="blog-post-title"><a href="https://alanorth.github.io/cgspace-notes/2016-05/">May, 2016</a></h2>
|
||||
@ -304,16 +331,16 @@
|
||||
<ol class="list-unstyled">
|
||||
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/posts/">Posts</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-09/">September, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-08/">August, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-07/">July, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-06/">June, 2019</a></li>
|
||||
|
||||
<li><a href="/cgspace-notes/2019-05/">May, 2019</a></li>
|
||||
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user