From e7cd054083bb636d138aeeb6b4cca1145b5d2e33 Mon Sep 17 00:00:00 2001 From: Alan Orth Date: Thu, 13 Sep 2018 16:15:01 +0300 Subject: [PATCH] Add notes for 2018-09-13 --- content/posts/2018-09.md | 37 +++++++++++++++++++++++++++++ docs/2018-09/index.html | 50 ++++++++++++++++++++++++++++++++++++---- docs/sitemap.xml | 10 ++++---- 3 files changed, 87 insertions(+), 10 deletions(-) diff --git a/content/posts/2018-09.md b/content/posts/2018-09.md index 047c0657f..fd3ad4f52 100644 --- a/content/posts/2018-09.md +++ b/content/posts/2018-09.md @@ -221,4 +221,41 @@ $ sudo docker run --name dspacedb -v dspacetest_data:/var/lib/postgresql/data -e - After forcing a complete re-indexing of OAI the mappings were fine - The `dateStamp` is most probably only updated when the item's metadata changes, not its mappings, so if Altmetric is relying on that we're in a tricky spot - We need to make sure that our OAI isn't publicizing stale data... I was going to post something on the dspace-tech mailing list, but never did +- Linode says that CGSpace (linode18) has had high CPU for the past two hours +- The top IP addresses today are: + +``` +# zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 | grep -E "13/Sep/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10 + 32 46.229.161.131 + 38 104.198.9.108 + 39 66.249.64.91 + 56 157.55.39.224 + 57 207.46.13.49 + 58 40.77.167.120 + 78 169.255.105.46 + 702 54.214.112.202 + 1840 50.116.102.77 + 4469 70.32.83.92 +``` + +- And the top two addresses seem to be re-using their Tomcat sessions properly: + +``` +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=70.32.83.92' dspace.log.2018-09-13 | sort | uniq +7 +$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=50.116.102.77' dspace.log.2018-09-13 | sort | uniq +2 +``` + +- So I'm not sure what's going on +- Valerio asked me if there's a way to get the page views and downloads from CGSpace +- I said no, but that we might be able to piggyback on the Atmire statlet REST API +- For example, when you expand the "statlet" at the bottom of an item like [10568/97103](https://cgspace.cgiar.org/handle/10568/97103) you can see the following request in the browser console: + +``` +https://cgspace.cgiar.org/rest/statlets?handle=10568/97103&_=1536844046540 +``` + +- That JSON file has the total page views and item downloads for the item... + diff --git a/docs/2018-09/index.html b/docs/2018-09/index.html index 9c0e9b854..a450208e7 100644 --- a/docs/2018-09/index.html +++ b/docs/2018-09/index.html @@ -18,7 +18,7 @@ I’m testing the new DSpace 5.8 branch in my Ubuntu 18.04 environment and I " /> - + Altmetric said it was somehow related to the OAI dateStamp not getting updated when the mappings changed, but I said that back in 2018-07 when this happened it was because the OAI was actually just not reflecting all the item’s mappings
  • After forcing a complete re-indexing of OAI the mappings were fine
  • The dateStamp is most probably only updated when the item’s metadata changes, not its mappings, so if Altmetric is relying on that we’re in a tricky spot
    • -
  • We need to make sure that our OAI isn’t publicizing stale data… I was going to post something on the dspace-tech mailing list, but never did -
    • +
  • We need to make sure that our OAI isn’t publicizing stale data… I was going to post something on the dspace-tech mailing list, but never did
    • +
  • Linode says that CGSpace (linode18) has had high CPU for the past two hours
    • +
  • The top IP addresses today are:
    • +
    # zcat --force /var/log/nginx/rest.log /var/log/nginx/rest.log.1 | grep -E "13/Sep/2018" | awk '{print $1}' | sort | uniq -c | sort -n | tail -n 10                                                                                                
+     32 46.229.161.131
+     38 104.198.9.108
+     39 66.249.64.91
+     56 157.55.39.224
+     57 207.46.13.49
+     58 40.77.167.120
+     78 169.255.105.46
+    702 54.214.112.202
+   1840 50.116.102.77
+   4469 70.32.83.92
+
    + + + +
    $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=70.32.83.92' dspace.log.2018-09-13 | sort | uniq
+7
+$ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=50.116.102.77' dspace.log.2018-09-13 | sort | uniq
+2
+
    + + + +
    https://cgspace.cgiar.org/rest/statlets?handle=10568/97103&_=1536844046540
+
    + + + + + diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 10818dff4..c613bd14a 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -4,7 +4,7 @@ https://alanorth.github.io/cgspace-notes/2018-09/ - 2018-09-12T17:02:14+03:00 + 2018-09-13T12:48:20+03:00 @@ -184,7 +184,7 @@ https://alanorth.github.io/cgspace-notes/ - 2018-09-12T17:02:14+03:00 + 2018-09-13T12:48:20+03:00 0 @@ -195,7 +195,7 @@ https://alanorth.github.io/cgspace-notes/tags/notes/ - 2018-09-12T17:02:14+03:00 + 2018-09-13T12:48:20+03:00 0 @@ -207,13 +207,13 @@ https://alanorth.github.io/cgspace-notes/posts/ - 2018-09-12T17:02:14+03:00 + 2018-09-13T12:48:20+03:00 0 https://alanorth.github.io/cgspace-notes/tags/ - 2018-09-12T17:02:14+03:00 + 2018-09-13T12:48:20+03:00 0