mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-15 19:37:04 +01:00
Add notes for 2018-01-24
This commit is contained in:
parent
4eabb20e99
commit
e02d82d56b
GPG Key ID:
0FB860CC9C45B1B9
@ -1047,3 +1047,70 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled
|
||||
```
|
||||
|
||||
- I can definitely design a test plan on this!
|
||||
|
||||
## 2018-01-24
|
||||
|
||||
- Looking at the REST requests, most of them are to expand all or metadata, but 5% are for retrieving bitstreams:
|
||||
|
||||
```
|
||||
# zcat --force /var/log/nginx/access.log.3.gz /var/log/nginx/access.log.4.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/library-access.log.4.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/rest.log.4.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/oai.log.4.gz /var/log/nginx/error.log.3.gz /var/log/nginx/error.log.4.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -E "^/rest" | grep -Eo "(retrieve|expand=[a-z].*)" | sort | uniq -c | sort -n
|
||||
1 expand=collections
|
||||
16 expand=all&limit=1
|
||||
45 expand=items
|
||||
775 retrieve
|
||||
5675 expand=all
|
||||
8633 expand=metadata
|
||||
```
|
||||
|
||||
- I finished creating the test plan for DSpace Test and ran it from my Linode with:
|
||||
|
||||
```
|
||||
$ jmeter -n -t DSpacePerfTest-dspacetest.cgiar.org.jmx -l 2018-01-24-1.jtl
|
||||
```
|
||||
|
||||
- Atmire responded to [my issue from two weeks ago](https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560) and said they will start looking into DSpace 5.8 compatibility for CGSpace
|
||||
- I set up a new Arch Linux Linode instance with 8192 MB of RAM and ran the test plan a few times to get a baseline:
|
||||
|
||||
```
|
||||
# lscpu
|
||||
# lscpu
|
||||
Architecture: x86_64
|
||||
CPU op-mode(s): 32-bit, 64-bit
|
||||
Byte Order: Little Endian
|
||||
CPU(s): 4
|
||||
On-line CPU(s) list: 0-3
|
||||
Thread(s) per core: 1
|
||||
Core(s) per socket: 1
|
||||
Socket(s): 4
|
||||
NUMA node(s): 1
|
||||
Vendor ID: GenuineIntel
|
||||
CPU family: 6
|
||||
Model: 63
|
||||
Model name: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
|
||||
Stepping: 2
|
||||
CPU MHz: 2499.994
|
||||
BogoMIPS: 5001.32
|
||||
Hypervisor vendor: KVM
|
||||
Virtualization type: full
|
||||
L1d cache: 32K
|
||||
L1i cache: 32K
|
||||
L2 cache: 4096K
|
||||
L3 cache: 16384K
|
||||
NUMA node0 CPU(s): 0-3
|
||||
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti retpoline fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
|
||||
# free -m
|
||||
total used free shared buff/cache available
|
||||
Mem: 7970 107 7759 1 103 7771
|
||||
Swap: 255 0 255
|
||||
# pacman -Syu
|
||||
# pacman -S git wget jre8-openjdk-headless mosh htop tmux
|
||||
# useradd -m test
|
||||
# su - test
|
||||
$ git clone -b ilri https://github.com/alanorth/dspace-performance-test.git
|
||||
$ wget http://www-us.apache.org/dist//jmeter/binaries/apache-jmeter-3.3.tgz
|
||||
$ tar xf apache-jmeter-3.3.tgz
|
||||
$ cd apache-jmeter-3.3/bin
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline.log
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline2.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline2.log
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline3.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline3.log
|
||||
```
|
||||
|
||||
@ -92,7 +92,7 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv
|
||||
|
||||
<meta property="article:published_time" content="2018-01-02T08:35:54-08:00"/>
|
||||
|
||||
<meta property="article:modified_time" content="2018-01-23T00:08:32+02:00"/>
|
||||
<meta property="article:modified_time" content="2018-01-23T18:08:57+02:00"/>
|
||||
|
||||
|
||||
|
||||
@ -194,9 +194,9 @@ Danny wrote to ask for help renewing the wildcard ilri.org certificate and I adv
|
||||
"@type": "BlogPosting",
|
||||
"headline": "January, 2018",
|
||||
"url": "https://alanorth.github.io/cgspace-notes/2018-01/",
|
||||
"wordCount": "5977",
|
||||
"wordCount": "6355",
|
||||
"datePublished": "2018-01-02T08:35:54-08:00",
|
||||
"dateModified": "2018-01-23T00:08:32+02:00",
|
||||
"dateModified": "2018-01-23T18:08:57+02:00",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "Alan Orth"
|
||||
@ -1418,6 +1418,76 @@ $ ./rest-find-collections.py 10568/1 | grep -i untitled
|
||||
<li>I can definitely design a test plan on this!</li>
|
||||
</ul>
|
||||
|
||||
<h2 id="2018-01-24">2018-01-24</h2>
|
||||
|
||||
<ul>
|
||||
<li>Looking at the REST requests, most of them are to expand all or metadata, but 5% are for retrieving bitstreams:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code># zcat --force /var/log/nginx/access.log.3.gz /var/log/nginx/access.log.4.gz /var/log/nginx/library-access.log.3.gz /var/log/nginx/library-access.log.4.gz /var/log/nginx/rest.log.3.gz /var/log/nginx/rest.log.4.gz /var/log/nginx/oai.log.3.gz /var/log/nginx/oai.log.4.gz /var/log/nginx/error.log.3.gz /var/log/nginx/error.log.4.gz | grep "21/Jan/2018" | grep "GET " | grep -v "/admin" | awk '{print $7}' | grep -E "^/rest" | grep -Eo "(retrieve|expand=[a-z].*)" | sort | uniq -c | sort -n
|
||||
1 expand=collections
|
||||
16 expand=all&limit=1
|
||||
45 expand=items
|
||||
775 retrieve
|
||||
5675 expand=all
|
||||
8633 expand=metadata
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>I finished creating the test plan for DSpace Test and ran it from my Linode with:</li>
|
||||
</ul>
|
||||
|
||||
<pre><code>$ jmeter -n -t DSpacePerfTest-dspacetest.cgiar.org.jmx -l 2018-01-24-1.jtl
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>Atmire responded to <a href="https://tracker.atmire.com/tickets-cgiar-ilri/view-ticket?id=560">my issue from two weeks ago</a> and said they will start looking into DSpace 5.8 compatibility for CGSpace</li>
|
||||
<li>I set up a new Arch Linux Linode instance with 8192 MB of RAM and ran the test plan a few times to get a baseline:<br /></li>
|
||||
</ul>
|
||||
|
||||
<pre><code># lscpu
|
||||
# lscpu
|
||||
Architecture: x86_64
|
||||
CPU op-mode(s): 32-bit, 64-bit
|
||||
Byte Order: Little Endian
|
||||
CPU(s): 4
|
||||
On-line CPU(s) list: 0-3
|
||||
Thread(s) per core: 1
|
||||
Core(s) per socket: 1
|
||||
Socket(s): 4
|
||||
NUMA node(s): 1
|
||||
Vendor ID: GenuineIntel
|
||||
CPU family: 6
|
||||
Model: 63
|
||||
Model name: Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
|
||||
Stepping: 2
|
||||
CPU MHz: 2499.994
|
||||
BogoMIPS: 5001.32
|
||||
Hypervisor vendor: KVM
|
||||
Virtualization type: full
|
||||
L1d cache: 32K
|
||||
L1i cache: 32K
|
||||
L2 cache: 4096K
|
||||
L3 cache: 16384K
|
||||
NUMA node0 CPU(s): 0-3
|
||||
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti retpoline fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
|
||||
# free -m
|
||||
total used free shared buff/cache available
|
||||
Mem: 7970 107 7759 1 103 7771
|
||||
Swap: 255 0 255
|
||||
# pacman -Syu
|
||||
# pacman -S git wget jre8-openjdk-headless mosh htop tmux
|
||||
# useradd -m test
|
||||
# su - test
|
||||
$ git clone -b ilri https://github.com/alanorth/dspace-performance-test.git
|
||||
$ wget http://www-us.apache.org/dist//jmeter/binaries/apache-jmeter-3.3.tgz
|
||||
$ tar xf apache-jmeter-3.3.tgz
|
||||
$ cd apache-jmeter-3.3/bin
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline.log
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline2.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline2.log
|
||||
$ ./jmeter -n -t ~/dspace-performance-test/DSpacePerfTest-dspacetest.cgiar.org.jmx -l ~/dspace-performance-test/2018-01-24-linode5451120-baseline3.jtl -j ~/dspace-performance-test/2018-01-24-linode5451120-baseline3.log
|
||||
</code></pre>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/2018-01/</loc>
|
||||
<lastmod>2018-01-23T00:08:32+02:00</lastmod>
|
||||
<lastmod>2018-01-23T18:08:57+02:00</lastmod>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
@ -144,7 +144,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||
<lastmod>2018-01-23T00:08:32+02:00</lastmod>
|
||||
<lastmod>2018-01-23T18:08:57+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -155,7 +155,7 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||
<lastmod>2018-01-23T00:08:32+02:00</lastmod>
|
||||
<lastmod>2018-01-23T18:08:57+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
@ -167,13 +167,13 @@
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/post/</loc>
|
||||
<lastmod>2018-01-23T00:08:32+02:00</lastmod>
|
||||
<lastmod>2018-01-23T18:08:57+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
<url>
|
||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||
<lastmod>2018-01-23T00:08:32+02:00</lastmod>
|
||||
<lastmod>2018-01-23T18:08:57+02:00</lastmod>
|
||||
<priority>0</priority>
|
||||
</url>
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user