alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2020-02-09

Browse Source
This commit is contained in:
Alan Orth
2020-02-09 14:08:16 +02:00
parent 009cc870ba
commit ed7ba21a18
9 changed files with 29518 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
content/posts/2020-02.md
View File

@ -339,4 +339,49 @@ $ perf-java-flames 11359
- If the width of the stacks indicates time, then it's clear that Hibernate takes longer...
- Apparently there is a "flame diff" tool, I wonder if we can use that to compare!
## 2020-02-09
- This weekend I did a lot more testing of indexing performance with our DSpace 5.8 branch, vanilla DSpace 5.10, and vanilla DSpace 6.4-SNAPSHOT:
```
# CGSpace 5.8
schedtool -D -e ~/dspace/bin/dspace index-discovery -b 385.72s user 131.16s system 19% cpu 43:21.18 total
schedtool -D -e ~/dspace/bin/dspace index-discovery -b 382.95s user 127.31s system 20% cpu 42:10.07 total
schedtool -D -e ~/dspace/bin/dspace index-discovery -b 368.56s user 143.97s system 20% cpu 42:22.66 total
schedtool -D -e ~/dspace/bin/dspace index-discovery -b 360.09s user 104.03s system 19% cpu 39:24.41 total
# Vanilla DSpace 5.10
schedtool -D -e ~/dspace510/bin/dspace index-discovery -b 236.19s user 59.70s system 3% cpu 2:03:31.14 total
schedtool -D -e ~/dspace510/bin/dspace index-discovery -b 232.41s user 50.38s system 3% cpu 2:04:16.00 total
# Vanilla DSpace 6.4-SNAPSHOT
schedtool -D -e ~/dspace63/bin/dspace index-discovery -b 5112.96s user 127.80s system 40% cpu 3:36:53.98 total
schedtool -D -e ~/dspace63/bin/dspace index-discovery -b 5112.96s user 127.80s system 40% cpu 3:21:0.0 total
```
- I generated better flame graphs for the DSpace indexing process by using `perf-record-stack` and filtering out the java process:
```
$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
$ export PERF_RECORD_SECONDS=60
$ export JAVA_OPTS="-XX:+PreserveFramePointer"
$ time schedtool -D -e ~/dspace/bin/dspace index-discovery -b &
# process id of java indexing process (not Tomcat)
$ perf-java-record-stack 169639
$ sudo perf script -i /tmp/perf-169639.data > out.dspace510-1
$ cat out.dspace510-1 | ../FlameGraph/stackcollapse-perf.pl | grep -E '^java' | ../FlameGraph/flamegraph.pl --color=java --hash > out.dspace510-1.svg
```
- All data recorded on my laptop with the same kernel, same boot, etc.
- CGSpace 5.8 (with Atmire patches):
![DSpace 5.8 (with Atmire modules) index-discovery flame graph](/cgspace-notes/2020/02/out.dspace58-2.svg)
- Vanilla DSpace 5.10:
![Vanilla DSpace 5.10 index-discovery flame graph](/cgspace-notes/2020/02/out.dspace510-3.svg)
- Vanilla DSpace 6.4-SNAPSHOT:
![Vanilla DSpace 6.4-SNAPSHOT index-discovery flame graph](/cgspace-notes/2020/02/out.dspace64-3.svg)
<!-- vim: set sw=2 ts=2: -->