Add notes for 2023-03-28

content/posts/2023-03.md

@@ -517,4 +517,35 @@ colly | awk '{print $1}' | sort | uniq -c | sort -h
 - I exported CGSpace to check for missing Initiative collection mappings
 - Start a harvest on AReS
 
+## 2023-03-27
+
+- The harvest on AReS was incredibly slow and I stopped it about half way twelve hours later
+  - Then I relied on the plugins to get missing items, which caused a high load on the server but actually worked fine
+- Continue working on thumbnails on DSpace
+
+## 2023-03-28
+
+- Regarding ImageMagick there are a few things I've learned
+  - The `-quality` setting does different things for different output formats, see: https://imagemagick.org/script/command-line-options.php#quality
+  - The `-compress` setting controls the compression algorithm for image data, and is unrelated to lossless/lossy
+    - On that note, `-compress lossless` for JPEGs refers to Lossless JPEG, which is not well defined or supported and should be avoided
+    - See: https://imagemagick.org/script/command-line-options.php#compress
+  - The way DSpace currently does its supersampling by exporting to a JPEG, then making a thumbnail of the JPEG, is a double lossy operation
+    - We should be exporting to something lossless like PNG, PPM, or MIFF, then making a thumbnail from that
+  - The PNG format is always lossless so the `-quality` setting controls compression and filtering, but has no effect on the appearance or signature of PNG images
+  - You can use `-quality n` with WebP's `-define webp:lossless=true`,  but I'm not sure about the interaction between ImageMagick quality and WebP lossless...
+    - Also, if converting from a lossless format to WebP lossless in the same command, ImageMagick will ignore quality settings
+  - The MIFF format is useful for piping between ImageMagick commands, but it is also lossless and the quality setting is ignored
+  - You can use a format specifier when piping between ImageMagick commands without writing a file
+  - For example, I want to create a lossless PNG from a distorted JPEG for comparison:
+
+```console
+$ magick convert reference.jpg -quality 85 jpg:- | convert - distorted-lossless.png
+```
+
+- If I convert the JPEG to PNG directly it will ignore the quality setting, so I set the quality and the output format, then pipe it to ImageMagick again to convert to lossless PNG
+- In an attempt to quantify the generation loss from DSpace's "JPG JPG" method of creating thumbnails I wrote a script called `generation-loss.sh` to test against a new "PNG JPG" method
+  - With my sample set of seventeen PDFs from CGSpace I found that _the "JPG JPG" method of thumbnailing results in scores an average of 1.6% lower than with the "PNG JPG" method_.
+  - The average file size with _the "PNG JPG" method was only 200 bytes larger_.
+
 <!-- vim: set sw=2 ts=2: -->