diff --git a/content/post/2015-12.md b/content/post/2015-12.md index 5b41c59ef..aa5ad4dcd 100644 --- a/content/post/2015-12.md +++ b/content/post/2015-12.md @@ -73,7 +73,7 @@ $ curl -o /dev/null -s -w %{time_total}\\n https://cgspace.cgiar.org/rest/handle - Last week it was an average of 8 seconds... now this is 1/4 of that - CCAFS noticed that one of their items displays only the Atmire statlets: https://cgspace.cgiar.org/handle/10568/42445 -![CCAFS item](2015/12/ccafs-item-no-metadata.png) +![CCAFS item](/cgspace-notes/2015/12/ccafs-item-no-metadata.png) - The authorizations for the item are all public READ, and I don't see any errors in dspace.log when browsing that item - I filed a ticket on Atmire's issue tracker @@ -122,10 +122,10 @@ postgres@linode01:~$ psql -c 'SELECT * from pg_stat_activity;' | grep cgspace | - I have reverted all the pgtune tweaks from the other day, as they didn't fix the stability issues, so I'd rather not have them introducing more variables into the equation - The PostgreSQL stats from Munin all point to something database-related with the DSpace 5 upgrade around mid–late November -![PostgreSQL bgwriter (year)](2015/12/postgres_bgwriter-year.png) -![PostgreSQL cache (year)](2015/12/postgres_cache_cgspace-year.png) -![PostgreSQL locks (year)](2015/12/postgres_locks_cgspace-year.png) -![PostgreSQL scans (year)](2015/12/postgres_scans_cgspace-year.png) +![PostgreSQL bgwriter (year)](/cgspace-notes/2015/12/postgres_bgwriter-year.png) +![PostgreSQL cache (year)](/cgspace-notes/2015/12/postgres_cache_cgspace-year.png) +![PostgreSQL locks (year)](/cgspace-notes/2015/12/postgres_locks_cgspace-year.png) +![PostgreSQL scans (year)](/cgspace-notes/2015/12/postgres_scans_cgspace-year.png) ## 2015-12-07 diff --git a/content/post/2016-01.md b/content/post/2016-01.md index 734a59482..efaa7c9ee 100644 --- a/content/post/2016-01.md +++ b/content/post/2016-01.md @@ -65,8 +65,8 @@ tags = ["Notes"] - Add five missing center-specific subjects to XMLUI item view ([#174](https://github.com/ilri/DSpace/issues/174)) - This [CCAFS item](https://cgspace.cgiar.org/handle/10568/67062) Before: -![XMLUI subjects before](2016/01/xmlui-subjects-before.png) +![XMLUI subjects before](/cgspace-notes/2016/01/xmlui-subjects-before.png) - After: -![XMLUI subjects after](2016/01/xmlui-subjects-after.png) +![XMLUI subjects after](/cgspace-notes/2016/01/xmlui-subjects-after.png) diff --git a/content/post/2016-02.md b/content/post/2016-02.md index 1bebd5f1e..d262820e5 100644 --- a/content/post/2016-02.md +++ b/content/post/2016-02.md @@ -11,7 +11,7 @@ tags = ["Notes"] - Lots of issues with spaces, newlines, etc causing the import to fail - I noticed we have a very *interesting* list of countries on CGSpace: -![CGSpace country list](2016/02/cgspace-countries.png) +![CGSpace country list](/cgspace-notes/2016/02/cgspace-countries.png) - Not only are there 49,000 countries, we have some blanks (25)... - Also, lots of things like "COTE D`LVOIRE" and "COTE D IVOIRE" @@ -108,8 +108,8 @@ $ ~/dspace/bin/dspace index-discovery -b - Finish cleaning up and importing ~400 DAGRIS items into CGSpace - Whip up some quick CSS to make the button in the submission workflow use the XMLUI theme's brand colors ([#154](https://github.com/ilri/DSpace/issues/154)) -![ILRI submission buttons](2016/02/submit-button-ilri.png) -![Drylands submission buttons](2016/02/submit-button-drylands.png) +![ILRI submission buttons](/cgspace-notes/2016/02/submit-button-ilri.png) +![Drylands submission buttons](/cgspace-notes/2016/02/submit-button-drylands.png) ## 2016-02-09 diff --git a/content/post/2016-03.md b/content/post/2016-03.md index 6a470bb47..78dbf5ced 100644 --- a/content/post/2016-03.md +++ b/content/post/2016-03.md @@ -39,7 +39,7 @@ Exception in thread "Lucene Merge Thread #19" org.apache.lucene.index.MergePolic - Start cleaning up the configuration for Atmire's CUA module ([#184](https://github.com/ilri/DSpace/issues/185)) - It is very messed up because some labels are incorrect, fields are missing, etc -![Mixed up label in Atmire CUA](2016/03/cua-label-mixup.png) +![Mixed up label in Atmire CUA](/cgspace-notes/2016/03/cua-label-mixup.png) - Update documentation for Atmire modules @@ -58,7 +58,7 @@ Exception in thread "Lucene Merge Thread #19" org.apache.lucene.index.MergePolic - Make titles in Discovery and Browse by more consistent (singular, sentence case, etc) ([#186](https://github.com/ilri/DSpace/issues/186)) - Also four or so center-specific subject strings were missing for Discovery -![Missing XMLUI string](2016/03/missing-xmlui-string.png) +![Missing XMLUI string](/cgspace-notes/2016/03/missing-xmlui-string.png) ## 2016-03-15 @@ -106,11 +106,11 @@ Exception in thread "Lucene Merge Thread #19" org.apache.lucene.index.MergePolic - Discuss thumbnails with Francesca from Bioversity - Some of their items end up with thumbnails that have a big white border around them: -![Excessive whitespace in thumbnail](2016/03/bioversity-thumbnail-bad.jpg) +![Excessive whitespace in thumbnail](/cgspace-notes/2016/03/bioversity-thumbnail-bad.jpg) - Turns out we can add `-trim` to the GraphicsMagick options to trim the whitespace -![Trimmed thumbnail](2016/03/bioversity-thumbnail-good.jpg) +![Trimmed thumbnail](/cgspace-notes/2016/03/bioversity-thumbnail-good.jpg) - Command used: @@ -136,14 +136,14 @@ $ gm convert -trim -quality 82 -thumbnail x300 -flatten Descriptor\ for\ Butia_E - Google says the first time it saw this particular error was September 29, 2015... so maybe it accidentally saw it somehow... - On a related note, we have 51,000 items indexed from the sitemap, but 500,000 items in the Google index, so we DEFINITELY have a problem with duplicate content -![CGSpace pages in Google index](2016/03/google-index.png) +![CGSpace pages in Google index](/cgspace-notes/2016/03/google-index.png) - Turns out this is a problem with DSpace's `robots.txt`, and there's a Jira ticket since December, 2015: https://jira.duraspace.org/browse/DS-2962 - I am not sure if I want to apply it yet - For now I've just set a bunch of these dynamic pages to not appear in search results by using the URL Parameters tool in Webmaster Tools -![URL parameters cause millions of dynamic pages](2016/03/url-parameters.png) -![Setting pages with the filter_0 param not to show in search results](2016/03/url-parameters2.png) +![URL parameters cause millions of dynamic pages](/cgspace-notes/2016/03/url-parameters.png) +![Setting pages with the filter_0 param not to show in search results](/cgspace-notes/2016/03/url-parameters2.png) - Move AVCD collection to new community and update `move_collection.sh` script: https://gist.github.com/alanorth/392c4660e8b022d99dfa - It seems Feedburner can do HTTPS now, so we might be able to update our feeds and simplify the nginx configs diff --git a/content/post/2016-05.md b/content/post/2016-05.md index 60933ef05..01bf3ee8f 100644 --- a/content/post/2016-05.md +++ b/content/post/2016-05.md @@ -33,7 +33,7 @@ GET /rest/handle/10568/NaN?expand=parentCommunityList,metadata HTTP/1.1 - Update nginx to 1.10.x branch on CGSpace - Fix a reference to `dc.type.output` in Discovery that I had missed when we migrated to `dc.type` last month ([#223](https://github.com/ilri/DSpace/pull/223)) -![Item type in Discovery results](2016/05/discovery-types.png) +![Item type in Discovery results](/cgspace-notes/2016/05/discovery-types.png) ## 2016-05-06 diff --git a/content/post/2016-06.md b/content/post/2016-06.md index 47d13f11f..bcc8d9ddc 100644 --- a/content/post/2016-06.md +++ b/content/post/2016-06.md @@ -270,7 +270,7 @@ $ ./delete-metadata-values.py -f dc.contributor.corporate -i Corporate-Authors-D - Re-deploy CGSpace and DSpace Test with latest June changes - Now the sharing and Altmetric bits are more prominent: -![DSpace 5.1 XMLUI With Altmetric Badge](2016/06/xmlui-altmetric-sharing.png) +![DSpace 5.1 XMLUI With Altmetric Badge](/cgspace-notes/2016/06/xmlui-altmetric-sharing.png) - Run all system updates on the servers and reboot - Start working on config changes for phase three of the metadata migrations diff --git a/content/post/2016-07.md b/content/post/2016-07.md index 97f71c8fc..3a23ee4ce 100644 --- a/content/post/2016-07.md +++ b/content/post/2016-07.md @@ -191,7 +191,7 @@ index.authority.ignore-variants=true - No changes to Discovery after reindexing... hmm. - Integrate and massively clean up About page ([#256](https://github.com/ilri/DSpace/pull/256)) -![About page](2016/07/cgspace-about-page.png) +![About page](/cgspace-notes/2016/07/cgspace-about-page.png) - The DSpace source code mentions the configuration key `discovery.index.authority.ignore-prefered.*` (with prefix of discovery, despite the docs saying otherwise), so I'm trying the following on DSpace Test: diff --git a/content/post/2016-08.md b/content/post/2016-08.md index 6dead3dbb..6258e187f 100644 --- a/content/post/2016-08.md +++ b/content/post/2016-08.md @@ -104,7 +104,7 @@ $ ln -sv ~/dspace/webapps/solr /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/sol - Finally got DSpace (5.5) running on Ubuntu 16.04, Tomcat 7, Java 8, PostgreSQL 9.5 via the updated Ansible stuff -![DSpace 5.5 on Ubuntu 16.04, Tomcat 7, Java 8, PostgreSQL 9.5](2016/08/dspace55-ubuntu16.04.png) +![DSpace 5.5 on Ubuntu 16.04, Tomcat 7, Java 8, PostgreSQL 9.5](/cgspace-notes/2016/08/dspace55-ubuntu16.04.png) ## 2016-08-14 @@ -114,7 +114,7 @@ $ ln -sv ~/dspace/webapps/solr /opt/brew/Cellar/tomcat/8.5.4/libexec/webapps/sol - Notes on NodeJS + nginx + systemd: https://gist.github.com/alanorth/51acd476891c67dfe27725848cf5ace1 -![ExpressJS running behind nginx](2016/08/nodejs-nginx.png) +![ExpressJS running behind nginx](/cgspace-notes/2016/08/nodejs-nginx.png) ## 2016-08-16 diff --git a/content/post/2016-09.md b/content/post/2016-09.md index 5e0a1f2ac..eb2dcc59b 100644 --- a/content/post/2016-09.md +++ b/content/post/2016-09.md @@ -27,7 +27,7 @@ distinguishedName: CN=Last\, First (ILRI),OU=ILRI Ethiopia Employees,OU=ILRI Eth - Changing the DSpace LDAP config to use `OU=ILRIHUB` seems to work: -![DSpace groups based on LDAP DN](2016/09/ilri-ldap-users.png) +![DSpace groups based on LDAP DN](/cgspace-notes/2016/09/ilri-ldap-users.png) - Notes for local PostgreSQL database recreation from production snapshot: @@ -327,9 +327,9 @@ Exception in thread "http-bio-127.0.0.1-8081-exec-25" java.lang.OutOfMemoryError - Perhaps these particular issues *are* memory issues, the munin graphs definitely show some weird purging/allocating behavior starting this week -![Tomcat JVM usage day](2016/09/tomcat_jvm-day.png) -![Tomcat JVM usage week](2016/09/tomcat_jvm-week.png) -![Tomcat JVM usage month](2016/09/tomcat_jvm-month.png) +![Tomcat JVM usage day](/cgspace-notes/2016/09/tomcat_jvm-day.png) +![Tomcat JVM usage week](/cgspace-notes/2016/09/tomcat_jvm-week.png) +![Tomcat JVM usage month](/cgspace-notes/2016/09/tomcat_jvm-month.png) - And really, we did reduce the memory of CGSpace in late 2015, so maybe we should just increase it again, now that our usage is higher and we are having memory errors in the logs - Oh great, the configuration on the actual server is different than in configuration management! @@ -346,7 +346,7 @@ JAVA_OPTS="-Djava.awt.headless=true -Xms3584m -Xmx3584m -XX:MaxPermSize=256m -XX - Looking at Google Webmaster Tools again, it seems the work I did on URL query parameters and blocking via the `X-Robots-Tag` HTTP header in March, 2016 seem to have had a positive effect on Google's index for CGSpace -![Google Webmaster Tools for CGSpace](2016/09/google-webmaster-tools-index.png) +![Google Webmaster Tools for CGSpace](/cgspace-notes/2016/09/google-webmaster-tools-index.png) ## 2016-09-16 @@ -421,8 +421,8 @@ $ ./delete-metadata-values.py -f cg.contributor.affiliation -i affiliations_pb-2 ``` - It actually works really well, and search results return much less hits now (before, after): -![CGSpace search with "OR" boolean logic](2016/09/cgspace-search.png) -![DSpace Test search with "AND" boolean logic](2016/09/dspacetest-search.png) +![CGSpace search with "OR" boolean logic](/cgspace-notes/2016/09/cgspace-search.png) +![DSpace Test search with "AND" boolean logic](/cgspace-notes/2016/09/dspacetest-search.png) - Found a way to improve the configuration of Atmire's Content and Usage Analysis (CUA) module for date fields diff --git a/content/post/2016-10.md b/content/post/2016-10.md index d572e2b53..29bd827c0 100644 --- a/content/post/2016-10.md +++ b/content/post/2016-10.md @@ -21,7 +21,7 @@ tags = ["Notes"] - With a blank `dc.contributor.author` column, DSpace wants to remove all non-ORCID authors and add the new ORCID authors - I added the [disclaimer text](https://github.com/ilri/DSpace/issues/234) to the About page, then added a footer link to the disclaimer's ID, but there is a Bootstrap issue that causes the page content to disappear when using in-page anchors: https://github.com/twbs/bootstrap/issues/1768 -![Bootstrap issue with in-page anchors](2016/10/bootstrap-issue.png) +![Bootstrap issue with in-page anchors](/cgspace-notes/2016/10/bootstrap-issue.png) - Looks like we'll just have to add the text to the About page (without a link) or add a separate page @@ -58,7 +58,7 @@ dspacetest=# \copy (select distinct text_value, count(*) as count from metadatav - Heap space is 2048m, and we have 5GB of RAM being used for OS cache (Solr!) so let's just bump the memory to 3072m - Magdalena from CCAFS asked why the colors in the thumbnails for these [two](https://cgspace.cgiar.org/handle/10568/71249) [items](https://cgspace.cgiar.org/handle/10568/71259) look different, even though they are the same in the PDF itself -![CMYK vs sRGB colors](2016/10/cmyk-vs-srgb.jpg) +![CMYK vs sRGB colors](/cgspace-notes/2016/10/cmyk-vs-srgb.jpg) - Turns out the first PDF was exported from InDesign using CMYK and the second one was using sRGB - Run all system updates on DSpace Test and reboot it @@ -205,8 +205,8 @@ UPDATE 0 - Looks much better now: -![CGSpace with old icons](2016/10/cgspace-icons.png) -![DSpace Test with Font Awesome icons](2016/10/dspacetest-fontawesome-icons.png) +![CGSpace with old icons](/cgspace-notes/2016/10/cgspace-icons.png) +![DSpace Test with Font Awesome icons](/cgspace-notes/2016/10/dspacetest-fontawesome-icons.png) - Run the same replacements on CGSpace diff --git a/content/post/2016-11.md b/content/post/2016-11.md index eff8bd96b..9d42b9fc2 100644 --- a/content/post/2016-11.md +++ b/content/post/2016-11.md @@ -9,7 +9,7 @@ tags = ["Notes"] - Add `dc.type` to the output options for Atmire's Listings and Reports module ([#286](https://github.com/ilri/DSpace/pull/286)) -![Listings and Reports with output type](2016/11/listings-and-reports.png) +![Listings and Reports with output type](/cgspace-notes/2016/11/listings-and-reports.png) ## 2016-11-02 @@ -82,7 +82,7 @@ $ ./fix-metadata-values.py -i /tmp/CRPs.csv -f cg.contributor.crp -t correct -m - Atmire's Listings and Reports module seems to be broken on DSpace 5.5 -![Listings and Reports broken in DSpace 5.5](2016/11/listings-and-reports-55.png) +![Listings and Reports broken in DSpace 5.5](/cgspace-notes/2016/11/listings-and-reports-55.png) - I've filed a ticket with Atmire - Thinking about batch updates for ORCIDs and authors @@ -260,8 +260,8 @@ X-Cocoon-Version: 2.2.0 - The Tomcat JVM heap looks really good after applying the Crawler Session Manager fix on DSpace Test last night: -![Tomcat JVM heap (day) after setting up the Crawler Session Manager](2016/11/dspacetest-tomcat-jvm-day.png) -![Tomcat JVM heap (week) after setting up the Crawler Session Manager](2016/11/dspacetest-tomcat-jvm-week.png) +![Tomcat JVM heap (day) after setting up the Crawler Session Manager](/cgspace-notes/2016/11/dspacetest-tomcat-jvm-day.png) +![Tomcat JVM heap (week) after setting up the Crawler Session Manager](/cgspace-notes/2016/11/dspacetest-tomcat-jvm-week.png) - Seems the default regex doesn't catch Baidu, though: diff --git a/content/post/2016-12.md b/content/post/2016-12.md index 60a50d63e..234f66aa1 100644 --- a/content/post/2016-12.md +++ b/content/post/2016-12.md @@ -444,8 +444,8 @@ dspace=# update metadatavalue set authority='2df8136e-d8f4-4142-b58c-562337cab76 - After enabling a sizable `shared_buffers` for CGSpace's PostgreSQL configuration the number of connections to the database dropped significantly -![postgres_bgwriter-week](2016/12/postgres_bgwriter-week.png) -![postgres_connections_ALL-week](2016/12/postgres_connections_ALL-week.png) +![postgres_bgwriter-week](/cgspace-notes/2016/12/postgres_bgwriter-week.png) +![postgres_connections_ALL-week](/cgspace-notes/2016/12/postgres_connections_ALL-week.png) - Looking at CIAT records from last week again, they have a lot of double authors like: @@ -484,8 +484,8 @@ UPDATE 35 - Checking in on CGSpace postgres stats again, looks like the `shared_buffers` change from a few days ago really made a big impact: -![postgres_bgwriter-week](2016/12/postgres_bgwriter-week-2016-12-13.png) -![postgres_connections_ALL-week](2016/12/postgres_connections_ALL-week-2016-12-13.png) +![postgres_bgwriter-week](/cgspace-notes/2016/12/postgres_bgwriter-week-2016-12-13.png) +![postgres_connections_ALL-week](/cgspace-notes/2016/12/postgres_connections_ALL-week-2016-12-13.png) - Looking at logs, it seems we need to evaluate which logs we keep and for how long - Basically the only ones we *need* are `dspace.log` because those are used for legacy statistics (need to keep for 1 month) @@ -593,8 +593,8 @@ Caused by: java.lang.NoSuchMethodError: com.atmire.statistics.generator.DSpaceOb - This is how DSpace works, and I need to ask if there is a way to override someone's submission, as the other reviewer seems to not be paying attention, or has perhaps taken the item from the task pool? - Run a batch edit to add "RANGELANDS" ILRI subject to all items containing the word "RANGELANDS" in their metadata for Peter Ballantyne -![Select all items with "rangelands" in metadata](2016/12/batch-edit1.png) -![Add RANGELANDS ILRI subject](2016/12/batch-edit2.png) +![Select all items with "rangelands" in metadata](/cgspace-notes/2016/12/batch-edit1.png) +![Add RANGELANDS ILRI subject](/cgspace-notes/2016/12/batch-edit2.png) ## 2016-12-18