alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Add notes for 2019-05-05

Browse Source
This commit is contained in:
Alan Orth
2019-05-05 16:45:12 +03:00
parent cfa5f3ddfb
commit 96d6602775
76 changed files with 10839 additions and 11300 deletions
Download Patch File Download Diff File Expand all files Collapse all files

60
docs/2017-07/index.html
View File

@ -39,7 +39,7 @@ Merge changes for WLE Phase II theme rename (#329)
Looking at extracting the metadata registries from ICARDA’s MEL DSpace database so we can compare fields with CGSpace
We can use PostgreSQL’s extended output format (-x) plus sed to format the output into quasi XML:
"/>
<meta name="generator" content="Hugo 0.55.3" />
<meta name="generator" content="Hugo 0.55.5" />
@ -155,32 +155,30 @@ We can use PostgreSQL&rsquo;s extended output format (-x) plus sed to format the
<ul>
<li>Adjust WLE Research Theme to include both Phase I and II on the submission form according to editor feedback (<a href="https://github.com/ilri/DSpace/pull/330">#330</a>)</li>
<li>Generate list of fields in the current CGSpace <code>cg</code> scheme so we can record them properly in the metadata registry:</li>
</ul>
<li><p>Generate list of fields in the current CGSpace <code>cg</code> scheme so we can record them properly in the metadata registry:</p>
<pre><code>$ psql dspace -x -c 'select element, qualifier, scope_note from metadatafieldregistry where metadata_schema_id=2 order by element, qualifier;' | sed -r 's:^-\[ RECORD (.*) \]-+$:&lt;/dc-type&gt;\n&lt;dc-type&gt;\n&lt;schema&gt;cg&lt;/schema&gt;:;s:([^ ]*) +\| (.*): &lt;\1&gt;\2&lt;/\1&gt;:;s:^$:&lt;/dc-type&gt;:;1s:&lt;/dc-type&gt;\n::' &gt; cg-types.xml
</code></pre>
</code></pre></li>
<ul>
<li>CGSpace was unavailable briefly, and I saw this error in the DSpace log file:</li>
</ul>
<li><p>CGSpace was unavailable briefly, and I saw this error in the DSpace log file:</p>
<pre><code>2017-07-05 13:05:36,452 ERROR org.dspace.storage.rdbms.DatabaseManager @ SQL connection Error -
org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
</code></pre>
</code></pre></li>
<ul>
<li>Looking at the <code>pg_stat_activity</code> table I saw there were indeed 98 active connections to PostgreSQL, and at this time the limit is 100, so that makes sense</li>
<li>Tsega restarted Tomcat and it&rsquo;s working now</li>
<li>Abenet said she was generating a report with Atmire&rsquo;s CUA module, so it could be due to that?</li>
<li>Looking in the logs I see this random error again that I should report to DSpace:</li>
</ul>
<li><p>Looking at the <code>pg_stat_activity</code> table I saw there were indeed 98 active connections to PostgreSQL, and at this time the limit is 100, so that makes sense</p></li>
<li><p>Tsega restarted Tomcat and it&rsquo;s working now</p></li>
<li><p>Abenet said she was generating a report with Atmire&rsquo;s CUA module, so it could be due to that?</p></li>
<li><p>Looking in the logs I see this random error again that I should report to DSpace:</p>
<pre><code>2017-07-05 13:50:07,196 ERROR org.dspace.statistics.SolrLogger @ COUNTRY ERROR: EU
</code></pre>
</code></pre></li>
<ul>
<li>Seems to come from <code>dspace-api/src/main/java/org/dspace/statistics/SolrLogger.java</code></li>
<li><p>Seems to come from <code>dspace-api/src/main/java/org/dspace/statistics/SolrLogger.java</code></p></li>
</ul>
<h2 id="2017-07-06">2017-07-06</h2>
@ -236,14 +234,12 @@ org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserve
<h2 id="2017-07-24">2017-07-24</h2>
<ul>
<li>Move two top-level communities to be sub-communities of ILRI Projects</li>
</ul>
<li><p>Move two top-level communities to be sub-communities of ILRI Projects</p>
<pre><code>$ for community in 10568/2347 10568/25209; do /home/cgspace.cgiar.org/bin/dspace community-filiator --set --parent=10568/27629 --child=&quot;$community&quot;; done
</code></pre>
</code></pre></li>
<ul>
<li>Discuss CGIAR Library data cleanup with Sisay and Abenet</li>
<li><p>Discuss CGIAR Library data cleanup with Sisay and Abenet</p></li>
</ul>
<h2 id="2017-07-27">2017-07-27</h2>
@ -279,27 +275,25 @@ org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserve
<h2 id="2017-07-31">2017-07-31</h2>
<ul>
<li>Looks like the final list of metadata corrections for CCAFS project tags will be:</li>
</ul>
<li><p>Looks like the final list of metadata corrections for CCAFS project tags will be:</p>
<pre><code>delete from metadatavalue where resource_type_id=2 and metadata_field_id=235 and text_value='PII-FP4_CRMWestAfrica';
update metadatavalue set text_value='FP3_VietnamLED' where resource_type_id=2 and metadata_field_id=134 and text_value='FP3_VeitnamLED';
update metadatavalue set text_value='PII-FP1_PIRCCA' where resource_type_id=2 and metadata_field_id=235 and text_value='PII-SEA_PIRCCA';
delete from metadatavalue where resource_type_id=2 and metadata_field_id=235 and text_value='PII-WA_IntegratedInterventions';
</code></pre>
</code></pre></li>
<ul>
<li>Now just waiting to run them on CGSpace, and then apply the modified input forms after Macaroni Bros give me an updated list</li>
<li>Temporarily increase the nginx upload limit to 200MB for Sisay to upload the CIAT presentations</li>
<li>Looking at CGSpace activity page, there are 52 Baidu bots concurrently crawling our website (I copied the activity page to a text file and grep it)!</li>
</ul>
<li><p>Now just waiting to run them on CGSpace, and then apply the modified input forms after Macaroni Bros give me an updated list</p></li>
<li><p>Temporarily increase the nginx upload limit to 200MB for Sisay to upload the CIAT presentations</p></li>
<li><p>Looking at CGSpace activity page, there are 52 Baidu bots concurrently crawling our website (I copied the activity page to a text file and grep it)!</p>
<pre><code>$ grep 180.76. /tmp/status | awk '{print $5}' | sort | uniq | wc -l
52
</code></pre>
</code></pre></li>
<ul>
<li>From looking at the <code>dspace.log</code> I see they are all using the same session, which means our Crawler Session Manager Valve is working</li>
<li><p>From looking at the <code>dspace.log</code> I see they are all using the same session, which means our Crawler Session Manager Valve is working</p></li>
</ul>