Update notes for 2016-09-19

content/2016-09.md

@@ -276,11 +276,10 @@ org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error
 ```
 
 - Looking at the top 20 IPs or so, most are Yahoo, MSN, Google, Baidu, TurnitIn (iParadigm), etc... do we have any real users?
-- Generate a list of all Affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:
+- Generate a list of all author affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:
 
 ```
-dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc)
- to /tmp/affiliations.csv with csv;
+dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
 ```
 
 - Looking into the Catalina logs again around the time of the first crash, I see:
@@ -387,3 +386,15 @@ Exception in thread "Thread-54216" org.apache.solr.client.solrj.impl.HttpSolrSer
 ```
 
 - I've sent a message to Atmire about the Solr error to see if it's related to their batch update module
+
+## 2016-09-19
+
+- Work on cleanups for author affiliations after Peter sent me his list of corrections/deletions:
+
+```
+$ ./fix-metadata-values.py -i affiliations_pb-322-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p fuuu
+$ ./delete-metadata-values.py -f cg.contributor.affiliation -i affiliations_pb-2-deletions.csv -m 211 -u dspace-d dspace-p fuuu
+```
+
+- After that we need to take the top ~300 and make a controlled vocabulary for it
+- I dumped a list of the top 300 affiliations from the database, sorted it alphabetically in OpenRefine, and created a controlled vocabulary for it ([#267](https://github.com/ilri/DSpace/pull/267))

public/2016-09/index.html

@@ -395,11 +395,10 @@ org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error
 
 <ul>
 <li>Looking at the top 20 IPs or so, most are Yahoo, MSN, Google, Baidu, TurnitIn (iParadigm), etc&hellip; do we have any real users?</li>
-<li>Generate a list of all Affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
+<li>Generate a list of all author affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
 </ul>
 
-<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc)
- to /tmp/affiliations.csv with csv;
+<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
 </code></pre>
 
 <ul>
@@ -519,6 +518,21 @@ Exception in thread &quot;Thread-54216&quot; org.apache.solr.client.solrj.impl.H
 
 <ul>
 <li>I&rsquo;ve sent a message to Atmire about the Solr error to see if it&rsquo;s related to their batch update module</li>
+</ul>
+
+<h2 id="2016-09-19">2016-09-19</h2>
+
+<ul>
+<li>Work on cleanups for author affiliations after Peter sent me his list of corrections/deletions:</li>
+</ul>
+
+<pre><code>$ ./fix-metadata-values.py -i affiliations_pb-322-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p fuuu
+$ ./delete-metadata-values.py -f cg.contributor.affiliation -i affiliations_pb-2-deletions.csv -m 211 -u dspace-d dspace-p fuuu
+</code></pre>
+
+<ul>
+<li>After that we need to take the top ~300 and make a controlled vocabulary for it</li>
+<li>I dumped a list of the top 300 affiliations from the database, sorted it alphabetically in OpenRefine, and created a controlled vocabulary for it (<a href="https://github.com/ilri/DSpace/pull/267">#267</a>)</li>
 </ul>
 
   </section>

public/index.xml

@@ -333,11 +333,10 @@ org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error
 
 <ul>
 <li>Looking at the top 20 IPs or so, most are Yahoo, MSN, Google, Baidu, TurnitIn (iParadigm), etc… do we have any real users?</li>
-<li>Generate a list of all Affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
+<li>Generate a list of all author affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
 </ul>
 
-<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc)
- to /tmp/affiliations.csv with csv;
+<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
 </code></pre>
 
 <ul>
@@ -458,6 +457,21 @@ Exception in thread "Thread-54216" org.apache.solr.client.solr
 <ul>
 <li>I’ve sent a message to Atmire about the Solr error to see if it’s related to their batch update module</li>
 </ul>
+
+<h2 id="2016-09-19">2016-09-19</h2>
+
+<ul>
+<li>Work on cleanups for author affiliations after Peter sent me his list of corrections/deletions:</li>
+</ul>
+
+<pre><code>$ ./fix-metadata-values.py -i affiliations_pb-322-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p fuuu
+$ ./delete-metadata-values.py -f cg.contributor.affiliation -i affiliations_pb-2-deletions.csv -m 211 -u dspace-d dspace-p fuuu
+</code></pre>
+
+<ul>
+<li>After that we need to take the top ~300 and make a controlled vocabulary for it</li>
+<li>I dumped a list of the top 300 affiliations from the database, sorted it alphabetically in OpenRefine, and created a controlled vocabulary for it (<a href="https://github.com/ilri/DSpace/pull/267">#267</a>)</li>
+</ul>
 </description>
     </item>
     

public/page/1/index.html

@@ -1 +1 @@
-<!DOCTYPE html><html><head><title>https://alanorth.github.io/cgspace-notes/</title><link rel="canonical" href="https://alanorth.github.io/cgspace-notes/"/><meta http-equiv="content-type" content="text/html; charset=utf-8" /><meta http-equiv="refresh" content="0; url=https://alanorth.github.io/cgspace-notes/" /></head></html>
+<!DOCTYPE html><html><head><link rel="canonical" href="https://alanorth.github.io/cgspace-notes/"/><meta http-equiv="content-type" content="text/html; charset=utf-8" /><meta http-equiv="refresh" content="0;url=https://alanorth.github.io/cgspace-notes/" /></head></html>

public/tags/notes/index.xml

@@ -333,11 +333,10 @@ org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error
 
 <ul>
 <li>Looking at the top 20 IPs or so, most are Yahoo, MSN, Google, Baidu, TurnitIn (iParadigm), etc… do we have any real users?</li>
-<li>Generate a list of all Affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
+<li>Generate a list of all author affiliations for Peter Ballantyne to go through, make corrections, and create a lookup list from:</li>
 </ul>
 
-<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc)
- to /tmp/affiliations.csv with csv;
+<pre><code>dspacetest=# \copy (select text_value, count(*) from metadatavalue where resource_type_id=2 and metadata_field_id=211 group by text_value order by count desc) to /tmp/affiliations.csv with csv;
 </code></pre>
 
 <ul>
@@ -458,6 +457,21 @@ Exception in thread "Thread-54216" org.apache.solr.client.solr
 <ul>
 <li>I’ve sent a message to Atmire about the Solr error to see if it’s related to their batch update module</li>
 </ul>
+
+<h2 id="2016-09-19">2016-09-19</h2>
+
+<ul>
+<li>Work on cleanups for author affiliations after Peter sent me his list of corrections/deletions:</li>
+</ul>
+
+<pre><code>$ ./fix-metadata-values.py -i affiliations_pb-322-corrections.csv -f cg.contributor.affiliation -t correct -m 211 -d dspace -u dspace -p fuuu
+$ ./delete-metadata-values.py -f cg.contributor.affiliation -i affiliations_pb-2-deletions.csv -m 211 -u dspace-d dspace-p fuuu
+</code></pre>
+
+<ul>
+<li>After that we need to take the top ~300 and make a controlled vocabulary for it</li>
+<li>I dumped a list of the top 300 affiliations from the database, sorted it alphabetically in OpenRefine, and created a controlled vocabulary for it (<a href="https://github.com/ilri/DSpace/pull/267">#267</a>)</li>
+</ul>
 </description>
     </item>