Add notes for 2018-03-21

content/posts/2018-03.md

@@ -348,3 +348,25 @@ java.lang.IllegalArgumentException: No choices plugin was configured for  field
 ```
 
 - I have to figure that one out...
+
+## 2018-03-21
+
+- Looks like the indexing gets confused that there is still data in the `authority` column
+- Unfortunately this causes those items to simply not be indexed, which users noticed because item counts were cut in half and old items showed up in RSS!
+- Since we've migrated the ORCID identifiers associated with the authority data to the `cg.creator.id` field we can nullify the authorities remaining in the database:
+
+```sql
+dspace=# UPDATE metadatavalue SET authority=NULL WHERE resource_type_id=2 AND metadata_field_id=3 AND authority IS NOT NULL;
+UPDATE 195463
+```
+
+- After this the indexing works as usual and item counts and facets are back to normal
+- Send Peter a list of all authors to correct:
+
+```sql
+dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element
+= 'contributor' and qualifier = 'author') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/authors.csv with csv header;
+COPY 56156
+```
+
+- Afterwards we'll want to do some batch tagging of ORCID identifiers to these names

docs/2018-03/index.html

@@ -20,7 +20,7 @@ Export a CSV of the IITA community metadata for Martin Mueller
 
 <meta property="article:published_time" content="2018-03-02T16:07:54&#43;02:00"/>
 
-<meta property="article:modified_time" content="2018-03-20T20:31:54&#43;02:00"/>
+<meta property="article:modified_time" content="2018-03-20T21:04:10&#43;02:00"/>
 
 
 
@@ -51,9 +51,9 @@ Export a CSV of the IITA community metadata for Martin Mueller
   "@type": "BlogPosting",
   "headline": "March, 2018",
   "url": "https://alanorth.github.io/cgspace-notes/2018-03/",
-  "wordCount": "1948",
+  "wordCount": "2113",
   "datePublished": "2018-03-02T16:07:54&#43;02:00",
-  "dateModified": "2018-03-20T20:31:54&#43;02:00",
+  "dateModified": "2018-03-20T21:04:10&#43;02:00",
   "author": {
     "@type": "Person",
     "name": "Alan Orth"
@@ -512,6 +512,32 @@ java.lang.IllegalArgumentException: No choices plugin was configured for  field
 <li>I have to figure that one out&hellip;</li>
 </ul>
 
+<h2 id="2018-03-21">2018-03-21</h2>
+
+<ul>
+<li>Looks like the indexing gets confused that there is still data in the <code>authority</code> column</li>
+<li>Unfortunately this causes those items to simply not be indexed, which users noticed because item counts were cut in half and old items showed up in RSS!</li>
+<li>Since we&rsquo;ve migrated the ORCID identifiers associated with the authority data to the <code>cg.creator.id</code> field we can nullify the authorities remaining in the database:</li>
+</ul>
+
+<pre><code class="language-sql">dspace=# UPDATE metadatavalue SET authority=NULL WHERE resource_type_id=2 AND metadata_field_id=3 AND authority IS NOT NULL;
+UPDATE 195463
+</code></pre>
+
+<ul>
+<li>After this the indexing works as usual and item counts and facets are back to normal</li>
+<li>Send Peter a list of all authors to correct:</li>
+</ul>
+
+<pre><code class="language-sql">dspace=# \copy (select distinct text_value, count(*) as count from metadatavalue where metadata_field_id = (select metadata_field_id from metadatafieldregistry where element
+= 'contributor' and qualifier = 'author') AND resource_type_id = 2 group by text_value order by count desc) to /tmp/authors.csv with csv header;
+COPY 56156
+</code></pre>
+
+<ul>
+<li>Afterwards we&rsquo;ll want to do some batch tagging of ORCID identifiers to these names</li>
+</ul>
+
   
 
   

docs/sitemap.xml

@@ -4,7 +4,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/2018-03/</loc>
-    <lastmod>2018-03-20T20:31:54+02:00</lastmod>
+    <lastmod>2018-03-20T21:04:10+02:00</lastmod>
   </url>
   
   <url>
@@ -154,7 +154,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/</loc>
-    <lastmod>2018-03-20T20:31:54+02:00</lastmod>
+    <lastmod>2018-03-20T21:04:10+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -165,7 +165,7 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
-    <lastmod>2018-03-20T20:31:54+02:00</lastmod>
+    <lastmod>2018-03-20T21:04:10+02:00</lastmod>
     <priority>0</priority>
   </url>
   
@@ -177,13 +177,13 @@
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
-    <lastmod>2018-03-20T20:31:54+02:00</lastmod>
+    <lastmod>2018-03-20T21:04:10+02:00</lastmod>
     <priority>0</priority>
   </url>
   
   <url>
     <loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
-    <lastmod>2018-03-20T20:31:54+02:00</lastmod>
+    <lastmod>2018-03-20T21:04:10+02:00</lastmod>
     <priority>0</priority>
   </url>