From d25ea402cae00aaf8e2427d6382e8f5cb0c0e19f Mon Sep 17 00:00:00 2001 From: Alan Orth Date: Sat, 6 Feb 2016 19:28:21 +0200 Subject: [PATCH] Add notes for 2016-02-06 Signed-off-by: Alan Orth --- content/2016-02.md | 33 ++++++++++++++++++++++++++++++ public/index.html | 8 +++++++- public/index.xml | 39 ++++++++++++++++++++++++++++++++++++ public/tags/notes/index.html | 8 +++++++- public/tags/notes/index.xml | 39 ++++++++++++++++++++++++++++++++++++ 5 files changed, 125 insertions(+), 2 deletions(-) diff --git a/content/2016-02.md b/content/2016-02.md index 7c435aec4..d003c7761 100644 --- a/content/2016-02.md +++ b/content/2016-02.md @@ -16,3 +16,36 @@ image = "../images/bg.jpg" - Not only are there 49,000 countries, we have some blanks (25)... - Also, lots of things like "COTE D`LVOIRE" and "COTE D IVOIRE" + +## 2016-02-06 + +- Found a way to get items with null/empty metadata values from SQL +- First, find the `metadata_field_id` for the field you want from the `metadatafieldregistry` table: + +``` +dspacetest=# select * from metadatafieldregistry; +``` + +- In this case our country field is 78 +- Now find all resources with type 2 (item) that have null/empty values for that field: + +``` +dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL); +``` + +- Then you can find the handle that owns it from its `resource_id`: + +``` +dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678'; +``` + +- It's 25 items so editing in the web UI is annoying, let's try SQL! + +``` +dspacetest=# delete from metadatavalue where metadata_field_id=78 and text_value=''; +DELETE 25 +``` + +- After that perhaps a regular `dspace index-discovery` (no -b) *should* suffice... +- Hmm, I indexed, cleared the Cocoon cache, and restarted Tomcat but the 25 "|||" countries are still there +- Maybe I need to do a full re-index... diff --git a/public/index.html b/public/index.html index f92a6fa27..25ae8d191 100644 --- a/public/index.html +++ b/public/index.html @@ -71,9 +71,15 @@
- 2016-02-05 Looking at some DAGRIS data for Abenet Yabowork Lots of issues with spaces, newlines, etc causing the import to fail I noticed we have a very interesting list of countries on CGSpace: Not only are there 49,000 countries, we have some blanks (25)… Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE” + 2016-02-05 Looking at some DAGRIS data for Abenet Yabowork Lots of issues with spaces, newlines, etc causing the import to fail I noticed we have a very interesting list of countries on CGSpace: Not only are there 49,000 countries, we have some blanks (25)… Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE” 2016-02-06 Found a way to get items with null/empty metadata values from SQL First, find the metadata_field_id for the field you want from the metadatafieldregistry table: dspacetest=# select * from metadatafieldregistry; In this case our country field is 78 Now find all resources with type 2 (item) that have null/empty values for that field: dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL); Then you can find the handle that owns it from its resource_id: dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678'; It’s 25 items so editing in the web UI is annoying, let’s try SQL!
+ + diff --git a/public/index.xml b/public/index.xml index c30189253..9cac82c32 100644 --- a/public/index.xml +++ b/public/index.xml @@ -31,6 +31,45 @@ <li>Not only are there 49,000 countries, we have some blanks (25)&hellip;</li> <li>Also, lots of things like &ldquo;COTE D`LVOIRE&rdquo; and &ldquo;COTE D IVOIRE&rdquo;</li> </ul> + +<h2 id="2016-02-06:124a59adbaa8ef13e1518d003fc03981">2016-02-06</h2> + +<ul> +<li>Found a way to get items with null/empty metadata values from SQL</li> +<li>First, find the <code>metadata_field_id</code> for the field you want from the <code>metadatafieldregistry</code> table:</li> +</ul> + +<pre><code>dspacetest=# select * from metadatafieldregistry; +</code></pre> + +<ul> +<li>In this case our country field is 78</li> +<li>Now find all resources with type 2 (item) that have null/empty values for that field:</li> +</ul> + +<pre><code>dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL); +</code></pre> + +<ul> +<li>Then you can find the handle that owns it from its <code>resource_id</code>:</li> +</ul> + +<pre><code>dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678'; +</code></pre> + +<ul> +<li>It&rsquo;s 25 items so editing in the web UI is annoying, let&rsquo;s try SQL!</li> +</ul> + +<pre><code>dspacetest=# delete from metadatavalue where metadata_field_id=78 and text_value=''; +DELETE 25 +</code></pre> + +<ul> +<li>After that perhaps a regular <code>dspace index-discovery</code> (no -b) <em>should</em> suffice&hellip;</li> +<li>Hmm, I indexed, cleared the Cocoon cache, and restarted Tomcat but the 25 &ldquo;|||&rdquo; countries are still there</li> +<li>Maybe I need to do a full re-index&hellip;</li> +</ul> diff --git a/public/tags/notes/index.html b/public/tags/notes/index.html index c40bba1df..057dbd441 100644 --- a/public/tags/notes/index.html +++ b/public/tags/notes/index.html @@ -75,9 +75,15 @@
- 2016-02-05 Looking at some DAGRIS data for Abenet Yabowork Lots of issues with spaces, newlines, etc causing the import to fail I noticed we have a very interesting list of countries on CGSpace: Not only are there 49,000 countries, we have some blanks (25)… Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE” + 2016-02-05 Looking at some DAGRIS data for Abenet Yabowork Lots of issues with spaces, newlines, etc causing the import to fail I noticed we have a very interesting list of countries on CGSpace: Not only are there 49,000 countries, we have some blanks (25)… Also, lots of things like “COTE D`LVOIRE” and “COTE D IVOIRE” 2016-02-06 Found a way to get items with null/empty metadata values from SQL First, find the metadata_field_id for the field you want from the metadatafieldregistry table: dspacetest=# select * from metadatafieldregistry; In this case our country field is 78 Now find all resources with type 2 (item) that have null/empty values for that field: dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL); Then you can find the handle that owns it from its resource_id: dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678'; It’s 25 items so editing in the web UI is annoying, let’s try SQL!
+ + diff --git a/public/tags/notes/index.xml b/public/tags/notes/index.xml index 8fa6356f3..3b5ae51f6 100644 --- a/public/tags/notes/index.xml +++ b/public/tags/notes/index.xml @@ -31,6 +31,45 @@ <li>Not only are there 49,000 countries, we have some blanks (25)&hellip;</li> <li>Also, lots of things like &ldquo;COTE D`LVOIRE&rdquo; and &ldquo;COTE D IVOIRE&rdquo;</li> </ul> + +<h2 id="2016-02-06:124a59adbaa8ef13e1518d003fc03981">2016-02-06</h2> + +<ul> +<li>Found a way to get items with null/empty metadata values from SQL</li> +<li>First, find the <code>metadata_field_id</code> for the field you want from the <code>metadatafieldregistry</code> table:</li> +</ul> + +<pre><code>dspacetest=# select * from metadatafieldregistry; +</code></pre> + +<ul> +<li>In this case our country field is 78</li> +<li>Now find all resources with type 2 (item) that have null/empty values for that field:</li> +</ul> + +<pre><code>dspacetest=# select resource_id from metadatavalue where resource_type_id=2 and metadata_field_id=78 and (text_value='' OR text_value IS NULL); +</code></pre> + +<ul> +<li>Then you can find the handle that owns it from its <code>resource_id</code>:</li> +</ul> + +<pre><code>dspacetest=# select handle from item, handle where handle.resource_id = item.item_id AND item.item_id = '22678'; +</code></pre> + +<ul> +<li>It&rsquo;s 25 items so editing in the web UI is annoying, let&rsquo;s try SQL!</li> +</ul> + +<pre><code>dspacetest=# delete from metadatavalue where metadata_field_id=78 and text_value=''; +DELETE 25 +</code></pre> + +<ul> +<li>After that perhaps a regular <code>dspace index-discovery</code> (no -b) <em>should</em> suffice&hellip;</li> +<li>Hmm, I indexed, cleared the Cocoon cache, and restarted Tomcat but the 25 &ldquo;|||&rdquo; countries are still there</li> +<li>Maybe I need to do a full re-index&hellip;</li> +</ul>