mirror of
https://github.com/ilri/cgspace-java-helpers.git
synced 2025-07-24 06:51:51 +02:00
Compare commits
14 Commits
3aa1503163
...
v5.2
Author | SHA1 | Date | |
---|---|---|---|
fd893d8c4e
|
|||
2263ac27e8
|
|||
cf7012d698
|
|||
7edc60e6ca
|
|||
fe2abc86c6
|
|||
e1d92ef2c7
|
|||
3e3c544cfa
|
|||
db9881faf6
|
|||
fa5fb60b5b
|
|||
44fb9a9f4d
|
|||
b790d5e4db
|
|||
08e7546a87
|
|||
ff076ecf50
|
|||
7a5dd1c094
|
24
.github/workflows/maven.yml
vendored
24
.github/workflows/maven.yml
vendored
@ -1,24 +0,0 @@
|
|||||||
# This workflow will build a Java project with Maven
|
|
||||||
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
|
|
||||||
|
|
||||||
name: Build
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [ dspace6 ]
|
|
||||||
pull_request:
|
|
||||||
branches: [ dspace6 ]
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build:
|
|
||||||
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v2
|
|
||||||
- name: Set up JDK 1.8
|
|
||||||
uses: actions/setup-java@v1
|
|
||||||
with:
|
|
||||||
java-version: 1.8
|
|
||||||
- name: Build with Maven
|
|
||||||
run: mvn -B package --file pom.xml
|
|
8
.travis.yml
Normal file
8
.travis.yml
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
dist: bionic
|
||||||
|
language: java
|
||||||
|
jdk:
|
||||||
|
- openjdk8
|
||||||
|
script:
|
||||||
|
- mvn package -B
|
||||||
|
|
||||||
|
# vim: ts=2 sw=2 et
|
48
README.md
48
README.md
@ -1,10 +1,10 @@
|
|||||||
# CGSpace Java Helpers [](https://github.com/ilri/cgspace-java-helpers/actions)
|
# CGSpace Java Helpers [](https://travis-ci.org/ilri/dspace-curation-tasks)
|
||||||
DSpace curation tasks and other Java-based helpers used on the [CGSpace](https://cgspace.cgiar.org) institutional repository:
|
DSpace curation tasks and other Java-based helpers used on the [CGSpace](https://cgspace.cgiar.org) institutional repository:
|
||||||
|
|
||||||
- **CountryCodeTagger**: add ISO 3166-1 Alpha2 country codes to items based on their existing country metadata
|
- **CountryCodeTagger**: add ISO 3166-1 Alpha2 country codes to items based on their existing country metadata
|
||||||
- **FixJpgJpgThumbnails**: Fix low-quality ".jpg.jpg" thumbnails by replacing them with their originals
|
- **FixJpgJpgThumbnails**: Fix low-quality ".jpg.jpg" thumbnails by replacing them with their originals
|
||||||
|
|
||||||
Tested on DSpace 6.3. Read more about the [DSpace curation system](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System).
|
Tested on DSpace 5.8. Read more about the [DSpace curation system](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System).
|
||||||
|
|
||||||
## Build and Install
|
## Build and Install
|
||||||
|
|
||||||
@ -15,7 +15,7 @@ To use these curation tasks in a DSpace project add the following dependency to
|
|||||||
<dependency>
|
<dependency>
|
||||||
<groupId>io.github.ilri.cgspace</groupId>
|
<groupId>io.github.ilri.cgspace</groupId>
|
||||||
<artifactId>cgspace-java-helpers</artifactId>
|
<artifactId>cgspace-java-helpers</artifactId>
|
||||||
<version>6.1-SNAPSHOT</version>
|
<version>5.1</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -31,18 +31,42 @@ $ mvn package
|
|||||||
Copy the resulting jar to the DSpace `lib` directory:
|
Copy the resulting jar to the DSpace `lib` directory:
|
||||||
|
|
||||||
```
|
```
|
||||||
$ cp target/cgspace-java-helpers-6.1-SNAPSHOT.jar ~/dspace/lib/
|
$ cp target/cgspace-java-helpers-5.1.jar ~/dspace/lib
|
||||||
```
|
```
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
Please refer to the appropriate README.md file:
|
Add the curation task to DSpace's `config/modules/curate.cfg`:
|
||||||
|
|
||||||
- Curation Tasks: [src/main/java/io/github/ilri/cgspace/ctasks/README.md](https://github.com/ilri/cgspace-java-helpers/blob/dspace6/src/main/java/io/github/ilri/cgspace/ctasks/README.md)
|
```
|
||||||
- Scripts: [src/main/java/io/github/ilri/cgspace/scripts/README.md](https://github.com/ilri/cgspace-java-helpers/blob/dspace6/src/main/java/io/github/ilri/cgspace/scripts/README.md)
|
plugin.named.org.dspace.curate.CurationTask = \
|
||||||
|
...
|
||||||
|
io.github.ilri.cgspace.ctasks.CountryCodeTagger = countrycodetagger \
|
||||||
|
io.github.ilri.cgspace.ctasks.CountryCodeTagger = countrycodetagger.force
|
||||||
|
```
|
||||||
|
|
||||||
## Todo
|
And then add a configuration file for the task in `config/modules/countrycodetagger.cfg`:
|
||||||
|
|
||||||
- Add a curation task to normalize DOIs to "https://doi.org" format
|
```
|
||||||
|
# name of the field containing ISO 3166-1 country names
|
||||||
|
iso3166.field = cg.coverage.country
|
||||||
|
|
||||||
|
# name of the field containing ISO 3166-1 Alpha2 country codes
|
||||||
|
iso3166-alpha2.field = cg.coverage.iso3166-alpha2
|
||||||
|
|
||||||
|
# only add country codes if an item doesn't have any (default false)
|
||||||
|
#forceupdate = false
|
||||||
|
```
|
||||||
|
|
||||||
|
*Note*: DSpace's curation system supports "profiles" where you can use the same task with different options, for example above I have a normal country code tagger and a "force" variant. To use the "force" variant you create a new configuration file with the overridden options in `config/modules/countrycodetagger.force.cfg`. The "force" profile clears all existing country codes and updates everything.
|
||||||
|
|
||||||
|
## Invocation
|
||||||
|
Once the jar is installed and you have added appropriate configuration in `~/dspace/config/modules`:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ ~/dspace/bin/dspace curate -t countrycodetagger -i 10568/3 -r - -l 500 -s object
|
||||||
|
```
|
||||||
|
|
||||||
|
*Note*: it is very important to set the cache limit (`-l`) and the database transaction scope to something sensible (`object`) if you're curating a community or collection with more than a few hundred items.
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
This project was initially created according to the [Maven Getting Started Guide](https://maven.apache.org/guides/getting-started/):
|
This project was initially created according to the [Maven Getting Started Guide](https://maven.apache.org/guides/getting-started/):
|
||||||
@ -51,6 +75,12 @@ This project was initially created according to the [Maven Getting Started Guide
|
|||||||
$ mvn -B archetype:generate -DgroupId=io.github.ilri.cgspace -DartifactId=cgspace-java-helpers -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4
|
$ mvn -B archetype:generate -DgroupId=io.github.ilri.cgspace -DartifactId=cgspace-java-helpers -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## TODO
|
||||||
|
|
||||||
|
- Make sure this doesn't work on items in the workflow
|
||||||
|
- Check for existence of metadata field before trying to add metadata
|
||||||
|
- Add tests
|
||||||
|
|
||||||
## License
|
## License
|
||||||
This work is licensed under the [GPLv3](https://www.gnu.org/licenses/gpl-3.0.en.html).
|
This work is licensed under the [GPLv3](https://www.gnu.org/licenses/gpl-3.0.en.html).
|
||||||
|
|
||||||
|
6
pom.xml
6
pom.xml
@ -6,7 +6,7 @@
|
|||||||
|
|
||||||
<groupId>io.github.ilri.cgspace</groupId>
|
<groupId>io.github.ilri.cgspace</groupId>
|
||||||
<artifactId>cgspace-java-helpers</artifactId>
|
<artifactId>cgspace-java-helpers</artifactId>
|
||||||
<version>6.1-SNAPSHOT</version>
|
<version>5.2</version>
|
||||||
|
|
||||||
<name>cgspace-java-helpers</name>
|
<name>cgspace-java-helpers</name>
|
||||||
<url>https://github.com/ilri/cgspace-java-helpers</url>
|
<url>https://github.com/ilri/cgspace-java-helpers</url>
|
||||||
@ -42,12 +42,12 @@
|
|||||||
<dependency>
|
<dependency>
|
||||||
<groupId>com.google.code.gson</groupId>
|
<groupId>com.google.code.gson</groupId>
|
||||||
<artifactId>gson</artifactId>
|
<artifactId>gson</artifactId>
|
||||||
<version>2.6.1</version>
|
<version>2.2.1</version>
|
||||||
</dependency>
|
</dependency>
|
||||||
<dependency>
|
<dependency>
|
||||||
<groupId>org.dspace</groupId>
|
<groupId>org.dspace</groupId>
|
||||||
<artifactId>dspace-api</artifactId>
|
<artifactId>dspace-api</artifactId>
|
||||||
<version>6.3</version>
|
<version>5.8</version>
|
||||||
<scope>provided</scope>
|
<scope>provided</scope>
|
||||||
</dependency>
|
</dependency>
|
||||||
</dependencies>
|
</dependencies>
|
||||||
|
@ -23,7 +23,7 @@ import org.apache.log4j.Logger;
|
|||||||
import org.dspace.authorize.AuthorizeException;
|
import org.dspace.authorize.AuthorizeException;
|
||||||
import org.dspace.content.DSpaceObject;
|
import org.dspace.content.DSpaceObject;
|
||||||
import org.dspace.content.Item;
|
import org.dspace.content.Item;
|
||||||
import org.dspace.content.MetadataValue;
|
import org.dspace.content.Metadatum;
|
||||||
import org.dspace.core.Constants;
|
import org.dspace.core.Constants;
|
||||||
import org.dspace.curate.AbstractCurationTask;
|
import org.dspace.curate.AbstractCurationTask;
|
||||||
import org.dspace.curate.Curator;
|
import org.dspace.curate.Curator;
|
||||||
@ -81,11 +81,7 @@ public class CountryCodeTagger extends AbstractCurationTask
|
|||||||
|
|
||||||
Item item = (Item)dso;
|
Item item = (Item)dso;
|
||||||
|
|
||||||
try {
|
alpha2Result = performAlpha2(item, config);
|
||||||
alpha2Result = performAlpha2(item, config);
|
|
||||||
} catch (SQLException throwables) {
|
|
||||||
throwables.printStackTrace();
|
|
||||||
}
|
|
||||||
|
|
||||||
setResult(alpha2Result.getResult());
|
setResult(alpha2Result.getResult());
|
||||||
report(alpha2Result.getResult());
|
report(alpha2Result.getResult());
|
||||||
@ -94,14 +90,15 @@ public class CountryCodeTagger extends AbstractCurationTask
|
|||||||
return alpha2Result.getStatus();
|
return alpha2Result.getStatus();
|
||||||
}
|
}
|
||||||
|
|
||||||
public CountryCodeTaggerResult performAlpha2(Item item, CountryCodeTaggerConfig config) throws IOException, SQLException {
|
public CountryCodeTaggerResult performAlpha2(Item item, CountryCodeTaggerConfig config) throws IOException
|
||||||
|
{
|
||||||
CountryCodeTaggerResult alpha2Result = new CountryCodeTaggerResult();
|
CountryCodeTaggerResult alpha2Result = new CountryCodeTaggerResult();
|
||||||
String itemHandle = item.getHandle();
|
String itemHandle = item.getHandle();
|
||||||
|
|
||||||
List<MetadataValue> itemCountries = itemService.getMetadataByMetadataString(item, config.iso3166Field);
|
Metadatum[] itemCountries = item.getMetadataByMetadataString(config.iso3166Field);
|
||||||
|
|
||||||
// skip items that don't have country metadata
|
// skip items that don't have country metadata
|
||||||
if (itemCountries.size() == 0) {
|
if (itemCountries.length == 0) {
|
||||||
alpha2Result.setResult(itemHandle + ": no countries, skipping.");
|
alpha2Result.setResult(itemHandle + ": no countries, skipping.");
|
||||||
alpha2Result.setStatus(Curator.CURATE_SKIP);
|
alpha2Result.setStatus(Curator.CURATE_SKIP);
|
||||||
} else {
|
} else {
|
||||||
@ -120,25 +117,25 @@ public class CountryCodeTagger extends AbstractCurationTask
|
|||||||
String[] iso3166Alpha2FieldParts = config.iso3166Alpha2Field.split("\\.");
|
String[] iso3166Alpha2FieldParts = config.iso3166Alpha2Field.split("\\.");
|
||||||
|
|
||||||
if (config.forceupdate) {
|
if (config.forceupdate) {
|
||||||
itemService.clearMetadata(Curator.curationContext(), item, iso3166Alpha2FieldParts[0], iso3166Alpha2FieldParts[1], iso3166Alpha2FieldParts[2], Item.ANY);
|
item.clearMetadata(iso3166Alpha2FieldParts[0], iso3166Alpha2FieldParts[1], iso3166Alpha2FieldParts[2], Item.ANY);
|
||||||
}
|
}
|
||||||
|
|
||||||
// check the item's country codes, if any
|
// check the item's country codes, if any
|
||||||
List<MetadataValue> itemAlpha2CountryCodes = itemService.getMetadataByMetadataString(item, config.iso3166Alpha2Field);
|
Metadatum[] itemAlpha2CountryCodes = item.getMetadataByMetadataString(config.iso3166Alpha2Field);
|
||||||
|
|
||||||
if (itemAlpha2CountryCodes.size() == 0) {
|
if (itemAlpha2CountryCodes.length == 0) {
|
||||||
List<String> newAlpha2Codes = new ArrayList<String>();
|
List<String> newAlpha2Codes = new ArrayList<String>();
|
||||||
for (MetadataValue itemCountry : itemCountries) {
|
for (Metadatum itemCountry : itemCountries) {
|
||||||
//check ISO 3166-1 countries
|
//check ISO 3166-1 countries
|
||||||
for (CountriesVocabulary.Country country : isocodesCountriesJson.countries) {
|
for (CountriesVocabulary.Country country : isocodesCountriesJson.countries) {
|
||||||
if (itemCountry.getValue().equalsIgnoreCase(country.getName()) || itemCountry.getValue().equalsIgnoreCase(country.get_official_name()) || itemCountry.getValue().equalsIgnoreCase(country.get_common_name())) {
|
if (itemCountry.value.equalsIgnoreCase(country.getName()) || itemCountry.value.equalsIgnoreCase(country.get_official_name()) || itemCountry.value.equalsIgnoreCase(country.get_common_name())) {
|
||||||
newAlpha2Codes.add(country.getAlpha_2());
|
newAlpha2Codes.add(country.getAlpha_2());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
//check CGSpace countries
|
//check CGSpace countries
|
||||||
for (CountriesVocabulary.Country country : cgspaceCountriesJson.countries) {
|
for (CountriesVocabulary.Country country : cgspaceCountriesJson.countries) {
|
||||||
if (itemCountry.getValue().equalsIgnoreCase(country.getCgspace_name())) {
|
if (itemCountry.value.equalsIgnoreCase(country.getCgspace_name())) {
|
||||||
newAlpha2Codes.add(country.getAlpha_2());
|
newAlpha2Codes.add(country.getAlpha_2());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -146,8 +143,9 @@ public class CountryCodeTagger extends AbstractCurationTask
|
|||||||
|
|
||||||
if (newAlpha2Codes.size() > 0) {
|
if (newAlpha2Codes.size() > 0) {
|
||||||
try {
|
try {
|
||||||
itemService.addMetadata(Curator.curationContext(), item, iso3166Alpha2FieldParts[0], iso3166Alpha2FieldParts[1], iso3166Alpha2FieldParts[2], "en_US", newAlpha2Codes);
|
// add metadata values (casting the List<String> to an array)
|
||||||
itemService.update(Curator.curationContext(), item);
|
item.addMetadata(iso3166Alpha2FieldParts[0], iso3166Alpha2FieldParts[1], iso3166Alpha2FieldParts[2], "en_US", newAlpha2Codes.toArray(new String[0]));
|
||||||
|
item.update();
|
||||||
} catch (SQLException | AuthorizeException sqle) {
|
} catch (SQLException | AuthorizeException sqle) {
|
||||||
config.log.debug(sqle.getMessage());
|
config.log.debug(sqle.getMessage());
|
||||||
alpha2Result.setResult(itemHandle + ": error");
|
alpha2Result.setResult(itemHandle + ": error");
|
||||||
|
@ -1,72 +0,0 @@
|
|||||||
# Curation Tasks
|
|
||||||
DSpace curation tasks used on the [CGSpace](https://cgspace.cgiar.org) institutional repository:
|
|
||||||
|
|
||||||
- **CountryCodeTagger**: add ISO 3166-1 Alpha2 country codes to items based on their existing country metadata
|
|
||||||
|
|
||||||
Tested on DSpace 6.3. Read more about the [DSpace curation system](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System).
|
|
||||||
|
|
||||||
## Build and Install
|
|
||||||
|
|
||||||
### Integrate into DSpace Build
|
|
||||||
To use these curation tasks in a DSpace project add the following dependency to `dspace/modules/additions/pom.xml`:
|
|
||||||
|
|
||||||
```
|
|
||||||
<dependency>
|
|
||||||
<groupId>io.github.ilri.cgspace</groupId>
|
|
||||||
<artifactId>cgspace-java-helpers</artifactId>
|
|
||||||
<version>6.1-SNAPSHOT</version>
|
|
||||||
</dependency>
|
|
||||||
```
|
|
||||||
|
|
||||||
The jar will be copied to all DSpace applications.
|
|
||||||
|
|
||||||
### Manual Build and Install
|
|
||||||
To build the standalone jar:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ mvn package
|
|
||||||
```
|
|
||||||
|
|
||||||
Copy the resulting jar to the DSpace `lib` directory:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ cp target/cgspace-java-helpers-6.1-SNAPSHOT.jar ~/dspace/lib/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
Add the curation task to DSpace's `config/modules/curate.cfg`:
|
|
||||||
|
|
||||||
```
|
|
||||||
plugin.named.org.dspace.curate.CurationTask = io.github.ilri.cgspace.ctasks.CountryCodeTagger = countrycodetagger
|
|
||||||
plugin.named.org.dspace.curate.CurationTask = io.github.ilri.cgspace.ctasks.CountryCodeTagger = countrycodetagger.force
|
|
||||||
```
|
|
||||||
|
|
||||||
And then add the following variables to your `local.cfg` or some other [configuration file that is included](https://wiki.lyrasis.org/display/DSDOC6x/Configuration+Reference#ConfigurationReference-IncludingotherPropertyFiles):
|
|
||||||
|
|
||||||
```
|
|
||||||
# name of the field containing ISO 3166-1 country names
|
|
||||||
countrycodetagger.iso3166.field = cg.coverage.country
|
|
||||||
|
|
||||||
# name of the field containing ISO 3166-1 Alpha2 country codes
|
|
||||||
countrycodetagger.iso3166-alpha2.field = cg.coverage.iso3166-alpha2
|
|
||||||
|
|
||||||
# only add country codes if an item doesn't have any (default false)
|
|
||||||
#countrycodetagger.forceupdate = false
|
|
||||||
```
|
|
||||||
|
|
||||||
*Note*: DSpace's curation system supports "profiles" where you can use the same task with different options, for example above I have a normal country code tagger task and a "force" variant. The "force" variant is the same task, but it looks for configuration variables using the `countrycodetagger.force` instead. To use the "force" variant you simply need to add these new variables with the `forceupdate` parameter overridden to the same configuration file where you put the other variables. The "force" profile clears all existing country codes and updates everything.
|
|
||||||
|
|
||||||
## Invocation
|
|
||||||
Once the jar is installed and you have added appropriate configuration in `~/dspace/config/modules`:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ ~/dspace/bin/dspace curate -t countrycodetagger -i 10568/3 -r - -s object
|
|
||||||
```
|
|
||||||
|
|
||||||
*Note*: it is very important to set the database transaction scope to something sensible (`object`) if you're curating a community or collection with more than a few hundred items.
|
|
||||||
|
|
||||||
## TODO
|
|
||||||
|
|
||||||
- Make sure this doesn't work on items in the workflow
|
|
||||||
- Check for existence of metadata field before trying to add metadata
|
|
||||||
- Add tests
|
|
@ -5,28 +5,18 @@ import org.dspace.authorize.AuthorizeException;
|
|||||||
import org.dspace.content.*;
|
import org.dspace.content.*;
|
||||||
import org.dspace.core.Constants;
|
import org.dspace.core.Constants;
|
||||||
import org.dspace.core.Context;
|
import org.dspace.core.Context;
|
||||||
import org.dspace.content.factory.ContentServiceFactory;
|
import org.dspace.handle.HandleManager;
|
||||||
import org.dspace.content.service.ItemService;
|
|
||||||
import org.dspace.handle.factory.HandleServiceFactory;
|
|
||||||
import org.dspace.handle.service.HandleService;
|
|
||||||
import org.dspace.content.service.BundleService;
|
|
||||||
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.sql.SQLException;
|
import java.sql.SQLException;
|
||||||
import java.util.Iterator;
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author Andrea Schweer schweer@waikato.ac.nz for the LCoNZ Institutional Research Repositories
|
* @author Andrea Schweer schweer@waikato.ac.nz for the LCoNZ Institutional Research Repositories
|
||||||
* @author Alan Orth for the International Livestock Research Institute
|
* @author Alan Orth for the International Livestock Research Institute
|
||||||
* @version 6.1
|
* @version 5.1-SNAPSHOT
|
||||||
* @since 5.1
|
* @since 5.1-SNAPSHOT
|
||||||
*/
|
*/
|
||||||
public class FixJpgJpgThumbnails {
|
public class FixJpgJpgThumbnails {
|
||||||
//note: static members belong to the class itself, not any one instance
|
|
||||||
public static ItemService itemService = ContentServiceFactory.getInstance().getItemService();
|
|
||||||
public static HandleService handleService = HandleServiceFactory.getInstance().getHandleService();
|
|
||||||
public static BundleService bundleService = ContentServiceFactory.getInstance().getBundleService();
|
|
||||||
|
|
||||||
public static void main(String[] args) {
|
public static void main(String[] args) {
|
||||||
String parentHandle = null;
|
String parentHandle = null;
|
||||||
@ -40,25 +30,25 @@ public class FixJpgJpgThumbnails {
|
|||||||
context.turnOffAuthorisationSystem();
|
context.turnOffAuthorisationSystem();
|
||||||
|
|
||||||
if (StringUtils.isBlank(parentHandle)) {
|
if (StringUtils.isBlank(parentHandle)) {
|
||||||
process(context, itemService.findAll(context));
|
process(context, Item.findAll(context));
|
||||||
} else {
|
} else {
|
||||||
DSpaceObject parent = handleService.resolveToObject(context, parentHandle);
|
DSpaceObject parent = HandleManager.resolveToObject(context, parentHandle);
|
||||||
if (parent != null) {
|
if (parent != null) {
|
||||||
switch (parent.getType()) {
|
switch (parent.getType()) {
|
||||||
case Constants.COLLECTION:
|
case Constants.COLLECTION:
|
||||||
process(context, itemService.findByCollection(context, (Collection) parent));
|
process(context, ((Collection) parent).getAllItems()); // getAllItems because we want to work on non-archived ones as well
|
||||||
break;
|
break;
|
||||||
case Constants.COMMUNITY:
|
case Constants.COMMUNITY:
|
||||||
List<Collection> collections = ((Community) parent).getCollections();
|
Collection[] collections = ((Community) parent).getCollections();
|
||||||
for (Collection collection : collections) {
|
for (Collection collection : collections) {
|
||||||
process(context, itemService.findAllByCollection(context, collection));
|
process(context, collection.getAllItems()); // getAllItems because we want to work on non-archived ones as well
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
case Constants.SITE:
|
case Constants.SITE:
|
||||||
process(context, itemService.findAll(context));
|
process(context, Item.findAll(context));
|
||||||
break;
|
break;
|
||||||
case Constants.ITEM:
|
case Constants.ITEM:
|
||||||
processItem(context, (Item) parent);
|
processItem((Item) parent);
|
||||||
context.commit();
|
context.commit();
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
@ -73,59 +63,36 @@ public class FixJpgJpgThumbnails {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private static void process(Context context, Iterator<Item> items) throws SQLException, IOException, AuthorizeException {
|
private static void process(Context context, ItemIterator items) throws SQLException, IOException, AuthorizeException {
|
||||||
while (items.hasNext()) {
|
while (items.hasNext()) {
|
||||||
Item item = items.next();
|
Item item = items.next();
|
||||||
processItem(context, item);
|
processItem(item);
|
||||||
itemService.update(context, item);
|
context.commit();
|
||||||
|
item.decache();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private static void processItem(Context context, Item item) throws SQLException, AuthorizeException, IOException {
|
private static void processItem(Item item) throws SQLException, AuthorizeException, IOException {
|
||||||
// Some bitstreams like Infographics are large JPGs and put in the ORIGINAL bundle on purpose so we shouldn't
|
Bundle[] thumbnailBundles = item.getBundles("THUMBNAIL");
|
||||||
// swap them.
|
|
||||||
List<MetadataValue> itemTypes = itemService.getMetadataByMetadataString(item, "dcterms.type");
|
|
||||||
boolean itemHasInfographic = false;
|
|
||||||
for (MetadataValue itemType: itemTypes) {
|
|
||||||
if (itemType.getValue().equals("Infographic")) {
|
|
||||||
itemHasInfographic = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
List<Bundle> thumbnailBundles = item.getBundles("THUMBNAIL");
|
|
||||||
for (Bundle thumbnailBundle : thumbnailBundles) {
|
for (Bundle thumbnailBundle : thumbnailBundles) {
|
||||||
List<Bitstream> thumbnailBundleBitstreams = thumbnailBundle.getBitstreams();
|
Bitstream[] thumbnailBundleBitstreams = thumbnailBundle.getBitstreams();
|
||||||
for (Bitstream thumbnailBitstream : thumbnailBundleBitstreams) {
|
for (Bitstream thumbnailBitstream : thumbnailBundleBitstreams) {
|
||||||
String thumbnailName = thumbnailBitstream.getName();
|
String thumbnailName = thumbnailBitstream.getName();
|
||||||
|
|
||||||
if (thumbnailName.toLowerCase().contains(".jpg.jpg")) {
|
if (thumbnailName.toLowerCase().contains(".jpg.jpg")) {
|
||||||
List<Bundle> originalBundles = item.getBundles("ORIGINAL");
|
Bundle[] originalBundles = item.getBundles("ORIGINAL");
|
||||||
for (Bundle originalBundle : originalBundles) {
|
for (Bundle originalBundle : originalBundles) {
|
||||||
List<Bitstream> originalBundleBitstreams = originalBundle.getBitstreams();
|
Bitstream[] originalBundleBitstreams = originalBundle.getBitstreams();
|
||||||
|
|
||||||
for (Bitstream originalBitstream : originalBundleBitstreams) {
|
for(Bitstream originalBitstream : originalBundleBitstreams) {
|
||||||
String originalName = originalBitstream.getName();
|
String originalName = originalBitstream.getName();
|
||||||
|
|
||||||
long originalBitstreamBytes = originalBitstream.getSize();
|
//check if the original file name is the same as the thumbnail name minus the extra ".jpg"
|
||||||
|
if (originalName.equalsIgnoreCase(StringUtils.removeEndIgnoreCase(thumbnailName, ".jpg")) && ("Generated Thumbnail".equals(thumbnailBitstream.getDescription()) || "IM Thumbnail".equals(thumbnailBitstream.getDescription()))) {
|
||||||
/*
|
|
||||||
- check if the original file name is the same as the thumbnail name minus the extra ".jpg"
|
|
||||||
- check if the thumbnail description indicates it was automatically generated
|
|
||||||
- check if the item has dc.type Infographic (JPG could be the "real" item!)
|
|
||||||
- check if the original bitstream is less than ~100KiB
|
|
||||||
- Note: in my tests there were 4022 items with ".jpg.jpg" thumbnails totaling 394549249
|
|
||||||
bytes for an average of about 98KiB so ~100KiB seems like a good cut off
|
|
||||||
*/
|
|
||||||
if (
|
|
||||||
originalName.equalsIgnoreCase(StringUtils.removeEndIgnoreCase(thumbnailName, ".jpg"))
|
|
||||||
&& ("Generated Thumbnail".equals(thumbnailBitstream.getDescription()) || "IM Thumbnail".equals(thumbnailBitstream.getDescription()))
|
|
||||||
&& !itemHasInfographic
|
|
||||||
&& originalBitstreamBytes < 100000
|
|
||||||
) {
|
|
||||||
System.out.println(item.getHandle() + ": replacing " + thumbnailName + " with " + originalName);
|
System.out.println(item.getHandle() + ": replacing " + thumbnailName + " with " + originalName);
|
||||||
|
|
||||||
//add the original bitstream to the THUMBNAIL bundle
|
//add the original bitstream to the THUMBNAIL bundle
|
||||||
bundleService.addBitstream(context, thumbnailBundle, originalBitstream);
|
thumbnailBundle.addBitstream(originalBitstream);
|
||||||
//remove the original bitstream from the ORIGINAL bundle
|
//remove the original bitstream from the ORIGINAL bundle
|
||||||
originalBundle.removeBitstream(originalBitstream);
|
originalBundle.removeBitstream(originalBitstream);
|
||||||
//remove the JpgJpg bitstream from the THUMBNAIL bundle
|
//remove the JpgJpg bitstream from the THUMBNAIL bundle
|
||||||
|
@ -1,41 +0,0 @@
|
|||||||
# Scripts
|
|
||||||
Java-based helpers used on the [CGSpace](https://cgspace.cgiar.org) institutional repository:
|
|
||||||
|
|
||||||
- **FixJpgJpgThumbnails**: Fix low-quality ".jpg.jpg" thumbnails by replacing them with their originals
|
|
||||||
|
|
||||||
Tested on DSpace 6.3. Read more about the [DSpace curation system](https://wiki.lyrasis.org/display/DSDOC5x/Curation+System).
|
|
||||||
|
|
||||||
## Build and Install
|
|
||||||
|
|
||||||
### Integrate into DSpace Build
|
|
||||||
To use these curation tasks in a DSpace project add the following dependency to `dspace/modules/additions/pom.xml`:
|
|
||||||
|
|
||||||
```
|
|
||||||
<dependency>
|
|
||||||
<groupId>io.github.ilri.cgspace</groupId>
|
|
||||||
<artifactId>cgspace-java-helpers</artifactId>
|
|
||||||
<version>6.1-SNAPSHOT</version>
|
|
||||||
</dependency>
|
|
||||||
```
|
|
||||||
|
|
||||||
The jar will be copied to all DSpace applications.
|
|
||||||
|
|
||||||
### Manual Build and Install
|
|
||||||
To build the standalone jar:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ mvn package
|
|
||||||
```
|
|
||||||
|
|
||||||
Copy the resulting jar to the DSpace `lib` directory:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ cp target/cgspace-java-helpers-6.1-SNAPSHOT.jar ~/dspace/lib/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Invocation
|
|
||||||
The script only takes one argument, which is a community, collection, or item:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ dspace dsrun io.github.ilri.cgspace.scripts.FixJpgJpgThumbnails 10568/83389
|
|
||||||
```
|
|
Reference in New Issue
Block a user