1
0
mirror of https://github.com/ilri/dspace-statistics-api.git synced 2024-10-08 18:04:18 +02:00
A simple REST API to expose Solr view and download statistics for items in a DSpace repository.
Go to file
2020-04-13 15:25:17 +03:00
contrib contrib: Update systemd unit files for refactor 2018-10-28 11:14:21 +02:00
dspace_statistics_api dspace_statistics_api/docs/index.html: Adjust help text 2020-03-02 14:30:16 +02:00
tests tests/dspacestatistics.sql: Update 2020-03-02 12:36:06 +02:00
.build.yml Disable emojis and animated output in CI 2019-03-17 23:39:38 +02:00
.flake8 Enable Flake8 validation in Hound CI 2018-11-04 00:48:06 +02:00
.gitignore Update docs to remove SQLite stuff 2018-09-25 00:56:01 +03:00
.hound.yml .hound.yml: Set pull requests to failed if build fails 2018-11-04 00:53:37 +02:00
.travis.yml .travis.yml: Don't build in a container 2020-03-29 16:36:47 +03:00
CHANGELOG.md CHANGELOG.md: Add changes for v1.2.1 2020-03-02 14:32:07 +02:00
LICENSE.txt Add GPLv3 license 2018-09-18 14:16:07 +03:00
Pipfile Add pytest-clarity to pipenv 2020-03-01 21:19:28 +02:00
Pipfile.lock Add pytest-clarity to pipenv 2020-03-01 21:19:28 +02:00
pytest.ini Add initial pytest configuration 2018-11-11 16:24:54 +02:00
README.md README.md: Update DSpace wiki URLs 2020-04-13 15:25:17 +03:00
requirements-dev.txt Update requirements 2020-03-01 21:21:13 +02:00
requirements.txt Update requirements 2020-03-01 21:21:13 +02:00
setup.cfg Add configuration for isort and black 2019-11-27 12:26:55 +02:00

DSpace Statistics API Build Status builds.sr.ht status

DSpace stores item view and download events in a Solr "statistics" core. This information is available for use in the various DSpace user interfaces, but is not exposed externally via any APIs. The DSpace 4/5/6 REST API, for example, only exposes information about communities, collections, item metadata, and bitstreams.

This project contains an indexer and a Falcon-based web application to make the statistics available via a simple REST API. You can read more about the Solr queries used to gather the item view and download statistics on the DSpace wiki.

If you use the DSpace Statistics API please cite:

Orth, A. 2018. DSpace statistics API. Nairobi, Kenya: ILRI. https://hdl.handle.net/10568/99143.

Requirements

Installation

Create a Python virtual environment and install the dependencies:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt

Running

Set up the environment variables for Solr and PostgreSQL:

$ export SOLR_SERVER=http://localhost:8080/solr
$ export DATABASE_NAME=dspacestatistics
$ export DATABASE_USER=dspacestatistics
$ export DATABASE_PASS=dspacestatistics
$ export DATABASE_HOST=localhost

Index the Solr statistics core to populate the PostgreSQL database:

$ python -m dspace_statistics_api.indexer

Run the REST API:

$ gunicorn dspace_statistics_api.app

Test to see if there are any statistics:

$ curl 'http://localhost:8000/items?limit=1'

Testing

Install development packages using pip:

$ pip install -r requirements-dev.txt

Run tests:

$ pytest

Deployment

There are example systemd service and timer units in the contrib directory. The API service listens on localhost by default so you will need to expose it publicly using a web server like nginx.

An example nginx configuration is:

server {
    #...

    location ~ /rest/statistics/?(.*) {
        access_log /var/log/nginx/statistics.log;
        proxy_pass http://statistics_api/$1$is_args$args;
    }
}

upstream statistics_api {
    server 127.0.0.1:5000;
}

This would expose the API at /rest/statistics.

Using the API

The API exposes the following endpoints:

  • GET /return a basic API documentation page.
  • GET /itemsreturn views and downloads for all items that Solr knows about¹. Accepts limit and page query parameters for pagination of results (limit must be an integer between 1 and 100, and page must be an integer greater than or equal to 0).
  • GET /item/idreturn views and downloads for a single item (id must be a UUID). Returns HTTP 404 if an item id is not found.

The item id is the internal uuid for an item. You can get these from the standard DSpace REST API.

¹ We are querying the Solr statistics core, which technically only knows about items that have either views or downloads. If an item is not present here you can assume it has zero views and zero downloads, but not necessarily that it does not exist in the repository.

Todo

  • Better logging
  • Version API
  • Use JSON in PostgreSQL
  • Add top items endpoint, perhaps /top/items or /items/top?
  • Make community and collection stats available
  • Check IDs in database to see if they are deleted...

License

This work is licensed under the GPLv3.

The license allows you to use and modify the work for personal and commercial purposes, but if you distribute the work you must provide users with a means to access the source code for the version you are distributing. Read more about the GPLv3 at TL;DR Legal.