Compare commits

...

209 Commits
v1.3.0 ... v6_x

Author SHA1 Message Date
Alan Orth 1912363899
Merge pull request #42 from ilri/renovate/pytest-7.x-lockfile
continuous-integration/drone/push Build is passing Details
Update dependency pytest to v7.4.4
2024-01-05 16:53:21 +03:00
Alan Orth cd3c024a77
Merge pull request #44 from ilri/renovate/flake8-7.x
Update dependency flake8 to v7
2024-01-05 16:53:03 +03:00
renovate[bot] e96c79bf2c
Update dependency flake8 to v7
continuous-integration/drone/push Build is passing Details
2024-01-05 01:26:48 +00:00
renovate[bot] d6330c7bd4
Update dependency pytest to v7.4.4
continuous-integration/drone/push Build is passing Details
2023-12-31 12:49:30 +00:00
Alan Orth 8c7a5c4047
Merge pull request #40 from ilri/renovate/isort-5.x-lockfile
continuous-integration/drone/push Build is passing Details
Update dependency isort to v5.13.2
2023-12-28 09:22:24 +03:00
Alan Orth a31c592fab
Merge pull request #41 from ilri/renovate/black-23.x-lockfile
Update dependency black to v23.12.1
2023-12-28 09:22:01 +03:00
renovate[bot] c7b179f1b5
Update dependency black to v23.12.1
continuous-integration/drone/push Build is passing Details
2023-12-23 00:02:01 +00:00
renovate[bot] 77c166c024
Update dependency isort to v5.13.2
continuous-integration/drone/push Build is passing Details
2023-12-13 23:34:34 +00:00
Alan Orth 7680b0f440
.github: update workflow
continuous-integration/drone/push Build is passing Details
- actions/checkout@v4
- actions/setup-python@v5
- python-version: '3.11'
2023-12-09 13:58:01 +03:00
Alan Orth e70a7a9675
Apply fixes from fixit
RewriteToLiteral: It's slower to call list() than using the empty literal
2023-12-09 13:57:55 +03:00
Alan Orth 24f90df13e
pyprojecy.toml: use new group syntax
poetry wants us to use `poetry add --group=dev` now, which generates
this syntax in the toml.
2023-12-09 13:57:45 +03:00
Alan Orth 780f2c1723
pyproject.toml: add fixit to dev dependencies 2023-12-09 13:57:31 +03:00
Alan Orth 53b58d4116
Merge pull request #38 from ilri/renovate/falcon-3.x
continuous-integration/drone/push Build is passing Details
Update dependency falcon to v3.1.3
2023-12-05 16:36:50 +03:00
renovate[bot] 19a6d2cea6
Update dependency falcon to v3.1.3
continuous-integration/drone/push Build is passing Details
2023-12-05 07:14:40 +00:00
Alan Orth 6c2bcda16f
Merge pull request #37 from ilri/renovate/black-23.x-lockfile
continuous-integration/drone/push Build is passing Details
Update dependency black to v23.11.0
2023-11-22 19:37:07 +03:00
renovate[bot] e4d9545b02
Update dependency black to v23.11.0
continuous-integration/drone/push Build is passing Details
2023-11-08 07:30:58 +00:00
Alan Orth 1f507d3074
Merge pull request #33 from ilri/renovate/actions-checkout-4.x
continuous-integration/drone/push Build is passing Details
Update actions/checkout action to v4
2023-10-25 12:22:47 +03:00
Alan Orth 82771d7b0c
Merge pull request #32 from ilri/renovate/psycopg2-2.x-lockfile
Update dependency psycopg2 to v2.9.9
2023-10-25 12:21:47 +03:00
Alan Orth 5ff3323f88
Merge pull request #31 from ilri/renovate/flake8-6.x-lockfile
Update dependency flake8 to v6.1.0
2023-10-25 12:20:32 +03:00
Alan Orth c7a871c2f1
Merge pull request #36 from ilri/renovate/gunicorn-21.x-lockfile
Update dependency gunicorn to v21.2.0
2023-10-25 12:20:15 +03:00
renovate[bot] b948283d40
Update dependency gunicorn to v21.2.0
continuous-integration/drone/push Build is passing Details
2023-10-25 09:12:04 +00:00
Alan Orth 124a05dcaf
Merge pull request #30 from ilri/renovate/gunicorn-21.x
continuous-integration/drone/push Build is passing Details
Update dependency gunicorn to v21
2023-10-25 12:11:21 +03:00
Alan Orth a2daf96fec
Merge pull request #29 from ilri/renovate/black-23.x-lockfile
Update dependency black to v23.10.1
2023-10-25 12:10:47 +03:00
Alan Orth 8634d53fa6
Merge pull request #28 from ilri/renovate/pytest-7.x-lockfile
Update dependency pytest to v7.4.3
2023-10-25 12:10:11 +03:00
renovate[bot] e2bfcef573
Update dependency pytest to v7.4.3
continuous-integration/drone/push Build is passing Details
2023-10-24 21:18:17 +00:00
renovate[bot] d64c4b8cbc
Update dependency black to v23.10.1
continuous-integration/drone/push Build is passing Details
2023-10-23 19:46:06 +00:00
renovate[bot] 3d91366412
Update actions/checkout action to v4
continuous-integration/drone/push Build is passing Details
2023-10-19 11:02:08 +00:00
renovate[bot] c3a4e2260b
Update dependency psycopg2 to v2.9.9
continuous-integration/drone/push Build is passing Details
2023-10-03 14:20:02 +00:00
renovate[bot] 10519997ac
Update dependency flake8 to v6.1.0
continuous-integration/drone/push Build is failing Details
2023-07-29 21:45:38 +00:00
renovate[bot] 4d7e9e9401
Update dependency gunicorn to v21
continuous-integration/drone/push Build is failing Details
2023-07-17 21:23:15 +00:00
Alan Orth fe9f98bcc0
dspace_statistics_api/util.py: format with black
continuous-integration/drone/push Build is passing Details
continuous-integration/drone Build is passing Details
2023-05-30 16:04:18 +03:00
Alan Orth 70f0d66c6e
poetry.lock: run poetry update 2023-05-30 16:03:51 +03:00
Alan Orth 913596c61d
pyproject.toml: bump flake8 2023-05-30 16:03:35 +03:00
Alan Orth 7cd762a5a2
Merge pull request #24 from ilri/renovate/black-23.x
continuous-integration/drone/push Build is passing Details
Update dependency black to v23
2023-05-30 11:26:12 +03:00
renovate[bot] 3811be18ef
Update dependency black to v23 2023-05-30 08:22:19 +00:00
Alan Orth a52818271c
Merge pull request #26 from ilri/renovate/pytest-7.x
Update dependency pytest to v7
2023-05-30 11:21:39 +03:00
renovate[bot] b643f60dd7
Update dependency pytest to v7 2023-05-30 08:08:39 +00:00
Alan Orth 7cec9a9545
Merge pull request #27 from ilri/renovate/postgres-15.x
Update postgres Docker tag to v15
2023-05-30 11:07:00 +03:00
renovate[bot] a9302506b6
Update postgres Docker tag to v15 2023-05-30 08:01:58 +00:00
Alan Orth b980602a03
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2023-05-30 11:00:46 +03:00
Alan Orth a4b4843036
poetry.lock: run poetry update 2023-05-30 11:00:09 +03:00
Alan Orth 7e334f6de8
Recommend Python 3.8+
I'm only using Python 3.8+ anyways, and 3.7 is end of life as of
next month. Some deps start wanting Python 3.8 so let's just bump
it.
2023-05-30 10:23:10 +03:00
Alan Orth 770f676fb5
Use PostgreSQL 14 for Drone CI and GitHub Actions 2023-05-30 10:14:47 +03:00
Alan Orth 6d5e3c350d
Remove .hound.yml
I haven't used this in years and now we have automatic tests via
GitHub Actions.
2023-05-30 10:14:06 +03:00
Alan Orth 531136183b
pyproject.toml: rework isort and black deps
We no longer need to gatekeep these for Python 3.6+, as all my dev
systems are running Python 3.11 and all my production systems are
running Python 3.8+.
2023-05-30 10:09:47 +03:00
Alan Orth 1a3d0350a5
.github/workflows: update GitHub Action workflow
We can use Poetry directly rather than installing deps with pip.
2023-05-30 09:39:00 +03:00
Alan Orth 25c4f05f16
Add renovate.json
Disable management of requirements.txt since I am using poetry.
2023-05-30 08:11:28 +03:00
Alan Orth 9fba8d1b81
Migrate isort config to pyproject.toml
continuous-integration/drone/push Build is failing Details
See: https://pycqa.github.io/isort/docs/configuration/black_compatibility.html
2022-12-20 15:13:44 +02:00
Alan Orth 568ced0f20
poetry.lock: run poetry update 2022-12-20 15:12:50 +02:00
Alan Orth 9cd93c9034 Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --with dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2022-11-21 10:51:43 +03:00
Alan Orth 83a2625987 CHANGELOG.md: add note about falcon 3.1.1 2022-11-21 10:51:43 +03:00
Alan Orth f591ed7162 Use falcon v3.1.1
Bug fix release with support for Python 3.11.

See: https://falcon.readthedocs.io/en/3.1.1/changes/3.1.1.html
2022-11-21 10:51:43 +03:00
Alan Orth bb0f267941
.github: use ubuntu-22.04 for actions
continuous-integration/drone/push Build is passing Details
Apparently 'ubuntu-latest' is still 20.04 and today is 2022-10-03,
which seems a bit old!

See: https://github.com/actions/runner-images
2022-10-03 19:57:56 +03:00
Alan Orth 0720605b6a
.github: update actions
Switch to newer checkout and setup-python actions, and enable the
pip cache for setup-python.
2022-10-03 19:43:41 +03:00
Alan Orth bcb97d025c
CHANGELOG.md: add note about PostgreSQL 12
continuous-integration/drone/push Build is passing Details
2022-05-31 13:05:41 +03:00
Alan Orth 0ff8490275
Use PostgreSQL 12 in CI
I migrated my production systems to PostgreSQL 12 months ago.
2022-05-31 13:05:01 +03:00
Alan Orth 0a8ac60ade
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2022-03-28 16:01:39 +03:00
Alan Orth 37527c21be
poetry.lock: run poetry update 2022-03-28 16:01:13 +03:00
Alan Orth eb660f8085
CHANGELOG.md: add note about Python version
continuous-integration/drone/push Build is passing Details
2022-03-28 15:16:02 +03:00
Alan Orth e7d780f511
Don't test on Python 3.6 anymore
Python 3.6 is deprecated in Falcon 3.1.0 and all of our production
and development hosts are running Python 3.8+ now.
2022-03-28 15:13:54 +03:00
Alan Orth c3b9a541b7
Bump version to 1.4.4-dev
continuous-integration/drone/push Build is passing Details
2022-03-26 19:10:47 +03:00
Alan Orth 1a1a14a25f
Version 1.4.3 2022-03-26 19:08:56 +03:00
Alan Orth c09fc789e8
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2022-03-26 19:06:54 +03:00
Alan Orth 134a4f1595
poetry.lock: run poetry update 2022-03-26 18:50:31 +03:00
Alan Orth 12ebd1aed5
pyproject.toml: falcon 3.1.0
Doesn't seem to have any breaking changes, only fixes and some new
compatability updates with new Pythons.

See: https://falcon.readthedocs.io/en/stable/changes/3.1.0.html
2022-03-26 18:49:33 +03:00
Alan Orth e5f3201b65
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2022-03-21 15:19:00 +03:00
Alan Orth c1ce4fe233
poetry.lock: run poetry update 2022-03-21 15:18:27 +03:00
Alan Orth b2eb1878a5
.github/workflows/python-app.yml: quote python version
continuous-integration/drone/push Build is passing Details
2022-01-30 18:57:10 +03:00
Alan Orth a0213c1c97
poetry.lock: run poetry update
continuous-integration/drone/push Build is passing Details
2022-01-30 13:52:20 +03:00
Alan Orth cd03ca2b36
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2022-01-30 13:50:49 +03:00
Alan Orth c48e6a79c7
pyproject.toml: update dependencies
We no longer need ipython because it's installed globally on all
my machines. Also, new major version of flake8 and black is no
longer a beta.
2022-01-30 13:49:36 +03:00
Alan Orth a2e1695ecc
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-12-19 14:18:42 +02:00
Alan Orth b683bf211c
.github/workflows/python-app.yml: use Python 3.10 2021-12-19 14:15:41 +02:00
Alan Orth 3ab48743d6
poetry.lock: run poetry update 2021-12-19 14:13:14 +02:00
Alan Orth 88173eaae9
README.md: fix link to actions
continuous-integration/drone/push Build is passing Details
2021-12-08 11:34:50 +02:00
Alan Orth f557d33f36
README.md: adjust intro
Use intro style from Python Black! This makes it easier to have the
badges displayed without wrapping and looks nicer.
2021-12-08 09:48:31 +02:00
Alan Orth ffc4ff4a5c
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-11-11 09:05:37 +02:00
Alan Orth 7551b34632
poetry.lock: run poetry update
This was previously failing for the past few days.
2021-11-11 09:05:03 +02:00
Alan Orth 5e71ec10eb
Remove pipenv
Poetry's working again.
2021-11-11 09:04:30 +02:00
Alan Orth f80d360cf9
Only install ipython on Python 3.7+
continuous-integration/drone/push Build is passing Details
2021-11-10 09:21:59 +02:00
Alan Orth e70b59ecfe
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with pipenv lock:

    $ pipenv lock -r > requirements.txt
    $ pipenv lock -r --dev > requirements-dev.txt
2021-11-09 22:52:21 +02:00
Alan Orth 4d0828b6c0
Add Pipenv configuration
I was having a problem with Poetry.
2021-11-09 22:51:23 +02:00
Alan Orth dabc4c0259
pyproject.toml: revert to my fork of falcon-swagger-ui
The Falcon 3 fix never actually got committed to rdidyik's fork. I
have submitted a new pull request and will use my fork until it is
merged.

See: https://github.com/rdidyk/falcon-swagger-ui/pull/21
2021-11-09 22:09:11 +02:00
Alan Orth 4fd8af07c3
.drone.yml: Fix job name 2021-11-09 17:36:48 +02:00
Alan Orth 4c5326a176
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-10-21 15:07:26 +03:00
Alan Orth 3b1ccafab4
poetry.lock: Run poetry update 2021-10-21 15:06:19 +03:00
Alan Orth 58b5ae82d3
pyproject.toml: Switch back to falcon-swagger-ui upstream
They merged my changes for Falcon 3.0.

See: https://github.com/rdidyk/falcon-swagger-ui/pull/20
2021-10-21 15:04:58 +03:00
Alan Orth 562aaeef7d
.drone.yml: Test on Python 3.10
continuous-integration/drone/push Build is failing Details
2021-10-11 20:11:32 +03:00
Alan Orth 5cdba6acb1
.drone.yml: Also install gcc for all Python containers
continuous-integration/drone/push Build is passing Details
We previously only needed gcc for typed-ast in Python 3.9, but now
we actually need gcc to compile psycopg2 in all of them.
2021-07-06 16:44:13 +03:00
Alan Orth dd0937179c
.drone.yml: Add libpq-dev to test container
continuous-integration/drone/push Build was killed Details
We need it to compile the psycopg2 Python library.
2021-07-06 16:41:17 +03:00
Alan Orth f0c6c004db
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-07-06 16:27:23 +03:00
Alan Orth 6843f0a8ac
poetry.lock: Run poetry update 2021-07-06 16:26:33 +03:00
Alan Orth f5fcfcc05a
pyproject.toml: Update psycopg2 version
I manually re-installed psycopg2@latest while troubleshooting an
issue with it not working after Arch Linux updated Python. That's
one down side of using the non-binary package.
2021-07-06 16:26:05 +03:00
Alan Orth e8ac74b6d1
pyproject.toml: Update some dev dependencies 2021-07-06 16:17:22 +03:00
Alan Orth 14fc14daee
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-06-22 20:53:07 +03:00
Alan Orth 871aae537a
poetry.lock: Sync changes 2021-06-22 20:52:15 +03:00
Alan Orth 2fada6c6ff
pyproject.toml: Use psycopg2 instead of psycopg2-binary
continuous-integration/drone/push Build is passing Details
According to the documentation the binary version is not meant to
be run in production. Since I'm in control of both my development
and production servers and can ensure that libpq-dev is installed
on both, I will use the source version of this module.

See: https://www.psycopg.org/docs/install.html#quick-install
2021-06-22 17:49:49 +03:00
Alan Orth ef0991e352
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-06-22 10:11:57 +03:00
Alan Orth 4502d6053c
poetry.lock: run poetry update
The following packages were updated:

> markupsafe (2.0.0 -> 2.0.1)
> certifi (2020.12.5 -> 2021.5.30)
> click (8.0.0 -> 8.0.1)
> decorator (5.0.7 -> 5.0.9)
> jinja2 (3.0.0 -> 3.0.1)
> prompt-toolkit (3.0.18 -> 3.0.19)
> urllib3 (1.26.4 -> 1.26.5)
> ipython (7.23.1 -> 7.24.1)
> psycopg2-binary (2.8.6 -> 2.9.1)
2021-06-22 10:10:29 +03:00
Alan Orth a524068cf6
Bump version to 1.4.3-dev
continuous-integration/drone/push Build is passing Details
2021-04-15 14:44:44 +03:00
Alan Orth 964d5dff06
Version 1.4.2 2021-04-15 14:23:07 +03:00
Alan Orth a9252d1771
Update requirements-dev.txt
Generated with poetry export:

    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-04-15 14:19:48 +03:00
Alan Orth a63687d516
poetry.lock: Run poetry update 2021-04-15 14:17:17 +03:00
Alan Orth 73dc3a292e
README.md: Remove TODO about Swagger
continuous-integration/drone/push Build is passing Details
I added the SwaggerUI interface a few months ago.
2021-04-06 20:28:10 +03:00
Alan Orth 1e742bad41
CHANGELOG.md: Add note about valid page tests
continuous-integration/drone/push Build is passing Details
2021-04-06 09:07:51 +03:00
Alan Orth 164008981e
CHANGELOG.md: Add notes about Falcon 3.0.0
continuous-integration/drone/push Build is passing Details
2021-04-06 08:58:00 +03:00
Alan Orth dd1769b954
tests: Fix totalPages
A few months ago I fixed the totalPages display to show 1 when we
only have one page of results (the page itself is still 0), but I
didn't update the tests.

See: 4f8cd1097b
2021-04-06 08:54:54 +03:00
Alan Orth b009820fb4
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-04-06 08:32:22 +03:00
Alan Orth 9830295978
poetry.lock: Run poetry update 2021-04-06 08:31:50 +03:00
Alan Orth c93a4d7455
pyproject.toml: Falcon 3.0.0
Release notes: https://falcon.readthedocs.io/en/latest/changes/3.0.0.html
2021-04-06 08:31:39 +03:00
Alan Orth 2f8e4f8a0a Changes for Falcon 3.0.0
Mostly it seems we just need to use resp.text instead of resp.body,
including in falcon-swagger-ui (I forked the upstream one to make
this change).

See: https://falcon.readthedocs.io/en/latest/changes/3.0.0.html
2021-04-06 08:30:28 +03:00
Alan Orth 0650c5985e
Add SPDX short license identifier to all Python files
continuous-integration/drone/push Build is passing Details
See: https://spdx.github.io/spdx-spec/appendix-V-using-SPDX-short-identifiers-in-source-files/
2021-03-22 13:42:42 +02:00
Alan Orth d814f1c4f0
CHANGELOG.md: Fix heading
continuous-integration/drone/push Build is passing Details
2021-03-21 19:50:39 +02:00
Alan Orth 00f30591c4
CHANGELOG.md: Add notes about GitHub Actions 2021-03-21 19:49:35 +02:00
Alan Orth acfe87b91a
Add GitHub Actions badge and remove sr.ht
continuous-integration/drone/push Build is passing Details
2021-03-21 11:48:05 +02:00
Alan Orth bc6d84dda2 Add GitHub Actions workflow
My first time setting up a PostgreSQL service container on GitHub
actions. Note that there are two different kinds of environment
variables: those passed to the Docker container, and those used by
the PostgreSQL utilities.

See: https://docs.github.com/en/actions/guides/creating-postgresql-service-containers
See: https://hub.docker.com/_/postgres
2021-03-21 11:44:39 +02:00
Alan Orth 889fb2f74a
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-03-21 08:59:41 +02:00
Alan Orth c42cd7a818
poetry.lock: Run poetry update 2021-03-21 08:59:04 +02:00
Alan Orth f8bba59d66
.gitignore: Ignore .egg-info
continuous-integration/drone/push Build is passing Details
2021-03-14 21:50:47 +02:00
Alan Orth b8cb752a29
CHANGELOG.md: Add note about updated poetry deps
continuous-integration/drone/push Build is passing Details
2021-03-11 11:23:18 +02:00
Alan Orth 09496aa2b5
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-03-11 11:22:05 +02:00
Alan Orth ff5dc7506d
poetry.lock: Run poetry update 2021-03-11 11:21:02 +02:00
Alan Orth 80a11ead97
Version 1.4.1
continuous-integration/drone/push Build is passing Details
2021-01-14 14:19:50 +02:00
Alan Orth a282c95933
CHANGELOG.md: Minor syntax fix 2021-01-14 14:15:57 +02:00
Alan Orth fd7cc36306
Update requirements-dev.txt
Generated with poetry export:

    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2021-01-14 14:14:16 +02:00
Alan Orth a20ff09570
poetry.lock: Run poetry update
All tests still pass.
2021-01-14 14:13:32 +02:00
Alan Orth fdc0e73088
tests: Sort imports with isort 2021-01-14 14:12:59 +02:00
Alan Orth b15afc9f39
CHANGELOG.md: Add note about UUIDs
continuous-integration/drone/push Build is passing Details
2021-01-05 12:41:21 +02:00
Alan Orth 2bc18ef719
README.md: Make a note about migrating UUIDs 2021-01-05 12:35:23 +02:00
Alan Orth 49751b53f0
dspace_statistics_api/indexer.py: Limit to UUIDs
We need to make sure that the indexer only tries to index UUIDs, as
opposed to legacy IDs that may have been left over from a migration
from earlier DSpace versions. For example, "98110-unmigrated", "-1"
etc.

For matching the UUIDs in Solr I decided that it is sufficient for
our use case to simply match thirty-six characters, where a UUID is
composed of thirty-two hexadecimal characters and four dashes. We
don't need to do any verification of "real" UUIDs because it would
be needlessly complex in our case.

See: https://github.com/ilri/dspace-statistics-api/issues/12
2021-01-05 12:30:27 +02:00
Alan Orth d1c177e146
.drone.yml: Add git to python container
continuous-integration/drone/push Build is passing Details
Now that I am installing my own fork of falcon-swagger-ui we need
to have git so we can install it with pip.
2020-12-27 14:22:23 +02:00
Alan Orth 33dc210452
dspace_statistics_api/docs/openapi.json: Minor edit
Better to leave the version in there because Swagger Editor doesn't
like it without. Also, change the example page parameter for POSTing
to /items and /collections, as it doesn't make sense to start on a
later page if we have less items than our limit.
2020-12-27 13:53:59 +02:00
Alan Orth 282d5f644a
Move unreleased change to v1.4.0 2020-12-27 12:52:24 +02:00
Alan Orth 05e0e8bdca
openapi.json: Set the API version from config
We don't need to hard code this in the JSON anymore since we are
reading and modifying it now for the server config anyways.
2020-12-27 12:48:13 +02:00
Alan Orth 2567bb8604
dspace_statistics_api/app.py: Format with black 2020-12-27 12:27:01 +02:00
Alan Orth 4af3c656a3
CHANGELOG.md: Add note about totalPages 2020-12-27 12:26:32 +02:00
Alan Orth 4f8cd1097b
Rework paging
The "totalPages" value in our response is calculated incorrectly.
Instead of casting to int and rounding, we should rather round up
to the next integer with math.ceil. This is a more correct way to
get the value.

Also update the indexer to use the same logic, although there the
values are printed with +1 so they are more readable.
2020-12-27 12:22:07 +02:00
Alan Orth a02211fd60
Update requirements
continuous-integration/drone/push Build is failing Details
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-12-25 13:03:32 +02:00
Alan Orth fc814593c7
Use my fork of falcon-swagger-ui
It has a newer Swagger UI (v3.38.0).
2020-12-25 12:57:58 +02:00
Alan Orth 7de1084f60
Add whitespace before vim modeline
continuous-integration/drone/push Build is passing Details
black wants this...
2020-12-24 13:12:06 +02:00
Alan Orth 6b78e82fe9
Add vim modeline to all tests 2020-12-24 13:11:12 +02:00
Alan Orth 4004515967
pyproject.toml: Update description
continuous-integration/drone/push Build is passing Details
2020-12-23 16:15:46 +02:00
Alan Orth d1229c2387
Adjust docs at root
Don't use a static HTML file anymore. Now I simply print an XHTML
page from the Falcon resource. This way I can use variables to add
in the API version as well as a link to the Swagger UI.

The list of API calls is still present on the README.md, though in
the long run I might move them to some dedicated documentation or
a GitHub wiki.
2020-12-23 16:12:50 +02:00
Alan Orth be83514de1
Re-work Swagger UI configuration
continuous-integration/drone/push Build is passing Details
It turns out that Swagger UI mostly does the "right" thing for our
use cases here, but it assumes that API paths are relative to the
root of the host where it is being served. This works in the local
development environment because we are serving on "/", but it does
not work in production where the API is deployed beneath the DSpace
REST API, for example at "/rest/statistics".

The solution here is to allow configuration of the DSpace Statistics
API path and use that when registering the Swagger UI as well as in
a new "server" block in the OpenAPI JSON schema.

By default it is configured to work out of the box in a development
environment. Set the DSPACE_STATISTICS_API_URL environment variable
to something like "/rest/statistics" when running in production.
2020-12-23 13:25:17 +02:00
Alan Orth 70b2ba83ba
Allow configuration of Swagger and OpenAPI JSON URL
continuous-integration/drone/push Build is passing Details
When running in production your statistics API might be deployed to
a path like /rest/statistics instead of at the root.
2020-12-22 12:50:03 +02:00
Alan Orth 893039bc6a
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-12-22 12:07:07 +02:00
Alan Orth a4628dde4e
Update requirements
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-22 11:45:39 +02:00
Alan Orth 68418ea053
dspace_statistics_api/docs/openapi.json: Add /status
Add a /status to the Swagger UI schema.
2020-12-22 11:41:47 +02:00
Alan Orth 6bbee7919e
Bump version to 1.4.0-dev 2020-12-22 11:31:46 +02:00
Alan Orth 8f0061ce29
CHANGELOG.md: Add note about the /status page 2020-12-22 11:30:50 +02:00
Alan Orth 4b1398c67f
Add /status route
Currently this only prints the API version.
2020-12-22 11:30:09 +02:00
Alan Orth a9d2a6d9be
CHANGELOG.md: Add note about Swagger UI 2020-12-22 11:21:46 +02:00
Alan Orth a35ecf2394
Add Swagger UI on /swagger
This includes a Swagger UI with an OpenAPI 3.0 JSON schema for easy
interactive demonstration and testing of the API. The JSON schema
was created with the standalone swagger-editor. Includes tests to
make sure that the /swagger and /docs/openapi.json paths are acce-
ssible.
2020-12-22 11:18:47 +02:00
Alan Orth 3e271c7852
tests/dspacestatistics.sql: Update data
continuous-integration/drone/push Build is passing Details
Add a new database snapshot with communities and collections.
2020-12-20 22:31:41 +02:00
Alan Orth d7ba14c590
tests: Add tests for communities and collections
Also, separate tests for items, communities, and collections into
their own files, leaving a single test for docs in its own file.
2020-12-20 22:12:13 +02:00
Alan Orth ab82e90773
dspace_statistics_api/stats.py: Use -isBot:true
continuous-integration/drone/push Build is passing Details
Minor change to bot filtering. We should use a negated match for
documents that have `isBot:true` rather than looking for documents
that are tagged with `isBot:false` (the distinction is subtle, but
important).
2020-12-20 16:56:03 +02:00
Alan Orth 8a1244d2d0
Update changelog and docs 2020-12-20 16:45:49 +02:00
Alan Orth 04f0756c7f
dspace_statistics_api/util.py: Add vim modeline 2020-12-20 16:31:52 +02:00
Alan Orth 830e4415f5
dspace_statistics_api/app.py: Run isort 2020-12-20 16:29:35 +02:00
Alan Orth 47b4eb3df7
Rename items.py to stats.py
It is no longer used only for item-related statistics functions.
2020-12-20 16:28:56 +02:00
Alan Orth 3339bf8d9c
Add communities and collections support to API
The basic logic is similar to items, where you can request single
item statistics with a UUID, all item statistics, and item statis-
tics for a list of items (optionally with a date range). Most of
the item code was re-purposed to work on "elements", which can be
items, communities, or collections depending on the request, with
the use of Falcon's `before` hooks to set the statistics scope so
we know how to behave for the current request.

Other than the minor difference in facet fields, another issue I
had with communities and collections is that the owningComm and
owningColl fields are multi-valued (unlike items' id field). This
means that, when you facet the results of your query, Solr returns
ids that seem unrelated, but are actually present in the field, so
I had to make sure I checked all returned ids to see if they were
in the user's POSTed elements list.

TODO:
  - Add tests
  - Revise docstrings
  - Refactor items.py as it is now generic
2020-12-20 16:14:46 +02:00
Alan Orth fba6f1ead1 CHANGELOG.md: Update unreleased changes
continuous-integration/drone/push Build is passing Details
2020-12-18 22:54:01 +02:00
Alan Orth 20c8ba0cf8 indexer.py: Add support for communities and collections
The logic to get views and downloads is very similar to that used
for items, but we facet by different fields. This uses a generic
function for indexing that takes an "indexType" and a "facetField"
parameter. The indexType parameter controls which database table
to insert into, and the facetField parameter indicates which field
to facet by in Solr.
2020-12-18 22:53:16 +02:00
Alan Orth b486f51dd7 indexer.py: Rename index functions for items
Start making plans for indexing communities and collections.
2020-12-18 22:53:16 +02:00
Alan Orth 787eec20ea
CHANGELOG.md: Add note about imports
continuous-integration/drone/push Build is passing Details
2020-12-18 22:52:14 +02:00
Alan Orth 9e6fcf279b
dspace_statistics_api/items.py: Format with black 2020-12-18 22:45:39 +02:00
Alan Orth 4dbf734a4b
Move all imports to top of file
A few months ago I had an issue setting up mocking because I was
trying to be clever importing these libraries only when I needed
them rather than at the global scope. Someone pointed out to me
that if the imports are at the top of the file Falcon will load
them once when the WSGI server starts, whereas if they are in the
on_get() or on_post() they will load for every request! Also, it
seems that PEP8 recommends keeping imports at the top of the file
anyways, so I will just do that.

Imports sorted with isort.

See: https://www.python.org/dev/peps/pep-0008/#imports
2020-12-18 22:42:06 +02:00
Alan Orth a0d0a47150
items.py: Add fl paramter to Solr queries
I forgot to add the fl parameter here as well.
2020-12-18 16:12:34 +02:00
Alan Orth 01e9756cf2
Update requirements
continuous-integration/drone/push Build is passing Details
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-18 11:20:17 +02:00
Alan Orth b2b4eb2939
poetry.lock: Run poetry update 2020-12-18 11:19:16 +02:00
Alan Orth 4bbbaa4af3
dspace_statistics_api/indexer.py: Use `fl` parameter
continuous-integration/drone/push Build is passing Details
I forgot to add the fl parameter to the downloads function.
2020-12-18 10:44:02 +02:00
Alan Orth 7e4d5f4b13
README.md: Minor edit to intro 2020-12-18 10:42:48 +02:00
Alan Orth 428172854d
README.md: Add TODO
continuous-integration/drone/push Build is passing Details
2020-12-17 20:44:25 +02:00
Alan Orth 2707cb37d5
CHANGELOG.md: Add note about fl parameter
continuous-integration/drone/push Build is failing Details
2020-12-17 12:27:11 +02:00
Alan Orth 2407aeec70
dspace_statistics_api/indexer.py: Use `fl` parameter
When indexing item views and downloads the only field we need is the
the id. The `fl` parameter tells Solr which fields to return in the
search results. This should theoretically be more efficient, though
I don't have any time to figure out how to measure it right now.
2020-12-17 12:25:28 +02:00
Alan Orth f3a0e3a671
CHANGELOG.md: Add note about ORDER BY
continuous-integration/drone/push Build is passing Details
2020-12-17 10:17:23 +02:00
Alan Orth 4590fc8708
dspace_statistics_api/app.py: Use ORDER BY in /items
Since we are paging through the results by limit/offset we need to
be sure that we are returning results deterministically.
2020-12-17 10:10:40 +02:00
Alan Orth 8b924cf450
Remove TravisCI config
continuous-integration/drone/push Build is passing Details
I will use other CIs since TravisCI changed their business model.
2020-12-15 09:38:51 +02:00
Alan Orth ea24c73a6a
.drone.yml: Install gcc for Python 3.9
It appears to be needed to compile typed-ast:

    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iast27/Include -I/usr/local/include/python3.9 -c ast27/Custom/typed_ast.c -o build/temp.linux-x86_64-3.9/ast27/Custom/typed_ast.o
    error: command 'gcc' failed: No such file or directory
    ----------------------------------------
    ERROR: Failed building wheel for typed-ast
2020-12-14 22:50:21 +02:00
Alan Orth cd98d33615
.drone.yml: Only install requirements-dev.txt
It seems that Poetry's --dev export includes both dev and non-dev
libraries so we don't need to install both.
2020-12-14 22:42:00 +02:00
Alan Orth 9d112266ca
Update requirements-dev.txt
Generated with poetry export:

    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-14 22:05:18 +02:00
Alan Orth 2b067050ff
Remove pytest-clarity
It is missing a six dependency which causes the build to fail. I
could simply add six to the virtualenv but it feels dirty. I don't
actually *need* pytest-clarity for anything so I'll just remove it.

See: https://github.com/darrenburns/pytest-clarity/issues/14
2020-12-14 22:05:03 +02:00
Alan Orth dc683f2d1c
pytest.ini: Change --strict to --strict-markers
This is deprecated since pytest 6.2.0.

See: https://docs.pytest.org/en/stable/deprecations.html#the-strict-command-line-option
2020-12-14 19:07:02 +02:00
Alan Orth f60f529bd7
Update requirements
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-14 15:41:40 +02:00
Alan Orth 7db8458201
poetry.lock: Run poetry update
[SKIP CI]
2020-12-14 15:40:06 +02:00
Alan Orth 707f878b94 Add .drone.yml
Uses multiple pipelines to test several versions of Python. A few
things to note:

- I use the -slim Python packages, which are smaller and yet still
have no problem installing psycopg2-binary with pip
- I have to start a PostgreSQL database service for each pipeline
separately
2020-12-14 15:38:50 +02:00
Alan Orth 930250352a
Update docs about POST /items 2020-12-13 20:09:20 +02:00
Alan Orth e27f30ba4d
README.md: Use travis-ci.com domain for badge link 2020-12-08 09:11:38 +02:00
Alan Orth 28d1917038
README.md: Use travis-ci.com domain for badge 2020-12-08 09:09:19 +02:00
Alan Orth fc6a9c2ad1
tests: Update for real data
Now that CGSpace is running DSpace 6 I will use some real UUIDs to
make things easier in the future.
2020-11-25 14:56:47 +02:00
Alan Orth 3125e96a16
Bump version to 1.3.2 2020-11-18 22:01:18 +02:00
Alan Orth 66143ff00f
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-11-18 21:59:33 +02:00
Alan Orth 2d15f12be9
poetry.lock: Run poetry update 2020-11-18 21:58:32 +02:00
Alan Orth 9218039e61
CHANGELOG.md: Add note about limit param 2020-11-18 21:57:12 +02:00
Alan Orth 88a8db6c78
Make sure limit is between 1 and 100
We were not properly checking whether the limit was actually less
than or equal to 100.
2020-11-18 21:55:54 +02:00
Alan Orth 3995eba0a7
CHANGELOG.md: Add note about Solr bot filter 2020-11-17 17:42:30 +02:00
Alan Orth 810508d038
dspace_statistics_api/indexer.py: Use -isBot:true
Minor change to bot filtering. We should use a negated match for
documents that have `isBot:true` rather than looking for documents
that are tagged with `isBot:false` (the distinction is subtle, but
important).
2020-11-17 17:40:08 +02:00
Alan Orth ecafab57cb
README.md: Update DSpace version note 2020-11-16 16:16:21 +02:00
Alan Orth 9c9431b58c
CHANGELOG.md: Add unreleased changes 2020-11-02 22:14:18 +02:00
Alan Orth 2d6520fc97
Fix limit in docs 2020-11-02 22:14:08 +02:00
Alan Orth 79a393d33f
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-11-02 22:10:29 +02:00
Alan Orth 149f6c418f
poetry.lock: Run poetry update 2020-11-02 22:00:29 +02:00
Alan Orth ca1582a8b6
Make sure limit is between 1 and 100
We were not properly checking whether the limit was greater than 0
in all cases.
2020-11-02 21:59:20 +02:00
Alan Orth 1904c243a4
Revert ".travis.yml: Use Ubuntu 20.04 "Focal" environment"
This reverts commit 0baa07f70a.

Focal only has PostgreSQL 12 installed, and we are not quite there
yet (our production has 9.6, testing has 10).
2020-10-29 00:11:28 +03:00
Alan Orth 0baa07f70a
.travis.yml: Use Ubuntu 20.04 "Focal" environment 2020-10-29 00:04:47 +03:00
Alan Orth 59214ffcb6
.travis.yml: Bump Python versions
Test Python 3.9 now that it was released, and allow tests to fail
on nightly builds.
2020-10-29 00:03:58 +03:00
Alan Orth 549b8bf1a7
dspace_statistics_api/docs/index.html: Fix version
We need to print it in the body, not the title.
2020-10-06 22:22:11 +03:00
Alan Orth 899a79b2e7
Version 1.3.1 2020-10-06 22:15:52 +03:00
Alan Orth 4c59469055
Update requirements
Generated with poetry export:

    $ poetry export --no-hashes -f requirements.txt > requirements.txt
    $ poetry export --no-hashes --dev -f requirements.txt > requirements-dev.txt

The `--no-hashes` is required to work around an issue with gunicorn
pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-10-06 22:11:54 +03:00
29 changed files with 3888 additions and 5310 deletions

View File

@ -1,21 +0,0 @@
image: archlinux
packages:
- python-poetry
- postgresql
sources:
- https://git.sr.ht/~alanorth/dspace-statistics-api
tasks:
- setup: |
id
psql --version
sudo su - postgres -c "initdb --locale en_US.UTF-8 -E UTF8 -D '/var/lib/postgres/data'"
sudo systemctl start postgresql
createuser -U postgres dspacestatistics
psql -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
createdb -U postgres -O dspacestatistics --encoding=UNICODE dspacestatistics
cd dspace-statistics-api
psql -U postgres -d dspacestatistics < tests/dspacestatistics.sql
poetry install --no-root
- test: |
cd dspace-statistics-api
poetry run pytest

117
.drone.yml Normal file
View File

@ -0,0 +1,117 @@
kind: pipeline
type: docker
name: python310
steps:
- name: setup
image: postgres:15-alpine
environment:
PGPASSWORD: postgres
commands:
- id
- psql --version
- sleep 5
- pg_isready -h database -U postgres -d dspacestatistics
- createuser -h database -U postgres dspacestatistics
- psql -h database -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
- psql -h database -U postgres -d dspacestatistics < tests/dspacestatistics.sql
- name: test
image: python:3.10-slim
environment:
PGPASSWORD: dspacestatistics
DATABASE_HOST: database
commands:
- id
- python -V
- apt update && apt install -y gcc git libpq-dev
- pip install -r requirements-dev.txt
- pytest
services:
- name: database
image: postgres:15-alpine
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: dspacestatistics
---
kind: pipeline
type: docker
name: python39
steps:
- name: setup
image: postgres:15-alpine
environment:
PGPASSWORD: postgres
commands:
- id
- psql --version
- sleep 5
- pg_isready -h database -U postgres -d dspacestatistics
- createuser -h database -U postgres dspacestatistics
- psql -h database -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
- psql -h database -U postgres -d dspacestatistics < tests/dspacestatistics.sql
- name: test
image: python:3.9-slim
environment:
PGPASSWORD: dspacestatistics
DATABASE_HOST: database
commands:
- id
- python -V
- apt update && apt install -y gcc git libpq-dev
- pip install -r requirements-dev.txt
- pytest
services:
- name: database
image: postgres:15-alpine
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: dspacestatistics
---
kind: pipeline
type: docker
name: python38
steps:
- name: database
image: postgres:15-alpine
detach: true
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: dspacestatistics
- name: setup
image: postgres:15-alpine
environment:
PGPASSWORD: postgres
commands:
- id
- psql --version
- sleep 5
- pg_isready -h database -U postgres -d dspacestatistics
- createuser -h database -U postgres dspacestatistics
- psql -h database -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
- psql -h database -U postgres -d dspacestatistics < tests/dspacestatistics.sql
- name: test
image: python:3.8-slim
environment:
PGPASSWORD: dspacestatistics
DATABASE_HOST: database
commands:
- id
- python -V
- apt update && apt install -y gcc git libpq-dev
- pip install -r requirements-dev.txt
- pytest
# vim: ts=2 sw=2 et

58
.github/workflows/python-app.yml vendored Normal file
View File

@ -0,0 +1,58 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
name: Build and Test
on: ['push', 'pull_request']
jobs:
build:
runs-on: ubuntu-22.04
services:
database:
image: postgres:15-alpine
env:
# password for postgres user in the Docker container
POSTGRES_PASSWORD: postgres
# default database to create
POSTGRES_DB: dspacestatistics
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- uses: actions/checkout@v4
- name: Install poetry
run: pipx install poetry
- uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'poetry'
- run: poetry install
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
poetry run flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
poetry run flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Set up PostgreSQL
run: |
pg_isready -U postgres -d dspacestatistics
createuser -U postgres dspacestatistics
psql -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
psql -U postgres -d dspacestatistics < tests/dspacestatistics.sql
env:
PGHOST: localhost
PGPASSWORD: postgres
- name: Test with pytest
run: |
poetry run pytest
env:
PGHOST: localhost
PGPASSWORD: dspacestatistics

1
.gitignore vendored
View File

@ -1,2 +1,3 @@
__pycache__
venv
*.egg-info

View File

@ -1,4 +0,0 @@
flake8:
enabled: true
config_file: .flake8
fail_on_violations: true

View File

@ -1,24 +0,0 @@
dist: bionic
language: python
python:
- "3.6"
- "3.7"
- "3.8"
- "3.8-dev" # 3.8 development branch
jobs:
allow_failures:
- python: "3.8-dev"
addons:
postgresql: "10"
before_script:
- psql --version
- createuser -U postgres dspacestatistics
- psql -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
- createdb -U postgres -O dspacestatistics --encoding=UNICODE dspacestatistics
- psql -U postgres -d dspacestatistics < tests/dspacestatistics.sql
install:
- "pip install -r requirements.txt"
- "pip install -r requirements-dev.txt"
script: pytest
# vim: ts=2 sw=2 et

View File

@ -4,6 +4,68 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## Unreleased
### Changed
- Update recommended Python version to 3.7+
- Use PostgreSQL 12 in CI
### Updated
- Falcon 3.1.1
## 1.4.3 - 2022-03-26
### Updated
- Update dependencies with `poetry update`
- Falcon 3.1.0, a minor change for us, but good to be using a current upstream
version
## 1.4.2 - 2021-04-14
### Updated
- Update dependencies with `poetry update`
- Falcon 3.0.0, a minor change for us, but good to be using a current upstream
version
### Fixed
- Bug in several of the "valid page" tests
### Added
- GitHub Actions workflow to build and test the API
## [1.4.1] - 2021-01-14
### Changed
- Limit Solr query to UUIDs to avoid errors with unmigrated legacy stats (https://github.com/ilri/dspace-statistics-api/issues/12)
### Updated
- Dev dependencies
## [1.4.0] - 2020-12-27
### Added
- indexer.py now indexes views and downloads for communities and collections
- API endpoints for /communities, /community/id, /collections, and /collections/id
- Swagger UI interface on /swagger
- /status page which lists the API version
### Changed
- Add ORDER BY to /items resource to make sure results are returned
deterministically
- Use `fl` parameter in indexer to return only the field we are faceting by
- Minor refactoring of imports for PEP8 style
- More correct calculation of `totalPages` parameter in REST API response
## [1.3.2] - 2020-11-18
### Fixed
- Minor issue with limit parameter (> 0)
- Minor issue with limit parameter (<= 100)
### Changed
- Minor refactor in Solr bot filtering
### Updated
- Run poetry update
## [1.3.1] - 2020-10-06
### Changed
- Fix issue with requirements.txt caused by poetry's export
## [1.3.0] - 2020-10-06
### Changed
- Minor refactoring of indexer
@ -43,7 +105,7 @@ and gunicorn 20.0.4
- Minor syntax issues highlighted by flake8
## [1.1.0] - 2019-05-05
## Updated
### Updated
- Falcon 2.0.0 (@alanorth)
## [1.0.0] - 2019-04-15
@ -61,7 +123,7 @@ and gunicorn 20.0.4
## [0.9.0] - 2019-01-22
### Updated
- pytest version 4.0.0
- Fix indexing of sharded statistics cores ([#10))
- Fix indexing of sharded statistics cores (#10)
- Handle case of missing views/downloads gracefully
## [0.8.1] - 2018-11-14

View File

@ -1,10 +1,18 @@
# DSpace Statistics API [![Build Status](https://travis-ci.org/ilri/dspace-statistics-api.svg?branch=master)](https://travis-ci.org/ilri/dspace-statistics-api) [![builds.sr.ht status](https://builds.sr.ht/~alanorth/dspace-statistics-api.svg)](https://builds.sr.ht/~alanorth/dspace-statistics-api?)
DSpace stores item view and download events in a Solr "statistics" core. This information is available for use in the various DSpace user interfaces, but is not exposed externally via any APIs. The DSpace 4/5/6 [REST API](https://wiki.lyrasis.org/display/DSDOC5x/REST+API), for example, only exposes information about communities, collections, item metadata, and bitstreams.
<h1 align="center">DSpace Statistics API</h1>
<p align="center">
<a href="https://ci.mjanja.ch/alanorth/dspace-statistics-api"><img alt="Build Status" src="https://ci.mjanja.ch/api/badges/alanorth/dspace-statistics-api/status.svg?ref=refs/heads/v6_x"></a>
<a href="https://github.com/ilri/dspace-statistics-api/actions"><img alt="Build and Test" src="https://github.com/ilri/dspace-statistics-api/actions/workflows/python-app.yml/badge.svg"></a>
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
</p>
DSpace stores item view and download events in a Solr "statistics" core. This information is available for use in the various DSpace user interfaces, but is not exposed externally via any APIs. The DSpace 4/5/6 [REST API](https://wiki.lyrasis.org/display/DSDOC5x/REST+API), for example, only exposes _metadata_ about communities, collections, items, and bitstreams.
- If your DSpace is version 4 or 5, use [dspace-statistics-api v1.1.1](https://github.com/ilri/dspace-statistics-api/releases/tag/v1.1.1)
- If your DSpace is version 6+, use [dspace-statistics-api v1.2.0 or greater](https://github.com/ilri/dspace-statistics-api/releases/tag/v1.2.0)
- Please make sure your statistics have been migrated from integers to UUIDs with the [solr-upgrade-statistics-6x](https://wiki.lyrasis.org/display/DSDOC6x/SOLR+Statistics+Maintenance) command
This project contains an indexer and a [Falcon-based](https://falcon.readthedocs.io/) web application to make the statistics available via a simple REST API. You can read more about the Solr queries used to gather the item view and download statistics on the [DSpace wiki](https://wiki.lyrasis.org/display/DSPACE/Solr).
This project contains an indexer and a [Falcon-based](https://falcon.readthedocs.io/) web application to make the item, community, and collection statistics available via a simple REST API. You can read more about the Solr queries used to gather the item view and download statistics on the [DSpace wiki](https://wiki.lyrasis.org/display/DSPACE/Solr).
If you use the DSpace Statistics API please cite:
@ -12,9 +20,9 @@ If you use the DSpace Statistics API please cite:
## Requirements
- Python 3.6+
- Python 3.8+
- PostgreSQL version 9.5+ (due to [`UPSERT` support](https://wiki.postgresql.org/wiki/UPSERT))
- DSpace with [Solr usage statistics enabled](https://wiki.lyrasis.org/display/DSDOC5x/SOLR+Statistics) (tested with 5.x)
- DSpace with [Solr usage statistics enabled](https://wiki.lyrasis.org/display/DSDOC5x/SOLR+Statistics) (tested with 5.8+ and 6.3)
## Installation
Create a Python virtual environment and install the dependencies:
@ -81,14 +89,20 @@ The API exposes the following endpoints:
- GET `/`return a basic API documentation page.
- GET `/items`return views and downloads for all items that Solr knows about¹. Accepts `limit` and `page` query parameters for pagination of results (`limit` must be an integer between 1 and 100, and `page` must be an integer greater than or equal to 0).
- POST `/items`return views and downloads for an arbitrary list of items. Accepts `limit`, `page`, `dateFrom`, and `dateTo` parameters².
- POST `/items`return views and downloads for an arbitrary list of items with an optional date range. Accepts `limit`, `page`, `dateFrom`, and `dateTo` parameters².
- GET `/item/id`return views and downloads for a single item (`id` must be a UUID). Returns HTTP 404 if an item id is not found.
- GET `/communities`return views and downloads for all communities that Solr knows about¹. Accepts `limit` and `page` query parameters for pagination of results (`limit` must be an integer between 1 and 100, and `page` must be an integer greater than or equal to 0).
- POST `/communities`return views and downloads for an arbitrary list of communities with an optional date range. Accepts `limit`, `page`, `dateFrom`, and `dateTo` parameters².
- GET `/community/id`return views and downloads for a single community (`id` must be a UUID). Returns HTTP 404 if a community id is not found.
- GET `/collections`return views and downloads for all collections that Solr knows about¹. Accepts `limit` and `page` query parameters for pagination of results (`limit` must be an integer between 1 and 100, and `page` must be an integer greater than or equal to 0).
- POST `/collections`return views and downloads for an arbitrary list of collections with an optional date range. Accepts `limit`, `page`, `dateFrom`, and `dateTo` parameters².
- GET `/collection/id`return views and downloads for a single collection (`id` must be a UUID). Returns HTTP 404 if an collection id is not found.
The item id is the *internal* UUID for an item. You can get these from the standard DSpace REST API.
The id is the *internal* UUID for an item, community, or collection. You can get these from the standard DSpace REST API.
¹ We are querying the Solr statistics core, which technically only knows about items that have either views or downloads. If an item is not present here you can assume it has zero views and zero downloads, but not necessarily that it does not exist in the repository.
¹ We are querying the Solr statistics core, which technically only knows about items, communities, or collections that have either views or downloads. If an item, community, or collection is not present here you can assume it has zero views and zero downloads, but not necessarily that it does not exist in the repository.
² POST requests to `/items` should be in JSON format with the following parameters:
² POST requests to `/items`, `/communities`, and `/collections` should be in JSON format with the following parameters (substitute the "items" list for communities or collections accordingly):
```
{
@ -109,11 +123,10 @@ The item id is the *internal* UUID for an item. You can get these from the stand
- Better logging
- Version API (or at least include a /version endpoint?)
- Probably use /status with a version in the response
- Use JSON in PostgreSQL
- Add top items endpoint, perhaps `/top/items` or `/items/top`?
- Actually we could add `/items?limit=10&sort=views`
- Make community and collection stats available
- Check IDs in database to see if they are deleted...
## License
This work is licensed under the [GPLv3](https://www.gnu.org/licenses/gpl-3.0.en.html).

View File

@ -1,23 +1,77 @@
import falcon
# SPDX-License-Identifier: GPL-3.0-only
import json
import math
import falcon
import psycopg2.extras
from falcon_swagger_ui import register_swaggerui_app
from .config import DSPACE_STATISTICS_API_URL, VERSION
from .database import DatabaseManager
from .items import get_downloads, get_views
from .util import validate_items_post_parameters
from .stats import get_downloads, get_views
from .util import set_statistics_scope, validate_post_parameters
class RootResource:
def on_get(self, req, resp):
resp.status = falcon.HTTP_200
resp.content_type = "text/html"
with open("dspace_statistics_api/docs/index.html", "r") as f:
resp.body = f.read()
docs_html = (
"<!DOCTYPE html>"
'<html lang="en-US">'
" <head>"
' <meta charset="UTF-8">'
" <title>DSpace Statistics API</title>"
" </head>"
" <body>"
f" <h1>DSpace Statistics API {VERSION}</h1>"
f" <p>This site is running the <a href=\"https://github.com/ilri/dspace-statistics-api\" title=\"DSpace Statistics API project\">DSpace Statistics API</a>. For more information see the project's README.md or the interactive <a href=\"{DSPACE_STATISTICS_API_URL + '/swagger'}\">Swagger UI</a> built into this API.</p>"
" </body>"
"</html"
)
resp.text = docs_html
class AllItemsResource:
class StatusResource:
def on_get(self, req, resp):
message = {"version": VERSION}
resp.status = falcon.HTTP_200
resp.media = message
class OpenAPIJSONResource:
def on_get(self, req, resp):
resp.status = falcon.HTTP_200
resp.content_type = "text/html"
with open("dspace_statistics_api/docs/openapi.json", "r") as f:
# Load the openapi.json schema
data = json.load(f)
# Swagger assumes your API is at the root of the current host unless
# you configure a "servers" block in the schema. The problem is that
# I want this to work in both development and production, so we need
# to make this configurable.
#
# If the DSPACE_STATISTICS_API_URL is configured then we will add a
# server entry to the openapi.json schema before sending it.
if DSPACE_STATISTICS_API_URL != "":
data["servers"] = [{"url": DSPACE_STATISTICS_API_URL}]
# Set the version in the schema so Swagger UI can display it
data["info"]["version"] = VERSION
resp.text = json.dumps(data)
class AllStatisticsResource:
@falcon.before(set_statistics_scope)
def on_get(self, req, resp):
"""Handles GET requests"""
# Return HTTPBadRequest if id parameter is not present and valid
limit = req.get_param_as_int("limit", min_value=0, max_value=100) or 100
limit = req.get_param_as_int("limit", min_value=1, max_value=100) or 100
page = req.get_param_as_int("page", min_value=0) or 0
offset = limit * page
@ -25,26 +79,26 @@ class AllItemsResource:
db.set_session(readonly=True)
with db.cursor() as cursor:
# get total number of items so we can estimate the pages
cursor.execute("SELECT COUNT(id) FROM items")
pages = round(cursor.fetchone()[0] / limit)
# get total number of communities/collections/items so we can estimate the pages
cursor.execute(f"SELECT COUNT(id) FROM {req.context.statistics_scope}")
pages = math.ceil(cursor.fetchone()[0] / limit)
# get statistics and use limit and offset to page through results
cursor.execute(
"SELECT id, views, downloads FROM items LIMIT %s OFFSET %s",
f"SELECT id, views, downloads FROM {req.context.statistics_scope} ORDER BY id LIMIT %s OFFSET %s",
[limit, offset],
)
# create a list to hold dicts of item stats
statistics = list()
# create a list to hold dicts of stats
statistics = []
# iterate over results and build statistics object
for item in cursor:
for result in cursor:
statistics.append(
{
"id": str(item["id"]),
"views": item["views"],
"downloads": item["downloads"],
"id": str(result["id"]),
"views": result["views"],
"downloads": result["downloads"],
}
)
@ -57,9 +111,15 @@ class AllItemsResource:
resp.media = message
@falcon.before(validate_items_post_parameters)
@falcon.before(set_statistics_scope)
@falcon.before(validate_post_parameters)
def on_post(self, req, resp):
"""Handles POST requests"""
"""Handles POST requests.
Uses two `before` hooks to set the statistics "scope" and validate the
POST parameters. The "scope" is the type of statistics we want, which
will be items, communities, or collections, depending on the request.
"""
# Build the Solr date string, ie: [* TO *]
if req.context.dateFrom and req.context.dateTo:
@ -73,10 +133,10 @@ class AllItemsResource:
# Helper variables to make working with pages/items/results easier and
# to make the code easier to understand
number_of_items: int = len(req.context.items)
pages: int = int(number_of_items / req.context.limit)
first_item: int = req.context.page * req.context.limit
last_item: int = first_item + req.context.limit
number_of_elements: int = len(req.context.elements)
pages: int = math.ceil(number_of_elements / req.context.limit)
first_element: int = req.context.page * req.context.limit
last_element: int = first_element + req.context.limit
# Get a subset of the POSTed items based on our limit. Note that Python
# list slicing and indexing are both zero based, but the first and last
# items in a slice can be confusing. See this ASCII diagram:
@ -87,20 +147,24 @@ class AllItemsResource:
# Slice position: 0 1 2 3 4 5 6
# Index position: 0 1 2 3 4 5
#
# So if we have a list items with 240 items:
# So if we have a list of items with 240 items:
#
# 1st set: items[0:100] would give items at indexes 0 to 99
# 2nd set: items[100:200] would give items at indexes 100 to 199
# 3rd set: items[200:300] would give items at indexes 200 to 239
items_subset: list = req.context.items[first_item:last_item]
elements_subset: list = req.context.elements[first_element:last_element]
views: dict = get_views(solr_date_string, items_subset)
downloads: dict = get_downloads(solr_date_string, items_subset)
views: dict = get_views(
solr_date_string, elements_subset, req.context.views_facet_field
)
downloads: dict = get_downloads(
solr_date_string, elements_subset, req.context.downloads_facet_field
)
# create a list to hold dicts of item stats
statistics = list()
# create a list to hold dicts of stats
statistics = []
# iterate over views dict to extract views and use the item id as an
# iterate over views dict to extract views and use the element id as an
# index to the downloads dict to extract downloads.
for k, v in views.items():
statistics.append({"id": k, "views": v, "downloads": downloads[k]})
@ -116,12 +180,11 @@ class AllItemsResource:
resp.media = message
class ItemResource:
def on_get(self, req, resp, item_id):
class SingleStatisticsResource:
@falcon.before(set_statistics_scope)
def on_get(self, req, resp, id_):
"""Handles GET requests"""
import psycopg2.extras
# Adapt Pythons uuid.UUID type to PostgreSQLs uuid
# See: https://www.psycopg.org/docs/extras.html
psycopg2.extras.register_uuid()
@ -132,18 +195,19 @@ class ItemResource:
with db.cursor() as cursor:
cursor = db.cursor()
cursor.execute(
"SELECT views, downloads FROM items WHERE id=%s", [str(item_id)]
f"SELECT views, downloads FROM {req.context.database} WHERE id=%s",
[str(id_)],
)
if cursor.rowcount == 0:
raise falcon.HTTPNotFound(
title="Item not found",
description=f'The item with id "{str(item_id)}" was not found.',
title=f"{req.context.statistics_scope} not found",
description=f'The {req.context.statistics_scope} with id "{str(id_)}" was not found.',
)
else:
results = cursor.fetchone()
statistics = {
"id": str(item_id),
"id": str(id_),
"views": results["views"],
"downloads": results["downloads"],
}
@ -151,9 +215,45 @@ class ItemResource:
resp.media = statistics
api = application = falcon.API()
api.add_route("/", RootResource())
api.add_route("/items", AllItemsResource())
api.add_route("/item/{item_id:uuid}", ItemResource())
app = application = falcon.App()
app.add_route("/", RootResource())
app.add_route("/status", StatusResource())
# Item routes
app.add_route("/items", AllStatisticsResource())
app.add_route("/item/{id_:uuid}", SingleStatisticsResource())
# Community routes
app.add_route("/communities", AllStatisticsResource())
app.add_route("/community/{id_:uuid}", SingleStatisticsResource())
# Collection routes
app.add_route("/collections", AllStatisticsResource())
app.add_route("/collection/{id_:uuid}", SingleStatisticsResource())
# Route to the Swagger UI Openapp schema
app.add_route("/docs/openapi.json", OpenAPIJSONResource())
# Path to host the Swagger UI. Keep in mind that Falcon will add a route for
# this automatically when we register Swagger and the path will be relative
# to the Falcon app like all other routes, not the absolute root.
SWAGGERUI_PATH = "/swagger"
# The *absolute* path to the OpenJSON schema. This must be absolute because
# it will be requested by the client and must resolve absolutely. Note: the
# name of this variable is misleading because it is actually the schema URL
# but we pass it into the register_swaggerui_app() function as the app_url
# parameter.
SWAGGERUI_API_URL = f"{DSPACE_STATISTICS_API_URL}/docs/openapi.json"
register_swaggerui_app(
app,
SWAGGERUI_PATH,
SWAGGERUI_API_URL,
config={
"supportedSubmitMethods": ["get", "post"],
},
uri_prefix=DSPACE_STATISTICS_API_URL,
)
# vim: set sw=4 ts=4 expandtab:

View File

@ -1,3 +1,5 @@
# SPDX-License-Identifier: GPL-3.0-only
import os
# Check if Solr connection information was provided in the environment
@ -9,4 +11,13 @@ DATABASE_PASS = os.environ.get("DATABASE_PASS", "dspacestatistics")
DATABASE_HOST = os.environ.get("DATABASE_HOST", "localhost")
DATABASE_PORT = os.environ.get("DATABASE_PORT", "5432")
# URL to DSpace Statistics API, which will be used as a prefix to API calls in
# the Swagger UI. An empty string will allow this to work out of the box in a
# local development environment, but for production it should be set to a value
# like "/rest/statistics", assuming that the statistics API is deployed next to
# the vanilla DSpace REST API.
DSPACE_STATISTICS_API_URL = os.environ.get("DSPACE_STATISTICS_API_URL", "")
VERSION = "1.4.4-dev"
# vim: set sw=4 ts=4 expandtab:

View File

@ -1,3 +1,5 @@
# SPDX-License-Identifier: GPL-3.0-only
import falcon
import psycopg2
import psycopg2.extras

View File

@ -1,37 +0,0 @@
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<title>DSpace Statistics API v1.3.0</title>
</head>
<body>
<h1>DSpace Statistics API</h1>
<p>This site is running the <a href="https://github.com/ilri/dspace-statistics-api" title="DSpace Statistics API project">DSpace Statistics API</a>. The following endpoints are available:</p>
<ul>
<li>GET <code>/</code>return a basic API documentation page.</li>
<li>GET <code>/items</code>return views and downloads for all items that Solr knows about¹. Accepts <code>limit</code> and <code>page</code> query parameters for pagination of results (<code>limit</code> must be an integer between 1 and 100, and <code>page</code> must be an integer greater than or equal to 0).</li>
<li>POST <code>/items</code>return views and downloads for an arbitrary list of items. Accepts <code>limit</code>, <code>page</code>, <code>dateFrom</code>, and <code>dateTo</code> parameters².</li>
<li>GET <code>/item/id</code>return views and downloads for a single item (<code>id</code> must be a UUID). Returns HTTP 404 if an item id is not found.</li>
</ul>
<p>The item id is the <em>internal</em> uuid for an item. You can get these from the standard DSpace REST API.</p>
<hr/>
<p>¹ We are querying the Solr statistics core, which technically only knows about items that have either views or downloads. If an item is not present here you can assume it has zero views and zero downloads, but not necessarily that it does not exist in the repository.</p>
<p>² POST requests to <code>/items</code> should be in JSON format with the following parameters:
<pre><code>{
"limit": 100, // optional, integer between 0 and 100, default 100
"page": 0, // optional, integer greater than 0, default 0
"dateFrom": "2020-01-01T00:00:00Z", // optional, default *
"dateTo": "2020-09-09T00:00:00Z", // optional, default *
"items": [
"f44cf173-2344-4eb2-8f00-ee55df32c76f",
"2324aa41-e9de-4a2b-bc36-16241464683e",
"8542f9da-9ce1-4614-abf4-f2e3fdb4b305",
"0fe573e7-042a-4240-a4d9-753b61233908"
]
}</code></pre>
</p>
</body>
</html>

View File

@ -0,0 +1,616 @@
{
"openapi": "3.0.3",
"info": {
"version": "1.4.4-dev",
"title": "DSpace Statistics API",
"description": "A [Falcon-based](https://falcon.readthedocs.io/) web application to make DSpace's item, community, and collection statistics available via a simple REST API. This Swagger interface is powered by [falcon-swagger-ui](https://github.com/rdidyk/falcon-swagger-ui).",
"license": {
"name": "GPLv3.0",
"url": "https://www.gnu.org/licenses/gpl-3.0.en.html"
}
},
"paths": {
"/item/{item_uuid}": {
"get": {
"summary": "Statistics for a specific item",
"operationId": "getItem",
"tags": [
"item"
],
"parameters": [
{
"name": "item_uuid",
"in": "path",
"required": true,
"description": "The UUID of the item to retrieve",
"schema": {
"type": "string",
"format": "uuid",
"example": "9596aeff-0b90-47d3-9fec-02d578920507"
}
}
],
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"404": {
"description": "Item not found"
}
}
}
},
"/items": {
"get": {
"summary": "Get statistics for all items",
"operationId": "getItems",
"tags": [
"items"
],
"parameters": [
{
"name": "limit",
"in": "query",
"description": "How many items to return at once (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100,
"example": 100
}
},
{
"name": "page",
"in": "query",
"description": "Page of results to start on (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0,
"example": 0
}
}
],
"responses": {
"200": {
"description": "A paged array of items",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"400": {
"description": "Bad request"
}
}
},
"post": {
"summary": "Get statistics for a list of items with an optional date range",
"operationId": "postItems",
"tags": [
"items"
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"limit": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100
},
"page": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0
},
"dateFrom": {
"type": "string",
"format": "date"
},
"dateTo": {
"type": "string",
"format": "date"
},
"items": {
"type": "array",
"items": {
"type": "string",
"format": "uuid"
}
}
},
"example": {
"limit": 100,
"page": 0,
"dateFrom": "2020-01-01T00:00:00Z",
"dateTo": "2020-12-31T00:00:00Z",
"items": [
"f44cf173-2344-4eb2-8f00-ee55df32c76f",
"2324aa41-e9de-4a2b-bc36-16241464683e",
"8542f9da-9ce1-4614-abf4-f2e3fdb4b305",
"0fe573e7-042a-4240-a4d9-753b61233908"
]
}
}
}
}
},
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"currentPage": {
"type": "integer",
"format": "int32"
},
"limit": {
"type": "integer",
"format": "int32"
},
"totalPages": {
"type": "integer",
"format": "int32"
},
"statistics": {
"$ref": "#/components/schemas/ListOfElements"
}
}
}
}
}
},
"400": {
"description": "Bad request"
}
}
}
},
"/community/{community_uuid}": {
"get": {
"summary": "Statistics for a specific community",
"operationId": "getCommunity",
"tags": [
"community"
],
"parameters": [
{
"name": "community_uuid",
"in": "path",
"required": true,
"description": "The UUID of the community to retrieve",
"schema": {
"type": "string",
"format": "uuid",
"example": "bde7139c-d321-46bb-aef6-ae70799e5edb"
}
}
],
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"404": {
"description": "Community not found"
}
}
}
},
"/communities": {
"get": {
"summary": "Get statistics for all communities",
"operationId": "getCommunities",
"tags": [
"communities"
],
"parameters": [
{
"name": "limit",
"in": "query",
"description": "How many communities to return at once (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100,
"example": 100
}
},
{
"name": "page",
"in": "query",
"description": "Zero-based page of results to start on (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0,
"example": 0
}
}
],
"responses": {
"200": {
"description": "A paged array of communities",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"400": {
"description": "Bad request"
}
}
},
"post": {
"summary": "Get statistics for a list of communities with an optional date range",
"operationId": "postCommunities",
"tags": [
"communities"
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"limit": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100
},
"page": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0
},
"dateFrom": {
"type": "string",
"format": "date"
},
"dateTo": {
"type": "string",
"format": "date"
},
"communities": {
"type": "array",
"items": {
"type": "string",
"format": "uuid"
}
}
},
"example": {
"limit": 100,
"page": 0,
"dateFrom": "2020-01-01T00:00:00Z",
"dateTo": "2020-12-31T00:00:00Z",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"8a8aeed1-077e-4360-bdf8-a5f3020193b1",
"47d0498a-203c-407d-afb8-1d44bf29badc",
"d3fe99a9-e27d-4035-9339-084c93228c82"
]
}
}
}
}
},
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"currentPage": {
"type": "integer",
"format": "int32"
},
"limit": {
"type": "integer",
"format": "int32"
},
"totalPages": {
"type": "integer",
"format": "int32"
},
"statistics": {
"$ref": "#/components/schemas/ListOfElements"
}
}
}
}
}
},
"400": {
"description": "Bad request"
}
}
}
},
"/collection/{collection_uuid}": {
"get": {
"summary": "Statistics for a specific collection",
"operationId": "getCollection",
"tags": [
"collection"
],
"parameters": [
{
"name": "collection_uuid",
"in": "path",
"required": true,
"description": "The UUID of the collection to retrieve",
"schema": {
"type": "string",
"format": "uuid",
"example": "49dc95d8-bf2f-4e68-b30f-41ea266c37ae"
}
}
],
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"404": {
"description": "Collection not found"
}
}
}
},
"/collections": {
"get": {
"summary": "Get statistics for all collections",
"operationId": "getCollections",
"tags": [
"collections"
],
"parameters": [
{
"name": "limit",
"in": "query",
"description": "How many collections to return at once (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100,
"example": 100
}
},
{
"name": "page",
"in": "query",
"description": "Zero-based page of results to start on (optional)",
"required": false,
"schema": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0,
"example": 0
}
}
],
"responses": {
"200": {
"description": "A paged array of collections",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
},
"400": {
"description": "Bad request"
}
}
},
"post": {
"summary": "Get statistics for a list of collections with an optional date range",
"operationId": "postCollections",
"tags": [
"collections"
],
"requestBody": {
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"limit": {
"type": "integer",
"format": "int32",
"minimum": 1,
"maximum": 100,
"default": 100
},
"page": {
"type": "integer",
"format": "int32",
"minimum": 0,
"default": 0
},
"dateFrom": {
"type": "string",
"format": "date"
},
"dateTo": {
"type": "string",
"format": "date"
},
"collections": {
"type": "array",
"items": {
"type": "string",
"format": "uuid"
}
}
},
"example": {
"limit": 100,
"page": 0,
"dateFrom": "2020-01-01T00:00:00Z",
"dateTo": "2020-12-31T00:00:00Z",
"collections": [
"5eeef6cf-b91b-42d0-9549-ea61bc8a758f",
"6aac3269-b4a9-4924-a24d-9e6ee2b410d2",
"551698dd-cd2b-4327-948e-54b5eb6deda5",
"39358713-bbaf-4149-a453-e2b18c09fd5d"
]
}
}
}
}
},
"responses": {
"200": {
"description": "Expected response to a valid request",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"currentPage": {
"type": "integer",
"format": "int32"
},
"limit": {
"type": "integer",
"format": "int32"
},
"totalPages": {
"type": "integer",
"format": "int32"
},
"statistics": {
"$ref": "#/components/schemas/ListOfElements"
}
}
}
}
}
},
"400": {
"description": "Bad request"
}
}
}
},
"/status": {
"get": {
"summary": "Get API status",
"operationId": "getStatus",
"tags": [
"status"
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"version": {
"type": "string",
"example": "1.4.0-dev"
}
}
}
}
}
},
"405": {
"description": "Method Not Allowed"
}
}
}
}
},
"components": {
"schemas": {
"SingleElementResponse": {
"type": "object",
"required": [
"id",
"views",
"downloads"
],
"properties": {
"id": {
"type": "string",
"format": "uuid"
},
"views": {
"type": "integer",
"example": 450
},
"downloads": {
"type": "integer",
"example": 1337
}
}
},
"ListOfElements": {
"type": "array",
"items": {
"$ref": "#/components/schemas/SingleElementResponse"
}
}
}
}
}

View File

@ -1,25 +1,9 @@
# SPDX-License-Identifier: GPL-3.0-only
#
# indexer.py
#
# Copyright 2018 Alan Orth.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# ---
#
# Connects to a DSpace Solr statistics core and ingests item views and downloads
# into a PostgreSQL database for use by other applications (like an API).
# Connects to a DSpace Solr statistics core and ingests views and downloads for
# communities, collections, and items into a PostgreSQL database.
#
# This script is written for Python 3.6+ and requires several modules that you
# can install with pip (I recommend using a Python virtual environment):
@ -28,6 +12,8 @@
#
# See: https://wiki.duraspace.org/display/DSPACE/Solr
import math
import psycopg2.extras
import requests
@ -36,7 +22,7 @@ from .database import DatabaseManager
from .util import get_statistics_shards
def index_views():
def index_views(indexType: str, facetField: str):
# get total number of distinct facets for items with a minimum of 1 view,
# otherwise Solr returns all kinds of weird ids that are actually not in
# the database. Also, stats are expensive, but we need stats.calcdistinct
@ -45,15 +31,16 @@ def index_views():
#
# see: https://lucene.apache.org/solr/guide/6_6/the-stats-component.html
solr_query_params = {
"q": "type:2",
"fq": "isBot:false AND statistics_type:view",
"q": f"type:2 AND {facetField}:/.{{36}}/",
"fq": "-isBot:true AND statistics_type:view",
"fl": facetField,
"facet": "true",
"facet.field": "id",
"facet.field": facetField,
"facet.mincount": 1,
"facet.limit": 1,
"facet.offset": 0,
"stats": "true",
"stats.field": "id",
"stats.field": facetField,
"stats.calcdistinct": "true",
"shards": shards,
"rows": 0,
@ -66,17 +53,17 @@ def index_views():
try:
# get total number of distinct facets (countDistinct)
results_totalNumFacets = res.json()["stats"]["stats_fields"]["id"][
results_totalNumFacets = res.json()["stats"]["stats_fields"][facetField][
"countDistinct"
]
except TypeError:
print("No item views to index, exiting.")
print(f"{indexType}: no views, exiting.")
exit(0)
# divide results into "pages" (cast to int to effectively round down)
# divide results into "pages" and round up to next integer
results_per_page = 100
results_num_pages = int(results_totalNumFacets / results_per_page)
results_num_pages = math.ceil(results_totalNumFacets / results_per_page)
results_current_page = 0
with DatabaseManager() as db:
@ -87,14 +74,15 @@ def index_views():
while results_current_page <= results_num_pages:
# "pages" are zero based, but one based is more human readable
print(
f"Indexing item views (page {results_current_page + 1} of {results_num_pages + 1})"
f"{indexType}: indexing views (page {results_current_page + 1} of {results_num_pages + 1})"
)
solr_query_params = {
"q": "type:2",
"fq": "isBot:false AND statistics_type:view",
"q": f"type:2 AND {facetField}:/.{{36}}/",
"fq": "-isBot:true AND statistics_type:view",
"fl": facetField,
"facet": "true",
"facet.field": "id",
"facet.field": facetField,
"facet.mincount": 1,
"facet.limit": results_per_page,
"facet.offset": results_current_page * results_per_page,
@ -108,12 +96,12 @@ def index_views():
# Solr returns facets as a dict of dicts (see json.nl parameter)
views = res.json()["facet_counts"]["facet_fields"]
# iterate over the 'id' dict and get the item ids and views
for item_id, item_views in views["id"].items():
data.append((item_id, item_views))
# iterate over the facetField dict and get the ids and views
for id_, views in views[facetField].items():
data.append((id_, views))
# do a batch insert of values from the current "page" of results
sql = "INSERT INTO items(id, views) VALUES %s ON CONFLICT(id) DO UPDATE SET views=excluded.views"
sql = f"INSERT INTO {indexType}(id, views) VALUES %s ON CONFLICT(id) DO UPDATE SET views=excluded.views"
psycopg2.extras.execute_values(cursor, sql, data, template="(%s, %s)")
db.commit()
@ -123,18 +111,19 @@ def index_views():
results_current_page += 1
def index_downloads():
def index_downloads(indexType: str, facetField: str):
# get the total number of distinct facets for items with at least 1 download
solr_query_params = {
"q": "type:0",
"fq": "isBot:false AND statistics_type:view AND bundleName:ORIGINAL",
"q": f"type:0 AND {facetField}:/.{{36}}/",
"fq": "-isBot:true AND statistics_type:view AND bundleName:ORIGINAL",
"fl": facetField,
"facet": "true",
"facet.field": "owningItem",
"facet.field": facetField,
"facet.mincount": 1,
"facet.limit": 1,
"facet.offset": 0,
"stats": "true",
"stats.field": "owningItem",
"stats.field": facetField,
"stats.calcdistinct": "true",
"shards": shards,
"rows": 0,
@ -147,17 +136,16 @@ def index_downloads():
try:
# get total number of distinct facets (countDistinct)
results_totalNumFacets = res.json()["stats"]["stats_fields"]["owningItem"][
results_totalNumFacets = res.json()["stats"]["stats_fields"][facetField][
"countDistinct"
]
except TypeError:
print("No item downloads to index, exiting.")
print(f"{indexType}: no downloads, exiting.")
exit(0)
# divide results into "pages" (cast to int to effectively round down)
results_per_page = 100
results_num_pages = int(results_totalNumFacets / results_per_page)
results_num_pages = math.ceil(results_totalNumFacets / results_per_page)
results_current_page = 0
with DatabaseManager() as db:
@ -168,14 +156,15 @@ def index_downloads():
while results_current_page <= results_num_pages:
# "pages" are zero based, but one based is more human readable
print(
f"Indexing item downloads (page {results_current_page + 1} of {results_num_pages + 1})"
f"{indexType}: indexing downloads (page {results_current_page + 1} of {results_num_pages + 1})"
)
solr_query_params = {
"q": "type:0",
"fq": "isBot:false AND statistics_type:view AND bundleName:ORIGINAL",
"q": f"type:0 AND {facetField}:/.{{36}}/",
"fq": "-isBot:true AND statistics_type:view AND bundleName:ORIGINAL",
"fl": facetField,
"facet": "true",
"facet.field": "owningItem",
"facet.field": facetField,
"facet.mincount": 1,
"facet.limit": results_per_page,
"facet.offset": results_current_page * results_per_page,
@ -189,12 +178,12 @@ def index_downloads():
# Solr returns facets as a dict of dicts (see json.nl parameter)
downloads = res.json()["facet_counts"]["facet_fields"]
# iterate over the 'owningItem' dict and get the item ids and downloads
for item_id, item_downloads in downloads["owningItem"].items():
data.append((item_id, item_downloads))
# iterate over the facetField dict and get the item ids and downloads
for id_, downloads in downloads[facetField].items():
data.append((id_, downloads))
# do a batch insert of values from the current "page" of results
sql = "INSERT INTO items(id, downloads) VALUES %s ON CONFLICT(id) DO UPDATE SET downloads=excluded.downloads"
sql = f"INSERT INTO {indexType}(id, downloads) VALUES %s ON CONFLICT(id) DO UPDATE SET downloads=excluded.downloads"
psycopg2.extras.execute_values(cursor, sql, data, template="(%s, %s)")
db.commit()
@ -211,13 +200,32 @@ with DatabaseManager() as db:
"""CREATE TABLE IF NOT EXISTS items
(id UUID PRIMARY KEY, views INT DEFAULT 0, downloads INT DEFAULT 0)"""
)
# create table to store community views and downloads
cursor.execute(
"""CREATE TABLE IF NOT EXISTS communities
(id UUID PRIMARY KEY, views INT DEFAULT 0, downloads INT DEFAULT 0)"""
)
# create table to store collection views and downloads
cursor.execute(
"""CREATE TABLE IF NOT EXISTS collections
(id UUID PRIMARY KEY, views INT DEFAULT 0, downloads INT DEFAULT 0)"""
)
# commit the table creation before closing the database connection
db.commit()
shards = get_statistics_shards()
index_views()
index_downloads()
# Index views and downloads for items, communities, and collections. Here the
# first parameter is the type of indexing to perform, and the second parameter
# is the field to facet by in Solr's statistics to get this information.
index_views("items", "id")
index_views("communities", "owningComm")
index_views("collections", "owningColl")
index_downloads("items", "owningItem")
index_downloads("communities", "owningComm")
index_downloads("collections", "owningColl")
# vim: set sw=4 ts=4 expandtab:

View File

@ -1,105 +0,0 @@
import requests
from .config import SOLR_SERVER
def get_views(solr_date_string: str, items: list):
"""
Get view statistics for a list of items from Solr.
:parameter solr_date_string (str): Solr date string, for example "[* TO *]"
:parameter items (list): a list of item IDs
:returns: A dict of item IDs and views
"""
from .util import get_statistics_shards
shards = get_statistics_shards()
# Join the UUIDs with "OR" and escape the hyphens for Solr
solr_items_string: str = " OR ".join(items).replace("-", r"\-")
solr_query_params = {
"q": f"id:({solr_items_string})",
"fq": f"type:2 AND isBot:false AND statistics_type:view AND time:{solr_date_string}",
"facet": "true",
"facet.field": "id",
"facet.mincount": 1,
"shards": shards,
"rows": 0,
"wt": "json",
"json.nl": "map", # return facets as a dict instead of a flat list
}
solr_url = SOLR_SERVER + "/statistics/select"
res = requests.get(solr_url, params=solr_query_params)
# Create an empty dict to store views
data = {}
# Solr returns facets as a dict of dicts (see the json.nl parameter)
views = res.json()["facet_counts"]["facet_fields"]
# iterate over the 'id' dict and get the item ids and views
for item_id, item_views in views["id"].items():
data[item_id] = item_views
# Check if any items have missing stats so we can set them to 0
if len(data) < len(items):
# List comprehension to get a list of item ids (keys) in the data
data_ids = [k for k, v in data.items()]
for item_id in items:
if item_id not in data_ids:
data[item_id] = 0
continue
return data
def get_downloads(solr_date_string: str, items: list):
"""
Get download statistics for a list of items from Solr.
:parameter solr_date_string (str): Solr date string, for example "[* TO *]"
:parameter items (list): a list of item IDs
:returns: A dict of item IDs and downloads
"""
from .util import get_statistics_shards
shards = get_statistics_shards()
# Join the UUIDs with "OR" and escape the hyphens for Solr
solr_items_string: str = " OR ".join(items).replace("-", r"\-")
solr_query_params = {
"q": f"owningItem:({solr_items_string})",
"fq": f"type:0 AND isBot:false AND statistics_type:view AND bundleName:ORIGINAL AND time:{solr_date_string}",
"facet": "true",
"facet.field": "owningItem",
"facet.mincount": 1,
"shards": shards,
"rows": 0,
"wt": "json",
"json.nl": "map", # return facets as a dict instead of a flat list
}
solr_url = SOLR_SERVER + "/statistics/select"
res = requests.get(solr_url, params=solr_query_params)
# Create an empty dict to store downloads
data = {}
# Solr returns facets as a dict of dicts (see the json.nl parameter)
downloads = res.json()["facet_counts"]["facet_fields"]
# Iterate over the 'owningItem' dict and get the item ids and downloads
for item_id, item_downloads in downloads["owningItem"].items():
data[item_id] = item_downloads
# Check if any items have missing stats so we can set them to 0
if len(data) < len(items):
# List comprehension to get a list of item ids (keys) in the data
data_ids = [k for k, v in data.items()]
for item_id in items:
if item_id not in data_ids:
data[item_id] = 0
continue
return data
# vim: set sw=4 ts=4 expandtab:

View File

@ -0,0 +1,126 @@
# SPDX-License-Identifier: GPL-3.0-only
import requests
from .config import SOLR_SERVER
from .util import get_statistics_shards
def get_views(solr_date_string: str, elements: list, facetField: str):
"""
Get view statistics for a list of elements from Solr. Depending on the req-
uest this could be items, communities, or collections.
:parameter solr_date_string (str): Solr date string, for example "[* TO *]"
:parameter elements (list): a list of IDs
:parameter facetField (str): Solr field to facet by, for example "id"
:returns: A dict of IDs and views
"""
shards = get_statistics_shards()
# Join the UUIDs with "OR" and escape the hyphens for Solr
solr_elements_string: str = " OR ".join(elements).replace("-", r"\-")
solr_query_params = {
"q": f"{facetField}:({solr_elements_string})",
"fq": f"type:2 AND -isBot:true AND statistics_type:view AND time:{solr_date_string}",
"fl": facetField,
"facet": "true",
"facet.field": facetField,
"facet.mincount": 1,
"shards": shards,
"rows": 0,
"wt": "json",
"json.nl": "map", # return facets as a dict instead of a flat list
}
solr_url = SOLR_SERVER + "/statistics/select"
res = requests.get(solr_url, params=solr_query_params)
# Create an empty dict to store views
data = {}
# Solr returns facets as a dict of dicts (see the json.nl parameter)
views = res.json()["facet_counts"]["facet_fields"]
# iterate over the facetField dict and ids and views
for id_, views in views[facetField].items():
# For items we can rely on Solr returning facets for the *only* the ids
# in our query, but for communities and collections, the owningComm and
# owningColl fields are multi-value so Solr will return facets with the
# values in our query as well as *any others* that happen to be present
# in the field (which looks like Solr returning unrelated results until
# you realize that the field is multi-value and this is correct).
#
# To work around this I make sure that each id in the returned dict are
# present in the elements list POSTed by the user.
if id_ in elements:
data[id_] = views
# Check if any ids have missing stats so we can set them to 0
if len(data) < len(elements):
# List comprehension to get a list of ids (keys) in the data
data_ids = [k for k, v in data.items()]
for element_id in elements:
if element_id not in data_ids:
data[element_id] = 0
continue
return data
def get_downloads(solr_date_string: str, elements: list, facetField: str):
"""
Get download statistics for a list of items from Solr. Depending on the req-
uest this could be items, communities, or collections.
:parameter solr_date_string (str): Solr date string, for example "[* TO *]"
:parameter elements (list): a list of IDs
:parameter facetField (str): Solr field to facet by, for example "id"
:returns: A dict of IDs and downloads
"""
shards = get_statistics_shards()
# Join the UUIDs with "OR" and escape the hyphens for Solr
solr_elements_string: str = " OR ".join(elements).replace("-", r"\-")
solr_query_params = {
"q": f"{facetField}:({solr_elements_string})",
"fq": f"type:0 AND -isBot:true AND statistics_type:view AND bundleName:ORIGINAL AND time:{solr_date_string}",
"fl": facetField,
"facet": "true",
"facet.field": facetField,
"facet.mincount": 1,
"shards": shards,
"rows": 0,
"wt": "json",
"json.nl": "map", # return facets as a dict instead of a flat list
}
solr_url = SOLR_SERVER + "/statistics/select"
res = requests.get(solr_url, params=solr_query_params)
# Create an empty dict to store downloads
data = {}
# Solr returns facets as a dict of dicts (see the json.nl parameter)
downloads = res.json()["facet_counts"]["facet_fields"]
# Iterate over the facetField dict and get the ids and downloads
for id_, downloads in downloads[facetField].items():
# Make sure that each id in the returned dict are present in the
# elements list POSTed by the user.
if id_ in elements:
data[id_] = downloads
# Check if any elements have missing stats so we can set them to 0
if len(data) < len(elements):
# List comprehension to get a list of ids (keys) in the data
data_ids = [k for k, v in data.items()]
for element_id in elements:
if element_id not in data_ids:
data[element_id] = 0
continue
return data
# vim: set sw=4 ts=4 expandtab:

View File

@ -1,4 +1,13 @@
# SPDX-License-Identifier: GPL-3.0-only
import datetime
import json
import re
import falcon
import requests
from .config import SOLR_SERVER
def get_statistics_shards():
@ -8,11 +17,6 @@ def get_statistics_shards():
Returns:
str:A list of Solr statistics shards separated by commas.
"""
import re
import requests
from .config import SOLR_SERVER
# Initialize an empty list for statistics core years
statistics_core_years = []
@ -58,8 +62,6 @@ def get_statistics_shards():
def is_valid_date(date):
import datetime
try:
# Solr date format is: 2020-01-01T00:00:00Z
# See: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
@ -73,12 +75,12 @@ def is_valid_date(date):
)
def validate_items_post_parameters(req, resp, resource, params):
"""Check the POSTed request parameters for the `/items` endpoint.
def validate_post_parameters(req, resp, resource, params):
"""Check the POSTed request parameters for the `/items`, `/communities` and
`/collections` endpoints.
Meant to be used as a `before` hook.
"""
import json
# Only attempt to read the POSTed request if its length is not 0 (or
# rather, in the Python sense, if length is not a False-y value).
@ -103,12 +105,12 @@ def validate_items_post_parameters(req, resp, resource, params):
# Parse the limit parameter from the POST request body
if "limit" in doc:
if isinstance(doc["limit"], int) and 0 < doc["limit"] < 100:
if isinstance(doc["limit"], int) and 0 < doc["limit"] <= 100:
req.context.limit = doc["limit"]
else:
raise falcon.HTTPBadRequest(
title="Invalid parameter",
description='The "limit" parameter is invalid. The value must be an integer between 0 and 100.',
description='The "limit" parameter is invalid. The value must be an integer between 1 and 100.',
)
else:
req.context.limit = 100
@ -125,14 +127,67 @@ def validate_items_post_parameters(req, resp, resource, params):
else:
req.context.page = 0
# Parse the list of items from the POST request body
if "items" in doc:
if isinstance(doc["items"], list) and len(doc["items"]) > 0:
req.context.items = doc["items"]
# Parse the list of elements from the POST request body
if req.context.statistics_scope in doc:
if (
isinstance(doc[req.context.statistics_scope], list)
and len(doc[req.context.statistics_scope]) > 0
):
req.context.elements = doc[req.context.statistics_scope]
else:
raise falcon.HTTPBadRequest(
title="Invalid parameter",
description='The "items" parameter is invalid. The value must be a comma-separated list of item UUIDs.',
description=f'The "{req.context.statistics_scope}" parameter is invalid. The value must be a comma-separated list of UUIDs.',
)
else:
req.context.items = list()
req.context.elements = []
def set_statistics_scope(req, resp, resource, params):
"""Set the statistics scope (item, collection, or community) of the request
as well as the appropriate database (for GET requests) and Solr facet fields
(for POST requests).
Meant to be used as a `before` hook.
"""
# Extract the scope from the request path. This is *guaranteed* to be one
# of the following values because we only send requests matching these few
# patterns to routes using this set_statistics_scope hook.
#
# Note: this regex is ordered so that "items" and "collections" match before
# "item" and "collection".
req.context.statistics_scope = re.findall(
r"^/(communities|community|collections|collection|items|item)", req.path
)[0]
# Set the correct database based on the statistics_scope. The database is
# used for all GET requests where statistics are returned directly from the
# database. In this case we can return early.
if req.method == "GET":
if re.findall(r"^(item|items)$", req.context.statistics_scope):
req.context.database = "items"
elif re.findall(r"^(community|communities)$", req.context.statistics_scope):
req.context.database = "communities"
elif re.findall(r"^(collection|collections)$", req.context.statistics_scope):
req.context.database = "collections"
# GET requests only need the scope and the database so we can return now
return
# If the current request is for a plural items, communities, or collections
# that includes a list of element ids POSTed with the request body then we
# need to set the Solr facet field so we can get the live results.
if req.method == "POST":
if req.context.statistics_scope == "items":
req.context.views_facet_field = "id"
req.context.downloads_facet_field = "owningItem"
elif req.context.statistics_scope == "communities":
req.context.views_facet_field = "owningComm"
req.context.downloads_facet_field = "owningComm"
elif req.context.statistics_scope == "collections":
req.context.views_facet_field = "owningColl"
req.context.downloads_facet_field = "owningColl"
# vim: set sw=4 ts=4 expandtab:

1327
poetry.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,25 +1,29 @@
[tool.poetry]
name = "dspace-statistics-api"
version = "1.3.0"
description = "A simple REST API to expose Solr view and download statistics for items in a DSpace repository."
version = "1.4.4-dev"
description = "A simple REST API to expose Solr view and download statistics for items, communities, and collections in a DSpace repository."
authors = ["Alan Orth <aorth@mjanja.ch>"]
license = "GPL-3.0-only"
[tool.poetry.dependencies]
python = "^3.6"
gunicorn = "^20.0.4"
falcon = "^2.0.0"
psycopg2-binary = "^2.8.6"
python = "^3.8.1"
gunicorn = "^21.0.0"
falcon = "3.1.3"
psycopg2 = "^2.9.1"
requests = "^2.24.0"
falcon-swagger-ui = {git = "https://github.com/alanorth/falcon-swagger-ui.git", rev="falcon3-update-swagger-ui"}
[tool.poetry.dev-dependencies]
ipython = { version = "^7.18.1", python = "^3.7" }
flake8 = "^3.8.4"
pytest = "^6.1.1"
isort = "^5.5.4"
black = "^20.8b1"
pytest-clarity = "^0.3.0-alpha.0"
[tool.poetry.group.dev.dependencies]
black = "^23.0.0"
fixit = "^2.1.0"
flake8 = "^7.0.0"
isort = "^5.9.1"
pytest = "^7.0.0"
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
[tool.isort]
profile = "black"
line_length=88

View File

@ -1,4 +1,4 @@
[pytest]
addopts= -rsxX -s -v --strict
addopts= -rsxX -s -v --strict-markers
filterwarnings =
error::UserWarning

9
renovate.json Normal file
View File

@ -0,0 +1,9 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:base"
],
"pip_requirements": {
"enabled": false
}
}

View File

@ -1,231 +1,33 @@
appdirs==1.4.4 \
--hash=sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128 \
--hash=sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41
appnope==0.1.0; python_version >= "3.7" and python_version < "4.0" and sys_platform == "darwin" \
--hash=sha256:5b26757dc6f79a3b7dc9fab95359328d5747fcb2409d331ea66d0272b90ab2a0 \
--hash=sha256:8b995ffe925347a2138d7ac0fe77155e4311a0ea6d6da4f5128fe4b3cbe5ed71
atomicwrites==1.4.0; sys_platform == "win32" \
--hash=sha256:6d1784dea7c0c8d4a5172b6c620f40b6e4cbfdf96d783691f2e1302a7b88e197 \
--hash=sha256:ae70396ad1a434f9c7046fd2dd196fc04b12f9e91ffb859164193be8b6168a7a
attrs==20.2.0 \
--hash=sha256:fce7fc47dfc976152e82d53ff92fa0407700c21acd20886a13777a0d20e655dc \
--hash=sha256:26b54ddbbb9ee1d34d5d3668dd37d6cf74990ab23c828c2888dccdceee395594
backcall==0.2.0; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:fbbce6a29f263178a1f7915c1940bde0ec2b2a967566fe1c65c1dfb7422bd255 \
--hash=sha256:5cbdbf27be5e7cfadb448baf0aa95508f91f2bbc6c6437cd9cd06e2a4c215e1e
black==20.8b1 \
--hash=sha256:1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea
certifi==2020.6.20 \
--hash=sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41 \
--hash=sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3
chardet==3.0.4 \
--hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
--hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae
click==7.1.2 \
--hash=sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc \
--hash=sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a
colorama==0.4.3; python_version >= "3.7" and python_version < "4.0" and sys_platform == "win32" or sys_platform == "win32" \
--hash=sha256:7d73d2a99753107a36ac6b455ee49046802e59d9d076ef8e47b61499fa29afff \
--hash=sha256:e96da0d330793e2cb9485e9ddfd918d456036c7149416295932478192f4436a1
dataclasses==0.6; python_version < "3.7" \
--hash=sha256:454a69d788c7fda44efd71e259be79577822f5e3f53f029a22d08004e951dc9f \
--hash=sha256:6988bd2b895eef432d562370bb707d540f32f7360ab13da45340101bc2307d84
decorator==4.4.2; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:41fa54c2a0cc4ba648be4fd43cff00aedf5b9465c9bf18d64325bc225f08f760 \
--hash=sha256:e3a62f0520172440ca0dcc823749319382e377f37f140a0b99ef45fecb84bfe7
falcon==2.0.0 \
--hash=sha256:733033ec80c896e30a43ab3e776856096836787197a44eb21022320a61311983 \
--hash=sha256:f93351459f110b4c1ee28556aef9a791832df6f910bea7b3f616109d534df06b \
--hash=sha256:e9efa0791b5d9f9dd9689015ea6bce0a27fcd5ecbcd30e6d940bffa4f7f03389 \
--hash=sha256:59d1e8c993b9a37ea06df9d72cf907a46cc8063b30717cdac2f34d1658b6f936 \
--hash=sha256:a5ebb22a04c9cc65081938ee7651b4e3b4d2a28522ea8ec04c7bdd2b3e9e8cd8 \
--hash=sha256:95bf6ce986c1119aef12c9b348f4dee9c6dcc58391bdd0bc2b0bf353c2b15986 \
--hash=sha256:aa184895d1ad4573fbfaaf803563d02f019ebdf4790e41cc568a330607eae439 \
--hash=sha256:74cf1d18207381c665b9e6292d65100ce146d958707793174b03869dc6e614f4 \
--hash=sha256:24adcd2b29a8ffa9d552dc79638cd21736a3fb04eda7d102c6cebafdaadb88ad \
--hash=sha256:18157af2a4fc3feedf2b5dcc6196f448639acf01c68bc33d4d5a04c3ef87f494 \
--hash=sha256:e3782b7b92fefd46a6ad1fd8fe63fe6c6f1b7740a95ca56957f48d1aee34b357 \
--hash=sha256:9712975adcf8c6e12876239085ad757b8fdeba223d46d23daef82b47658f83a9 \
--hash=sha256:54f2cb4b687035b2a03206dbfc538055cc48b59a953187b0458aa1b574d47b53 \
--hash=sha256:eea593cf466b9c126ce667f6d30503624ef24459f118c75594a69353b6c3d5fc
flake8==3.8.4 \
--hash=sha256:749dbbd6bfd0cf1318af27bf97a14e28e5ff548ef8e5b1566ccfb25a11e7c839 \
--hash=sha256:aadae8761ec651813c24be05c6f7b4680857ef6afaae4651a4eccaef97ce6c3b
gunicorn==20.0.4 \
--hash=sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c \
--hash=sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626
idna==2.10 \
--hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 \
--hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6
importlib-metadata==2.0.0; python_version < "3.8" \
--hash=sha256:cefa1a2f919b866c5beb7c9f7b0ebb4061f30a8a9bf16d609b000e2dfaceb9c3 \
--hash=sha256:77a540690e24b0305878c37ffd421785a6f7e53c8b5720d211b211de8d0e95da
iniconfig==1.0.1 \
--hash=sha256:80cf40c597eb564e86346103f609d74efce0f6b4d4f30ec8ce9e2c26411ba437 \
--hash=sha256:e5f92f89355a67de0595932a6c6c02ab4afddc6fcdc0bfc5becd0d60884d3f69
ipython==7.18.1; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:2e22c1f74477b5106a6fb301c342ab8c64bb75d702e350f05a649e8cb40a0fb8 \
--hash=sha256:a331e78086001931de9424940699691ad49dfb457cea31f5471eae7b78222d5e
ipython-genutils==0.2.0; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:72dd37233799e619666c9f639a9da83c34013a73e8bbc79a7a6348d93c61fab8 \
--hash=sha256:eb2e116e75ecef9d4d228fdc66af54269afa26ab4463042e33785b887c628ba8
isort==5.5.4 \
--hash=sha256:36f0c6659b9000597e92618d05b72d4181104cf59472b1c6a039e3783f930c95 \
--hash=sha256:ba040c24d20aa302f78f4747df549573ae1eaf8e1084269199154da9c483f07f
jedi==0.17.2; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:98cc583fa0f2f8304968199b01b6b4b94f469a1f4a74c1560506ca2a211378b5 \
--hash=sha256:86ed7d9b750603e4ba582ea8edc678657fb4007894a12bcf6f4bb97892f31d20
mccabe==0.6.1 \
--hash=sha256:ab8a6258860da4b6677da4bd2fe5dc2c659cff31b3ee4f7f5d64e79735b80d42 \
--hash=sha256:dd8d182285a0fe56bace7f45b5e7d1a6ebcbf524e8f3bd87eb0f125271b8831f
mypy-extensions==0.4.3 \
--hash=sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d \
--hash=sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8
packaging==20.4 \
--hash=sha256:998416ba6962ae7fbd6596850b80e17859a5753ba17c32284f67bfff33784181 \
--hash=sha256:4357f74f47b9c12db93624a82154e9b120fa8293699949152b22065d556079f8
parso==0.7.1; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:97218d9159b2520ff45eb78028ba8b50d2bc61dcc062a9682666f2dc4bd331ea \
--hash=sha256:caba44724b994a8a5e086460bb212abc5a8bc46951bf4a9a1210745953622eb9
pathspec==0.8.0 \
--hash=sha256:7d91249d21749788d07a2d0f94147accd8f845507400749ea19c1ec9054a12b0 \
--hash=sha256:da45173eb3a6f2a5a487efba21f050af2b41948be6ab52b6a1e3ff22bb8b7061
pexpect==4.8.0; python_version >= "3.7" and python_version < "4.0" and sys_platform != "win32" \
--hash=sha256:0b48a55dcb3c05f3329815901ea4fc1537514d6ba867a152b581d69ae3710937 \
--hash=sha256:fc65a43959d153d0114afe13997d439c22823a27cefceb5ff35c2178c6784c0c
pickleshare==0.7.5; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:9649af414d74d4df115d5d718f82acb59c9d418196b7b4290ed47a12ce62df56 \
--hash=sha256:87683d47965c1da65cdacaf31c8441d12b8044cdec9aca500cd78fc2c683afca
pluggy==0.13.1 \
--hash=sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d \
--hash=sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0
prompt-toolkit==3.0.7; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:83074ee28ad4ba6af190593d4d4c607ff525272a504eb159199b6dd9f950c950 \
--hash=sha256:822f4605f28f7d2ba6b0b09a31e25e140871e96364d1d377667b547bb3bf4489
psycopg2-binary==2.8.6 \
--hash=sha256:11b9c0ebce097180129e422379b824ae21c8f2a6596b159c7659e2e5a00e1aa0 \
--hash=sha256:d14b140a4439d816e3b1229a4a525df917d6ea22a0771a2a78332273fd9528a4 \
--hash=sha256:1fabed9ea2acc4efe4671b92c669a213db744d2af8a9fc5d69a8e9bc14b7a9db \
--hash=sha256:f5ab93a2cb2d8338b1674be43b442a7f544a0971da062a5da774ed40587f18f5 \
--hash=sha256:b4afc542c0ac0db720cf516dd20c0846f71c248d2b3d21013aa0d4ef9c71ca25 \
--hash=sha256:e74a55f6bad0e7d3968399deb50f61f4db1926acf4a6d83beaaa7df986f48b1c \
--hash=sha256:0deac2af1a587ae12836aa07970f5cb91964f05a7c6cdb69d8425ff4c15d4e2c \
--hash=sha256:ad20d2eb875aaa1ea6d0f2916949f5c08a19c74d05b16ce6ebf6d24f2c9f75d1 \
--hash=sha256:950bc22bb56ee6ff142a2cb9ee980b571dd0912b0334aa3fe0fe3788d860bea2 \
--hash=sha256:b8a3715b3c4e604bcc94c90a825cd7f5635417453b253499664f784fc4da0152 \
--hash=sha256:d1b4ab59e02d9008efe10ceabd0b31e79519da6fb67f7d8e8977118832d0f449 \
--hash=sha256:ac0c682111fbf404525dfc0f18a8b5f11be52657d4f96e9fcb75daf4f3984859 \
--hash=sha256:7d92a09b788cbb1aec325af5fcba9fed7203897bbd9269d5691bb1e3bce29550 \
--hash=sha256:aaa4213c862f0ef00022751161df35804127b78adf4a2755b9f991a507e425fd \
--hash=sha256:c2507d796fca339c8fb03216364cca68d87e037c1f774977c8fc377627d01c71 \
--hash=sha256:ee69dad2c7155756ad114c02db06002f4cded41132cc51378e57aad79cc8e4f4 \
--hash=sha256:e82aba2188b9ba309fd8e271702bd0d0fc9148ae3150532bbb474f4590039ffb \
--hash=sha256:d5227b229005a696cc67676e24c214740efd90b148de5733419ac9aaba3773da \
--hash=sha256:a0eb43a07386c3f1f1ebb4dc7aafb13f67188eab896e7397aa1ee95a9c884eb2 \
--hash=sha256:e1f57aa70d3f7cc6947fd88636a481638263ba04a742b4a37dd25c373e41491a \
--hash=sha256:833709a5c66ca52f1d21d41865a637223b368c0ee76ea54ca5bad6f2526c7679 \
--hash=sha256:ba28584e6bca48c59eecbf7efb1576ca214b47f05194646b081717fa628dfddf \
--hash=sha256:6a32f3a4cb2f6e1a0b15215f448e8ce2da192fd4ff35084d80d5e39da683e79b \
--hash=sha256:0e4dc3d5996760104746e6cfcdb519d9d2cd27c738296525d5867ea695774e67 \
--hash=sha256:cec7e622ebc545dbb4564e483dd20e4e404da17ae07e06f3e780b2dacd5cee66 \
--hash=sha256:ba381aec3a5dc29634f20692349d73f2d21f17653bda1decf0b52b11d694541f \
--hash=sha256:a0c50db33c32594305b0ef9abc0cb7db13de7621d2cadf8392a1d9b3c437ef77 \
--hash=sha256:2dac98e85565d5688e8ab7bdea5446674a83a3945a8f416ad0110018d1501b94 \
--hash=sha256:bd1be66dde2b82f80afb9459fc618216753f67109b859a361cf7def5c7968729 \
--hash=sha256:8cd0fb36c7412996859cb4606a35969dd01f4ea34d9812a141cd920c3b18be77 \
--hash=sha256:42ec1035841b389e8cc3692277a0bd81cdfe0b65d575a2c8862cec7a80e62e52 \
--hash=sha256:7312e931b90fe14f925729cde58022f5d034241918a5c4f9797cac62f6b3a9dd
ptyprocess==0.6.0; python_version >= "3.7" and python_version < "4.0" and sys_platform != "win32" \
--hash=sha256:d7cc528d76e76342423ca640335bd3633420dc1366f258cb31d05e865ef5ca1f \
--hash=sha256:923f299cc5ad920c68f2bc0bc98b75b9f838b93b599941a6b63ddbc2476394c0
py==1.9.0 \
--hash=sha256:366389d1db726cd2fcfc79732e75410e5fe4d31db13692115529d34069a043c2 \
--hash=sha256:9ca6883ce56b4e8da7e79ac18787889fa5206c79dcc67fb065376cd2fe03f342
pycodestyle==2.6.0 \
--hash=sha256:2295e7b2f6b5bd100585ebcb1f616591b652db8a741695b3d8f5d28bdc934367 \
--hash=sha256:c58a7d2815e0e8d7972bf1803331fb0152f867bd89adf8a01dfd55085434192e
pyflakes==2.2.0 \
--hash=sha256:0d94e0e05a19e57a99444b6ddcf9a6eb2e5c68d3ca1e98e90707af8152c90a92 \
--hash=sha256:35b2d75ee967ea93b55750aa9edbbf72813e06a66ba54438df2cfac9e3c27fc8
pygments==2.7.1; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:307543fe65c0947b126e83dd5a61bd8acbd84abec11f43caebaf5534cbc17998 \
--hash=sha256:926c3f319eda178d1bd90851e4317e6d8cdb5e292a3386aac9bd75eca29cf9c7
pyparsing==2.4.7 \
--hash=sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b \
--hash=sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1
pytest==6.1.1 \
--hash=sha256:7a8190790c17d79a11f847fba0b004ee9a8122582ebff4729a082c109e81a4c9 \
--hash=sha256:8f593023c1a0f916110285b6efd7f99db07d59546e3d8c36fc60e2ab05d3be92
pytest-clarity==0.3.0a0 \
--hash=sha256:5cc99e3d9b7969dfe17e5f6072d45a917c59d363b679686d3c958a1ded2e4dcf
regex==2020.9.27 \
--hash=sha256:d23a18037313714fb3bb5a94434d3151ee4300bae631894b1ac08111abeaa4a3 \
--hash=sha256:84e9407db1b2eb368b7ecc283121b5e592c9aaedbe8c78b1a2f1102eb2e21d19 \
--hash=sha256:5f18875ac23d9aa2f060838e8b79093e8bb2313dbaaa9f54c6d8e52a5df097be \
--hash=sha256:ae91972f8ac958039920ef6e8769277c084971a142ce2b660691793ae44aae6b \
--hash=sha256:9a02d0ae31d35e1ec12a4ea4d4cca990800f66a917d0fb997b20fbc13f5321fc \
--hash=sha256:ebbe29186a3d9b0c591e71b7393f1ae08c83cb2d8e517d2a822b8f7ec99dfd8b \
--hash=sha256:4707f3695b34335afdfb09be3802c87fa0bc27030471dbc082f815f23688bc63 \
--hash=sha256:9bc13e0d20b97ffb07821aa3e113f9998e84994fe4d159ffa3d3a9d1b805043b \
--hash=sha256:f1b3afc574a3db3b25c89161059d857bd4909a1269b0b3cb3c904677c8c4a3f7 \
--hash=sha256:5533a959a1748a5c042a6da71fe9267a908e21eded7a4f373efd23a2cbdb0ecc \
--hash=sha256:1fe0a41437bbd06063aa184c34804efa886bcc128222e9916310c92cd54c3b4c \
--hash=sha256:c570f6fa14b9c4c8a4924aaad354652366577b4f98213cf76305067144f7b100 \
--hash=sha256:eda4771e0ace7f67f58bc5b560e27fb20f32a148cbc993b0c3835970935c2707 \
--hash=sha256:60b0e9e6dc45683e569ec37c55ac20c582973841927a85f2d8a7d20ee80216ab \
--hash=sha256:088afc8c63e7bd187a3c70a94b9e50ab3f17e1d3f52a32750b5b77dbe99ef5ef \
--hash=sha256:eaf548d117b6737df379fdd53bdde4f08870e66d7ea653e230477f071f861121 \
--hash=sha256:41bb65f54bba392643557e617316d0d899ed5b4946dccee1cb6696152b29844b \
--hash=sha256:8d69cef61fa50c8133382e61fd97439de1ae623fe943578e477e76a9d9471637 \
--hash=sha256:f2388013e68e750eaa16ccbea62d4130180c26abb1d8e5d584b9baf69672b30f \
--hash=sha256:4318d56bccfe7d43e5addb272406ade7a2274da4b70eb15922a071c58ab0108c \
--hash=sha256:a6f32aea4260dfe0e55dc9733ea162ea38f0ea86aa7d0f77b15beac5bf7b369d
requests==2.24.0 \
--hash=sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898 \
--hash=sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b
six==1.15.0 \
--hash=sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced \
--hash=sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259
termcolor==1.1.0 \
--hash=sha256:1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b
toml==0.10.1 \
--hash=sha256:bda89d5935c2eac546d648028b9901107a595863cb36bae0c73ac804a9b4ce88 \
--hash=sha256:926b612be1e5ce0634a2ca03470f95169cf16f939018233a670519cb4ac58b0f
traitlets==5.0.4; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:9664ec0c526e48e7b47b7d14cd6b252efa03e0129011de0a9c1d70315d4309c3 \
--hash=sha256:86c9351f94f95de9db8a04ad8e892da299a088a64fd283f9f6f18770ae5eae1b
typed-ast==1.4.1 \
--hash=sha256:73d785a950fc82dd2a25897d525d003f6378d1cb23ab305578394694202a58c3 \
--hash=sha256:aaee9905aee35ba5905cfb3c62f3e83b3bec7b39413f0a7f19be4e547ea01ebb \
--hash=sha256:0c2c07682d61a629b68433afb159376e24e5b2fd4641d35424e462169c0a7919 \
--hash=sha256:4083861b0aa07990b619bd7ddc365eb7fa4b817e99cf5f8d9cf21a42780f6e01 \
--hash=sha256:269151951236b0f9a6f04015a9004084a5ab0d5f19b57de779f908621e7d8b75 \
--hash=sha256:24995c843eb0ad11a4527b026b4dde3da70e1f2d8806c99b7b4a7cf491612652 \
--hash=sha256:fe460b922ec15dd205595c9b5b99e2f056fd98ae8f9f56b888e7a17dc2b757e7 \
--hash=sha256:4e3e5da80ccbebfff202a67bf900d081906c358ccc3d5e3c8aea42fdfdfd51c1 \
--hash=sha256:249862707802d40f7f29f6e1aad8d84b5aa9e44552d2cc17384b209f091276aa \
--hash=sha256:8ce678dbaf790dbdb3eba24056d5364fb45944f33553dd5869b7580cdbb83614 \
--hash=sha256:c9e348e02e4d2b4a8b2eedb48210430658df6951fa484e59de33ff773fbd4b41 \
--hash=sha256:bcd3b13b56ea479b3650b82cabd6b5343a625b0ced5429e4ccad28a8973f301b \
--hash=sha256:d5d33e9e7af3b34a40dc05f498939f0ebf187f07c385fd58d591c533ad8562fe \
--hash=sha256:0666aa36131496aed8f7be0410ff974562ab7eeac11ef351def9ea6fa28f6355 \
--hash=sha256:d205b1b46085271b4e15f670058ce182bd1199e56b317bf2ec004b6a44f911f6 \
--hash=sha256:6daac9731f172c2a22ade6ed0c00197ee7cc1221aa84cfdf9c31defeb059a907 \
--hash=sha256:498b0f36cc7054c1fead3d7fc59d2150f4d5c6c56ba7fb150c013fbc683a8d2d \
--hash=sha256:715ff2f2df46121071622063fc7543d9b1fd19ebfc4f5c8895af64a77a8c852c \
--hash=sha256:fc0fea399acb12edbf8a628ba8d2312f583bdbdb3335635db062fa98cf71fca4 \
--hash=sha256:d43943ef777f9a1c42bf4e552ba23ac77a6351de620aa9acf64ad54933ad4d34 \
--hash=sha256:8c8aaad94455178e3187ab22c8b01a3837f8ee50e09cf31f1ba129eb293ec30b
typing-extensions==3.7.4.3 \
--hash=sha256:dafc7639cde7f1b6e1acc0f457842a83e722ccca8eef5270af2d74792619a89f \
--hash=sha256:7cb407020f00f7bfc3cb3e7881628838e69d8f3fcab2f64742a5e76b2f841918 \
--hash=sha256:99d4073b617d30288f569d3f13d2bd7548c3a7e4c8de87db09a9d29bb3a4a60c
urllib3==1.25.10 \
--hash=sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461 \
--hash=sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a
wcwidth==0.2.5; python_version >= "3.7" and python_version < "4.0" \
--hash=sha256:beb4802a9cebb9144e99086eff703a642a13d6a0052920003a230f3294bbe784 \
--hash=sha256:c4d647b99872929fdb7bdcaa4fbe7f01413ed3d98077df798530e5b04f116c83
zipp==3.3.0; python_version < "3.8" \
--hash=sha256:eed8ec0b8d1416b2ca33516a37a08892442f3954dee131e92cfd92d8fe3e7066 \
--hash=sha256:64ad89efee774d1897a58607895d80789c59778ea02185dd846ac38394a8642b
atomicwrites==1.4.1 ; python_version >= "3.8" and python_version < "4.0" and sys_platform == "win32"
attrs==23.1.0 ; python_version >= "3.8" and python_version < "4.0"
black==22.12.0 ; python_version >= "3.8" and python_version < "4.0"
certifi==2023.5.7 ; python_version >= "3.8" and python_version < "4.0"
charset-normalizer==3.1.0 ; python_version >= "3.8" and python_version < "4.0"
click==8.1.3 ; python_version >= "3.8" and python_version < "4.0"
colorama==0.4.6 ; python_version >= "3.8" and python_version < "4.0" and (sys_platform == "win32" or platform_system == "Windows")
falcon-swagger-ui @ git+https://github.com/alanorth/falcon-swagger-ui.git@falcon3-update-swagger-ui ; python_version >= "3.8" and python_version < "4.0"
falcon==3.1.1 ; python_version >= "3.8" and python_version < "4.0"
flake8==4.0.1 ; python_version >= "3.8" and python_version < "4.0"
gunicorn==20.1.0 ; python_version >= "3.8" and python_version < "4.0"
idna==3.4 ; python_version >= "3.8" and python_version < "4.0"
iniconfig==2.0.0 ; python_version >= "3.8" and python_version < "4.0"
isort==5.12.0 ; python_version >= "3.8" and python_version < "4.0"
jinja2==3.1.2 ; python_version >= "3.8" and python_version < "4.0"
markupsafe==2.1.2 ; python_version >= "3.8" and python_version < "4.0"
mccabe==0.6.1 ; python_version >= "3.8" and python_version < "4.0"
mypy-extensions==1.0.0 ; python_version >= "3.8" and python_version < "4.0"
packaging==23.1 ; python_version >= "3.8" and python_version < "4.0"
pathspec==0.11.1 ; python_version >= "3.8" and python_version < "4.0"
platformdirs==3.5.1 ; python_version >= "3.8" and python_version < "4.0"
pluggy==1.0.0 ; python_version >= "3.8" and python_version < "4.0"
psycopg2==2.9.6 ; python_version >= "3.8" and python_version < "4.0"
py==1.11.0 ; python_version >= "3.8" and python_version < "4.0"
pycodestyle==2.8.0 ; python_version >= "3.8" and python_version < "4.0"
pyflakes==2.4.0 ; python_version >= "3.8" and python_version < "4.0"
pytest==6.2.5 ; python_version >= "3.8" and python_version < "4.0"
requests==2.31.0 ; python_version >= "3.8" and python_version < "4.0"
setuptools==67.8.0 ; python_version >= "3.8" and python_version < "4.0"
toml==0.10.2 ; python_version >= "3.8" and python_version < "4.0"
tomli==2.0.1 ; python_version >= "3.8" and python_full_version < "3.11.0a7"
typing-extensions==4.6.2 ; python_version >= "3.8" and python_version < "3.10"
urllib3==2.0.2 ; python_version >= "3.8" and python_version < "4.0"

View File

@ -1,66 +1,12 @@
certifi==2020.6.20 \
--hash=sha256:8fc0819f1f30ba15bdb34cceffb9ef04d99f420f68eb75d901e9560b8749fc41 \
--hash=sha256:5930595817496dd21bb8dc35dad090f1c2cd0adfaf21204bf6732ca5d8ee34d3
chardet==3.0.4 \
--hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
--hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae
falcon==2.0.0 \
--hash=sha256:733033ec80c896e30a43ab3e776856096836787197a44eb21022320a61311983 \
--hash=sha256:f93351459f110b4c1ee28556aef9a791832df6f910bea7b3f616109d534df06b \
--hash=sha256:e9efa0791b5d9f9dd9689015ea6bce0a27fcd5ecbcd30e6d940bffa4f7f03389 \
--hash=sha256:59d1e8c993b9a37ea06df9d72cf907a46cc8063b30717cdac2f34d1658b6f936 \
--hash=sha256:a5ebb22a04c9cc65081938ee7651b4e3b4d2a28522ea8ec04c7bdd2b3e9e8cd8 \
--hash=sha256:95bf6ce986c1119aef12c9b348f4dee9c6dcc58391bdd0bc2b0bf353c2b15986 \
--hash=sha256:aa184895d1ad4573fbfaaf803563d02f019ebdf4790e41cc568a330607eae439 \
--hash=sha256:74cf1d18207381c665b9e6292d65100ce146d958707793174b03869dc6e614f4 \
--hash=sha256:24adcd2b29a8ffa9d552dc79638cd21736a3fb04eda7d102c6cebafdaadb88ad \
--hash=sha256:18157af2a4fc3feedf2b5dcc6196f448639acf01c68bc33d4d5a04c3ef87f494 \
--hash=sha256:e3782b7b92fefd46a6ad1fd8fe63fe6c6f1b7740a95ca56957f48d1aee34b357 \
--hash=sha256:9712975adcf8c6e12876239085ad757b8fdeba223d46d23daef82b47658f83a9 \
--hash=sha256:54f2cb4b687035b2a03206dbfc538055cc48b59a953187b0458aa1b574d47b53 \
--hash=sha256:eea593cf466b9c126ce667f6d30503624ef24459f118c75594a69353b6c3d5fc
gunicorn==20.0.4 \
--hash=sha256:cd4a810dd51bf497552cf3f863b575dabd73d6ad6a91075b65936b151cbf4f9c \
--hash=sha256:1904bb2b8a43658807108d59c3f3d56c2b6121a701161de0ddf9ad140073c626
idna==2.10 \
--hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 \
--hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6
psycopg2-binary==2.8.6 \
--hash=sha256:11b9c0ebce097180129e422379b824ae21c8f2a6596b159c7659e2e5a00e1aa0 \
--hash=sha256:d14b140a4439d816e3b1229a4a525df917d6ea22a0771a2a78332273fd9528a4 \
--hash=sha256:1fabed9ea2acc4efe4671b92c669a213db744d2af8a9fc5d69a8e9bc14b7a9db \
--hash=sha256:f5ab93a2cb2d8338b1674be43b442a7f544a0971da062a5da774ed40587f18f5 \
--hash=sha256:b4afc542c0ac0db720cf516dd20c0846f71c248d2b3d21013aa0d4ef9c71ca25 \
--hash=sha256:e74a55f6bad0e7d3968399deb50f61f4db1926acf4a6d83beaaa7df986f48b1c \
--hash=sha256:0deac2af1a587ae12836aa07970f5cb91964f05a7c6cdb69d8425ff4c15d4e2c \
--hash=sha256:ad20d2eb875aaa1ea6d0f2916949f5c08a19c74d05b16ce6ebf6d24f2c9f75d1 \
--hash=sha256:950bc22bb56ee6ff142a2cb9ee980b571dd0912b0334aa3fe0fe3788d860bea2 \
--hash=sha256:b8a3715b3c4e604bcc94c90a825cd7f5635417453b253499664f784fc4da0152 \
--hash=sha256:d1b4ab59e02d9008efe10ceabd0b31e79519da6fb67f7d8e8977118832d0f449 \
--hash=sha256:ac0c682111fbf404525dfc0f18a8b5f11be52657d4f96e9fcb75daf4f3984859 \
--hash=sha256:7d92a09b788cbb1aec325af5fcba9fed7203897bbd9269d5691bb1e3bce29550 \
--hash=sha256:aaa4213c862f0ef00022751161df35804127b78adf4a2755b9f991a507e425fd \
--hash=sha256:c2507d796fca339c8fb03216364cca68d87e037c1f774977c8fc377627d01c71 \
--hash=sha256:ee69dad2c7155756ad114c02db06002f4cded41132cc51378e57aad79cc8e4f4 \
--hash=sha256:e82aba2188b9ba309fd8e271702bd0d0fc9148ae3150532bbb474f4590039ffb \
--hash=sha256:d5227b229005a696cc67676e24c214740efd90b148de5733419ac9aaba3773da \
--hash=sha256:a0eb43a07386c3f1f1ebb4dc7aafb13f67188eab896e7397aa1ee95a9c884eb2 \
--hash=sha256:e1f57aa70d3f7cc6947fd88636a481638263ba04a742b4a37dd25c373e41491a \
--hash=sha256:833709a5c66ca52f1d21d41865a637223b368c0ee76ea54ca5bad6f2526c7679 \
--hash=sha256:ba28584e6bca48c59eecbf7efb1576ca214b47f05194646b081717fa628dfddf \
--hash=sha256:6a32f3a4cb2f6e1a0b15215f448e8ce2da192fd4ff35084d80d5e39da683e79b \
--hash=sha256:0e4dc3d5996760104746e6cfcdb519d9d2cd27c738296525d5867ea695774e67 \
--hash=sha256:cec7e622ebc545dbb4564e483dd20e4e404da17ae07e06f3e780b2dacd5cee66 \
--hash=sha256:ba381aec3a5dc29634f20692349d73f2d21f17653bda1decf0b52b11d694541f \
--hash=sha256:a0c50db33c32594305b0ef9abc0cb7db13de7621d2cadf8392a1d9b3c437ef77 \
--hash=sha256:2dac98e85565d5688e8ab7bdea5446674a83a3945a8f416ad0110018d1501b94 \
--hash=sha256:bd1be66dde2b82f80afb9459fc618216753f67109b859a361cf7def5c7968729 \
--hash=sha256:8cd0fb36c7412996859cb4606a35969dd01f4ea34d9812a141cd920c3b18be77 \
--hash=sha256:42ec1035841b389e8cc3692277a0bd81cdfe0b65d575a2c8862cec7a80e62e52 \
--hash=sha256:7312e931b90fe14f925729cde58022f5d034241918a5c4f9797cac62f6b3a9dd
requests==2.24.0 \
--hash=sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898 \
--hash=sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b
urllib3==1.25.10 \
--hash=sha256:e7983572181f5e1522d9c98453462384ee92a0be7fac5f1413a1e35c56cc0461 \
--hash=sha256:91056c15fa70756691db97756772bb1eb9678fa585d9184f24534b100dc60f4a
certifi==2023.5.7 ; python_version >= "3.8" and python_version < "4.0"
charset-normalizer==3.1.0 ; python_version >= "3.8" and python_version < "4.0"
falcon-swagger-ui @ git+https://github.com/alanorth/falcon-swagger-ui.git@falcon3-update-swagger-ui ; python_version >= "3.8" and python_version < "4.0"
falcon==3.1.1 ; python_version >= "3.8" and python_version < "4.0"
gunicorn==20.1.0 ; python_version >= "3.8" and python_version < "4.0"
idna==3.4 ; python_version >= "3.8" and python_version < "4.0"
jinja2==3.1.2 ; python_version >= "3.8" and python_version < "4.0"
markupsafe==2.1.2 ; python_version >= "3.8" and python_version < "4.0"
psycopg2==2.9.6 ; python_version >= "3.8" and python_version < "4.0"
requests==2.31.0 ; python_version >= "3.8" and python_version < "4.0"
setuptools==67.8.0 ; python_version >= "3.8" and python_version < "4.0"
urllib3==2.0.2 ; python_version >= "3.8" and python_version < "4.0"

View File

@ -1,6 +0,0 @@
[isort]
multi_line_output=3
include_trailing_comma=True
force_grid_wrap=0
use_parentheses=True
line_length=88

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,379 @@
# SPDX-License-Identifier: GPL-3.0-only
import json
from unittest.mock import patch
import pytest
from falcon import testing
from dspace_statistics_api.app import app
@pytest.fixture
def client():
return testing.TestClient(app)
def test_get_collection(client):
"""Test requesting a single collection."""
response = client.simulate_get("/collection/8ea4b611-1f59-4d4e-b78d-a9921a72cfe7")
response_doc = json.loads(response.text)
assert isinstance(response_doc["downloads"], int)
assert isinstance(response_doc["id"], str)
assert isinstance(response_doc["views"], int)
assert response.status_code == 200
def test_get_missing_collection(client):
"""Test requesting a single non-existing collection."""
response = client.simulate_get("/collection/508abe0a-689f-402e-885d-2f6b02e7a39c")
assert response.status_code == 404
def test_get_collections(client):
"""Test requesting 100 collections."""
response = client.simulate_get("/collections", query_string="limit=100")
response_doc = json.loads(response.text)
assert isinstance(response_doc["currentPage"], int)
assert isinstance(response_doc["totalPages"], int)
assert isinstance(response_doc["statistics"], list)
assert response.status_code == 200
def test_get_collections_invalid_limit(client):
"""Test requesting 100 collections with an invalid limit parameter."""
response = client.simulate_get("/collections", query_string="limit=101")
assert response.status_code == 400
def test_get_collections_invalid_page(client):
"""Test requesting 100 collections with an invalid page parameter."""
response = client.simulate_get("/collections", query_string="page=-1")
assert response.status_code == 400
@pytest.mark.xfail
def test_post_collections_valid_dateFrom(client):
"""Test POSTing a request to /collections with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_valid_dateFrom_mocked(client):
"""Mock test POSTing a request to /collections with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
get_views_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 21,
"260548c8-fda4-4dc8-a979-03495753cdd5": 0,
}
get_downloads_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 575,
"260548c8-fda4-4dc8-a979-03495753cdd5": 899,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_invalid_dateFrom(client):
"""Test POSTing a request to /collections with an invalid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_collections_valid_dateTo(client):
"""Test POSTing a request to /collections with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_valid_dateTo_mocked(client):
"""Mock test POSTing a request to /collections with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
get_views_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 21,
"260548c8-fda4-4dc8-a979-03495753cdd5": 0,
}
get_downloads_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 575,
"260548c8-fda4-4dc8-a979-03495753cdd5": 899,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_invalid_dateTo(client):
"""Test POSTing a request to /collections with an invalid dateTo parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_collections_valid_limit(client):
"""Test POSTing a request to /collections with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 1
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 1
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
def test_post_collections_valid_limit_mocked(client):
"""Mock test POSTing a request to /collections with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
get_views_return_value = {"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 21}
get_downloads_return_value = {"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 575}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 1
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 1
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
def test_post_collections_invalid_limit(client):
"""Test POSTing a request to /collections with an invalid limit parameter in the request body."""
request_body = {
"limit": -1,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_collections_valid_page(client):
"""Test POSTing a request to /collections with a valid page parameter in the request body."""
request_body = {
"page": 0,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert response.json["totalPages"] == 1
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_valid_page_mocked(client):
"""Mock test POSTing a request to /collections with a valid page parameter in the request body."""
request_body = {
"page": 0,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
get_views_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 21,
"260548c8-fda4-4dc8-a979-03495753cdd5": 0,
}
get_downloads_return_value = {
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7": 575,
"260548c8-fda4-4dc8-a979-03495753cdd5": 899,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_collections_invalid_page(client):
"""Test POSTing a request to /collections with an invalid page parameter in the request body."""
request_body = {
"page": -1,
"collections": [
"8ea4b611-1f59-4d4e-b78d-a9921a72cfe7",
"260548c8-fda4-4dc8-a979-03495753cdd5",
],
}
response = client.simulate_post("/collections", json=request_body)
assert response.status_code == 400
# vim: set sw=4 ts=4 expandtab:

View File

@ -0,0 +1,379 @@
# SPDX-License-Identifier: GPL-3.0-only
import json
from unittest.mock import patch
import pytest
from falcon import testing
from dspace_statistics_api.app import app
@pytest.fixture
def client():
return testing.TestClient(app)
def test_get_community(client):
"""Test requesting a single community."""
response = client.simulate_get("/community/bde7139c-d321-46bb-aef6-ae70799e5edb")
response_doc = json.loads(response.text)
assert isinstance(response_doc["downloads"], int)
assert isinstance(response_doc["id"], str)
assert isinstance(response_doc["views"], int)
assert response.status_code == 200
def test_get_missing_community(client):
"""Test requesting a single non-existing community."""
response = client.simulate_get("/item/dec6bfc6-efeb-4f74-8436-79fa80bb5c21")
assert response.status_code == 404
def test_get_communities(client):
"""Test requesting 100 communities."""
response = client.simulate_get("/communities", query_string="limit=100")
response_doc = json.loads(response.text)
assert isinstance(response_doc["currentPage"], int)
assert isinstance(response_doc["totalPages"], int)
assert isinstance(response_doc["statistics"], list)
assert response.status_code == 200
def test_get_communities_invalid_limit(client):
"""Test requesting 100 communities with an invalid limit parameter."""
response = client.simulate_get("/communities", query_string="limit=101")
assert response.status_code == 400
def test_get_communities_invalid_page(client):
"""Test requesting 100 communities with an invalid page parameter."""
response = client.simulate_get("/communities", query_string="page=-1")
assert response.status_code == 400
@pytest.mark.xfail
def test_post_communities_valid_dateFrom(client):
"""Test POSTing a request to /communities with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_valid_dateFrom_mocked(client):
"""Mock test POSTing a request to /communities with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
get_views_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 309,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 0,
}
get_downloads_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 400,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 290,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_invalid_dateFrom(client):
"""Test POSTing a request to /communities with an invalid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_communities_valid_dateTo(client):
"""Test POSTing a request to /communities with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_valid_dateTo_mocked(client):
"""Mock test POSTing a request to /communities with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
get_views_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 21,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 0,
}
get_downloads_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 575,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 899,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_invalid_dateTo(client):
"""Test POSTing a request to /communities with an invalid dateTo parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_communities_valid_limit(client):
"""Test POSTing a request to /communities with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 1
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 1
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
def test_post_communities_valid_limit_mocked(client):
"""Mock test POSTing a request to /communities with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
get_views_return_value = {"bde7139c-d321-46bb-aef6-ae70799e5edb": 200}
get_downloads_return_value = {"bde7139c-d321-46bb-aef6-ae70799e5edb": 309}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 1
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 1
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
def test_post_communities_invalid_limit(client):
"""Test POSTing a request to /communities with an invalid limit parameter in the request body."""
request_body = {
"limit": -1,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 400
@pytest.mark.xfail
def test_post_communities_valid_page(client):
"""Test POSTing a request to /communities with a valid page parameter in the request body."""
request_body = {
"page": 0,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert response.json["totalPages"] == 1
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_valid_page_mocked(client):
"""Mock test POSTing a request to communities with a valid page parameter in the request body."""
request_body = {
"page": 0,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
get_views_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 21,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 0,
}
get_downloads_return_value = {
"bde7139c-d321-46bb-aef6-ae70799e5edb": 575,
"2a920a61-b08a-4642-8e5d-2639c6702b1f": 899,
}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
):
with patch(
"dspace_statistics_api.app.get_downloads",
return_value=get_downloads_return_value,
):
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert isinstance(response.json["totalPages"], int)
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
assert isinstance(response.json["statistics"][1]["views"], int)
assert isinstance(response.json["statistics"][1]["downloads"], int)
def test_post_communities_invalid_page(client):
"""Test POSTing a request to /communities with an invalid page parameter in the request body."""
request_body = {
"page": -1,
"communities": [
"bde7139c-d321-46bb-aef6-ae70799e5edb",
"2a920a61-b08a-4642-8e5d-2639c6702b1f",
],
}
response = client.simulate_post("/communities", json=request_body)
assert response.status_code == 400
# vim: set sw=4 ts=4 expandtab:

50
tests/test_api_docs.py Normal file
View File

@ -0,0 +1,50 @@
# SPDX-License-Identifier: GPL-3.0-only
import pytest
from falcon import testing
from dspace_statistics_api.app import app
@pytest.fixture
def client():
return testing.TestClient(app)
def test_get_docs(client):
"""Test requesting the documentation at the root."""
response = client.simulate_get("/")
assert isinstance(response.content, bytes)
assert response.status_code == 200
def test_get_openapi_json(client):
"""Test requesting the OpenAPI JSON schema."""
response = client.simulate_get("/docs/openapi.json")
assert isinstance(response.content, bytes)
assert response.status_code == 200
def test_get_swagger_ui(client):
"""Test requesting the Swagger UI."""
response = client.simulate_get("/swagger")
assert isinstance(response.content, bytes)
assert response.status_code == 200
def test_get_status(client):
"""Test requesting the status page."""
response = client.simulate_get("/status")
assert isinstance(response.content, bytes)
assert response.status_code == 200
# vim: set sw=4 ts=4 expandtab:

View File

@ -1,29 +1,23 @@
from falcon import testing
# SPDX-License-Identifier: GPL-3.0-only
import json
import pytest
from unittest.mock import patch
from dspace_statistics_api.app import api
import pytest
from falcon import testing
from dspace_statistics_api.app import app
@pytest.fixture
def client():
return testing.TestClient(api)
def test_get_docs(client):
"""Test requesting the documentation at the root."""
response = client.simulate_get("/")
assert isinstance(response.content, bytes)
assert response.status_code == 200
return testing.TestClient(app)
def test_get_item(client):
"""Test requesting a single item."""
response = client.simulate_get("/item/c3910974-c3a5-4053-9dce-104aa7bb1621")
response = client.simulate_get("/item/fd8a46d5-1480-4e69-b187-cd3db96d8e4d")
response_doc = json.loads(response.text)
assert isinstance(response_doc["downloads"], int)
@ -70,13 +64,13 @@ def test_get_items_invalid_page(client):
@pytest.mark.xfail
def test_post_items_valid_dateFrom(client):
"""Test POSTing a request with a valid dateFrom parameter in the request body."""
"""Test POSTing a request to /items with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -94,23 +88,23 @@ def test_post_items_valid_dateFrom(client):
def test_post_items_valid_dateFrom_mocked(client):
"""Mock test POSTing a request with a valid dateFrom parameter in the request body."""
"""Mock test POSTing a request to /items with a valid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00Z",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
get_views_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 21,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 0,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 21,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 0,
}
get_downloads_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 575,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 899,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 575,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 899,
}
with patch(
@ -134,13 +128,13 @@ def test_post_items_valid_dateFrom_mocked(client):
def test_post_items_invalid_dateFrom(client):
"""Test POSTing a request with an invalid dateFrom parameter in the request body."""
"""Test POSTing a request to /items with an invalid dateFrom parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -151,13 +145,13 @@ def test_post_items_invalid_dateFrom(client):
@pytest.mark.xfail
def test_post_items_valid_dateTo(client):
"""Test POSTing a request with a valid dateTo parameter in the request body."""
"""Test POSTing a request to /items with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -175,23 +169,23 @@ def test_post_items_valid_dateTo(client):
def test_post_items_valid_dateTo_mocked(client):
"""Mock test POSTing a request with a valid dateTo parameter in the request body."""
"""Mock test POSTing a request to /items with a valid dateTo parameter in the request body."""
request_body = {
"dateTo": "2020-01-01T00:00:00Z",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
get_views_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 21,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 0,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 21,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 0,
}
get_downloads_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 575,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 899,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 575,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 899,
}
with patch(
@ -215,13 +209,13 @@ def test_post_items_valid_dateTo_mocked(client):
def test_post_items_invalid_dateTo(client):
"""Test POSTing a request with an invalid dateTo parameter in the request body."""
"""Test POSTing a request to /items with an invalid dateTo parameter in the request body."""
request_body = {
"dateFrom": "2020-01-01T00:00:00",
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -232,13 +226,13 @@ def test_post_items_invalid_dateTo(client):
@pytest.mark.xfail
def test_post_items_valid_limit(client):
"""Test POSTing a request with a valid limit parameter in the request body."""
"""Test POSTing a request to /items with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -254,18 +248,18 @@ def test_post_items_valid_limit(client):
def test_post_items_valid_limit_mocked(client):
"""Mock test POSTing a request with a valid limit parameter in the request body."""
"""Mock test POSTing a request to /items with a valid limit parameter in the request body."""
request_body = {
"limit": 1,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
get_views_return_value = {"c3910974-c3a5-4053-9dce-104aa7bb1620": 21}
get_downloads_return_value = {"c3910974-c3a5-4053-9dce-104aa7bb1620": 575}
get_views_return_value = {"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 21}
get_downloads_return_value = {"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 575}
with patch(
"dspace_statistics_api.app.get_views", return_value=get_views_return_value
@ -286,13 +280,13 @@ def test_post_items_valid_limit_mocked(client):
def test_post_items_invalid_limit(client):
"""Test POSTing a request with an invalid limit parameter in the request body."""
"""Test POSTing a request to /items with an invalid limit parameter in the request body."""
request_body = {
"limit": -1,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -303,13 +297,13 @@ def test_post_items_invalid_limit(client):
@pytest.mark.xfail
def test_post_items_valid_page(client):
"""Test POSTing a request with a valid page parameter in the request body."""
"""Test POSTing a request to /items with a valid page parameter in the request body."""
request_body = {
"page": 0,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
@ -318,7 +312,7 @@ def test_post_items_valid_page(client):
assert response.status_code == 200
assert response.json["limit"] == 100
assert response.json["currentPage"] == 0
assert response.json["totalPages"] == 0
assert response.json["totalPages"] == 1
assert len(response.json["statistics"]) == 2
assert isinstance(response.json["statistics"][0]["views"], int)
assert isinstance(response.json["statistics"][0]["downloads"], int)
@ -327,23 +321,23 @@ def test_post_items_valid_page(client):
def test_post_items_valid_page_mocked(client):
"""Mock test POSTing a request with a valid page parameter in the request body."""
"""Mock test POSTing a request to /items with a valid page parameter in the request body."""
request_body = {
"page": 0,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
get_views_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 21,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 0,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 21,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 0,
}
get_downloads_return_value = {
"c3910974-c3a5-4053-9dce-104aa7bb1620": 575,
"887cc5f8-b5e7-4a2f-9053-49c91ab81313": 899,
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d": 575,
"e53a2eab-1e31-448d-907b-3656ca4e86c1": 899,
}
with patch(
@ -367,16 +361,19 @@ def test_post_items_valid_page_mocked(client):
def test_post_items_invalid_page(client):
"""Test POSTing a request with an invalid page parameter in the request body."""
"""Test POSTing a request to /items with an invalid page parameter in the request body."""
request_body = {
"page": -1,
"items": [
"c3910974-c3a5-4053-9dce-104aa7bb1620",
"887cc5f8-b5e7-4a2f-9053-49c91ab81313",
"fd8a46d5-1480-4e69-b187-cd3db96d8e4d",
"e53a2eab-1e31-448d-907b-3656ca4e86c1",
],
}
response = client.simulate_post("/items", json=request_body)
assert response.status_code == 400
# vim: set sw=4 ts=4 expandtab: