1
0
mirror of https://github.com/ilri/dspace-statistics-api.git synced 2024-11-14 18:37:03 +01:00
Commit Graph

527 Commits

Author SHA1 Message Date
2407aeec70
dspace_statistics_api/indexer.py: Use fl parameter
When indexing item views and downloads the only field we need is the
the id. The `fl` parameter tells Solr which fields to return in the
search results. This should theoretically be more efficient, though
I don't have any time to figure out how to measure it right now.
2020-12-17 12:25:28 +02:00
f3a0e3a671
CHANGELOG.md: Add note about ORDER BY
All checks were successful
continuous-integration/drone/push Build is passing
2020-12-17 10:17:23 +02:00
4590fc8708
dspace_statistics_api/app.py: Use ORDER BY in /items
Since we are paging through the results by limit/offset we need to
be sure that we are returning results deterministically.
2020-12-17 10:10:40 +02:00
8b924cf450
Remove TravisCI config
All checks were successful
continuous-integration/drone/push Build is passing
I will use other CIs since TravisCI changed their business model.
2020-12-15 09:38:51 +02:00
ea24c73a6a
.drone.yml: Install gcc for Python 3.9
It appears to be needed to compile typed-ast:

    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iast27/Include -I/usr/local/include/python3.9 -c ast27/Custom/typed_ast.c -o build/temp.linux-x86_64-3.9/ast27/Custom/typed_ast.o
    error: command 'gcc' failed: No such file or directory
    ----------------------------------------
    ERROR: Failed building wheel for typed-ast
2020-12-14 22:50:21 +02:00
cd98d33615
.drone.yml: Only install requirements-dev.txt
It seems that Poetry's --dev export includes both dev and non-dev
libraries so we don't need to install both.
2020-12-14 22:42:00 +02:00
9d112266ca
Update requirements-dev.txt
Generated with poetry export:

    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-14 22:05:18 +02:00
2b067050ff
Remove pytest-clarity
It is missing a six dependency which causes the build to fail. I
could simply add six to the virtualenv but it feels dirty. I don't
actually *need* pytest-clarity for anything so I'll just remove it.

See: https://github.com/darrenburns/pytest-clarity/issues/14
2020-12-14 22:05:03 +02:00
dc683f2d1c
pytest.ini: Change --strict to --strict-markers
This is deprecated since pytest 6.2.0.

See: https://docs.pytest.org/en/stable/deprecations.html#the-strict-command-line-option
2020-12-14 19:07:02 +02:00
f60f529bd7
Update requirements
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-12-14 15:41:40 +02:00
7db8458201
poetry.lock: Run poetry update
[SKIP CI]
2020-12-14 15:40:06 +02:00
707f878b94 Add .drone.yml
Uses multiple pipelines to test several versions of Python. A few
things to note:

- I use the -slim Python packages, which are smaller and yet still
have no problem installing psycopg2-binary with pip
- I have to start a PostgreSQL database service for each pipeline
separately
2020-12-14 15:38:50 +02:00
930250352a
Update docs about POST /items 2020-12-13 20:09:20 +02:00
e27f30ba4d
README.md: Use travis-ci.com domain for badge link 2020-12-08 09:11:38 +02:00
28d1917038
README.md: Use travis-ci.com domain for badge 2020-12-08 09:09:19 +02:00
fc6a9c2ad1
tests: Update for real data
Now that CGSpace is running DSpace 6 I will use some real UUIDs to
make things easier in the future.
2020-11-25 14:56:47 +02:00
3125e96a16
Bump version to 1.3.2 2020-11-18 22:01:18 +02:00
66143ff00f
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-11-18 21:59:33 +02:00
2d15f12be9
poetry.lock: Run poetry update 2020-11-18 21:58:32 +02:00
9218039e61
CHANGELOG.md: Add note about limit param 2020-11-18 21:57:12 +02:00
88a8db6c78
Make sure limit is between 1 and 100
We were not properly checking whether the limit was actually less
than or equal to 100.
2020-11-18 21:55:54 +02:00
3995eba0a7
CHANGELOG.md: Add note about Solr bot filter 2020-11-17 17:42:30 +02:00
810508d038
dspace_statistics_api/indexer.py: Use -isBot:true
Minor change to bot filtering. We should use a negated match for
documents that have `isBot:true` rather than looking for documents
that are tagged with `isBot:false` (the distinction is subtle, but
important).
2020-11-17 17:40:08 +02:00
ecafab57cb
README.md: Update DSpace version note 2020-11-16 16:16:21 +02:00
9c9431b58c
CHANGELOG.md: Add unreleased changes 2020-11-02 22:14:18 +02:00
2d6520fc97
Fix limit in docs 2020-11-02 22:14:08 +02:00
79a393d33f
Update requirements
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

The `--without-hashes` is required to work around an issue with
gunicorn pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-11-02 22:10:29 +02:00
149f6c418f
poetry.lock: Run poetry update 2020-11-02 22:00:29 +02:00
ca1582a8b6
Make sure limit is between 1 and 100
We were not properly checking whether the limit was greater than 0
in all cases.
2020-11-02 21:59:20 +02:00
1904c243a4
Revert ".travis.yml: Use Ubuntu 20.04 "Focal" environment"
This reverts commit 0baa07f70a.

Focal only has PostgreSQL 12 installed, and we are not quite there
yet (our production has 9.6, testing has 10).
2020-10-29 00:11:28 +03:00
0baa07f70a
.travis.yml: Use Ubuntu 20.04 "Focal" environment 2020-10-29 00:04:47 +03:00
59214ffcb6
.travis.yml: Bump Python versions
Test Python 3.9 now that it was released, and allow tests to fail
on nightly builds.
2020-10-29 00:03:58 +03:00
549b8bf1a7
dspace_statistics_api/docs/index.html: Fix version
We need to print it in the body, not the title.
2020-10-06 22:22:11 +03:00
899a79b2e7
Version 1.3.1 2020-10-06 22:15:52 +03:00
4c59469055
Update requirements
Generated with poetry export:

    $ poetry export --no-hashes -f requirements.txt > requirements.txt
    $ poetry export --no-hashes --dev -f requirements.txt > requirements-dev.txt

The `--no-hashes` is required to work around an issue with gunicorn
pulling in a dependency on setuptools that poetry ignores.

See: https://github.com/python-poetry/poetry/issues/1584
2020-10-06 22:11:54 +03:00
4e9064329d
Bump version to 1.3.0 2020-10-06 21:33:38 +03:00
4958d5d2e9
pyproject.toml: Fix email 2020-10-06 18:46:31 +03:00
923ed0a434
tests/test_api.py: Add tests for /items POST handlers
This adds tests for the new /items POST handler, both with mocked
data and a live connection to a Solr statistics core. Tests that
only work when Solr is available are marked with XFAIL so that they
don't turn the whole test suite red.

In each test I try to assert as many parameters as we can know for
each response so that we cover all expectations. For example, when
we test a valid limit parameter we should test whether the response
not only has the same limit parameter, but that the number of items
has actually been limited and the number of pages has been adjusted
accordingly.

See: https://docs.pytest.org/en/stable/skipping.html
2020-10-06 16:51:53 +03:00
5acd927210
dspace_statistics_api: Sort imports with isort 2020-10-06 15:12:13 +03:00
630fa0d5fb
dspace_statistics_api/util.py: Fix f-strings
flake8 raised this warning:

    F541 f-string is missing placeholders
2020-10-06 15:11:12 +03:00
58d2b8d4ed
dspace_statistics_api/items.py: Move util import
Move util import from global scope because it causes tests to fail.
We don't need the set up the Solr connection unless we're actually
trying to use the get_views and get_downloads methods, either when
running the API in production or during tests where the connection
has been set up.
2020-10-06 15:07:00 +03:00
e6572d9469
.build.yml: Use poetry instead of pipenv 2020-10-05 22:37:42 +03:00
85fca81611
Update requirements
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
2020-10-05 22:33:27 +03:00
c81a8d03d7
Remove pipenv configuration
I was having issues with re-creating an environment from scratch:

    ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'

Switching to Poetry for now.
2020-10-05 22:32:01 +03:00
2923a3b325
Add Poetry configuration
I was having some problems with pipenv when trying to install a
clean environment:

    ModuleNotFoundError: No module named 'virtualenv.seed.via_app_data'
2020-10-05 22:30:35 +03:00
d4518d62ad
dspace_statistics_api/app.py: Refactor for testability
I thought it was clever to only import these in the on_post handler
because they aren't needed elsewhere, but it turns out that this is
not a common pattern and even causes problems with testability.

First, if the imports are at the top of the file as PEP8 recommends,
then the WSGI server will import them once when it loads the app and
they remain in memory for the lifecycle of the app. If the imports
are in the on_post handler they would be re-imported on every request!

Second, this pattern of importing in a method makes it tricky to use
object patching in mocks.

See: https://www.python.org/dev/peps/pep-0008/#imports
2020-10-05 20:43:50 +03:00
3a98de78e3
dspace_statistics_api/items.py: Remove executable bit
We don't need to execute this on the command line.
2020-10-05 14:33:36 +03:00
b26439daf3
CHANGELOG.md: Add note about Python dependencies 2020-09-27 11:16:37 +03:00
9e898ba54f
Update requirements
Generated from pipenv with:

  $ pipenv lock -r > requirements.txt
  $ pipenv lock -r -d > requirements-dev.txt
2020-09-27 11:13:38 +03:00
716d65030c
Pipfile.lock: Run pipenv update 2020-09-27 11:13:04 +03:00