1
0
mirror of https://github.com/ilri/dspace-statistics-api.git synced 2024-11-14 18:37:03 +01:00
Commit Graph

527 Commits

Author SHA1 Message Date
5a53b57b3b
Refactor /items POST handler to use a before hook
This allows us to do the dirty work of parsing, validating, and
setting local variables from the POST parameters outside of the
on_post function. We then share the parameters via the req.context
object. Functionally it is the same, but readability is better
and it's a neat trick that I could use elsewhere.

See: https://falcon.readthedocs.io/en/stable/user/faq.html#how-can-i-pass-data-from-a-hook-to-a-responder-and-between-hooks
2020-09-26 18:40:52 +03:00
3ceb9a6eb0
dspace_statistics_api/items.py: Fix flake8 warning
According to flake8 we need to use a different syntax for strings
with backslash escape sequences:

> As of Python 3.6, a backslash-character pair that is not a valid
> escape sequence now generates a DeprecationWarning. This will
> eventually become a SyntaxError.

The warning was:

    W605 invalid escape sequence '\-'

See: https://www.flake8rules.com/rules/W605.html
2020-09-26 12:22:06 +03:00
946f0749e2
dspace_statistics_api/app.py: Use bounded_stream in on_post
For reasons I don't quite understand, we need to use bounded_stream
in the on_post request handler in order to use simulate_post() with
the testing client in Falcon 2.0.0. Normal runtime operation via
gunicorn does not have any issues with stream.

See: https://github.com/falconry/falcon/issues/1720
See: https://github.com/falconry/falcon/issues/1554
2020-09-26 11:50:57 +03:00
b06651d1ec
dspace_statistics_api/indexer.py: Fix Python comment 2020-09-25 13:35:05 +03:00
a0ee181361
dspace_statistics_api/docs/index.html: Fix whitespace 2020-09-25 13:33:45 +03:00
f58c209609
dspace_statistics_api/indexer.py: Update comment
I don't remember why we needed the stats, but it seems that it was
because without them there is no way to know how many results were
returned and therefore no way to know how many pages we'll need to
iterate over. Having the total number allows us to use a limit and
and offset to page through them deterministically.
2020-09-25 13:25:34 +03:00
6dbff1e78f
README.md: Capitalize UUID 2020-09-25 13:03:15 +03:00
731226ec15
README.md: Update for POST /items functionality 2020-09-25 13:01:09 +03:00
2201d3df4e
dspace_statistics_api/docs/index.html: Minor HTML syntax issue 2020-09-25 12:55:39 +03:00
2c0436f845
Update API docs HTML for /items POST functionality 2020-09-25 12:53:30 +03:00
f1e939481b
dspace_statistics_api/items.py: Remove shebang
This was originally a standalone script I was testing interactively.
2020-09-25 12:39:00 +03:00
4d8026a3d0
Add missing dspace_statistics_api/items.py
This was meant to be added with the new /items POST changes.
2020-09-25 12:30:06 +03:00
de1f462ad2
CHANGELOG.md: Add more notes about /items 2020-09-25 12:29:19 +03:00
8a6bbfd527
CHANGELOG.md: Add note about /items 2020-09-25 12:27:33 +03:00
73c71fa8a0
dspace_statistics_api: Add support for date ranges to /items
You can now POST a JSON request to /items with a list of items and
a date range. This allows the possibility to get view and download
statistics for arbitrary items and arbitrary date ranges.

The JSON request should be in the following format:

    {
        "limit": 100,
        "page": 0,
        "dateFrom": "2020-01-01T00:00:00Z",
        "dateTo": "2020-09-09T00:00:00Z",
        "items": [
            "f44cf173-2344-4eb2-8f00-ee55df32c76f",
            "2324aa41-e9de-4a2b-bc36-16241464683e",
            "8542f9da-9ce1-4614-abf4-f2e3fdb4b305",
            "0fe573e7-042a-4240-a4d9-753b61233908"
        ]
    }

The limit, page, and date parameters are all optional. By default
it will use a limit of 100, page 0, and [* TO *] Solr date range.
2020-09-25 12:21:11 +03:00
7a5e14716d
CHANGELOG.md: Add note about indexer refactoring 2020-09-24 12:08:21 +03:00
21b500b4f7
dspace_statistics_api/util.py: Use docstring for get_statistics_shards
It seems better to use a docstring instead of a comment because it
can potentially be used by IDEs or documentation generators.
2020-09-24 12:07:31 +03:00
495386856b
Refactor indexer
Move the get_statistics_shards() method to a utility module so it
can be used by other things.
2020-09-24 12:03:12 +03:00
8e87f80e9a
dspace_statistics_api/indexer.py: Remove duplicate solr_url variable
This is declared twice and it never changes.
2020-09-24 11:54:31 +03:00
c4bf8bf698
README.md: Add TODO note about sorting by views or downloads 2020-09-24 11:53:23 +03:00
6ff95bb5f2
dspace_statistics_api/indexer.py: Remove SolrClient reference
We stopped using SolrClient in favor of vanilla requests.
2020-09-24 11:30:31 +03:00
0c8fb21f80
README.md: Update DSpace wiki URLs 2020-04-13 15:25:17 +03:00
b359c2466f
.travis.yml: Don't build in a container
I didn't realize these LXD containers are not available on AMD64.
Now I understand why the build was so slow: because it was ARM64!
2020-03-29 16:36:47 +03:00
0eaed3e8c4
.travis.yml: Use Python 3.8-dev instead of master
See: https://docs.travis-ci.com/user/languages/python/#specifying-python-versions
2020-03-29 16:26:57 +03:00
70e96214c8
.travis.yml: Go→Python
Fix incorrect language.
2020-03-29 16:24:39 +03:00
cab9f16dbc
.travis.yml: Try to run in an LXD container
According to the build environment documentation we need to specify
an OS of Linux in order to get a container instead of a VM.

See: https://docs.travis-ci.com/user/reference/overview/
2020-03-29 16:24:00 +03:00
bd49e1d1f6
.travis.yml: Correctly specify PostgreSQL 10
See: https://docs.travis-ci.com/user/database-setup/#postgresql
2020-03-29 16:19:08 +03:00
144ed9a7c4
.travis.yml: Use PostgreSQL 10.0
Production is still PostgreSQL 9.6, but I have been using 10.0 in
local development and staging environments.
2020-03-29 16:08:29 +03:00
48eef8c8e3
.travis.yml: Test on Python master
But allow failures!
2020-03-29 16:07:44 +03:00
fa9325e8a3 CHANGELOG.md: Add changes for v1.2.1 2020-03-02 14:32:07 +02:00
998e833470 dspace_statistics_api/docs/index.html: Adjust help text 2020-03-02 14:30:16 +02:00
dd8252601f README.md: Adjust API help text 2020-03-02 14:29:13 +02:00
9a9555853f README.md: Add note about versions 2020-03-02 14:28:22 +02:00
385e92cc5e README.md: Update
Remove TODOs that I've recently completed and update introduction.
2020-03-02 14:25:47 +02:00
b0e6481961 tests/dspacestatistics.sql: Update
New database snapshot that uses UUIDs.
2020-03-02 12:36:06 +02:00
f96a903be3 README.md: Update Python requirement 2020-03-02 11:47:03 +02:00
fcf8fa4c29 CHANGELOG.md: Minor syntax and spelling changes 2020-03-02 11:45:26 +02:00
5dd50ff998 CHANGELOG.md: Version 1.2.0
This version only works with DSpace 6+ where the internal item id-
entifiers are UUIDs instead of integers. Version 1.1.1 was the last
version to work with DSpace 4 and 5.
2020-03-02 11:34:58 +02:00
6704e7375f CHANGELOG.md: Add note about Python dependencies 2020-03-02 11:34:13 +02:00
37630d8dac CHANGELOG.md: Add note about DSpace 6+ UUIDs 2020-03-02 11:27:10 +02:00
0ef071a91d dspace_statistics_api: Use f-strings instead of format()
We had previously been avoiding the f-strings because we needed to
run on Python 3.5 and they were only available in Python 3.6+, but
now the black formatter requires Python 3.6 and all our systems are
running Python 3.6+ anyways.
2020-03-02 11:24:29 +02:00
9e7dd28156 dspace_statistics_api/app.py: Use parameterized SQL queries
This is a better way to run SQL queries because psycopg2 takes care
of the quoting for us.
2020-03-02 11:16:05 +02:00
60e6ea57b1 tests/test_api.py: Use UUID
DSpace 6+ uses a UUID for item identifiers instead of an integer so
we need to adapt our tests accordingly. The Python UUID object must
be cast to a string to use it elsewhere in the code.
2020-03-02 11:10:41 +02:00
5955868b9a dspace_statistics_api/app.py: Use UUID
DSpace 6+ uses a UUID for item identifiers instead of an integer so
we need to adapt our PostgreSQL queries to use those. Note that we
can no longer sort results in the "all items" endpoint by ID. Also,
we need to use parameterized psycopg2 queries instead of strings to
support queries with UUIDs properly. To use the Python UUID objects
elsewhere in the code we need to make sure that we cast them to str.
2020-03-02 11:06:48 +02:00
250fd8164f dspace_statistics_api/indexer.py: Use UUID
DSpace 6+ uses a UUID for item identifiers instead of an integer so
we need to update the PostgreSQL schema accordingly. Solr still re-
fers to them as "id" in its schema so we don't need to change anyt-
hing there.
2020-03-01 21:22:10 +02:00
82be1a4d00 Update requirements
Generated from pipenv with:

  $ pipenv lock -r > requirements.txt
  $ pipenv lock -r -d > requirements-dev.txt
2020-03-01 21:21:13 +02:00
0615064e3d Add pytest-clarity to pipenv
Makes pytest output easier to understand.
2020-03-01 21:19:28 +02:00
76be1b749a Run pipenv update 2020-03-01 21:13:32 +02:00
92146fe426
tests/test_api.py: Format with black 2019-12-14 12:39:58 +02:00
440b2f2dfa
Pipfile.lock: Run pipenv update 2019-12-14 12:38:11 +02:00