1
0
mirror of https://github.com/ilri/dspace-statistics-api.git synced 2025-05-23 20:42:25 +02:00

Compare commits

...

33 Commits

Author SHA1 Message Date
5a2a7d684c CHANGELOG.md: Move unreleased changes to version 0.8.1 2018-11-14 09:37:00 +02:00
18276e910f CHANGELOG.md: Add notes about pipenv 2018-11-14 09:36:13 +02:00
8de8c2765f Merge pull request from ilri/update-dependencies
Update dependencies
2018-11-14 09:34:45 +02:00
11a1755e59 Update requirements.txt
Generated from pipenv with:

    $ pipenv lock -r > requirements.txt
2018-11-14 09:19:47 +02:00
a835b0fdc5 Re-create pipenv environment from scratch
When I originally created the pipenv environment I used the standard
pip requirements.txt that I already had, which captured all the mod-
ules and their exact versions at the time. This makes it hard to se-
parate the project's actual dependencies from the dependencies' dep-
endencies, complicating the Pipfile and making it hard to update mo-
dule versions later.

I've re-created the environment with the following commands:

    $ pipenv install gunicorn falcon psycopg2-binary git+https://github.com/alanorth/SolrClient.git@kazoo-2.5.0#egg=SolrClient
    $ pipenv install --dev ipython flake8 pytest
2018-11-14 09:07:32 +02:00
a88600c92b README.md: Add note about GPLv3 2018-11-13 12:34:31 +02:00
019d9242c9 Merge pull request from ilri/use-pip
Rework to use pip instead of pipenv
2018-11-12 09:17:16 +02:00
f4d7312a3f CHANGELOG.md: Add unreleased changes 2018-11-12 09:02:04 +02:00
9c46cfc7e2 Use Python 3.7 for pipenv
Now that I'm only using pipenv locally it shouldn't create problems
for people. They can still just create a vanilla virtualenv and use
pip to install the dependencies.
2018-11-12 08:54:54 +02:00
c1c2e319ac README.md: Rework to use pip instead of pipenv
Pipenv is great for local development, but I don't think many people
are using it yet. I can use it locally and on Travis, but still keep
vanilla requirements.txt for use with pip. The requirements.txt file
can be generated easily from pipenv itself:

    $ pipenv lock -r > requirements.txt

The same for the development requirements:

    $ pipenv lock -r -d > requirements-dev.txt
2018-11-12 08:49:02 +02:00
0895b4f469 Add requirements-dev.txt for pip
Generated with pipenv lock -r -d. Will be used for separating the
development dependencies.
2018-11-12 08:48:45 +02:00
dcfef06a65 Pipfile.lock: Run pipenv update 2018-11-12 08:20:47 +02:00
13736d6359 CHANGELOG.md: Move unreleased changes to verison 0.8.0 2018-11-11 17:16:26 +02:00
4fc64edeb8 Merge pull request from ilri/pytest
Pytest
2018-11-11 17:14:49 +02:00
2a8901dc4f CHANGELOG.md: Update notes 2018-11-11 17:10:45 +02:00
e25c974796 README.md: We have tests now 2018-11-11 17:08:51 +02:00
ffc62e9ee6 tests/test_api.py: Use response.text for all json.loads()
This allows the code to work in Python 3.5 as well as 3.6+.
2018-11-11 17:05:31 +02:00
556c5ae088 tests/test_api.py: Use response.text instead of content
Falcon's response content is raw bytes, while its text is a string.
Let's use the latter so we can use json.loads() in Python 3.5, 3.6,
and 3.7 with the same code.

See: https://falcon.readthedocs.io/en/stable/api/testing.html
2018-11-11 17:01:17 +02:00
d94134f80a tests/test_api.py: Try to add workaround for Python 3.5
In Python 3.5 it seems that json.loads() cannot decode a bytes, but
it works in Python 3.6 and 3.7. Let's try a workaround to see if we
can get it working on both Python 3.5 and 3.6+.

See: https://docs.python.org/3.5/library/json.html#json.loads
See: https://docs.python.org/3.6/library/json.html#json.loads
2018-11-11 17:00:20 +02:00
586231eb2d .travis.yml: Use PostgreSQL 9.5
Default PostgreSQL in Travis CI is 9.2 which is very old, so let's
try to use 9.5.

See: https://docs.travis-ci.com/user/database-setup/#postgresql
2018-11-11 16:41:35 +02:00
766b77a3b6 .travis.yml: Use PostgreSQL directly
It seems that Travis CI already has a PostgreSQL service running.
2018-11-11 16:35:28 +02:00
1959e8154e .travis.yml: Use localhost for Docker's PostgreSQL ports
See: https://docs.travis-ci.com/user/docker/
2018-11-11 16:28:18 +02:00
d40b2f0b2e Test API using pytest and PostgreSQL on Travis
First attempt at getting the Travis Docker setup correct. Inspired
by the Travis pipenv setup used in Responder.

See: https://docs.travis-ci.com/user/docker/
See: https://github.com/kennethreitz/responder/blob/master/.travis.yml
2018-11-11 16:25:16 +02:00
061d0a8f5f CHANGELOG.md: Add API tests to unreleased changes 2018-11-11 16:24:54 +02:00
e57660ff88 Add initial pytest configuration
From: https://github.com/kennethreitz/responder/blob/master/pytest.ini
2018-11-11 16:24:54 +02:00
5c8756bede Add pytest to pipenv development packages 2018-11-11 16:24:54 +02:00
bae9fb80e4 Add initial API tests
Test the basic assumptions of the API like response codes and types.
2018-11-11 16:24:54 +02:00
8a65d99e08 .travis.yml: Don't limit builds to master
This is good in theory but it means we can't trigger builds for other
branches on the fly from the Travis web interface.
2018-11-11 16:21:48 +02:00
d479b7dc6c CHANGELOG.md: Syntax fixes 2018-11-11 00:08:44 +02:00
40aac8bf89 Merge pull request from ilri/database-error-handling
Database error handling
2018-11-11 00:07:25 +02:00
53ba6f2936 CHANGELOG.md: Add database try/except to unreleased changes 2018-11-11 00:05:49 +02:00
140cc4cb07 README.md: Remove TODO for database try/except
Now database connection errors are properly excepted and raised.
2018-11-11 00:04:28 +02:00
d5d2d2149b dspace_statistics_api/database.py: Raise HTTP 500 on error
Properly except on database connection error and raise an HTTP 500
instead of spamming the console/log with twenty lines of text.
2018-11-10 23:58:58 +02:00
12 changed files with 77478 additions and 134 deletions

@ -3,9 +3,17 @@ python:
- "3.5"
- "3.6"
- "3.7-dev"
script: pip install -r requirements.txt
branches:
only:
- master
addons:
postgresql: "9.5"
before_script:
- psql --version
- createuser -U postgres dspacestatistics
- psql -U postgres -c "ALTER USER dspacestatistics WITH PASSWORD 'dspacestatistics'"
- createdb -U postgres -O dspacestatistics --encoding=UNICODE dspacestatistics
- psql -U postgres -d dspacestatistics < tests/dspacestatistics.sql
install:
- "pip install pipenv --upgrade-strategy=only-if-needed"
- "pipenv install --dev"
script: pytest
# vim: ts=2 sw=2 et

@ -4,46 +4,61 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### [0.7.0] - 2018-11-07
## Added
## [0.8.1] - 2018-11-14
### Changed
- README.md to recommend using vanilla Python virtual environments and pip instead of pipenv
- Regenerate pipenv environment to capture only direct dependencies
### Added
- `requirements-dev.txt` for installing development packages with pip
## [0.8.0] - 2018-11-11
### Changed
- Properly handle database connection errors
### Added
- API tests with pytest
## [0.7.0] - 2018-11-07
### Added
- Ability to configure PostgreSQL database port with DATABASE_PORT environment variable (defaults to 5432)
- Hound CI configuration to validate pull requests against PEP 8 code style with Flake8
- Configuration for [pipenv](https://pipenv.readthedocs.io/en/latest/)
## Changed
### Changed
- Use a database management class with Python context management to automatically open/close connections and cursors
## Changed
### Changed
- Validate code against PEP 8 style guide with Flake8
### [0.6.1] - 2018-10-31
## Added
## [0.6.1] - 2018-10-31
### Added
- API documentation at root path (/)
### [0.6.0] - 2018-10-31
## Changed
## [0.6.0] - 2018-10-31
### Changed
- Refactor project structure (note breaking changes to API and indexing invocation, see contrib and README.md)
### [0.5.2] - 2018-10-28
## Changed
## [0.5.2] - 2018-10-28
### Changed
- Update library versions in requirements.txt
### [0.5.1] - 2018-10-24
## Changed
## [0.5.1] - 2018-10-24
### Changed
- Use Python's native json instead of ujson
### [0.5.0] - 2018-10-24
## Added
## [0.5.0] - 2018-10-24
### Added
- Example nginx configuration to README.md
## Changed
### Changed
- Don't initialize Solr connection in API
### [0.4.3] - 2018-10-17
## Changed
## [0.4.3] - 2018-10-17
### Changed
- Use pip install as script for Travis CI
## Improved
### Improved
- Documentation for deployment and testing
## [0.4.2] - 2018-10-04
@ -57,7 +72,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [0.4.1] - 2018-09-26
### Changed
- Use execute_values() to batch insert records to PostgreSQL
- Use `execute_values()` to batch insert records to PostgreSQL
## [0.4.0] - 2018-09-25
### Fixed

19
Pipfile

@ -4,22 +4,15 @@ verify_ssl = true
name = "pypi"
[packages]
certifi = "==2018.10.15"
chardet = "==3.0.4"
falcon = "==1.4.1"
gunicorn = "==19.9.0"
idna = "==2.7"
kazoo = "==2.5.0"
"psycopg2-binary" = "==2.7.5"
python-mimeparse = "==1.6.0"
requests = "==2.20.0"
six = "==1.11.0"
gunicorn = "*"
falcon = "*"
"psycopg2-binary" = "*"
solrclient = {ref = "kazoo-2.5.0", git = "https://github.com/alanorth/SolrClient.git"}
"urllib3" = "==1.24"
[dev-packages]
"flake8" = "*"
ipython = "*"
"flake8" = "*"
pytest = "*"
[requires]
python_version = "3.6"
python_version = "3.7"

161
Pipfile.lock generated

@ -1,11 +1,11 @@
{
"_meta": {
"hash": {
"sha256": "74430260b3271348f65792cc7f9cadc5d2036abc4a5fc958524239656ffabb4f"
"sha256": "a846fdab4de5765a7e7fc19424a97a6196248e29f87285cf81fd76e8e9ae3e28"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.6"
"python_version": "3.7"
},
"sources": [
{
@ -16,22 +16,6 @@
]
},
"default": {
"certifi": {
"hashes": [
"sha256:339dc09518b07e2fa7eda5450740925974815557727d6bd35d319c1524a04a4c",
"sha256:6d58c986d22b038c8c0df30d639f23a3e6d172a05c3583e766f4c0b785c0986a"
],
"index": "pypi",
"version": "==2018.10.15"
},
"chardet": {
"hashes": [
"sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
"sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
],
"index": "pypi",
"version": "==3.0.4"
},
"falcon": {
"hashes": [
"sha256:0a66b33458fab9c1e400a9be1a68056abda178eb02a8cb4b8f795e9df20b053b",
@ -48,96 +32,76 @@
"index": "pypi",
"version": "==19.9.0"
},
"idna": {
"hashes": [
"sha256:156a6814fb5ac1fc6850fb002e0852d56c0c8d2531923a51032d1b70760e186e",
"sha256:684a38a6f903c1d71d6d5fac066b58d7768af4de2b832e426ec79c30daa94a16"
],
"index": "pypi",
"version": "==2.7"
},
"kazoo": {
"hashes": [
"sha256:8db774f7bdece7d0dc7decb21539ff0852e42c2ffe1c28d7f1ff6f9292a1c3a4",
"sha256:a5fa2e400c5068cfee9e86b35cf0dab8232b574152d8e3590d823b3e2426ab5e"
],
"index": "pypi",
"version": "==2.5.0"
},
"psycopg2-binary": {
"hashes": [
"sha256:04afb59bbbd2eab3148e6816beddc74348078b8c02a1113ea7f7822f5be4afe3",
"sha256:098b18f4d8857a8f9b206d1dc54db56c2255d5d26458917e7bcad61ebfe4338f",
"sha256:0bf855d4a7083e20ead961fda4923887094eaeace0ab2d76eb4aa300f4bbf5bd",
"sha256:197dda3ffd02057820be83fe4d84529ea70bf39a9a4daee1d20ffc74eb3d042e",
"sha256:278ef63afb4b3d842b4609f2c05ffbfb76795cf6a184deeb8707cd5ed3c981a5",
"sha256:3cbf8c4fc8f22f0817220891cf405831559f4d4c12c4f73913730a2ea6c47a47",
"sha256:4305aed922c4d9d6163ab3a41d80b5a1cfab54917467da8168552c42cad84d32",
"sha256:47ee296f704fb8b2a616dec691cdcfd5fa0f11943955e88faa98cbd1dc3b3e3d",
"sha256:4a0e38cb30457e70580903367161173d4a7d1381eb2f2cfe4e69b7806623f484",
"sha256:4d6c294c6638a71cafb82a37f182f24321f1163b08b5d5ca076e11fe838a3086",
"sha256:4f3233c366500730f839f92833194fd8f9a5c4529c8cd8040aa162c3740de8e5",
"sha256:5221f5a3f4ca2ddf0d58e8b8a32ca50948be9a43351fda797eb4e72d7a7aa34d",
"sha256:5c6ca0b507540a11eaf9e77dee4f07c131c2ec80ca0cffa146671bf690bc1c02",
"sha256:789bd89d71d704db2b3d5e67d6d518b158985d791d3b2dec5ab85457cfc9677b",
"sha256:7b94d29239efeaa6a967f3b5971bd0518d2a24edd1511edbf4a2c8b815220d07",
"sha256:89bc65ef3301c74cf32db25334421ea6adbe8f65601ea45dcaaf095abed910bb",
"sha256:89d6d3a549f405c20c9ae4dc94d7ed2de2fa77427a470674490a622070732e62",
"sha256:97521704ac7127d7d8ba22877da3c7bf4a40366587d238ec679ff38e33177498",
"sha256:a395b62d5f44ff6f633231abe568e2203b8fabf9797cd6386aa92497df912d9a",
"sha256:a6d32c37f714c3f34158f3fa659f3a8f2658d5f53c4297d45579b9677cc4d852",
"sha256:a89ee5c26f72f2d0d74b991ce49e42ddeb4ac0dc2d8c06a0f2770a1ab48f4fe0",
"sha256:b4c8b0ef3608e59317bfc501df84a61e48b5445d45f24d0391a24802de5f2d84",
"sha256:b5fcf07140219a1f71e18486b8dc28e2e1b76a441c19374805c617aa6d9a9d55",
"sha256:b86f527f00956ecebad6ab3bb30e3a75fedf1160a8716978dd8ce7adddedd86f",
"sha256:be4c4aa22ba22f70de36c98b06480e2f1697972d49eb20d525f400d204a6d272",
"sha256:c2ac7aa1a144d4e0e613ac7286dae85671e99fe7a1353954d4905629c36b811c",
"sha256:de26ef4787b5e778e8223913a3e50368b44e7480f83c76df1f51d23bd21cea16",
"sha256:e70ebcfc5372dc7b699c0110454fc4263967f30c55454397e5769eb72c0eb0ce",
"sha256:eadbd32b6bc48b67b0457fccc94c86f7ccc8178ab839f684eb285bb592dc143e",
"sha256:ecbc6dfff6db06b8b72ae8a2f25ff20fbdcb83cb543811a08f7cb555042aa729"
"sha256:036bcb198a7cc4ce0fe43344f8c2c9a8155aefa411633f426c8c6ed58a6c0426",
"sha256:1d770fcc02cdf628aebac7404d56b28a7e9ebec8cfc0e63260bd54d6edfa16d4",
"sha256:1fdc6f369dcf229de6c873522d54336af598b9470ccd5300e2f58ee506f5ca13",
"sha256:21f9ddc0ff6e07f7d7b6b484eb9da2c03bc9931dd13e36796b111d631f7135a3",
"sha256:247873cda726f7956f745a3e03158b00de79c4abea8776dc2f611d5ba368d72d",
"sha256:3aa31c42f29f1da6f4fd41433ad15052d5ff045f2214002e027a321f79d64e2c",
"sha256:475f694f87dbc619010b26de7d0fc575a4accf503f2200885cc21f526bffe2ad",
"sha256:4b5e332a24bf6e2fda1f51ca2a57ae1083352293a08eeea1fa1112dc7dd542d1",
"sha256:570d521660574aca40be7b4d532dfb6f156aad7b16b5ed62d1534f64f1ef72d8",
"sha256:59072de7def0690dd13112d2bdb453e20570a97297070f876fbbb7cbc1c26b05",
"sha256:5f0b658989e918ef187f8a08db0420528126f2c7da182a7b9f8bf7f85144d4e4",
"sha256:649199c84a966917d86cdc2046e03d536763576c0b2a756059ae0b3a9656bc20",
"sha256:6645fc9b4705ae8fbf1ef7674f416f89ae1559deec810f6dd15197dfa52893da",
"sha256:6872dd54d4e398d781efe8fe2e2d7eafe4450d61b5c4898aced7610109a6df75",
"sha256:6ce34fbc251fc0d691c8d131250ba6f42fd2b28ef28558d528ba8c558cb28804",
"sha256:73920d167a0a4d1006f5f3b9a3efce6f0e5e883a99599d38206d43f27697df00",
"sha256:8a671732b87ae423e34b51139628123bc0306c2cb85c226e71b28d3d57d7e42a",
"sha256:8d517e8fda2efebca27c2018e14c90ed7dc3f04d7098b3da2912e62a1a5585fe",
"sha256:9475a008eb7279e20d400c76471843c321b46acacc7ee3de0b47233a1e3fa2cf",
"sha256:96947b8cd7b3148fb0e6549fcb31258a736595d6f2a599f8cd450e9a80a14781",
"sha256:abf229f24daa93f67ac53e2e17c8798a71a01711eb9fcdd029abba8637164338",
"sha256:b1ab012f276df584beb74f81acb63905762c25803ece647016613c3d6ad4e432",
"sha256:b22b33f6f0071fe57cb4e9158f353c88d41e739a3ec0d76f7b704539e7076427",
"sha256:b3b2d53274858e50ad2ffdd6d97ce1d014e1e530f82ec8b307edd5d4c921badf",
"sha256:bab26a729befc7b9fab9ded1bba9c51b785188b79f8a2796ba03e7e734269e2e",
"sha256:daa1a593629aa49f506eddc9d23dc7f89b35693b90e1fbcd4480182d1203ea90",
"sha256:dd111280ce40e89fd17b19c1269fd1b74a30fce9d44a550840e86edb33924eb8",
"sha256:e0b86084f1e2e78c451994410de756deba206884d6bed68d5a3d7f39ff5fea1d",
"sha256:eb86520753560a7e89639500e2a254bb6f683342af598088cb72c73edcad21e6",
"sha256:ff18c5c40a38d41811c23e2480615425c97ea81fd7e9118b8b899c512d97c737"
],
"index": "pypi",
"version": "==2.7.5"
"version": "==2.7.6.1"
},
"python-mimeparse": {
"hashes": [
"sha256:76e4b03d700a641fd7761d3cd4fdbbdcd787eade1ebfac43f877016328334f78",
"sha256:a295f03ff20341491bfe4717a39cd0a8cc9afad619ba44b77e86b0ab8a2b8282"
],
"index": "pypi",
"version": "==1.6.0"
},
"requests": {
"hashes": [
"sha256:99dcfdaaeb17caf6e526f32b6a7b780461512ab3f1d992187801694cba42770c",
"sha256:a84b8c9ab6239b578f22d1c21d51b696dcfe004032bb80ea832398d6909d7279"
],
"index": "pypi",
"version": "==2.20.0"
},
"six": {
"hashes": [
"sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
"sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
],
"index": "pypi",
"version": "==1.11.0"
},
"solrclient": {
"git": "https://github.com/alanorth/SolrClient.git",
"ref": "c629e3475be37c82770b2be61748be7e29882648"
},
"urllib3": {
"hashes": [
"sha256:41c3db2fc01e5b907288010dec72f9d0a74e37d6994e6eb56849f59fea2265ae",
"sha256:8819bba37a02d143296a4d032373c4dd4aca11f6d4c9973335ca75f9c8475f59"
],
"index": "pypi",
"version": "==1.24"
}
},
"develop": {
"atomicwrites": {
"hashes": [
"sha256:0312ad34fcad8fac3704d441f7b317e50af620823353ec657a53e981f92920c0",
"sha256:ec9ae8adaae229e4f8446952d204a3e4b5fdd2d099f9be3aaf556120135fb3ee"
],
"version": "==1.2.1"
},
"attrs": {
"hashes": [
"sha256:10cbf6e27dbce8c30807caf056c8eb50917e0eaafe86347671b57254006c3e69",
"sha256:ca4be454458f9dec299268d472aaa5a11f67a4ff70093396e1ceae9c76cf4bbb"
],
"version": "==18.2.0"
},
"backcall": {
"hashes": [
"sha256:38ecd85be2c1e78f77fd91700c76e14667dc21e2713b63876c0eb901196e01e4",
@ -189,6 +153,14 @@
],
"version": "==0.6.1"
},
"more-itertools": {
"hashes": [
"sha256:c187a73da93e7a8acc0001572aebc7e3c69daf7bf6881a2cea10650bd4420092",
"sha256:c476b5d3a34e12d40130bc2f935028b5f636df8f372dc2c1c01dc19681b2039e",
"sha256:fcbfeaea0be121980e15bc97b3817b5202ca73d0eae185b4550cbfce2a3ebb3d"
],
"version": "==4.3.0"
},
"parso": {
"hashes": [
"sha256:35704a43a3c113cce4de228ddb39aab374b8004f4f2407d070b6a2ca784ce8a2",
@ -211,6 +183,13 @@
],
"version": "==0.7.5"
},
"pluggy": {
"hashes": [
"sha256:447ba94990e8014ee25ec853339faf7b0fc8050cdc3289d4d71f7f410fb90095",
"sha256:bde19360a8ec4dfd8a20dcb811780a30998101f078fc7ded6162f0076f50508f"
],
"version": "==0.8.0"
},
"prompt-toolkit": {
"hashes": [
"sha256:c1d6aff5252ab2ef391c2fe498ed8c088066f66bc64a8d5c095bbf795d9fec34",
@ -226,6 +205,13 @@
],
"version": "==0.6.0"
},
"py": {
"hashes": [
"sha256:bf92637198836372b520efcba9e020c330123be8ce527e535d185ed4b6f45694",
"sha256:e76826342cefe3c3d5f7e8ee4316b80d1dd8a300781612ddbc765c17ba25a6c6"
],
"version": "==1.7.0"
},
"pycodestyle": {
"hashes": [
"sha256:cbc619d09254895b0d12c2c691e237b2e91e9b2ecf5e84c26b35400f93dcfb83",
@ -247,12 +233,19 @@
],
"version": "==2.2.0"
},
"pytest": {
"hashes": [
"sha256:3f193df1cfe1d1609d4c583838bea3d532b18d6160fd3f55c9447fdca30848ec",
"sha256:e246cf173c01169b9617fc07264b7b1316e78d7a650055235d6d897bc80d9660"
],
"index": "pypi",
"version": "==3.10.1"
},
"six": {
"hashes": [
"sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9",
"sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb"
],
"index": "pypi",
"version": "==1.11.0"
},
"traitlets": {

@ -9,11 +9,14 @@ This project contains an indexer and a [Falcon-based](https://falcon.readthedocs
- PostgreSQL version 9.5+ (due to [`UPSERT` support](https://wiki.postgresql.org/wiki/UPSERT))
- DSpace with [Solr usage statistics enabled](https://wiki.duraspace.org/display/DSDOC5x/SOLR+Statistics) (tested with 5.x)
## Installation and Testing
Create a Python virtual environment and install the dependencies using [`pipenv`](https://github.com/pypa/pipenv):
## Installation
Create a Python virtual environment and install the dependencies:
$ pipenv install --dev
$ pipenv shell
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt
## Running
Set up the environment variables for Solr and PostgreSQL:
@ -35,6 +38,15 @@ Test to see if there are any statistics:
$ curl 'http://localhost:8000/items?limit=1'
## Testing
Install development packages using pip:
$ pip install -r requirements-dev.txt
Run tests:
$ pytest
## Deployment
There are example systemd service and timer units in the `contrib` directory. The API service listens on localhost by default so you will need to expose it publicly using a web server like nginx.
@ -71,11 +83,11 @@ The item id is the *internal* id for an item. You can get these from the standar
## Todo
- Better logging
- Tests
- Check if database exists (try/except)
- Version API
- Use JSON in PostgreSQL
- Switch to [Python 3.6+ f-string syntax](https://realpython.com/python-f-strings/)
## License
This work is licensed under the [GPLv3](https://www.gnu.org/licenses/gpl-3.0.en.html).
The license allows you to use and modify the work for personal and commercial purposes, but if you distribute the work you must provide users with a means to access the source code for the version you are distributing. Read more about the [GPLv3 at TL;DR Legal](https://tldrlegal.com/license/gnu-general-public-license-v3-(gpl-3)).

@ -3,6 +3,7 @@ from .config import DATABASE_USER
from .config import DATABASE_PASS
from .config import DATABASE_HOST
from .config import DATABASE_PORT
import falcon
import psycopg2
import psycopg2.extras
@ -14,7 +15,13 @@ class DatabaseManager():
self._connection_uri = 'dbname={} user={} password={} host={} port={}'.format(DATABASE_NAME, DATABASE_USER, DATABASE_PASS, DATABASE_HOST, DATABASE_PORT)
def __enter__(self):
self._connection = psycopg2.connect(self._connection_uri, cursor_factory=psycopg2.extras.DictCursor)
try:
self._connection = psycopg2.connect(self._connection_uri, cursor_factory=psycopg2.extras.DictCursor)
except psycopg2.OperationalError:
title = '500 Internal Server Error'
description = 'Could not connect to database'
raise falcon.HTTPInternalServerError(title, description)
return self._connection
def __exit__(self, exc_type, exc_value, exc_traceback):

4
pytest.ini Normal file

@ -0,0 +1,4 @@
[pytest]
addopts= -rsxX -s -v --strict
filterwarnings =
error::UserWarning

25
requirements-dev.txt Normal file

@ -0,0 +1,25 @@
-i https://pypi.org/simple
atomicwrites==1.2.1
attrs==18.2.0
backcall==0.1.0
decorator==4.3.0
flake8==3.6.0
ipython-genutils==0.2.0
ipython==7.1.1
jedi==0.13.1
mccabe==0.6.1
more-itertools==4.3.0
parso==0.3.1
pexpect==4.6.0 ; sys_platform != 'win32'
pickleshare==0.7.5
pluggy==0.8.0
prompt-toolkit==2.0.7
ptyprocess==0.6.0
py==1.7.0
pycodestyle==2.4.0
pyflakes==2.0.0
pygments==2.2.0
pytest==3.10.1
six==1.11.0
traitlets==4.3.2
wcwidth==0.1.7

@ -1,13 +1,7 @@
-i https://pypi.org/simple
certifi==2018.10.15
chardet==3.0.4
falcon==1.4.1
git+https://github.com/alanorth/SolrClient.git@c629e3475be37c82770b2be61748be7e29882648#egg=solrclient
gunicorn==19.9.0
idna==2.7
kazoo==2.5.0
psycopg2-binary==2.7.5
psycopg2-binary==2.7.6.1
python-mimeparse==1.6.0
requests==2.20.0
six==1.11.0
urllib3==1.24

0
tests/__init__.py Normal file

77226
tests/dspacestatistics.sql Normal file

File diff suppressed because it is too large Load Diff

67
tests/test_api.py Normal file

@ -0,0 +1,67 @@
from falcon import testing
import json
import pytest
from dspace_statistics_api.app import api
@pytest.fixture
def client():
return testing.TestClient(api)
def test_get_docs(client):
'''Test requesting the documentation at the root.'''
response = client.simulate_get('/')
assert isinstance(response.content, bytes)
assert response.status_code == 200
def test_get_item(client):
'''Test requesting a single item.'''
response = client.simulate_get('/item/17')
response_doc = json.loads(response.text)
assert isinstance(response_doc['downloads'], int)
assert isinstance(response_doc['id'], int)
assert isinstance(response_doc['views'], int)
assert response.status_code == 200
def test_get_missing_item(client):
'''Test requesting a single non-existing item.'''
response = client.simulate_get('/item/1')
assert response.status_code == 404
def test_get_items(client):
'''Test requesting 100 items.'''
response = client.simulate_get('/items', query_string='limit=100')
response_doc = json.loads(response.text)
assert isinstance(response_doc['currentPage'], int)
assert isinstance(response_doc['totalPages'], int)
assert isinstance(response_doc['statistics'], list)
assert response.status_code == 200
def test_get_items_invalid_limit(client):
'''Test requesting 100 items with an invalid limit parameter.'''
response = client.simulate_get('/items', query_string='limit=101')
assert response.status_code == 400
def test_get_items_invalid_page(client):
'''Test requesting 100 items with an invalid page parameter.'''
response = client.simulate_get('/items', query_string='page=-1')
assert response.status_code == 400