alanorth/csv-metadata-quality
1
0
Fork 0
mirror of https://github.com/ilri/csv-metadata-quality.git synced 2025-05-09 14:46:00 +02:00
Code Releases Activity
349 Commits 5 Branches 18 Tags
773a0a2695f9d9ab2597862c04407e7664de9a11
Commit Graph

349 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alan Orth
23b95fa368 .travis.yml: Use Ubuntu 20.04 "Focal" environment 2020-10-29 00:14:54 +03:00
Alan Orth
6985f76aa3 .travis.yml: Bump Python versions 
Test Python 3.9 now that it was released, and allow tests to fail
on nightly builds.
 2020-10-29 00:14:36 +03:00
Alan Orth
98a6a19e12 Update requirements-dev.txt 
Generated with poetry export:

    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:48:46 +03:00
Alan Orth
f4914c414f Only install ipython on Python 3.7+ 2020-10-06 17:48:16 +03:00
Alan Orth
d352fe8017 Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:21:33 +03:00
Alan Orth
f13c360084 Update poetry package dependencies 2020-10-06 17:20:16 +03:00
Alan Orth
7cfd4c0b59 csv_metadata_quality: Move scoped imports to global 
According to PEP8 we should avoid scoped imports unless you have a
good reason. Here there are two cases where we do (issn and isbn),
but I will move the others to the global scope.
 2020-10-06 17:11:39 +03:00
Alan Orth
826509ddcf poetry.lock: Run poetry update 
List of updated modules:

  - Updating numpy (1.19.1 -> 1.19.2)
  - Updating pygments (2.6.1 -> 2.7.1)
  - Updating pandas (1.1.1 -> 1.1.2)

All tests still pass according to pytest.
 2020-09-26 12:18:23 +03:00
Alan Orth
22b5c0f7a1 CHANGELOG.md: Add note about dependencies update 2020-09-08 15:04:40 +03:00
Alan Orth
774e274b32 poetry.lock: Run poetry update 
Update dependencies to latest version:

  - Updating attrs (19.3.0 -> 20.2.0)
  - Updating more-itertools (8.4.0 -> 8.5.0)
  - Updating openpyxl (3.0.4 -> 3.0.5)
  - Updating parso (0.7.0 -> 0.7.1)
  - Updating sqlalchemy (1.3.18 -> 1.3.19)
  - Updating urllib3 (1.25.9 -> 1.25.10)
  - Updating agate-dbf (0.2.1 -> 0.2.2)
  - Updating agate-sql (0.5.4 -> 0.5.5)
  - Updating jedi (0.17.1 -> 0.17.2)
  - Updating numpy (1.19.0 -> 1.19.1)
  - Updating prompt-toolkit (3.0.5 -> 3.0.7)
  - Updating regex (2020.6.8 -> 2020.7.14)
  - Updating traitlets (4.3.3 -> 5.0.4)
  - Updating ipython (7.16.1 -> 7.18.1)
  - Updating pandas (1.0.5 -> 1.1.1)
  - Updating python-stdnum (1.13 -> 1.14)

All tests still pass according to pytest.
 2020-09-08 15:04:00 +03:00
Alan Orth
db474a802f README.md: Use badge from travis-ci.com 2020-08-04 11:12:28 +03:00
Alan Orth
e241f8461b CHANGELOG.md: Add notes 2020-07-06 14:10:46 +03:00
Alan Orth
431e6331c8 csv_metadata_quality/check.py: Format with black 2020-07-06 14:10:19 +03:00
Alan Orth
cb07d357d4 Version 0.4.2 v0.4.2 2020-07-06 14:04:34 +03:00
Alan Orth
65cd48a26f CHANGELOG.md: Update changes 2020-07-06 14:00:21 +03:00
Alan Orth
0f883f640c Remove pipenv 2020-07-06 13:59:49 +03:00
Alan Orth
f4c5c5781e README.md: Switch to poetry 2020-07-06 13:59:11 +03:00
Alan Orth
6aa784ad8c Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-07-06 13:57:07 +03:00
Alan Orth
7b8da94f41 poetry.lock: Update Python dependencies 2020-07-06 13:56:31 +03:00
Alan Orth
2a1566af62 csv_metadata_quality/check.py: Parameterize AGROVOC request 2020-07-06 13:44:46 +03:00
Alan Orth
5fcaa63bd5 csv_metadata_quality/check.py: Prune requests cache once 
We only need to prune the requests cache once before using it, not
for every value we check.
 2020-07-06 13:42:19 +03:00
Alan Orth
aa9e23b46c pyproject.toml: Update license specifier 
We need to use valid SPDX license identifiers.
 2020-06-09 14:22:53 +03:00
Alan Orth
73acb1661f Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-05-31 17:51:16 +03:00
Alan Orth
2a068fddc4 .build.yml: Fix test 2020-05-31 17:44:37 +03:00
Alan Orth
c6c2f13e88 .build.yml: Fix poetry install invocation 
Poetry apparently installs dev dependencies by default.
 2020-05-31 17:37:09 +03:00
Alan Orth
56f16e37ed .build.yml: Use poetry in SourceHut CI 2020-05-31 17:35:04 +03:00
Alan Orth
0c44b967b6 Add poetry project file and lock 
I want to try to use poetry instead of pipenv because pipenv takes
forever to do dependency resolution sometimes. Also, I have had a
few issues with Python modules like black that don't have releases
other than pre-releases, and even including the project itself in
the dependencies (pip install -e . ...?). My initial experience is
that poetry handles this better.
 2020-05-31 17:33:40 +03:00
Alan Orth
8a267bb40b .travis.yml: Try to build with Python 3.8-dev 
But allow failures.
 2020-03-29 16:40:11 +03:00
Alan Orth
8fda8f1ef1 Pipfile.lock: Run pipenv update 
All tests still passing.
 2020-03-20 16:22:04 +02:00
Alan Orth
5e471813e8 CHANGELOG.md: Add note about python dependencies 2020-01-29 12:41:43 +02:00
Alan Orth
79244b9ac3 Pipfile.lock: Run pipenv update 2020-01-29 12:39:12 +02:00
Alan Orth
5e81a33482 CHANGELOG.md: Add note about field names 2020-01-16 12:37:11 +02:00
Alan Orth
28b5996aa6 Output field name for more fixes and checks 
This helps identify which field has the error.
 2020-01-16 12:35:11 +02:00
Alan Orth
40ba9bae6c README.md: Adjust heading size 2020-01-15 12:26:11 +02:00
Alan Orth
0b2d211455 Version 0.4.1 v0.4.1 2020-01-15 12:19:42 +02:00
Alan Orth
7f1df0b47c Support Python 3.6 and 3.7 again 2020-01-15 12:19:17 +02:00
Alan Orth
365ecda324 Add utility function to check normalization 
Python's built-in unicodedata library includes the is_normalized()
function starting with Python 3.8. This utility function allows us
to do the same thing with earlier Python versions.

See: https://docs.python.org/3/library/unicodedata.html
 2020-01-15 12:17:52 +02:00
Alan Orth
550ce7fb7e .travis.yml: Only test Python 3.8 
The Unicode normalization feature requires Python 3.8 because the
unicodedata.is_normalized() function only appears there. If I find
another way to check if a string is normalized without normalizing
it first I will drop the requirements back down to Python 3.6.

See: https://docs.python.org/3/library/unicodedata.html
 2020-01-15 11:57:21 +02:00
Alan Orth
705127fd28 Version 0.4.0 v0.4.0 2020-01-15 11:44:56 +02:00
Alan Orth
894e0a196d setup.py: Change Python requirements 
The `unicodedata.is_normalized()` function requires Python 3.8.

See: https://docs.python.org/3/library/unicodedata.html
 2020-01-15 11:43:25 +02:00
Alan Orth
87181bc7b8 Run black, isort, and flake8. 2020-01-15 11:41:31 +02:00
Alan Orth
8de5d862b6 CHANGELOG.md: Add note about Unicode normalization 2020-01-15 11:40:40 +02:00
Alan Orth
49e3543878 Add Unicode normalization 
This will check all strings for un-normalized Unicode characters.
Normalization is done using NFC. This includes tests and updated
sample data (data/test.csv).

See: https://withblue.ink/2019/03/11/why-you-need-to-normalize-unicode-strings.html
 2020-01-15 11:37:54 +02:00
Alan Orth
403b253762 CHANGELOG.md: Update python library versions 2020-01-15 10:58:44 +02:00
Alan Orth
c5fbaf407a Update python requirements 
Generated using pipenv:

  $ pipenv lock -r > requirements.txt
  $ pipenv lock -r -d > requirements-dev.txt
 2020-01-15 10:51:58 +02:00
Alan Orth
4f81f6c83c Pipfile.lock: Run pipenv update 2020-01-15 10:51:19 +02:00
Alan Orth
4b9d1e060f setup.py: Add Python 3.8 classifier 2019-12-14 12:56:11 +02:00
Alan Orth
c8a71e3143 Pipfile.lock: Run pipenv update 2019-12-14 12:53:39 +02:00
Alan Orth
7964d98ca5 Pipfile: Specify exact version of black 
Black only releases pre-release versions, which causes issues with
pipenv. Instead of always running pipenv with "--pre" and potenti-
ally letting in some other pre-release versions for other depende-
ncies, I would rather specify the latest black version explicitly.

See: https://github.com/psf/black/issues/517
See: https://github.com/microsoft/vscode-python/issues/5171
 2019-12-14 12:41:28 +02:00
Alan Orth
64ffc2f1da .travis.yml: Install packages from requirements.txt too 2019-11-14 23:42:28 +02:00