alanorth
/
csv-metadata-quality
mirror of https://github.com/ilri/csv-metadata-quality.git
1
0
Fork 0
Code Releases Activity
368 Commits 7 Branches 17 Tags 5.4 MiB
Commit Graph

368 Commits

Author SHA1 Message Date
Alan Orth d3880a9dfa
Remove Python 3.6 support
continuous-integration/drone/push Build is passing Details 
Pandas 1.2.0 apparently requires Python 3.7.1+.
 2021-01-03 15:51:53 +02:00
Alan Orth 7edb8b19d7
tests/test_check.py: Reformat with black 2021-01-03 15:50:21 +02:00
Alan Orth a6709c7f82
Update requirements
continuous-integration/drone/push Build is failing Details 
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

I am trying `--without-hashes` to work around an error on pip install
when running on Python 3.6 in Travis:

    ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==.
 2021-01-03 15:42:00 +02:00
Alan Orth d489ea4609
poetry.lock: Run poetry update 2021-01-03 15:41:08 +02:00
Alan Orth 96634cbb67
pytest.ini: Change --strict to --strict-markers 
This is deprecated since pytest 6.2.0.

See: https://docs.pytest.org/en/stable/deprecations.html#the-strict-command-line-option
 2021-01-03 15:40:14 +02:00
Alan Orth 29e67a0887
Add tests for unnecessary multi-value separators 2021-01-03 15:37:18 +02:00
Alan Orth 32cea2055f
data/test.csv: Add unnecessary multi-value separator 2021-01-03 15:33:04 +02:00
Alan Orth 0dc66c5c4e
Expand check/fix for multi-value separators 
I just came across some metadata that had unnecessary multi-value
separators at the end of a field, causing a blank value to be used.

For example: "Kenya||Tanzania||"
 2021-01-03 15:30:03 +02:00
Alan Orth c26ad83534
.github: Test CLI invocation 2020-12-14 23:47:09 +02:00
Alan Orth 72ca9d99bf
setup.py: Add Python 3.9 
[SKIP CI]
 2020-12-14 23:44:35 +02:00
Alan Orth ae33a9b793
Add .drone.yml 2020-12-14 23:42:23 +02:00
Alan Orth fc0367bfc8
README.md: Update note about Python version 2020-12-08 10:52:24 +02:00
Alan Orth e33b285034
README.md: Add GitHub Actions badge 2020-12-08 10:48:31 +02:00
Alan Orth 349fca03b8
.github/workflows/python-app.yml: Rename 
This name is displayed in the badge so it should be something more
relevant.
 2020-12-08 10:46:39 +02:00
Alan Orth 52d8904870
Remove .travis.yml 
They changed their free tier and I might as well use GitHub Actions
for ILRI stuff anyways.
 2020-12-08 10:41:36 +02:00
Alan Orth 971c69e535
Create python-app.yml 
Try GitHub Actions for Python 3.8 using GitHub's Python example.
 2020-12-08 10:38:52 +02:00
Alan Orth f8cc233e25
.travis.yml: Use Amazon Graviton2 ARM environment 
These are the new hotness and should have faster build times.

See: https://blog.travis-ci.com/2020-09-11-arm-on-aws
 2020-12-06 10:49:03 +02:00
Alan Orth aa7b7a9592
Update requirements 
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

I am trying `--without-hashes` to work around an error on pip install
when running on Python 3.6 in Travis:

    ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==.
 2020-11-03 07:42:45 +02:00
Alan Orth 57b455bde7
poetry.lock: Run poetry update 2020-11-03 07:40:56 +02:00
Alan Orth 23b95fa368
.travis.yml: Use Ubuntu 20.04 "Focal" environment 2020-10-29 00:14:54 +03:00
Alan Orth 6985f76aa3
.travis.yml: Bump Python versions 
Test Python 3.9 now that it was released, and allow tests to fail
on nightly builds.
 2020-10-29 00:14:36 +03:00
Alan Orth 98a6a19e12
Update requirements-dev.txt 
Generated with poetry export:

    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:48:46 +03:00
Alan Orth f4914c414f
Only install ipython on Python 3.7+ 2020-10-06 17:48:16 +03:00
Alan Orth d352fe8017
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:21:33 +03:00
Alan Orth f13c360084
Update poetry package dependencies 2020-10-06 17:20:16 +03:00
Alan Orth 7cfd4c0b59
csv_metadata_quality: Move scoped imports to global 
According to PEP8 we should avoid scoped imports unless you have a
good reason. Here there are two cases where we do (issn and isbn),
but I will move the others to the global scope.
 2020-10-06 17:11:39 +03:00
Alan Orth 826509ddcf
poetry.lock: Run poetry update 
List of updated modules:

  - Updating numpy (1.19.1 -> 1.19.2)
  - Updating pygments (2.6.1 -> 2.7.1)
  - Updating pandas (1.1.1 -> 1.1.2)

All tests still pass according to pytest.
 2020-09-26 12:18:23 +03:00
Alan Orth 22b5c0f7a1
CHANGELOG.md: Add note about dependencies update 2020-09-08 15:04:40 +03:00
Alan Orth 774e274b32
poetry.lock: Run poetry update 
Update dependencies to latest version:

  - Updating attrs (19.3.0 -> 20.2.0)
  - Updating more-itertools (8.4.0 -> 8.5.0)
  - Updating openpyxl (3.0.4 -> 3.0.5)
  - Updating parso (0.7.0 -> 0.7.1)
  - Updating sqlalchemy (1.3.18 -> 1.3.19)
  - Updating urllib3 (1.25.9 -> 1.25.10)
  - Updating agate-dbf (0.2.1 -> 0.2.2)
  - Updating agate-sql (0.5.4 -> 0.5.5)
  - Updating jedi (0.17.1 -> 0.17.2)
  - Updating numpy (1.19.0 -> 1.19.1)
  - Updating prompt-toolkit (3.0.5 -> 3.0.7)
  - Updating regex (2020.6.8 -> 2020.7.14)
  - Updating traitlets (4.3.3 -> 5.0.4)
  - Updating ipython (7.16.1 -> 7.18.1)
  - Updating pandas (1.0.5 -> 1.1.1)
  - Updating python-stdnum (1.13 -> 1.14)

All tests still pass according to pytest.
 2020-09-08 15:04:00 +03:00
Alan Orth db474a802f
README.md: Use badge from travis-ci.com 2020-08-04 11:12:28 +03:00
Alan Orth e241f8461b
CHANGELOG.md: Add notes 2020-07-06 14:10:46 +03:00
Alan Orth 431e6331c8
csv_metadata_quality/check.py: Format with black 2020-07-06 14:10:19 +03:00
Alan Orth cb07d357d4
Version 0.4.2 2020-07-06 14:04:34 +03:00
Alan Orth 65cd48a26f
CHANGELOG.md: Update changes 2020-07-06 14:00:21 +03:00
Alan Orth 0f883f640c
Remove pipenv 2020-07-06 13:59:49 +03:00
Alan Orth f4c5c5781e
README.md: Switch to poetry 2020-07-06 13:59:11 +03:00
Alan Orth 6aa784ad8c
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-07-06 13:57:07 +03:00
Alan Orth 7b8da94f41
poetry.lock: Update Python dependencies 2020-07-06 13:56:31 +03:00
Alan Orth 2a1566af62
csv_metadata_quality/check.py: Parameterize AGROVOC request 2020-07-06 13:44:46 +03:00
Alan Orth 5fcaa63bd5
csv_metadata_quality/check.py: Prune requests cache once 
We only need to prune the requests cache once before using it, not
for every value we check.
 2020-07-06 13:42:19 +03:00
Alan Orth aa9e23b46c
pyproject.toml: Update license specifier 
We need to use valid SPDX license identifiers.
 2020-06-09 14:22:53 +03:00
Alan Orth 73acb1661f
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-05-31 17:51:16 +03:00
Alan Orth 2a068fddc4
.build.yml: Fix test 2020-05-31 17:44:37 +03:00
Alan Orth c6c2f13e88
.build.yml: Fix poetry install invocation 
Poetry apparently installs dev dependencies by default.
 2020-05-31 17:37:09 +03:00
Alan Orth 56f16e37ed
.build.yml: Use poetry in SourceHut CI 2020-05-31 17:35:04 +03:00
Alan Orth 0c44b967b6
Add poetry project file and lock 
I want to try to use poetry instead of pipenv because pipenv takes
forever to do dependency resolution sometimes. Also, I have had a
few issues with Python modules like black that don't have releases
other than pre-releases, and even including the project itself in
the dependencies (pip install -e . ...?). My initial experience is
that poetry handles this better.
 2020-05-31 17:33:40 +03:00
Alan Orth 8a267bb40b
.travis.yml: Try to build with Python 3.8-dev 
But allow failures.
 2020-03-29 16:40:11 +03:00
Alan Orth 8fda8f1ef1
Pipfile.lock: Run pipenv update 
All tests still passing.
 2020-03-20 16:22:04 +02:00
Alan Orth 5e471813e8
CHANGELOG.md: Add note about python dependencies 2020-01-29 12:41:43 +02:00
Alan Orth 79244b9ac3
Pipfile.lock: Run pipenv update 2020-01-29 12:39:12 +02:00