alanorth/csv-metadata-quality
1
0
Fork 0
mirror of https://github.com/ilri/csv-metadata-quality.git synced 2024-11-25 23:28:18 +01:00
Code Releases Activity
312 Commits 4 Branches 17 Tags 2 MiB
3dbe656f9f
Commit Graph

312 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alan Orth
32cea2055f
data/test.csv: Add unnecessary multi-value separator 2021-01-03 15:33:04 +02:00
Alan Orth
0dc66c5c4e
Expand check/fix for multi-value separators 
I just came across some metadata that had unnecessary multi-value
separators at the end of a field, causing a blank value to be used.

For example: "Kenya||Tanzania||"
 2021-01-03 15:30:03 +02:00
Alan Orth
c26ad83534
.github: Test CLI invocation 2020-12-14 23:47:09 +02:00
Alan Orth
72ca9d99bf
setup.py: Add Python 3.9 
[SKIP CI]
 2020-12-14 23:44:35 +02:00
Alan Orth
ae33a9b793
Add .drone.yml 2020-12-14 23:42:23 +02:00
Alan Orth
fc0367bfc8
README.md: Update note about Python version 2020-12-08 10:52:24 +02:00
Alan Orth
e33b285034
README.md: Add GitHub Actions badge 2020-12-08 10:48:31 +02:00
Alan Orth
349fca03b8
.github/workflows/python-app.yml: Rename 
This name is displayed in the badge so it should be something more
relevant.
 2020-12-08 10:46:39 +02:00
Alan Orth
52d8904870
Remove .travis.yml 
They changed their free tier and I might as well use GitHub Actions
for ILRI stuff anyways.
 2020-12-08 10:41:36 +02:00
Alan Orth
971c69e535
Create python-app.yml 
Try GitHub Actions for Python 3.8 using GitHub's Python example.
 2020-12-08 10:38:52 +02:00
Alan Orth
f8cc233e25
.travis.yml: Use Amazon Graviton2 ARM environment 
These are the new hotness and should have faster build times.

See: https://blog.travis-ci.com/2020-09-11-arm-on-aws
 2020-12-06 10:49:03 +02:00
Alan Orth
aa7b7a9592
Update requirements 
Generated with poetry export:

    $ poetry export --without-hashes -f requirements.txt > requirements.txt
    $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt

I am trying `--without-hashes` to work around an error on pip install
when running on Python 3.6 in Travis:

    ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==.
 2020-11-03 07:42:45 +02:00
Alan Orth
57b455bde7
poetry.lock: Run poetry update 2020-11-03 07:40:56 +02:00
Alan Orth
23b95fa368
.travis.yml: Use Ubuntu 20.04 "Focal" environment 2020-10-29 00:14:54 +03:00
Alan Orth
6985f76aa3
.travis.yml: Bump Python versions 
Test Python 3.9 now that it was released, and allow tests to fail
on nightly builds.
 2020-10-29 00:14:36 +03:00
Alan Orth
98a6a19e12
Update requirements-dev.txt 
Generated with poetry export:

    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:48:46 +03:00
Alan Orth
f4914c414f
Only install ipython on Python 3.7+ 2020-10-06 17:48:16 +03:00
Alan Orth
d352fe8017
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-10-06 17:21:33 +03:00
Alan Orth
f13c360084
Update poetry package dependencies 2020-10-06 17:20:16 +03:00
Alan Orth
7cfd4c0b59
csv_metadata_quality: Move scoped imports to global 
According to PEP8 we should avoid scoped imports unless you have a
good reason. Here there are two cases where we do (issn and isbn),
but I will move the others to the global scope.
 2020-10-06 17:11:39 +03:00
Alan Orth
826509ddcf
poetry.lock: Run poetry update 
List of updated modules:

  - Updating numpy (1.19.1 -> 1.19.2)
  - Updating pygments (2.6.1 -> 2.7.1)
  - Updating pandas (1.1.1 -> 1.1.2)

All tests still pass according to pytest.
 2020-09-26 12:18:23 +03:00
Alan Orth
22b5c0f7a1
CHANGELOG.md: Add note about dependencies update 2020-09-08 15:04:40 +03:00
Alan Orth
774e274b32
poetry.lock: Run poetry update 
Update dependencies to latest version:

  - Updating attrs (19.3.0 -> 20.2.0)
  - Updating more-itertools (8.4.0 -> 8.5.0)
  - Updating openpyxl (3.0.4 -> 3.0.5)
  - Updating parso (0.7.0 -> 0.7.1)
  - Updating sqlalchemy (1.3.18 -> 1.3.19)
  - Updating urllib3 (1.25.9 -> 1.25.10)
  - Updating agate-dbf (0.2.1 -> 0.2.2)
  - Updating agate-sql (0.5.4 -> 0.5.5)
  - Updating jedi (0.17.1 -> 0.17.2)
  - Updating numpy (1.19.0 -> 1.19.1)
  - Updating prompt-toolkit (3.0.5 -> 3.0.7)
  - Updating regex (2020.6.8 -> 2020.7.14)
  - Updating traitlets (4.3.3 -> 5.0.4)
  - Updating ipython (7.16.1 -> 7.18.1)
  - Updating pandas (1.0.5 -> 1.1.1)
  - Updating python-stdnum (1.13 -> 1.14)

All tests still pass according to pytest.
 2020-09-08 15:04:00 +03:00
Alan Orth
db474a802f
README.md: Use badge from travis-ci.com 2020-08-04 11:12:28 +03:00
Alan Orth
e241f8461b
CHANGELOG.md: Add notes 2020-07-06 14:10:46 +03:00
Alan Orth
431e6331c8
csv_metadata_quality/check.py: Format with black 2020-07-06 14:10:19 +03:00
Alan Orth
cb07d357d4
Version 0.4.2 2020-07-06 14:04:34 +03:00
Alan Orth
65cd48a26f
CHANGELOG.md: Update changes 2020-07-06 14:00:21 +03:00
Alan Orth
0f883f640c
Remove pipenv 2020-07-06 13:59:49 +03:00
Alan Orth
f4c5c5781e
README.md: Switch to poetry 2020-07-06 13:59:11 +03:00
Alan Orth
6aa784ad8c
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-07-06 13:57:07 +03:00
Alan Orth
7b8da94f41
poetry.lock: Update Python dependencies 2020-07-06 13:56:31 +03:00
Alan Orth
2a1566af62
csv_metadata_quality/check.py: Parameterize AGROVOC request 2020-07-06 13:44:46 +03:00
Alan Orth
5fcaa63bd5
csv_metadata_quality/check.py: Prune requests cache once 
We only need to prune the requests cache once before using it, not
for every value we check.
 2020-07-06 13:42:19 +03:00
Alan Orth
aa9e23b46c
pyproject.toml: Update license specifier 
We need to use valid SPDX license identifiers.
 2020-06-09 14:22:53 +03:00
Alan Orth
73acb1661f
Update requirements 
Generated with poetry export:

    $ poetry export -f requirements.txt > requirements.txt
    $ poetry export --dev -f requirements.txt > requirements-dev.txt
 2020-05-31 17:51:16 +03:00
Alan Orth
2a068fddc4
.build.yml: Fix test 2020-05-31 17:44:37 +03:00
Alan Orth
c6c2f13e88
.build.yml: Fix poetry install invocation 
Poetry apparently installs dev dependencies by default.
 2020-05-31 17:37:09 +03:00
Alan Orth
56f16e37ed
.build.yml: Use poetry in SourceHut CI 2020-05-31 17:35:04 +03:00
Alan Orth
0c44b967b6
Add poetry project file and lock 
I want to try to use poetry instead of pipenv because pipenv takes
forever to do dependency resolution sometimes. Also, I have had a
few issues with Python modules like black that don't have releases
other than pre-releases, and even including the project itself in
the dependencies (pip install -e . ...?). My initial experience is
that poetry handles this better.
 2020-05-31 17:33:40 +03:00
Alan Orth
8a267bb40b
.travis.yml: Try to build with Python 3.8-dev 
But allow failures.
 2020-03-29 16:40:11 +03:00
Alan Orth
8fda8f1ef1
Pipfile.lock: Run pipenv update 
All tests still passing.
 2020-03-20 16:22:04 +02:00
Alan Orth
5e471813e8
CHANGELOG.md: Add note about python dependencies 2020-01-29 12:41:43 +02:00
Alan Orth
79244b9ac3
Pipfile.lock: Run pipenv update 2020-01-29 12:39:12 +02:00
Alan Orth
5e81a33482
CHANGELOG.md: Add note about field names 2020-01-16 12:37:11 +02:00
Alan Orth
28b5996aa6
Output field name for more fixes and checks 
This helps identify which field has the error.
 2020-01-16 12:35:11 +02:00
Alan Orth
40ba9bae6c
README.md: Adjust heading size 2020-01-15 12:26:11 +02:00
Alan Orth
0b2d211455
Version 0.4.1 2020-01-15 12:19:42 +02:00
Alan Orth
7f1df0b47c
Support Python 3.6 and 3.7 again 2020-01-15 12:19:17 +02:00
Alan Orth
365ecda324
Add utility function to check normalization 
Python's built-in unicodedata library includes the is_normalized()
function starting with Python 3.8. This utility function allows us
to do the same thing with earlier Python versions.

See: https://docs.python.org/3/library/unicodedata.html
 2020-01-15 12:17:52 +02:00