Commit Graph

132 Commits

Author SHA1 Message Date
6f396f392f Revert "Move style.min.css to css/v1/style.min.css"
This reverts commit 8f6d337d2d.

We are using Heroku now so we don't need this phony version.
2021-03-14 20:40:33 +02:00
f292a4902f
.drone.yml: Install git for some pip deps
All checks were successful
continuous-integration/drone/push Build is passing
2021-03-14 19:32:14 +02:00
b43b995a90
Add drone.yml for Drone CI
Some checks failed
continuous-integration/drone/push Build is failing
2021-03-14 18:33:42 +02:00
aa862620d7
README.md: Try a relative link for screenshot.png 2021-03-14 16:15:21 +02:00
f42a83c7ab
Remove Google App Engine config 2021-03-14 16:07:26 +02:00
1d46d490cb
README.md: Add build badge from GitHub Actions 2021-03-14 15:57:59 +02:00
4f48af7f24
Add GitHub workflow to build
For now only builds, as I'm not sure how to test the web application
yet.
2021-03-14 15:55:58 +02:00
e8dc08bcac
README.md: Update text 2021-03-14 13:54:03 +02:00
f4ddf4a7b5
README.md: Update intro text 2021-03-14 13:11:39 +02:00
d37654206f
README.md: Fix screenshot link 2021-03-14 13:08:50 +02:00
69501cbacb
Add README.md with screenshot and license 2021-03-14 13:07:24 +02:00
2a9ec1c3f3
LICENSE.txt: Use GPLv3 instead of AGPLv3
I would rather have the source publishing requirements be triggered
on distribution than on web hosting.
2021-03-14 13:05:32 +02:00
be9143204c
Add Procfile for Heroku
See: https://devcenter.heroku.com/articles/python-gunicorn
2021-03-14 12:29:28 +02:00
55815cf4c0
runtime.txt: Use Python 3.9.2
Actually it seems they do have Python 3.9.2.

See: https://devcenter.heroku.com/articles/python-support
2021-03-14 12:25:49 +02:00
73a13145b6
Add runtime.txt
Apparently to deploy on Heroku we need this. And they only support
Python 3.7? Damn...
2021-03-14 12:24:03 +02:00
8f6d337d2d
Move style.min.css to css/v1/style.min.css
Trying to break Google App Engine's aggressive caching.
2021-03-14 12:04:28 +02:00
aabb783d99
app.yaml: Set Cache-Control header to private for CSS
Google App Engine agressively caches stuff. They are currently serving
a 24-hour old version of my CSS after multiple updates and re-deploys.
Ughhh. From their docs:

> After a file is transmitted with a given expiration time, there is
> generally no way to clear it out of web-proxy caches, even if the user
> clears their own browser cache. Re-deploying a new version of the app
> will not reset any caches. Therefore, if you ever plan to modify a
> static file, it should have a short (less than one hour) expiration
> time. In most cases, the default 10-minute expiration time is
> appropriate.

The only way to break this for now is to change the CSS *directory*.
In the future I think we have to be sure to set the private cache
control header, which lets browsers cache it, but not public CDNs.

See: https://cloud.google.com/appengine/docs/standard/python3/how-requests-are-handled
2021-03-14 11:58:57 +02:00
bd31ac912e
Regenerate static assets 2021-03-14 11:42:07 +02:00
61040ea4a5
Use an ILRI theme 2021-03-14 11:41:48 +02:00
d483f7fc0b
.gitignore: Ignore sqlite requests response cache 2021-03-14 11:36:46 +02:00
c07716abcb
.gcloudignore: Ignore sqlite requests response cache 2021-03-14 11:36:09 +02:00
36b43a06b9
templates/index.html: Add link to test.csv
Offer the user to test with this file.
2021-03-14 11:34:55 +02:00
25aa74ba14
app.yaml: Try to serve static CSS directly
Google App Engine is currently caching an old version of my CSS, so
I am trying to get it to use the correct version. Let's try serving
it directly from the filesystem.

See: https://cloud.google.com/appengine/docs/standard/php/getting-started/serving-static-files
2021-03-14 10:36:57 +02:00
a2ace2d331
.gcloudignore: Don't upload source dir
We only need this during build.
2021-03-14 10:31:46 +02:00
94cdd7fb50
Regenerate requirements
Generated using poetry:

  $ poetry export --without-hashes -f requirements.txt > requirements.txt
  $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
2021-03-14 10:09:18 +02:00
74bde23567
poetry.lock: Run poetry update 2021-03-14 10:08:44 +02:00
4e52d1bcc9
Add configurable requests cache directory
As I expected, on Google App Engine we can't write the cache file
to the current working directory. I modified csv-metadata-quality
CLI to check for the REQUESTS_CACHE_DIR environment variable so we
don't really have to do anything different other than setting the
variable.
2021-03-14 10:06:39 +02:00
e7dd8d1421
Add AGROVOC lookup support
This works locally, but I don't think it will work on App Engine
because csv-metadata-quality uses requests-cache and creates the
agrovoc-response-cache.sqlite file in the current working directory.
2021-03-13 23:49:24 +02:00
9aab2ae83f
templates/index.html: Remove basic and experimental labels 2021-03-13 23:34:34 +02:00
6c3804d55b
Add support for skipping fields ("-x") 2021-03-13 23:34:11 +02:00
122d9fd53c
Add support for experimental checks ("-e") 2021-03-13 23:01:11 +02:00
198acdb1a7
Major refactor
Re-work upload and file processing so they are in the same Python
function. Now I will start exposing other command line options in
the form, like unsafe fixes, excluding fields, etc. Now I see tha
t it is easier to save the POSTed file and process it in the same
function so I don't have to pass around the other POSTed form val
ues as URL query parameters.

Now, as a result of changing the flow above, I also had to make a
change to the way I show the results page. Instead of processing
the file and returning the rendered results to the user directly,
I process the file, save the rendered results to /tmp, and return
a redirect to the user to the results page.
2021-03-13 22:11:26 +02:00
adc2d06094
main.py: Add note about using /tmp
I originally wanted to use an "uploads" directory or something, but
it seems we can only write to /tmp on Google App Engine. They really
want you to buy storage or database services! This is memory mapped
so it disappears when you re-deploy.
2021-03-13 22:09:17 +02:00
bc256b242d
static/css/style.min.css: Regenerate 2021-03-13 22:07:53 +02:00
f82cb6ce05
source/scss/main.scss: Increase container width
We need more space for the log on the results page and we actually
don't even need to worry about people running this on a phone.
2021-03-13 22:06:43 +02:00
4bdec3b889
app.yaml: Use Python 3.9
Python 3.9 is apparently now generally available.
2021-03-13 14:12:14 +02:00
f79be86361
templates/index.html: Use Bootstrap form components 2021-03-13 13:50:46 +02:00
3715c5e976
Use new commit for csv-metadata-quality
This one doesn't treat the fixing of invalid multi-value separators
as "unsafe".
2021-03-13 13:02:47 +02:00
8603ec4bca
main.py: Rework command line args
Turns out we only need to use sys.argv when we were trying to run
the csv-metadata-quality module directly in Python by importing it.
2021-03-13 12:52:22 +02:00
0471820f3a
main.py: Actually use sys.argv
I set this but never actually passed it to the subprocess. Now I'm
wondering if I actually need it, or if that was just when I was tr-
ying to import the csv-metadata-quality module?
2021-03-13 12:49:01 +02:00
f1de00d7ca
Regenerate requirements
Generated using poetry:

  $ poetry export --without-hashes -f requirements.txt > requirements.txt
  $ poetry export --without-hashes --dev -f requirements.txt > requirements-dev.txt
2021-03-12 23:11:33 +02:00
31e699bb8b
poetry.lock: Sync lockfile 2021-03-12 23:10:55 +02:00
982b47459c
Use gunicorn to run on Google App Engine
See: https://cloud.google.com/appengine/docs/flexible/python/quickstart
2021-03-12 23:10:29 +02:00
38ace23f97
.gitignore: Add __pycache__ 2021-03-12 23:09:29 +02:00
e5f57802f3
Don't ignore uploads directory
We're using /tmp now since we apparently can't create and upload to
arbitrary directories on Google App Engine (?).
2021-03-12 23:08:28 +02:00
69abc3d764
Add downloading of cleaned CSV file 2021-03-12 23:00:48 +02:00
37cee1d3f0
Add configuration for isort
See: https://sourcery.ai/blog/python-best-practices/
2021-03-12 22:59:57 +02:00
126bf74ea0
main.py: Remove comments 2021-03-12 21:28:24 +02:00
54ea340b80
poetry.lock: Run poetry update 2021-03-12 21:24:51 +02:00
75c5e48aa3
templates/header.html: Link to home in navbar 2021-03-12 21:24:31 +02:00