alanorth/csv-metadata-quality
1
0
Fork 0
mirror of https://github.com/ilri/csv-metadata-quality.git synced 2024-11-22 05:45:02 +01:00
Code Releases Activity

csv_metadata_quality/check.py: requests cache

Browse Source 
Allow overriding the directory for the requests cache. In the case
of csv-metadata-quality-web, which currently runs on Google's App
Engine, we can only write to /tmp.
This commit is contained in:
Alan Orth 2021-03-14 09:07:35 +02:00
parent 1008acf35e
commit 0e9176f0a6
Signed by: alanorth
GPG Key ID: 0FB860CC9C45B1B9
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
csv_metadata_quality/check.py
View File

@ -1,3 +1,4 @@
import os
import re import re
from datetime import datetime, timedelta from datetime import datetime, timedelta
@ -242,7 +243,11 @@ def agrovoc(field, field_name):
# enable transparent request cache with thirty days expiry # enable transparent request cache with thirty days expiry
expire_after = timedelta(days=30) expire_after = timedelta(days=30)
requests_cache.install_cache("agrovoc-response-cache", expire_after=expire_after) # Allow overriding the location of the requests cache, just in case we are
# running in an environment where we can't write to the current working di-
# rectory (for example from csv-metadata-quality-web).
REQUESTS_CACHE_DIR = os.environ.get("REQUESTS_CACHE_DIR", ".")
requests_cache.install_cache(f"{REQUESTS_CACHE_DIR}/agrovoc-response-cache", expire_after=expire_after)
# prune old cache entries # prune old cache entries
requests_cache.core.remove_expired_responses() requests_cache.core.remove_expired_responses()