alanorth/cgspace-notes
1
0
Fork 0
mirror of https://github.com/alanorth/cgspace-notes.git synced 2025-01-27 05:49:12 +01:00
Code Issues Projects Releases Wiki Activity

Update notes for 2019-11-14

Browse Source
This commit is contained in:
Alan Orth
2019-11-14 09:47:49 +02:00
parent 0848710257
commit 519d9c84a9
3 changed files with 40 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
content/posts/2019-11.md
View File

@ -288,5 +288,19 @@ $ http "http://localhost:8081/solr/statistics/select?q=userAgent:/Scrapoo\/[0-9]
```
- Nice, so searching with regex in Solr with `//` syntax works for those digits!
- I realized that it's easier to search Solr from curl via POST using this syntax:
```
$ curl -s "http://localhost:8081/solr/statistics/select" -d "q=userAgent:*Scrapoo*&rows=0")
```
- If the parameters include something like "[0-9]" then curl interprets it as a range and will make ten requests
- You can disable this using the `-g` option, but there are other benefits to searching with POST, for example it seems that I have less issues with escaping special parameters when using Solr's regex search:
```
$ curl -s 'http://localhost:8081/solr/statistics/select' -d 'q=userAgent:/Postgenomic(\s|\+)v2/&rows=2'
```
- I updated the `check-spider-hits.sh` script to use the POST syntax, and I'm evaluating the feasability of including the regex search patterns from the spider agent file, as I had been filtering them out due to differences in PCRE and Solr regex syntax and issues with shell handling
<!-- vim: set sw=2 ts=2: -->