mirror of
https://github.com/alanorth/cgspace-notes.git
synced 2024-11-25 08:00:18 +01:00
Update notes for 2018-11-05
This commit is contained in:
parent
6f561ce4b5
commit
9d81dc3176
@ -232,5 +232,14 @@ $ grep -c -E 'session_id=[A-Z0-9]{32}:ip_addr=2a03:2880:11ff' dspace.log.2018-11
|
|||||||
|
|
||||||
- I added the "most-popular" pages to the list that return `X-Robots-Tag: none` to try to inform bots not to index or follow those pages
|
- I added the "most-popular" pages to the list that return `X-Robots-Tag: none` to try to inform bots not to index or follow those pages
|
||||||
- Also, I implemented an nginx rate limit of twelve requests per minute on all dynamic pages... I figure a human user might legitimately request one every five seconds
|
- Also, I implemented an nginx rate limit of twelve requests per minute on all dynamic pages... I figure a human user might legitimately request one every five seconds
|
||||||
|
- I wrote a small Python script [add-dc-rights.py](https://gist.github.com/alanorth/4ff81d5f65613814a66cb6f84fdf1fc5) to add usage rights (`dc.rights`) to CGSpace items based on the CSV Hector gave me from MARLO:
|
||||||
|
|
||||||
|
```
|
||||||
|
$ ./add-dc-rights.py -i /tmp/marlo.csv -db dspace -u dspace -p 'fuuu'
|
||||||
|
```
|
||||||
|
|
||||||
|
- The file `marlo.csv` was cleaned up and formatted in Open Refine
|
||||||
|
- 165 of the items in their 2017 data are from CGSpace!
|
||||||
|
- I will add the data to CGSpace this week
|
||||||
|
|
||||||
<!-- vim: set sw=2 ts=2: -->
|
<!-- vim: set sw=2 ts=2: -->
|
||||||
|
@ -23,7 +23,7 @@ Today these are the top 10 IPs:
|
|||||||
" />
|
" />
|
||||||
<meta property="og:type" content="article" />
|
<meta property="og:type" content="article" />
|
||||||
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2018-11/" /><meta property="article:published_time" content="2018-11-01T16:41:30+02:00"/>
|
<meta property="og:url" content="https://alanorth.github.io/cgspace-notes/2018-11/" /><meta property="article:published_time" content="2018-11-01T16:41:30+02:00"/>
|
||||||
<meta property="article:modified_time" content="2018-11-04T12:18:52+02:00"/>
|
<meta property="article:modified_time" content="2018-11-04T22:45:00+02:00"/>
|
||||||
|
|
||||||
<meta name="twitter:card" content="summary"/>
|
<meta name="twitter:card" content="summary"/>
|
||||||
<meta name="twitter:title" content="November, 2018"/>
|
<meta name="twitter:title" content="November, 2018"/>
|
||||||
@ -52,9 +52,9 @@ Today these are the top 10 IPs:
|
|||||||
"@type": "BlogPosting",
|
"@type": "BlogPosting",
|
||||||
"headline": "November, 2018",
|
"headline": "November, 2018",
|
||||||
"url": "https://alanorth.github.io/cgspace-notes/2018-11/",
|
"url": "https://alanorth.github.io/cgspace-notes/2018-11/",
|
||||||
"wordCount": "992",
|
"wordCount": "1057",
|
||||||
"datePublished": "2018-11-01T16:41:30+02:00",
|
"datePublished": "2018-11-01T16:41:30+02:00",
|
||||||
"dateModified": "2018-11-04T12:18:52+02:00",
|
"dateModified": "2018-11-04T22:45:00+02:00",
|
||||||
"author": {
|
"author": {
|
||||||
"@type": "Person",
|
"@type": "Person",
|
||||||
"name": "Alan Orth"
|
"name": "Alan Orth"
|
||||||
@ -368,6 +368,16 @@ Today these are the top 10 IPs:
|
|||||||
<ul>
|
<ul>
|
||||||
<li>I added the “most-popular” pages to the list that return <code>X-Robots-Tag: none</code> to try to inform bots not to index or follow those pages</li>
|
<li>I added the “most-popular” pages to the list that return <code>X-Robots-Tag: none</code> to try to inform bots not to index or follow those pages</li>
|
||||||
<li>Also, I implemented an nginx rate limit of twelve requests per minute on all dynamic pages… I figure a human user might legitimately request one every five seconds</li>
|
<li>Also, I implemented an nginx rate limit of twelve requests per minute on all dynamic pages… I figure a human user might legitimately request one every five seconds</li>
|
||||||
|
<li>I wrote a small Python script <a href="https://gist.github.com/alanorth/4ff81d5f65613814a66cb6f84fdf1fc5">add-dc-rights.py</a> to add usage rights (<code>dc.rights</code>) to CGSpace items based on the CSV Hector gave me from MARLO:</li>
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
<pre><code>$ ./add-dc-rights.py -i /tmp/marlo.csv -db dspace -u dspace -p 'fuuu'
|
||||||
|
</code></pre>
|
||||||
|
|
||||||
|
<ul>
|
||||||
|
<li>The file <code>marlo.csv</code> was cleaned up and formatted in Open Refine</li>
|
||||||
|
<li>165 of the items in their 2017 data are from CGSpace!</li>
|
||||||
|
<li>I will add the data to CGSpace this week</li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<!-- vim: set sw=2 ts=2: -->
|
<!-- vim: set sw=2 ts=2: -->
|
||||||
|
@ -4,7 +4,7 @@
|
|||||||
|
|
||||||
<url>
|
<url>
|
||||||
<loc>https://alanorth.github.io/cgspace-notes/2018-11/</loc>
|
<loc>https://alanorth.github.io/cgspace-notes/2018-11/</loc>
|
||||||
<lastmod>2018-11-04T12:18:52+02:00</lastmod>
|
<lastmod>2018-11-04T22:45:00+02:00</lastmod>
|
||||||
</url>
|
</url>
|
||||||
|
|
||||||
<url>
|
<url>
|
||||||
@ -194,7 +194,7 @@
|
|||||||
|
|
||||||
<url>
|
<url>
|
||||||
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
<loc>https://alanorth.github.io/cgspace-notes/</loc>
|
||||||
<lastmod>2018-11-04T12:18:52+02:00</lastmod>
|
<lastmod>2018-11-04T22:45:00+02:00</lastmod>
|
||||||
<priority>0</priority>
|
<priority>0</priority>
|
||||||
</url>
|
</url>
|
||||||
|
|
||||||
@ -205,7 +205,7 @@
|
|||||||
|
|
||||||
<url>
|
<url>
|
||||||
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
<loc>https://alanorth.github.io/cgspace-notes/tags/notes/</loc>
|
||||||
<lastmod>2018-11-04T12:18:52+02:00</lastmod>
|
<lastmod>2018-11-04T22:45:00+02:00</lastmod>
|
||||||
<priority>0</priority>
|
<priority>0</priority>
|
||||||
</url>
|
</url>
|
||||||
|
|
||||||
@ -217,13 +217,13 @@
|
|||||||
|
|
||||||
<url>
|
<url>
|
||||||
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
<loc>https://alanorth.github.io/cgspace-notes/posts/</loc>
|
||||||
<lastmod>2018-11-04T12:18:52+02:00</lastmod>
|
<lastmod>2018-11-04T22:45:00+02:00</lastmod>
|
||||||
<priority>0</priority>
|
<priority>0</priority>
|
||||||
</url>
|
</url>
|
||||||
|
|
||||||
<url>
|
<url>
|
||||||
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
<loc>https://alanorth.github.io/cgspace-notes/tags/</loc>
|
||||||
<lastmod>2018-11-04T12:18:52+02:00</lastmod>
|
<lastmod>2018-11-04T22:45:00+02:00</lastmod>
|
||||||
<priority>0</priority>
|
<priority>0</priority>
|
||||||
</url>
|
</url>
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user