deadlink-crawler

commit cc5c2daedf04d7b2d16a717c0c54547e4062bc80
parent 7fef91a7208c5e7a2665f6355dc6439428e93b41
Author: Stefan <stefan@eliteinformatiker.de>
Date:   Thu, 24 Jan 2013 14:43:19 +0100

improved README a bit

Diffstat:
M
README.md
 | 
5
++++-

1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/README.md b/README.md
@@ -15,8 +15,11 @@ Additional options available are:
 
 ```bash
 # Crawl all subsites of http://stefan-koch.name/ for deadlinks (including external deadlinks)
+# Wait one second between opening each URL
 python2.7 crawler.py --wait 1 --restrict http://stefan-koch.name/.* http://stefan-koch.name/
-# Crawl all article pages of example.com for deadlinks. We assume that there are linked articles on the main page
+
+# Crawl all article pages of example.com for deadlinks.
+# We assume that there are linked articles on the main page
 python2.7 crawler.py --restrict http://example.com/article/.+ http://example.com/
 ```