msRoot:~ ioannis$ echo "ls -l" | at midnight
job 2 at Tue Aug 16 00:00:00 2011
wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com/
<b>Download an entire website</b>
-p parameter tells wget to include all files, including images.
-e robots=off you don't want wget to obey by the robots.txt file
-U mozilla as your browsers identity.
--random-wait to let wget chose a random number of seconds to wait, avoid get into black list.
Other Useful wget Parameters:
--limit-rate=20k limits the rate at which it downloads files.
-b continues wget after logging out.
-o $HOME/wget_log.txt logs the output
----------------------------------------------------------------------
cal
msRoot:~ ioannis$ cal 2011
job 2 at Tue Aug 16 00:00:00 2011
wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com/
<b>Download an entire website</b>
-p parameter tells wget to include all files, including images.
-e robots=off you don't want wget to obey by the robots.txt file
-U mozilla as your browsers identity.
--random-wait to let wget chose a random number of seconds to wait, avoid get into black list.
Other Useful wget Parameters:
--limit-rate=20k limits the rate at which it downloads files.
-b continues wget after logging out.
-o $HOME/wget_log.txt logs the output
----------------------------------------------------------------------
cal
msRoot:~ ioannis$ cal 2011
No comments:
Post a Comment