Beginners tutorial for Wget

From now on , i will start a series of tutorial on advanced linux commands... planning to make the tutorials simple and easy


Here goes.. first tutorial is on wget...




WGET


1.What is wget ?


wget is a superb utility for  downloading files from the Web. 
its really cool , versastile and very poweful and provides so many options to download files from a server.


Its available free for many platforms including windows... ok ok , lets get started....



2.what protocols supported ?


  It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies


3. Some Important notes on  Wget's features


  I am listing some basic and important features users should know about wget :

  1. Wget  can work in the background, so users can write scripts to automate downloads , schedule these scripts and whatever creative they can go for...            
  2.  Wget can download entire pages  for offline viewing.                                
  3.  Wget  can restart broken downloads



4. Syntax


   wget [OPTION]... [URL]...




5.Examples


Let me explain first with basic examples and goto complex options later...



1. To download a basic web page



Type wget http://harisoft.net.tf

this will download the webpageof http://harisoft.net.tf and save to disk.



Output:





b.Recursive Download


The "-r" option allows wget to download a resource, search that content for links to other resources, and then download those resources. 

 wget -r  http://pcideas.blogspot.com/


will download all pages and save it to disk



c) Resume Downloads


 You can continue previousl interruped downloads (broken downloads ) using the -c option. 



wget -c <download address that was interrupted>



d)To Save download to a different name


use -O option


wget -O myweb.html pcideas.blogspot.com

this will save the download  to  myweb.html



e)Mirror download the entire site


using -m option you can download the entire site to disk


the documentation of wget says that -m option will  " Turn on options suitable for mirroring. This option turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. It is currently equivalent to -r -N -l inf -nr. "


 wget -m  http://pcideas.blogspot.com/

f)Download in the Background

For  big downloads, use the wget  -b option  as shown below

 wget -b  http://pcideas.blogspot.com/


now download will start in background and  Output will be written to `wget-log'
 then you can examine the  log file using more to check everything has downloaded correctly.


more wget-log






I hope this tutorial will give an basic introduction to beginners for using wget, for more options refer the GNU documentation for WGET.






Comments

Post a Comment

Popular posts from this blog

How to upgrade Huawei WA1003A to a Feature Rich Firmware

Image
I am one of those guys who still have the Huaweii WA1003a Router provided by BSNL and its still working perfectly (guess a solid build) , but its nearly 7 years passed and  no new firmware update was provided , on asking for newer version of firmware in huawei forums... the reply was shocking as one of Huawei staff replied "Sorry , Quidway 1003A is a terminal product not a DATACOM router."(Source: http://forum.huawei.com/jive4/thread.jspa?threadID=319714&tstart=1&orderStr=9). Ok, Finally i decided to put Third party Feature Rich firmware called   RouterTech Firmware The upgrade process is a little difficult , but people who have knowledge of linux commands should handle this fairly easily. THIS UPGRADE PROCESS IS NOT FOR NOVICES AND ANY MISTAKES MAY BRICK THE ROUTER AND MAKE IT USELESS. ALSO , MAKE SURE YOU HAVE ACCESS TO OTHER  MEANS OF INTERNET CONNECTION LIKE MOBILE GPRS / 3G AVAILABLE  IN THE EVENT OF YOU BRICKING THE ROUTER... YOU CAN USE ...
Read more

Converting OVA file to VMX file

Ever downloaded OVA files and found that your virtualization player refuses to open it , then you should use this simple utility called OVF Tool. for example , if u want convert ova type file to vmware file ,  u can use a command like this Step 1: download ovftool from here http://communities.vmware.com/community/vmtn/automationtools/ovf step 2: Since this is a command line tool , after installation , go to the installation directory and type ovftool if command is recognized , then ovftool works step 3: conversion u can use the ovftool like this as shown below assuming srcfile.ova - is source file , which is in ova format destfile_name.vmx - is destination file , which should be in vmx format usage : ovftool srcfile.ova destfile_name.vmx
Read more

A simple and easy Spark standalone program for beginners.

A simple and easy Spark standalone program for beginners.... Courtsey: Hariprasad Bhaskaran Aim : To demonstrate writing a simple spark scala program use sbt to build it create dependencies for spark-core and spark-sql in build.sbt create a final jar that can be submitted using spark-submit... Program code below: MyAdd2.scala ================= import org.apache.spark.SparkContext import org.apache.spark.SparkConf import org.apache.spark.sql._ import org.apache.spark.sql.hive.HiveContext import org.apache.spark.sql.hive._ import org.apache.spark.sql.SQLContext import org.apache.spark.sql.SQLContext._ object MyAdd2 { def main(args: Array[String] ) { val conf = new SparkConf().setAppName("t1").setMaster("local") val sc2 = new SparkContext(conf) val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc2) import sqlContext.implicits._ val mylist = sc2.parallelize(List( 1 ,2,3,4,5,6,7,8,9,100) ) val mylistDF = mylist.map( x =...
Read more