How to Download Your Website Using WGET for Windows

by Richard Baxter on 3rd February 2009

Ever had that terrifying feeling you’ve lost your blog? Perhaps your WordPress installation got hacked, or you’ve got to move to a new web host because your old ones screwed up with a “database upgrade”. Either way there’s an almost infinite array of reasons to download and backup a copy of your website, and precisely zero reasons to neglect doing it.

If you’re a Linux user, there are lots of guides out there on how to use WGET, the free network utility to retrieve files from the World Wide Web using HTTP and FTP, but no guides to doing so with Windows. Unless you fancy installing Ubuntu or Crunchbang, here’s a handy guide to downloading your site using WGET in Windows.

Summary: Here’s how to install and use WGET in Windows

Download WGET.
Make WGET a command you can run from any directory in Command Prompt.
Restart the command terminal and test WGET.
Make a directory to download your site to.
Use the commands listed in this article to download your site.

Down to the details – get started:

Download WGET

Download and save WGET to your desktop. You can get wget.exe here. I recommend downloading WGET for Windows (win32) from Ugent.be as it’s the most up to date version I could find. For info, you can also get WGET from Brothersoft. Avoid the WGET for Windows (http://gnuwin32.sourceforge.net/packages/wget.htm) download page, because their installer doesn’t work with Windows versions later than Vista!

Make WGET a command you can run from any directory in Command Prompt

If you want to be able to run WGET from any directory inside the command terminal, you’ll need to learn about the path command to work out where to copy your new executable.

First, open a command terminal by selecting “run” in the start menu (if you’re using Windows XP) and typing “cmd”. If you’re running Windows Vista go to “All Programs > Accessories > Command Prompt” from the start bar.

You’ll see something that looks like this:

We’re going to move wget.exe into a Windows directory that will allow WGET to be run from anywhere. First, we need to find out which directory that should be. Type:

path

into the command prompt to find out:

Thanks to the “Path” environment variable, we know that we need to copy wget.exe to either the C:WindowsSystem32 directory or the C:Windows directory.

Go ahead and copy WGET to either of the directories you see in your Command Terminal.

Restart the command terminal and test WGET

If you want to test WGET is working properly, restart your terminal and type:

wget -h

If you’ve copied the file to the right place, you’ll see a help file appear with all of the available commands

Make a directory to download your site to

Seeing that we’ll be working in Command Prompt, let’s create a download directory just for WGET downloads. *If you’re familiar with Command Terminal basics, just skip this step. Change to the C: and use to make a directory:

md

Change to your new directory and you’re ready to do some downloading!

cd site-download

Use these commands to download your site using WGET

Ok, the fun bit begins. Once you’ve got WGET installed and you’ve created a new directory, all you have to do is learn some of the finer points of WGET arguments to make sure you get what you need.

I found two particularly useful resources for WGET usage. The Gnu.org WGET Manual and About.com’s Linux WGET guide are definitely the best.

After some research I came up with a set of instructions to WGET to recursively mirror your site, download all the images, CSS and JavaScript, localise all of the URLS (so the site works on your local machine), and save all the pages as a .html file.

To mirror your site execute this command:

wget -r http://www.yoursite.com

To mirror the site and localise all of the urls:

wget --convert-links -r http://www.yoursite.com

To mirror the site and save the files as .html:

wget --html-extension -r http://www.yoursite.com

Stop WGET from being blocked

Some web servers are set up to deny WGET’s default user agent – for obvious, bandwidth saving reasons. You could try changing your user agent to get round this. Try er, pretending to be Googlebot:

wget --user-agent="Googlebot/2.1 (+http://www.googlebot.com/bot.html)" -r http://www.yoursite.com

And finally, here’s WGET downloading my website:

On that last note, lots of hosting companies block WGET. Mine included! Took me a while to be able to back my own site up but now, I feel pretty safe that I have backups of the database, the plugins, the images and even the HTML of the site itself.

Like this? Sign up to read more

Responses

Rarst 4th February 2009 at 20:40

That’s quite a twist on site backup. :) I use less inventive way – Cobian Backup to backup folders from my FTP account. Fetches only needed folders (theme, plugins, images).
Matthew 4th February 2009 at 23:04

nice article and wget is very useful tool to be aware of. You might also like Unison.
Gab Goldenberg 4th February 2009 at 23:24

Nifty technique and unique guide here Richard. Way to contribute to the community.

2 points:
1) Recursively mirror your site – huh? Try that again in English please, Mr. Englishman ;P
2) How do Mac users do this?
richardbaxterseo 6th February 2009 at 09:21

Hey Gab – ok – the -r command is the mirror command. Recursively, follow all links. As for the MAC – no idea dude! I keep meaning to get hold of a MAC to learn how. If I come across the answer, I’ll post it here. Thanks for dropping by!
Dan 23rd March 2009 at 02:08

This works for me: wget -e robots=off -E -r -k -l inf -p –restrict-file-names=windows -H -K -D [Your Blog Name].wordpress.com,[Your Blog Name].files.wordpress.com –random-wait http://[Your Blog Name].wordpress.com
Owen English 7th April 2009 at 09:24

Hey can anyone help. Trying to download files using wget v 1.10.2 from the command prompt gives this (filenames blanked for commercial reasons):

–2009-04-07 07:53:52– http://www.medistat-software.net/*******/******.***
Resolving http://www.medistat-software.net... seconds 0.00, 77.92.81.1
Caching http://www.medistat-software.net => 77.92.81.1
Connecting to http://www.medistat-software.net|77.92.81.1|:80… seconds 0.00, Closed fd 1936
failed: Connection timed out.
Releasing 0x009259d8 (new refcount 1).
Retrying.

This actually works from about 75% of my clients but the other 25% get this error.

Help – what does it mean/ What is ‘Closed fd 1936’ ??

Cheers Owen E
Ivan 30th August 2009 at 20:16

Hello,

very nice article indeed. However, it does not help me download my site or any site. I copied and pasted all the commands here with the same result. Only the index file and a js file (http://jscook.sourceforge.net/JSCookMenu/). Why can't I download the site?

Please help.
richardbaxter 30th August 2009 at 20:39

I've found that some sites don't respond correctly unless you add a user agent to the request. Have you tried that?
Ivan 30th August 2009 at 21:32

How do I add an user agent and what is it?

I cannot download my site only with wget. WinHTTrack did it. Other sites can be downloaded with wget, but mine nada.

Here is the url: http://www.all-e-services.com (still in development phase)

Thank you!
richardbaxter 30th August 2009 at 21:43

No problem Ivan,

I suggest you follow point 6 in the post, as it may be your web host is blocking WGET's standard user agent.

Good luck!
Darius 6th May 2010 at 19:17

Suggested wget download website
is corrupt

http://users.ugent.be/~bpuype/wget/#download

Hidden installation under Vista
no apps, no interface

Can you check it under Vista ?
randomstranger 13th May 2010 at 12:16

Thanks Dan, that totally worked!
Manali 14th July 2010 at 09:07

It helped me in all ways… Details are informative…
Thanks!
thank 19th July 2010 at 12:33

i’m saving a site that’s about to be deleted as i type!

thanks for sharing.
Cameron Fraser 1st August 2010 at 15:05

the site I’m trying to archive has “?” in the links and wget saves files on windows replacing
the ? with “@” – but it leaves the links with “?” so the links don’t match the filenames.
Also “@” is problematic since it looks like an email link to the browser. Any workaround
for this?
thanks
HP Bryce 3rd September 2010 at 08:25

This looks like a great 11th hour plan B option. I am sure changing the ? is easier than starting over! However if there is a way to get rid of the ? that would be great.
ashish 10th February 2011 at 05:21

thanks. wget helped me save the site that i wanted.
php ide 15th February 2011 at 15:23

great tutorial, a problem i have is download a page that auto redirects to the home page, so wget only downloads the home page, can i set wget not to follow redirects??
Matthijn 26th February 2011 at 19:29

Mac users can just install wget through something like macports.
Anonymous 8th May 2011 at 21:41

This is the first guide on wget I have read that is fairly up-to-date and actually helpful. Thank you.

Everything is working fine for me but I was wondering if it is possible to save directory/folder names as html files?

Example:

http://www.sitename.com/directory-name-1/ would be saved as /site-name/directory-name-1/index.html

Is there no way to save as /site-name/directory-name-1.html ?

Any help would be greatly appreciated!
hoowha 6th June 2011 at 16:14

Goooooooooooooooooooooooooooooooooooooooood!
the option –user-agent is what I looking for.
Thanks!
Macaroni Schotel 7th June 2011 at 17:41

Arrh … my hosting company block WGET. I can’t try this awesome tool!
Agentursoftware 29th December 2011 at 14:09

Thanks Richard,

Between your steps and the other comments I finally got my problem worked out with downloading images from external site.

For Windows users though – you might find it easier with WinHTTrack
Evan 3rd February 2012 at 19:49

Awesome! Thanks!
Pferdeklinik 23rd February 2012 at 09:29

Better late than never. Having recovered from a hack I’ve been looking for ways to protect and better backup in future.

Thanks for the great info…
BaZz 7th March 2012 at 13:13

Thank you for this useful article.

Please note: the 2 versions of wget you are linking are different in command-line options!
PC 1st June 2012 at 17:16

Buy a windows computer.
bluerika08 9th July 2012 at 14:55

thank you so much for the instructions!!!!! really! :D
Adam 8th August 2012 at 18:34

Why use wget and not just FTP your site locally for backup?
Kellie 5th September 2012 at 03:32

Terrific instructions, very easy to follow.

worked like a charm :D
Helper 17th September 2012 at 20:41

this is fake page plz don’t downlod wget it’s hacking VIRUS if clik the file dosent show any thing … I SAY IT’S VIRUS
rerevisionist 23rd September 2012 at 14:20

Nobody here seems to know about WHTT, a free site grabber. (Possibly the omission is because it’s Windows? – I don’t know). Once you work out how it operates, you can grab assorted sites from around Internet, for future reference. This includes forums – I have a copy of the nukelies forum, which I uploaded for reference – but any forum can be downloaded, though ones with milions of postings are best avoided, partly becvause of the size, partly the low quality.