We earn commission when you buy through affiliate links.
This does not influence our reviews or recommendations.Learn more.
One of the frequently used utilities by sysadmin is wget.
It can be very handy during web-related troubleshooting.
What is wget command?
wget command is a popular Unix/Linux command-line utility for fetching the content from the web.
It is free to use and provides a non-interactive way to download files from the web.
The wget command supports HTTPS, HTTP, and FTP protocols out of the box.
Moreover, you’ve got the option to also use HTTP proxies with it.
How does wget help you troubleshoot?
There are many ways.
Or, you want to verify intranet websites.
Or, you want to download a certain page to verify the content.
In the background, thewgetwill run and finish their assigned job.
It can also be used to get the entire website on your local machines.
It can follow links inXHTML and HTMLpages to create a local version.
To do so, it has to download the page recursively.
Lets see them in action.
The syntax of the wget is as below.
Download a webpage
Lets have a go at download a page.
Download multiple files
Handy when you have to download multiple files at once.
This can give you an idea about automating files download through some scripts.
Lets venture to download Python 3.8.1 and 3.5.1 files.
So, as you’re able to guess, the syntax is as below.
You just have to ensure giving space between URLs.
Using the–limit-rateoption, you could limit the download speed.
Here is the output of downloading the Nodejs file.
It took 0.05 seconds to download 13.92 MB files.
Now, lets make a run at limit the speed to 500K.
Reducing the bandwidth took longer to download 28 seconds.
Imagine, your users are complaining about slow download, and you know their web link bandwidth is low.
you’re able to quickly try–limit-rateto simulate the issue.
This is expected, but what if you dont want to stare at your terminal?
Well, you could use-bargument to start the wget in the background.
By default, wget will throw an error when a certificate is not valid.
The above example is for the URL where cert is expired.
As you’re free to see it has suggested using–no-check-certificatewhich will ignore any cert validation.
Cool, isnt it?
HTTP Response Header
See the HTTP response header of a given site on the terminal.
Using-Swill print the header, as you might see below forCoursera.
Or specific browsers user-agent.
This is doable by specifying–user-agent.
The below example is for the user agent as MyCustomUserAgent.
In this situation,–headerwould be useful.
But you’ve got the option to take advantage of proxy to connect.
Dont forget to update $PROXYHOST:PORT variable with the actual ones.
Connect using a specific TLS protocol
Usually, I would recommend usingOpenSSLto test the TLS protocol.
But, you’re free to use wget too.
wget –secure-protocol=TLSv1_2 https://example.com
The above will force wget to connect over TLS 1.2.
Conclusion
Knowing the necessary command can help you at work.
I hope the above gives you an idea of what you’re free to do withwget.
