How to install and use wget on Mac

Wget is a non-interactive command-line utility to download resources from a specified URL. Due to non-interaction, wget can operate in the background or before the user logs in. The program is specially designed for poor connections, making this tool particularly robust in unstable conditions. Although wget is not built into macOS, users can easily download and install with Homebrew, the best Mac package manager available.

1. Download and install Homebrew

To install Homebrew, open the Terminal window and execute the following command, which is obtained from the Homebrew website (reference link https://brew.sh/):

 /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" 

Users can see the command called curl, another command-line utility to download files from a URL, built-in with the Ruby installation included in macOS.

2. Install wget

When the installation is complete, we will use Homebrew to install wget. In Terminal, run the following command to download and install wget:

 brew install wget 

Users will receive direct updates about the download process and install any required dependencies to run wget on the system.

If Homebrew is already installed, remember to run the brew update command to get the latest copies of all the recipes (software packages).

3. Use wget

The purpose of wget is to download content from the URL. It is a quick and simple non-interactive tool to download files from any publicly accessible URL.

Download a single file

Like the curl command, wget takes the remote resource through the URL and saves it to a specified location on the computer. The operation command structure is as follows:

 wget -O path/to/local.copy http://example.com/url/to/download.html 

The above command will save the specified file in the URL to the specified location on the computer. If the -O flag is excluded, the specified URL will be downloaded to the current directory.

Download a recursive directory

To download the entire directory tree with wget, users need to use the -r / - recursive and -np / - no-parent flags as follows:

 wget -e robots=off -r -np https://www.w3.org/History/19921103-hypertext/hypertext/ 

This command will cause wget to follow any links found on documents in the specified directory, download recursively the entire URL specified.

The command also includes -e robots = off , ignoring the restrictions in the robots.txt file . In general, disabling robot.txt to prevent shortened downloads is a bad idea.

4. Other wget flags

In addition to the above flags, some of the following wget flags are the most useful:

Download control

1. wget -X / absolute / path / to / directory will exclude a specific directory on the remote server.
2. wget -nH deletes the hostname directory. Remember, the hostname is part of the URL containing the domain name and ends with a TLD like '.com'. For example, the folder called www.w3.org in the previous example will be ignored, and start downloading with the 'History' folder instead.
3. wget --cut-dirs = # ignore the number of directories specified under the URL before starting to download files. For example, -nH --cut-dirs = 1 will change the specified path of 'ftp.xemacs.org/pub/xemacs/' to '/ seeacs /', reducing the number of empty parent directories in the process download locally.
4. wget -R index.html / wget --reject index.html will ignore all files that match the specified file name. In this case, this flag will exclude all index files. The * character can be used as a wildcard, like * .png, will ignore all PNG extension files.
5. wget -i file specifies the target URL from an input file. The input file must be an HTML file or parsed in HTML format with additional --force-html flag.
6. wget -nc / wget --no-clobber will not overwrite files that already exist at the destination.
7. wget -c / wget --continue will continue to download downloaded files in part.
8. wget -t 10 will attempt to download resources up to 10 times before reporting a failure.

Adjust the logging level

1. wget -d allows output debugging.
2. wget -o path / to / log.tx t allows to output log to the specified directory instead of displaying standard output when logging in.
3. wget -q disables all wget output, including error messages.
4. wget -v allows to display wget's default detail output.
5. wget --no-verbose turns off the log message but displays an error message.

The article covers most of the use of wget, but this download has even more capabilities, you can learn more during use.

Hope you are succesful.