We often have to download various files from the Internet, for example, executable program files, script files, archives with source codes. But it is not always necessary to do this through the browser. In many situations it is much easier to perform all actions through the terminal. Because this way you can automate the process. On the other hand, webmasters from time to time have to test the availability of websites, check the sent and received headers, and more.

To solve such problems and tasks of such a circle, you can use the curl utility. It allows you to solve a much wider range of tasks, among which even an imitation of user actions on the site. In this article we will look at how to use curl, what it is and why we need this program.

What is CURL?

In fact, curl is more than just a command line utility for macOS, Linux or Windows. This is a set of libraries that implement the basic capabilities of working with URL pages and file transfer. The library supports working with protocols: FTP, FTPS, HTTP, HTTPS, TFTP, SCP, SFTP, Telnet, DICT, LDAP, as well as POP3, IMAP and SMTP. It is great for simulating user actions on pages and other operations with URL addresses.

CURL Utility

Before proceeding to the description of how the curl linux command can be used, let’s take a look at the utility itself and its main options that we need. The utility syntax is very simple:

$ curl [options] [link]

Use CURL to Download File

The most common task is downloading files. Download the file is very simple. To do this, simply transfer the file name or the html page to the utility in the parameters:

curl [url]
Example: curl http://example.com

But here you will find one surprise, the entire contents of the file will be sent to the standard output. To write it to any file use:

curl -o [url]
Example: curl -o http://example.com

Read more: How to Create a Tar GZip file via Terminal?

Use CURL to Download Multiple Files

If necessary, one command can download several files:

curl -O [url 1] [url 2] [url 3]
Example: curl -O http://example.com/index1.html http://example.com/index2.html http://example.com/index3.html

Use CURL for File Transfer

Uploading files is quite simple, but the utility allows you to perform other actions, such as sending files to an ftp server. For this there is an option -T:

curl -T [filename] [destination url]
Example: curl -T osxtips.txt http://example.com/

Authentication with CURL

If the server requires authentication of one of the common types, for example, HTTP Basic or FTP, then curl can very easily cope with this task. To specify authentication data, simply enter them with a colon in the -u option:

curl -u username:password [url]
Example: curl -u osxtips:password http://example.com/

Using Proxy

If you need to use a proxy server to download files, then it is also very simple. It is enough to set the proxy server address in the -x option:

 curl -x proxy_server:proxy_port -L url
Example: curl -x 125.119.175.48:8909 -L http://example.com/

CURL’s Options

Now consider the main options:

-a, –append        Append to target file when uploading (F/SFTP)

-E, –cert CERT[:PASSWD]  Client certificate file and password (SSL)

-K, –config FILE   Read config from FILE

-C, –continue-at OFFSET  Resumed transfer OFFSET

-b, –cookie STRING/FILE  Read cookies from STRING/FILE (H)

-c, –cookie-jar FILE  Write cookies to FILE after operation (H)

-d, –data DATA     HTTP POST data (H)

-D, –dump-header FILE  Write the received headers to FILE

-f, –fail          Fail silently (no output at all) on HTTP errors (H)

-F, –form CONTENT  Specify HTTP multipart POST data (H)

-P, –ftp-port ADR  Use PORT with given address instead of PASV (F)

-G, –get           Send the -d data with a HTTP GET (H)

-g, –globoff       Disable URL sequences and ranges using {} and []

-H, –header LINE   Pass custom header LINE to server (H)

-I, –head          Show document info only

-h, –help          This help text

-0, –http1.0       Use HTTP 1.0 (H)

-i, –include       Include protocol headers in the output (H/F)

-k, –insecure      Allow connections to SSL sites without certs (H)

-4, –ipv4          Resolve name to IPv4 address

-6, –ipv6          Resolve name to IPv6 address

-j, –junk-session-cookies  Ignore session cookies read from file (H)

-l, –list-only     List only mode (F/POP3)

-L, –location      Follow redirects (H)

-M, –manual        Display the full manual

-m, –max-time SECONDS  Maximum time allowed for the transfer

-n, –netrc         Must read .netrc for user name and password

-:, –next          Allows the following URL to use a separate set of options

-N, –no-buffer     Disable buffering of the output stream

-o, –output FILE   Write to FILE instead of stdout

-#, –progress-bar  Display transfer progress as a progress bar

-x, –proxy [PROTOCOL://]HOST[:PORT]  Use proxy on given port

-U, –proxy-user USER[:PASSWORD]  Proxy user and password

-p, –proxytunnel   Operate through a HTTP proxy tunnel (using CONNECT)

-Q, –quote CMD     Send command(s) to server before transfer (F/SFTP)

-r, –range RANGE   Retrieve only the bytes within RANGE

-e, –referer       Referer URL (H)

-J, –remote-header-name  Use the header-provided filename (H)

-O, –remote-name   Write output to a file named as the remote file

-R, –remote-time   Set the remote file’s time on the local output

-X, –request COMMAND  Specify request command to use

Conclusion

This article describes the basic commands for working with CURL. If suddenly you want to expand your knowledge of working with the utility, check the table above or type this command in Command Line:

man curl

In this article we looked at how to use curl, why we need this utility and its main features. Despite their similarity with wget, they are very different. The curl command is designed more to analyze and simulate various actions on the server, while wget is more suitable for downloading files and crawling sites.

It’s the best time to ask questions and give us your feedback in comments.

Author

Hi there! My Name is Vincent Lago! My goal is to share insightful reviews, guides, and manuals for people looking to know more about current solutions for Apple hardware and OS. I have years of technological background knowledge including owning a custom iMacs & PCs which was using for different types of business. And now, I want to share my experience with you.

Write A Comment