Curl Command in Linux with Examples

Linux curl command is used to download or upload data to a server via supported protocols such as HTTP, FTP, IMAP, SFTP, TFTP, IMAP, POP3, SCP, etc. It is a remote utility, so it works without user interaction.

The data transfer from one place to another is one of the vital and most used tasks of a computer system. However, there are many GUI tools available for data transfer. But, when working on the command-line, it becomes a bit complicated. The curl utility allows us to transfer data via the command line.

Curl is a tool of command line to get and send data such as files with URL syntax. It supports all protocols that are supported by libcurl because it uses libcurl. Curl provides its support for HTTPS and implements SSL certificate verification when a secure protocol is mentioned by default, like HTTPS. Curl will bring the remote server certificate and check the CA certificate to save the remote server validity from guaranteeing the remote server is the one it requests to be when curl links to a remote server by HTTPS.

There are many options to define a CA certificate, like --capath and --cacert. The option, i.e., --cacert, can be utilized for specifying the location of the file of the CA certificate store. If a CA certificate file isn't available in the Windows platform, curl will search for the "curl-ca-bundle.crt" CA certificate file name in the below sequence:

• Directory in which the curl program is present.
• Current working directory.
• System directory (Windows).
• Directory (Windows).
• Directories are mentioned in the %PATH% environment variables.

Curl will give an error message when the remote server is utilizing a self-signed certificate or when the remote server certificate isn't signed via a CA mentioned in the CA cert file. --insecure or -k option can be utilized for skipping certificate verification. The remote CA certificate can alternatively be included in the CA certificate store file if the remote server is valid.

Curl provides a busload of helpful tricks like file transfer resume, cookies, SSL connections, HTTP post, FTP upload, user authentication, proxy support, and more. Curl is mechanized by libcurl for every transfer-related aspect.

What is libcurl?

libcurl is a client-side free URL transfer library. It supports SMB, SMTP, SCP, RTMP, RTSP, POP3, MQTT, LDAP, Kerberos, IMAP, HTTPS, HTTP proxy tunneling, HTTP PUT, HTTP POST, HTTP/1, Gopher, FTPS, FTP, DICT, and cookies. The library also supports user-plus-password authentication, proxies, LDAPS, HTTPS certificates, HTTP form-based upload, FTP uploading, file transfer resume, TFTP, Telnet, SFTP, and file URL scheme.

libcurl is compact. It identically creates and works on several platforms, including Microsoft Windows, UnixWare, Ultrix, Tru64, Symbian, Solaris, RISC OS, QNX Neutrino, OS/2, OpenBSD, NetWare, NetBSD, macOS, Linux, IRIX, iOS, HURD, HP-UX, FreeBSD, DOS, Darwin, OpenVMS, BlackBerry 10, BlackBerry Tablet OS, BeOS, Android, AmigaOS, and AIX.

The libcurl library is IPv6 compatible, thread-safe, and free. Bindings are also available for 50+ languages, including Python, PHP, Java, and C/C++.
The libcurl library supports SChannel, gskit on IBM i, NSS, mbed TLS, GnuTLS on Windows, Secure Trasport on iOS and macOS, TLS/SSL through OpenSSL, rustls, BearSSL, wolfSSL, AmiSSL, libressl, and Boringssl.

Syntax:

The basic syntax for using curl is as follows:

From the above syntax,

URL:

The URL syntax is a general protocol-dependent URL. We can specify multiple URLs as follows:

Options:

The curl command supports the following command-line options:

--abstract-unix-socket <path>: It is used to connect via abstract Unix domain socket instead of a network.

--anyauth: It is used to command curl for authenticating the method by itself for using the most secure method. This utility is an optional utility "--basic, --digest, --ntlm, and negotiate," which is used to set a specific authentication method.

-a, --append: It is used to upload files. It appends the file rather than overwriting it. If the given file does not exist on the server, it will create one.

--basic: It specifies the use of HTTP basic authentication. It is the default option of the curl command. It is useful for overriding the previous settings.

--cacert <file>: It is specified for using the particular certificate file to verify the peer. The files may have several CA certificates. The standard format for the certificates is PEM, so, all the certificates must be in it.

--capath <dir>: It is specified for using the particular directory to verify the peer. We can specify multiple paths by separating them by a colon (:) such as "path: path2:path3". The standard format for the certificates is PEM, so, all the certificates must be in it.

--cert-status: It is used to verify the status of the server certificate. It uses Certificate Status Request or OCSP stapling TLS.

--cert-type <type>: It specifies the provided curl certificate type. These certificates can be in PEM, DER, and ENG format. The default value is PEM. If it is specified multiple times, the last value will be taken by the curl.

-E, --cert <certificate[: password]>: It is specified for using the client certificate file when getting a file via any of SSL-based protocol such as HTTPS, FTPS, etc.

--ciphers <list of ciphers>: It is used to select the ciphers to use in the connection.

--compressed-ssh: It is used to enable built-in SSH compression. This option will take as a request by the server, so the server may or may not accept it.

--compressed: It is used to request a compressed response using the curl algorithms and saves the uncompressed document. This option will send a report for unsupported encoding curl.

-K, --config <file>: It is used to describe a text file for reading curl arguments. The curl will use the command line arguments from the text file.

--connect-timeout <seconds>: It is used to specify the maximum time in seconds for curl connection time-out.

--connect-to <HOST1:PORT1:HOST2:PORT2>: It is used to create a request to the given pair of host and port; otherwise, it will connect to the next pair. This option is a handy tool for making direct requests at a specific server.

-C, --continue-at <offset>: It is used to continue or resume a previous file transfer at the given offset.

-c, --cookie-jar <filename>: It is used to specify a particular file to which we want to write all cookies after a successful operation.

-b, --cookie <data>: It is used to forward data to the HTTP server in the Cookie header.

--create-dirs: It is used for conjunction with the '-o? option, It will create the required local directory hierarchy.

--crlf (FTP SMTP): It is used to convert LF to CRLF in the upload. It is a handy tool for MVS (OS/390).

--crlfile <file>: It is used to specify (in PEM format) with a Certificate Revocation List.

--data-ascii <data>: It is an alias for the ?-d? option.

--delegation <LEVEL>: It is used to set LEVEL to acknowledge server what it is allowed to delegate when it comes to user credentials.

--digest: It is used to enable HTTP Digest authentication.

-q, --disable: If used as the first argument, It will ignore the curlrc config file.

--dns-interface <interface>: It is used to acknowledge the server to send the outgoing DNS requests.

--dns-servers <addresses>: It is used to specify the DNS servers instead of default servers.

-f, --fail: It is used to make the curl silently fail on server errors.

-F, --form <name=content>: It is used emulate a filled-in form submitted by user.

-P, --ftp-port <address>: It is used to reverse the default listener roles when connecting with FTP.

--ftp-ssl-ccc-mode <active/passive>: It is used to set the CCC mode.

-G, --get: It is used to specify data with ?-d? option to be used in an HTTP GET request instead of POST request.

-h, --help: It is used to show the help manual having a brief description of the usage and support options.

-0, --http1.0: It is specified for using the HTTP version 1.0.

--ignore-content-length: It is used to ignore the Content-Length header.

-i, --include: It is used to include the HTTP responses headers.

-4, --ipv4: It is used to resolve names to Ipv4 addresses.

-6, --ipv6: It is used to resolve names to Ipv6 addresses.

Protocols of curl

Curl supports several protocols:

• DICT: It enables us to look up words with only dictionaries.
• FTP (S): Curl supports FTP protocol using a lot of levers and tweaks. With or without TLS.
• FILE: It allows us to write and read the local file. Curl doesn't allow to remotely access file://URL, but it will work when active on Microsoft Windows with the native UNC approach.
• GOPHER (S): It retrieves files.
• HTTP (S): Curl supports HTTP using numerous variations and options. It can speak the 0.9, 1.0, 1.1, 2, and 3 HTTP versions depending on the accurate command line and build options.
• IMAP (S): Curl downloads emails for us with the mail-reading protocol. With or without TLS.
• LDAP (S): Curl can implement directory lookup for us with or without using TLS.
• MQTT: Curl supports the 3 version of MQTT. Downloading on MQTT is the same as "subscribe" to any topic, and posting/uploading is the same as "publish" on any topic. Over TLS, MQTT isn't supported.
• POP3 (S): Downloading through a pop3 server represents receiving mail without or with TLS.
• RTMP (S): The RTMP or Realtime Messaging Protocol is mainly used for server streaming media.
• RTSP: Curl supports the 1.0 downloads of RTSP.
• SCP: Curl supports the two scp transfer version of SCP.
• SFTP: Curl supports SFTP over the 2 version of SSH.
• SMB (S): Curl supports the 1 version of SMB for download and upload.
• SMTP (S): Uploading contents to any SMTP server means transferring an email without or with TLS.
• TELNET: Informing curl to bring a telnet URL begins an interactive session in which it transfers what it reads over stdin and results what the server transfers it.
• TFTP: Curl can implement TFTP uploads and downloads.

Installation of the curl Command

The curl command comes with most of the Linux distributions. But, if the system does not carry the curl by default. You need to install it manually. To install the curl, execute the following commands:

Update the system by executing the following commands:

Now, install the curl utility by executing the below command:

Verify the installation by executing the below command:

The above command will display the installed version of the curl command.

Fetch the content of the specified URL

To fetch the content of any specific URL, execute the curl command, followed by the URL. Consider the below command:

The above command will fetch the page data of the specified page. Consider the below snap of the output:

From the above output, we can see the page data of the given URL are being fetched. To stop the execution, press the CTRL+C keys.

Save data in a Specific File

To save the data in a specific file, pass the '-o' option followed by directory, file name, and URL as follows:

Consider the below command:

The above command will save the page data in 'linux.html' file under " /home/javatpoint/Documents/" directory. Consider the below output:

From the above command, we can see the total amount of downloaded data, received data, average time, and some other statistics about the data.

To verify the downloaded data, open the file by executing the cat command

Consider the below snap of output:

Download a File From the Web

One of the interesting and fascinating uses of curl is that we can download a file from the web. To download a file from the web, copy the download link and paste it with the curl command. We can pass other arguments as well to make it more specific. For example, download the latest version of ubuntu, copy the download link of the ubuntu from its official website and paste it with curl command as follows:

The above command will download the Ubuntu 20.04 to the specified directory. Provide the proper file extension; otherwise, it will download the file in a different format. Consider the below output:

From the above output, the ubuntu.iso file is being downloaded. We can see the download time, file size, download speed, and other statistics. To stop the execution, any time press the CTRL+D keys.

Resume the Interrupted Downloads

There may be a chance that the downloads can be interrupted for some reason. We can resume downloads by using the curl command. To resume the interrupted file, pass the '-C' option with curl command as follows:

The above command will resume the download of the specified URL.

Download Multiple Files

To download the multiple files, specify the multiple URLs separated a by space as follows:

The above command will download the data from both URLs, respectively.

Query HTTP Headers

The HTTP headers contain additional information; it allows the webserver to download this information. To query the HTTP headers from a website, execute the command with '-I' option as follows:

The above command will produce the below output: