PowerShell cURL: A Comprehensive Guide

cURL is a versatile command-line tool for transferring data using various network protocols. With its capabilities to interact with APIs, web services, and servers, cURL has become an essential tool for developers and sysadmins alike.

In this comprehensive guide, we will explore using cURL in PowerShell on Windows to take advantage of its full potential.

An Overview of cURL

cURL stands for "Client URL". It allows you to connect to various servers and services to transfer data easily through the command line interface (CLI).

Some key features and capabilities of cURL include:

• Support for multiple protocols: HTTP, HTTPS, FTP, FTPS, SMTP, POP3, IMAP etc.
• SSL/TLS connectivity for encrypted transfers
• Upload and download files from remote servers
• Fill out and submit web forms
• POST JSON data to APIs
• Follow redirects
• Save response data to a file
• Pass custom headers
• Authentication with usernames and passwords

With all these features, cURL can handle most data transfer needs programmatically.

Now let‘s see how we can use it effectively in PowerShell on Windows.

Installing cURL on Windows

If you‘re using the latest PowerShell 7 and above, cURL comes built-in so no extra installation is needed.

You can verify the version using:

$PSVersionTable

For older PowerShell versions, here are a few ways to get cURL:

• Upgrade to PowerShell 7 or later
• Install Git for Windows which bundles cURL
• Manually install cURL binaries using a package manager like Chocolatey

Let‘s check if cURL is available and works properly:

curl --version

This should display details about the cURL version if it‘s correctly set up.

Using cURL in PowerShell

The basic syntax of cURL on the command line is:

curl [options] [URL]

For example, to GET the contents of a web page:

curl https://www.example.com 

This will display the full HTML content of example.com.

You can save the output to file instead of printing to the screen:

curl https://www.example.com -o example.html

Similarly, you can download a file:

curl -o example.zip https://download.example.com/example.zip

These are just simple usage examples – cURL supports a wide range of advanced options. Let‘s go through them in detail.

1. GET and POST Requests

• GET request is the default HTTP request method in cURL. We‘ve already seen examples above.

• For POST requests, use -d or --data option:

curl -d ‘key1=value1&key2=value2‘ https://example.com/form

This will POST the data to the URL.

Similarly you can read the post data from a file instead of typing on the command line:

curl -d @data.txt https://example.com/form

2. Passing Headers

To send custom HTTP headers with the request, use -H or --header option:

curl -H ‘Authorization: Bearer token‘ https://api.example.com/user

You can pass multiple headers by using the option multiple times:

curl -H ‘Accept: application/json‘ -H ‘API-Key: xxx‘ https://api.example.com

3. HTTP Authentication

For basic access authentication, use -u or --user option:

curl -u myusername https://example.com/protected

It will prompt for password interactively.

To provide both username and password inline:

curl -u myusername:password https://example.com

For NTLM and Negotiate (SPNEGO) authentication mostly used in enterprise setups with AD, use --ntlm and --negotiate:

curl --ntlm --user username:password https://intranet.example.com

curl --negotiate -u : https://sharepoint.example.com

4. Handling HTTP Responses

By default cURL displays the response body.

To extract specific information from the response headers:

curl -I https://www.example.com 

This will show only the response headers containing status code, content type etc.

To get effective URL after redirects:

curl -L -I https://tinyurl.com/app 

To follow maximum of 5 redirects:

curl --max-redirs 5 https://example.com

To fail on HTTP error status codes:

curl --fail https://invalid-link.com/

This will return exit code 22 on HTTP errors.

5. Downloading Files

We already saw a basic file download example earlier. Here are some additional options:

Resume interrupted download:

curl -C - -o file.zip https://example.com/file.zip 

Download multiple files in parallel:

curl -O URL1 -O URL2  

The files will be saved with their remote names.

Limit download speed:

curl --limit-rate 100K https://cdn.example.com/movie.mp4

This regulates the transfer speed to 100 kilobytes per second.

6. Uploading Data and Files

To upload data, POST it with -d or --data as shown earlier.

For file uploads, read from a file:

curl -F ‘image=@photo.jpg‘ https://upload.example.com/

This will upload photo.jpg with form field name image.

You can also pass multiple files in the same command:

curl -F ‘media=@file1.avi‘ -F ‘media=@file2.avi‘ https://upload.example.com/ 

7. Security and Encryption

By default cURL will attempt to negotiate TLS automatically.

To force TLS regardless:

curl -1 https://testssl-expire.disig.sk/index.en.html

To ignore invalid certificates and hosts:

curl -k https://self-signed.badssl.com/

However, this will make transfers susceptible to man-in-the-middle attacks.

For client certificates, pass the PEM files:

curl --cert client.pem https://example.com

If the PEM file requires a passphrase, add --key passwd option.

8. Proxies and Tunnels

To use an HTTP proxy for all requests:

curl -x proxy-server.example.com:8080 https://example.net/

Similar option is available for SOCKS5 proxies.

For basic proxy authentication:

curl -U proxyuser:proxypass -x proxy.example.com:3128 https://wikipedia.org

To tunnel data over SSH:

curl -p 999 -u sshuser sftp://example.com:2222  

Here we tunnel local port 999 through SSH to remote server‘s port 2222.

9. API Interactions

A common use case for cURL is interacting with web APIs.

For JSON APIs, set header to application/json and pass JSON data with -d:

curl -H ‘Content-Type: application/json‘ -d ‘{"key":"value"}‘ https://api.example.com

For XML APIs, set header to application/xml and pass XML body:

curl -H ‘Content-Type: application/xml‘ -d ‘<data>Hello</data>‘ https://api.example.com

To use query string parameters, append them to the URL:

curl ‘https://api.example.com/users?page=2&per_page=100‘

Overall, cURL lets you work seamlessly with REST, SOAP, and other APIs.

10. Automate with Scripts

You can automate sequences of cURL commands in Bash scripts:

#!/bin/bash

curl -X GET "https://api.example.com/users" -H "Accept: application/json"
curl -X POST "https://api.example.com/users" -H "Content-Type: application/json"

Save it as script.sh and run with:

./script.sh

In PowerShell scripts, use cmdlets like Invoke-WebRequest / Invoke-RestMethod instead which offer more features.

This covers the most essential cURL usage options – there are many more available for advanced operations.

Common Issues with cURL on Windows

When running cURL in PowerShell, you may face encoding issues in the output or slashes being stripped from data.

This happens because PowerShell tries to analyze the cURL output which runs into problems with non-ASCII characters.

To avoid it, wrap the cURL command in cmd.exe shell:

cmd /c curl https://example.com

Now PowerShell won‘t interfere and you‘ll see raw output from cURL.

Another option is to save output to file instead of printing it with -o filename.ext parameter.

Alternatives for cURL on Windows

While cURL is ubiquitous, PowerShell offers its own alternative – Invoke-WebRequest and Invoke-RestMethod cmdlets.

The syntax is quite similar:

Invoke-WebRequest -Uri https://example.com

Invoke-RestMethod -Uri https://api.example.com  

Benefits over plain cURL:

• Automatic conversion from JSON/XML
• Custom object output
• Support for certificates stored in certificate store
• Retry logic, progress tracking
• Windows authentication helpers

So If you intend to mainly use PowerShell for scripting, it may be preferable over cURL binaries.

Other alternatives like System.Net.HttpWebRequest/.NET also come with their own pros and cons.

Conclusion

With this detailed walkthrough of using cURL for various HTTP requests, file transfers and API interactions – you should be able to leverage its full potential for automation tasks.

cURL is cross-platform by nature and acts as a common ground between Windows, Linux and macOS while bringing speed, security and versatility to handle both adhoc and complex requests.

By mastering cURL alongside native PowerShell tools like Invoke-RestMethod, you strengthen your arsenal to tackle any kind of HTTP traffic from the Windows command line.