{"id":44745,"date":"2022-10-14T03:01:51","date_gmt":"2022-10-13T17:01:51","guid":{"rendered":"http:\/\/www.rjmprogramming.com.au\/ITblog\/?p=44745"},"modified":"2022-10-17T17:49:31","modified_gmt":"2022-10-17T07:49:31","slug":"curl-multiple-urls-on-command-line-tutorial","status":"publish","type":"post","link":"https:\/\/www.rjmprogramming.com.au\/ITblog\/curl-multiple-urls-on-command-line-tutorial\/","title":{"rendered":"Curl Multiple URLs on Command Line Tutorial"},"content":{"rendered":"
\"Curl<\/a>

Curl Multiple URLs on Command Line Tutorial<\/p><\/div>\n

Continuing on from yesterday’s Curl via PHP to CSV on Command Line Tutorial<\/a> with more Linux or Mac OS X command line curl<\/a> brilliance, did you know that you can run several URLs from the one curl command line statement?<\/p>\n

Take a look at our tutorial picture<\/a>‘s scenario as a simple example of what is possible here with command line curl …<\/p>\n


\n$ echo \"The rain\" > page1.html
\n$ echo \"In Spain\" > page2.html
\n$ echo \"Falls mainly\" > page3.html
\n$ echo \"On the plain\" > page4.html
\n$ curl http:\/\/localhost:8888\/page[1-4].html 2> \/dev\/null | grep -v '-'
\nThe rain
\nIn Spain
\nFalls mainly
\nOn the plain
\n$
\n<\/code><\/p>\n

Cute, huh?!<\/p>\n

\n

\n

Previous relevant Curl via PHP to CSV on Command Line Tutorial<\/a> is shown below.<\/p>\n

\"Curl<\/a>

Curl via PHP to CSV on Command Line Tutorial<\/p><\/div>\n

Like with XML to CSV Command Line Report Tutorial<\/a> today’s tutorial uses a Linux or Mac OS X (Terminal application) command line to end up with a CSV<\/a> (comma separated values) file, which can be input for just about any spreadsheet application you care to mention.<\/p>\n

Today’s tutorial lays to rest a misnomer that an avid reader of this blog may have come to believe, that being that the marvellous curl<\/a> can not be used with more than the one ? and & argument involved because of the dual purpose & has in “command line land” meaning “send this command into the background”. We can, use that & if that & appears within a curl<\/a> command “-d” switch as per …<\/p>\n

\n\n\n
-d, –data <data><\/td>\n\n

(HTTP) Sends the specified data in a POST request to the HTTP server, in a way that can emulate as if a user has filled in an HTML form and pressed the submit button. Note that the data is sent exactly as specified with no extra processing (with all newlines cut off). The data is expected to be “url-encoded”. This will cause curl to pass the data to the server using the content-type application\/x-www-form-urlencoded. Compare to -F\/–form. If this option is used more than once on the same command line, the data pieces specified will be merged together with a separating “&” character. Thus, using ‘-d name=daniel -d skill=lousy’ would generate a POST chunk that looks like ‘name=daniel&skill=lousy’.<\/p>\n

If you start the data with the “@” character, the rest should be a file name to read the data from, or “-” (dash) if you want curl to read the data from stdin. The contents of the file must already be url-encoded. Multiple files can also be specified. Posting data from a file named ‘foobar’ would thus be done with “–data @foo-bar”.<\/p>\n

-d\/–data is the same as –data-ascii. To post data purely binary, you should instead use the –data-binary option. To URL-encode the value of a form field you may use –data-urlencode.<\/p>\n

If this option is used several times, the ones following the first will append data.<\/td>\n<\/tr>\n<\/table>\n<\/blockquote>\n

Notice how this “-d” switch uses POST to send data. That needs a serverside webpage as its destination, hence the PHP interpret_curl_d.php<\/a> code (making use of the $_POST array) that we make dual purpose today …<\/p>\n

    \n
  1. can support “surfing the net” mode of use (eg. HTTP:\/\/www.rjmprogramming.com.au\/PHP\/interpret_curl_d.php?column1=value%201&column2=value%202&column3=value%203<\/a>) from a web browser address bar …
    \n