TigerTronics logoTigerTronics team 2053

Creating CGI Programs with Bash: Handling POST Data

<-- Page 2: Getting Started | Page 3: Working with Files -->

Differences between POST and GET

In the previous page, handling data from forms submitted using GET was discussed, and from that page your should be able to figure out how to write a Bash CGI that collects and can do things with data submitted with GET. GET however, is only one of the ways a form may submit its data. The other way is POST.

GET data is sent as part of the URL. You may notice when browsing URLs such as this nicely color-coded example:

http://www.example.com/cgi-bin/myProgram.sh?action=buy&item=Big%20Table&price=32.99&sub=Submit

The green part of this URL is the site and path to the CGI program, and the actual CGI program is in red. Everything in blue is GET data. As you can see, GET data is identified by a ? after the CGI that will handle the data. The data is made up of the form form_element_name=value separated by ampersands (&). In the example above, we see that an HTML form contained elements named "action", "item", "price", and "sub" and their values were "buy", "Big Table" (remember spaces are turned into %20 when the data is submitted), "32.99", and "Submit".


Note: One nice thing about GET data is that it doesn't necessarily have to come from an HTML form. Since it is simply part of a URL, you can create links to CGI scripts that accept GET data and simple pass the data you want to the program in the URL of the link, without having to create a form and have the user click "submit."

For example, say I had a CGI program that displayed a page that showed either the date or time, depending on what something called "whatToShow" equaled. If "whatToShow" equals "date" it shows the date, if it equals "time" it shows the time. One way to access this CGI would be to create an HTML form and have the user submit this, as shown below:
<form action="myprogram.sh" method="get">
What do you want to do?<br>
<input type="radio" name="whatToDo" value="date">Show the Date<br>
<input type="radio" name="whatToDo" value="time">Show the Time<br>
<input type="submit" name="submit" value="Go">
</form>

Alternatively, since GET data is just part of the URL, I could just create two links to the CGI, one for the date and one for the time, and get the same result, but with less typing and a different look:
<a href="myprogram.sh?whatToDo=date">Show the Date</a><br>
<a href="myprogram.sh?whatToDo=time">Show the Time</a>

This trick is very useful for times when your CGI program only accepts some simple, predefined input that doesn't require a whole form to get, and you want the user to access the CGI by just clicking a link, instead of having to fill out and submit an entire form. This trick is also useful for debugging CGIs that accept GET data: you can keep running them with different data without having to keep changing your HTML form; just change the URL to make the new data you want to submit your CGI.

GET has several benefits: it is easy to use, you can send get data in a regular link (see the above's "Note"), and you can see the data being transmitted (great for debugging). However, GET has some drawbacks:
  • It can only send a limited amount of data. URLs (and GET itself) are limited in how long they can be, and this limits how much data GET can send.
  • The data sent in GET is clearly visible in the URL. While this is great for debugging, it may be undesirable in the final product.
  • GET can't send any type of data. Its limited to regular characters. So, you can't upload files with GET (it would be impracticable to do this given the limit on the amount of data you can send with GET, anyway).

Fortunately, an alternative to GET exists which solves most of these problems: POST. POST data is not sent as part of the URL. Instead, it is sent as its own separate stream. POST can send any type of data, can send any amount of data (no limitations of the amount of data like with GET), and the data isn't easily visible to the user in the URL. (Note: Just because the data sent using POST isn't easily visible to the user doesn't mean it is sent securely. Unless you are using an encrypted connection (HTTPS), POST data can be viewed without too much difficult by anyone between your user and your web server. Don't think sending passwords via POST is secure just because someone can't look at the URL to see the password. Unless you are running over an encrypted connection, POST is really no more secure or safe than GET).

Since POST data can be any length and any type, it is not put into an environmental variable on the web server like GET data is. This means our methods for handling GET data sent from a browser, which involved using echo and sed to pull values out of the QUERY_STRING environmental variable won't work for POST data. The following code will read both POST and GET data and assign the values from the POST and GET data into variables in your Bash program:

#This code for getting code from post data is from http://oinkzwurgl.org/bash_cgi and #was written by Phillippe Kehi <phkehi@gmx.net> and flipflip industries # (internal) routine to store POST data function cgi_get_POST_vars() { # check content type # FIXME: not sure if we could handle uploads with this.. [ "${CONTENT_TYPE}" != "application/x-www-form-urlencoded" ] && \ echo "Warning: you should probably use MIME type "\ "application/x-www-form-urlencoded!" 1>&2 # save POST variables (only first time this is called) [ -z "$QUERY_STRING_POST" \ -a "$REQUEST_METHOD" = "POST" -a ! -z "$CONTENT_LENGTH" ] && \ read -n $CONTENT_LENGTH QUERY_STRING_POST return } # (internal) routine to decode urlencoded strings function cgi_decodevar() { [ $# -ne 1 ] && return local v t h # replace all + with whitespace and append %% t="${1//+/ }%%" while [ ${#t} -gt 0 -a "${t}" != "%" ]; do v="${v}${t%%\%*}" # digest up to the first % t="${t#*%}" # remove digested part # decode if there is anything to decode and if not at end of string if [ ${#t} -gt 0 -a "${t}" != "%" ]; then h=${t:0:2} # save first two chars t="${t:2}" # remove these v="${v}"`echo -e \\\\x${h}` # convert hex to special char fi done # return decoded string echo "${v}" return } # routine to get variables from http requests # usage: cgi_getvars method varname1 [.. varnameN] # method is either GET or POST or BOTH # the magic varible name ALL gets everything function cgi_getvars() { [ $# -lt 2 ] && return local q p k v s # get query case $1 in GET) [ ! -z "${QUERY_STRING}" ] && q="${QUERY_STRING}&" ;; POST) cgi_get_POST_vars [ ! -z "${QUERY_STRING_POST}" ] && q="${QUERY_STRING_POST}&" ;; BOTH) [ ! -z "${QUERY_STRING}" ] && q="${QUERY_STRING}&" cgi_get_POST_vars [ ! -z "${QUERY_STRING_POST}" ] && q="${q}${QUERY_STRING_POST}&" ;; esac shift s=" $* " # parse the query data while [ ! -z "$q" ]; do p="${q%%&*}" # get first part of query string k="${p%%=*}" # get the key (variable name) from it v="${p#*=}" # get the value from it q="${q#$p&*}" # strip first part from query string # decode and evaluate var if requested [ "$1" = "ALL" -o "${s/ $k /}" != "$s" ] && \ eval "$k=\"`cgi_decodevar \"$v\"`\"" done return } # register all GET and POST variables cgi_getvars BOTH ALL
(Credit: The above code was written by Phillippe Kehi and came from http://oinkzwurgl.org/bash_cgi.)

How this code works is not as important as what is does. With that in mind, we'll skip a discussion on how the code works and instead jump right into how to use it in your programs:

This code should be early in your CGI program, probably right after the two echo lines that identify the output of your CGI as HTML (echo "Content-type: text/html" and echo ""). The above code consists of three functions, but the magic line is the last one, cgi_getvars BOTH ALL. This line uses the other three lines to create variables in your program that are named the same as the form elements they correspond to, and with the values of those form elements. By using BOTH and ALL, the code will get all form elements and will work with both GET and POST data.

Here's an example of how to use this code. Suppose you have a web page that allows users to enter comments and some additional information. You know some people type a lot, so you don't want to use GET, since your users might type more than GET supports. So you use POST. Your HTML form on you web page looks like this:
<form action="/cgi-bin/submit.sh" method="post">
What is your name? <input type="text" name="name"></input><br>
Did you like the product you received?<br>
Yes <input type="radio" name="likeProduct" value="yes"></input>
No <input type="radio" name="likeProduct" value="no" ></input><br>
Please provide some feedback on your opinions of our product:<br>
<textarea name="comments"></textarea><br>
<input type="submit" name="sub" value="Submit"></input>
</form>
Which will produce a page that looks like this (in this example, the user has already filled in the form):
Example of above form, filled in by a user.

If submit.sh, the CGI our form runs to process the data it sends, contained the code above for POST data processing, it would end up containing variables named "name", "likeProduct", and "comments", since these were the names of form elements in the HTML form that invoked the CGI, and the CGI code copies the form element names into Bash variables of the same name. Since the code also also sets the variables to the values from the form, our script ends up with variables like this:
  • name="John Doe"
  • likeProduct="yes"
  • comments="I think it is great, it saves me lots of time and looks cool."
As you can see, this is a useful piece of code, since it essentially maps form input with variable names in Bash. You can use the values of these variables however you want in your CGI programs.
<-- Page 2: Getting Started | Page 3: Working with Files -->




TigerTronics is primarily sponsored by
NASA