Ruby has special syntax for making strings, arrays and system commands easier to write. They allow you to use different characters as delimiters so you can minimize escaping in your literals.

The syntax

The syntax for the % literals is a percent symbol (%) a letter which defines what kind of literal it is (Q, q, w, x, r) a delimiter,  the content, and the closing delimiter.

The delimiter can be any character, and is defined as whatever is immediately after the letter in the syntax. For example %Q!content! , the delimiter is the ! and it surrounds the content. There are special cases when the delimiter is { or (, the closing delimiter will be } or ) respectively.

%Q and %q (Percent Q): Strings

%Q!Some String of “Characters”! <==> ” Some String of \”Characters\” “

%Q is the equivalent to a double-quoted ruby string. #{expression} evaluation works just like in double-quoted strings, even if you use %Q{} as your delimiter!

You can also leave off the Q and it will have the same functionality. I recommend leaving the Q in to be more clear.

%q!Some String of “Characters”! <==> ‘Some String of Characters’

The %q is just like %Q, but acts the same as a single-quoted string. Whatever is inside the delimiters is returned as a string.

You can remember %Q is for strings because it acts like Quotes.

More info here: http://docs.huihoo.com/ruby/ruby-man-1.4/syntax.html#string

%W (Percent W): Arrays

%W(North South East West) <==> ["North", "South", "East", "West"]

%W (and %w) allow you to create an Array of strings without using quotes and commas.

The delimiter rules are the same as strings, but typically parentheses are used. The content inside the delimiters are split by white-space, and put into an array. This is great if you have a hard coded list of single word strings.

You can remember %W is by thinking of it as a White-space divided Array.

More info here : http://docs.huihoo.com/ruby/ruby-man-1.4/syntax.html#array

%x (Percent x): System Execution

%x{ ls /usr/local } <==> `ls /usr/local`

%x allows you to call system commands, equivilent to wrapping the command in `s (grave accents). The benefit of the $x{} syntax is you don’t have to escape your accents in commands that use them.

You can remember to use X because it eXecutes a command.

More info here: http://docs.huihoo.com/ruby/ruby-man-1.4/syntax.html#command

%r (Percent r): Regular Expressions

%r{/usr/bin/} <==> /\/user\/bin\//

%r is really handy for regular expressions that contain /s (forward slashes) which are the default delimiter for regular expressions and have to be escaped.

Remember to use %r with regular expressions.

More info here: http://docs.huihoo.com/ruby/ruby-man-1.4/syntax.html#regexp

I hope this information is helpful. Please leave a comment if this helped or if I left something out.

Here comes Validifier to save the day! Validifier makes your Flash embed code Valid XHTML! There are a lot of different reasons embed code fails validation, Validifier takes care of all of them. Just paste your current code into the box, click Validify! and use the code provided. It’s Valid, Compatible, and Free!

This is the first product released by Done21, check out Almost Done: The Done21 Blog for more information on upcoming tools and product releases!

It may not be immediately obvious from the CherryPy documentation, but CherryPy includes all the tools needed to implement HTTP authentication, both Digest and Basic. You can see what the CherryPy website has documented about authentication at the CherryPy Builtin Tools Page. The best documentation for this is available in the book: CherryPy Essentials: Rapid Python Web Application Development

Basic vs Digest

HTTP supports two types of authentication, basic and digest. To a user accessing a protected page through a web browser, both types look the same. The browser will prompt the user for a username and password. The difference is how the client (browser) sends the password to the server.

With basic authentication, the client sends the password in clear-text, so any malicious attacker who can see the request can see the password. With digest authentication, the client does not send the password, but rather a digest based on the password and other factors. The server will also compute the digest using the known correct password. If the digest sent by the client matches the one calculated by the server, authentication succeeds.

The catch with digest authentication is the users’ passwords must be stored stored in clear-text on the server** in order to calculate the digest. On the flipside, basic authentication must send the password in clear-text through the network. If you are going to use basic authentication, it is recommended you use SSL to prevent the password from being sniffed.

** This is not entirely true, The digest algorithm allows you to store a partially digested password on the server side, however CherryPy has no easy way to specify this.

Digest Authentication

The following code creates a simple application with a public page and a secure page using digest authentication. The server configuration defines that the /secure url is to be protected with digest authentication (via the tools.digest_auth Tool available in CherryPy)

import cherrypy
 
class RootServer:
    @cherrypy.expose
    def index(self):
        return """This is a public page!"""
 
class SecureServer:
    @cherrypy.expose
    def index(self):
    return "This is a secure section"
 
if __name__ == '__main__':
    users = {"admin": "secretPassword",
             "editor": "otherPassword"}
 
    conf = {'/secure': {'tools.digest_auth.on': True,
                        'tools.digest_auth.realm': 'Some site',
                        'tools.digest_auth.users': users}}
    root = RootServer()
    root.secure = SecureServer()
    cherrypy.quickstart(root, '/', config=conf)

Run this code and visit http://localhost:8080/ . Now visit http://localhost:8080/secure. When your browser prompts you to do so, enter “admin” and “secretPassword” for the username and password respectively.

The /secure configuration section sets up the digest authentication with the following options:

• ‘tools.digest_auth.on’: This boolean enables digest authentication for the given section (in this case /secure)
• ‘tools.digest_auth.realm’: This string defines the realm to supply to the client.
• ‘tools.digest_auth.users’: This dictionary defines users and passwords ({user: password}). This can also take two other forms, please see below.

There is a bug in the CherryPy Digest Authentication tool that prevents authentication when the request is a post request with a body. See Cherrypy Digest Auth POST Problem (and Solution!)

Basic Authentication

We are going to use the same code to implement basic authentication, but we will change the configuration variables.

import cherrypy
 
class RootServer:
    @cherrypy.expose
    def index(self):
        return """This is a public page!"""
 
class SecureServer:
    @cherrypy.expose
    def index(self):
    return "This is a secure section"
 
if __name__ == '__main__':
    users = {"admin": "secretPassword",
             "editor": "otherPassword"}
 
    def clear_text(passwd):
        return passwd
 
    conf = {'/secure': {'tools.basic_auth.on': True,
                        'tools.basic_auth.realm': 'Some site2',
                        'tools.basic_auth.users': users,
                        'tools.basic_auth.encrypt': clear_text}}
    root = RootServer()
    root.secure = SecureServer()
    cherrypy.quickstart(root, '/', config=conf)

Run this code as before, and supply the same username and password. Can’t tell the difference? Remember, the difference is in how it is transmitted. Here are the options we supplied to enable basic authentication:

• ‘tools.basic_auth.on’: This boolean enables digest authentication for the given section (in this case /secure)
• ‘tools.basic_auth.realm’: This string defines the realm to supply to the client.
• ‘tools.basic_auth.users’: This dictionary defines users and passwords ({user: password}). This can also take two other forms, please see below.
• ‘tools.basic_auth.encrypt’: This takes a function that encrypts the user-supplied password to match a pre-encrypted password in the user dict. (In our case, the function clear_text we defined just passes the password through, allowing us to store password in clear-text for ease of demonstration). If you do not supply this option, it will be assumed the passwords are stored in MD5.

Defining Users

You can supply either a dict or function to the ‘tools.******_auth.users’ option to define your users. There are three ways to supply users: as a dict ( {user:password} ), as a function that returns a dict ( {user:password} ), or as a function that takes a username as an argument, and returns a password.

Remember that password could mean the clear-text password, or an encrypted password, depending on whether your using basic or digest, and the encryption option specified.

Using a dictionary

Simply supply a dictionary of {username: password} to the users option of the authentication configuration.

users = {'user1': 'password1',
         'user2': 'password2}
 
conf = {'/secure': {'tools.digest_auth.on': True,
                     'tools.digest_auth.realm': 'Some site',
                     'tools.digest_auth.users': users}}

Using a function that returns a dictionary

Supply a function that takes no arguments and returns a dictionary containing all usernames and passwords

def get_users():
    return {'user1': 'password1',
            'user2': 'password2}
 
conf = {'/secure': {'tools.digest_auth.on': True,
                     'tools.digest_auth.realm': 'Some site',
                     'tools.digest_auth.users': get_users}}

Using a Function Which Takes a Username (Most Useful)

Many times, applications have so many users that it does not make sense to pass the list of all users around like the options above. I have not seen this feature documented anywhere, but the code allows it. You can specify a function that takes a username string as an argument, and returns the password for that user. This function could consult a database or other mechanism, and allows your application to add and remove user more easily.

def find_password(username):
    #INSERT PASSWORD LOOKUP LOGIC HERE
    password = db.find_password_for_user(username)
    return password
 
conf = {'/secure': {'tools.digest_auth.on': True,
                     'tools.digest_auth.realm': 'Some site',
                     'tools.digest_auth.users': find_password}}

Retrieving the Username

Once authenticated your application will likely want to know which user is logged in. The username of the user who has successfully authenticated will be available at ‘cherrypy.request.login’. This variable wil be False if authentication failed.

In Closing

It may not be well documented, but the tools for basic and digest authentication are already built in to CherryPy. You just have to know how to use them, and now you do.

If there are any problems with the code or this post in general, leave me a comment and I will make it right.

The Problem

In Cherrypy 3, when utlizing Digest Authentication (via tools.digest_auth), if the first request to a protected area comes from a POST request (with HTTP body), the client will not be able to authenticate properly.

Example: Posting from an HTML form to a protected action, the client will be prompted for credentials. After supplying a good username and password, Cherrypy will reject and the request and ask for credentials again.

The Cause

First a primer on Digest Authentication:

The Process

1. The client requests a protected url URL (eg: /secret)
2. The server tries to authenticate the request from the HTTP headers for the URL. There are not credentials, so this fails. The server replies with an HTTP Status code of 401: Not Authorized
3. The client recieves this requests and gets a username and password (from an operator or database or other source)
4. The client requests the URL again, but adds the credentials to the “Authorization” HTTP Header.
5. The server tries again to authenticate. If the credentials are good, the resource at /secret is served. If authentication fails, The server responds with 401 …
6. rinse and repeat..

The Digest

Unlike HTTP Basic Authentication, the password is not sent in the Authorization header in clear text. Instead, a specified Digest Algorithm is used to create a digest of the password. The client calculates and sends the digest, then the server calculates the digest. If they match, the authorization succeeds.

Among other factors, the inputs to the digest are the username, password, HTTP method, and path.

The Initial Request

When the client makes the initial request (and the request contains a body), the Digest Authentication Tool tries to authenticate, and fails (due to lack of the Authorization header). The tool raises a cherrypy.HTTPError with status code 401 to signal to the client that credentials are required.

The Leftovers

At the point the HTTPError is raised, request.process_body has not been called, so the request body (if there is one) will remain in the socket. When the next request comes from the client, the leftover body from the last request will be prepended to the new request.

So when the client sends  “POST /secret HTTP/1.1 …”, Cherrypy receives “bodyfromlastrequestPOST /secret HTTP/1.1 …” When the line is parsed, request.method is set to “bodyfromlastrequestPOST” instead of “POST”

Since the digest is created using the HTTP method, Cherrypy creates the wrong digest, and the digests don’t match. The authentication fails. The next time this cycle repeats, the body sent in this request will show up, and you won’t be able to log in.

The Solution

The way to fix this is to make sure the body is processed (or cleared from the socket). You can either modify the Cherrypy code, or monkey-patch the code from your application. I don’t like to modify library code because it becomes hard to track, especially when installing the application on multiple servers.

Option A: Patch Cherrypy library files

You can update the cherrypy library file: cherrypy/lib/auth.py

 def digest_auth(realm, users):
    """If auth fails, raise 401 with a digest authentication header.
 
    realm: a string containing the authentication realm.
    users: a dict of the form: {username: password} or a callable returning a dict.
    """
    if check_auth(users, realm=realm):
        return
 
    # inform the user-agent this path is protected
    cherrypy.response.headers['www-authenticate'] = httpauth.digestAuth(realm)
 
    #Added: Process body to clean up requesy
    cherrypy.request.process_body()
 
    raise cherrypy.HTTPError(401, "You are not authorized to access that resource")

The added call to cherrypy.request.process_body() will clean out the request body so the next request will not be fed the body from the last request

Option B: Monkey Patch the code from your application (recommended)

Run the following code before starting your server

import cherrypy
def replace_digest_auth():
    # alias old auth method
    _old = cherrypy.tools.digest_auth.callable
    def new(*args,**kwargs):
        # New method to wrap the old method
        # Catch HTTPErrors with status == 401 and clean up with process_body
        try:
            _old(*args, **kwargs)
        except cherrypy.HTTPError, e:
           if e.args[0] == 401:
               cherrypy.request.process_body()
           raise
 
    # Replace the digest_auth callable with our new safe method
    cherrypy.tools.digest_auth.callable=new
 
# Do the replacement
replace_digest_auth()
# Clean up file
del replace_digest_auth

What this code does is wrap the existing authorization method in a method that cleans up if authentication fails. Notice that though we catch the HTTPError we re-raise it so cherrypy can handle it appropriately.

In Closing

I would definitely consider this a Cherrypy bug, and I will look into whether a ticket has been filed for it. If you need it now, this will do. If you experience any problems, please let me know.

Leave a comment if you need more information or if you want more tips like this.