Jim Hoskins

When using Digest Authentication with Cherrypy, if a client tries to authenticate a session from a page with a POST body, the credentials will be rejected even if they are correct. This is due to how Cherrypy’s digest_auth tool handles the initial request and request body.

The Problem

In Cherrypy 3, when utlizing Digest Authentication (via tools.digest_auth), if the first request to a protected area comes from a POST request (with HTTP body), the client will not be able to authenticate properly.

Example: Posting from an HTML form to a protected action, the client will be prompted for credentials. After supplying a good username and password, Cherrypy will reject and the request and ask for credentials again.

The Cause

First a primer on Digest Authentication:

The Process

1. The client requests a protected url URL (eg: /secret)
2. The server tries to authenticate the request from the HTTP headers for the URL. There are not credentials, so this fails. The server replies with an HTTP Status code of 401: Not Authorized
3. The client recieves this requests and gets a username and password (from an operator or database or other source)
4. The client requests the URL again, but adds the credentials to the “Authorization” HTTP Header.
5. The server tries again to authenticate. If the credentials are good, the resource at /secret is served. If authentication fails, The server responds with 401 …
6. rinse and repeat..

The Digest

Unlike HTTP Basic Authentication, the password is not sent in the Authorization header in clear text. Instead, a specified Digest Algorithm is used to create a digest of the password. The client calculates and sends the digest, then the server calculates the digest. If they match, the authorization succeeds.

Among other factors, the inputs to the digest are the username, password, HTTP method, and path.

The Initial Request

When the client makes the initial request (and the request contains a body), the Digest Authentication Tool tries to authenticate, and fails (due to lack of the Authorization header). The tool raises a cherrypy.HTTPError with status code 401 to signal to the client that credentials are required.

The Leftovers

At the point the HTTPError is raised, request.process_body has not been called, so the request body (if there is one) will remain in the socket. When the next request comes from the client, the leftover body from the last request will be prepended to the new request.

So when the client sends  “POST /secret HTTP/1.1 …”, Cherrypy receives “bodyfromlastrequestPOST /secret HTTP/1.1 …” When the line is parsed, request.method is set to “bodyfromlastrequestPOST” instead of “POST”

Since the digest is created using the HTTP method, Cherrypy creates the wrong digest, and the digests don’t match. The authentication fails. The next time this cycle repeats, the body sent in this request will show up, and you won’t be able to log in.

The Solution

The way to fix this is to make sure the body is processed (or cleared from the socket). You can either modify the Cherrypy code, or monkey-patch the code from your application. I don’t like to modify library code because it becomes hard to track, especially when installing the application on multiple servers.

Option A: Patch Cherrypy library files

You can update the cherrypy library file: cherrypy/lib/auth.py

 def digest_auth(realm, users):
    """If auth fails, raise 401 with a digest authentication header.
 
    realm: a string containing the authentication realm.
    users: a dict of the form: {username: password} or a callable returning a dict.
    """
    if check_auth(users, realm=realm):
        return
 
    # inform the user-agent this path is protected
    cherrypy.response.headers['www-authenticate'] = httpauth.digestAuth(realm)
 
    #Added: Process body to clean up requesy
    cherrypy.request.process_body()
 
    raise cherrypy.HTTPError(401, "You are not authorized to access that resource")

The added call to cherrypy.request.process_body() will clean out the request body so the next request will not be fed the body from the last request

Option B: Monkey Patch the code from your application (recommended)

Run the following code before starting your server

import cherrypy
def replace_digest_auth():
    # alias old auth method
    _old = cherrypy.tools.digest_auth.callable
    def new(*args,**kwargs):
        # New method to wrap the old method
        # Catch HTTPErrors with status == 401 and clean up with process_body
        try:
            _old(*args, **kwargs)
        except cherrypy.HTTPError, e:
           if e.args[0] == 401:
               cherrypy.request.process_body()
           raise
 
    # Replace the digest_auth callable with our new safe method
    cherrypy.tools.digest_auth.callable=new
 
# Do the replacement
replace_digest_auth()
# Clean up file
del replace_digest_auth

What this code does is wrap the existing authorization method in a method that cleans up if authentication fails. Notice that though we catch the HTTPError we re-raise it so cherrypy can handle it appropriately.

In Closing

I would definitely consider this a Cherrypy bug, and I will look into whether a ticket has been filed for it. If you need it now, this will do. If you experience any problems, please let me know.

Leave a comment if you need more information or if you want more tips like this.

Tags: Bug, Cherrypy, Digest, Hack, Python, REST

This entry was posted on Friday, July 18th, 2008 at 8:04 pm and is filed under Cherrypy, Python. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.