Using SCGI with Apache 2.x and Python

This guide describes how to use SCGI with Apache 2.x, to serve an application written in Python. The SCGI package also comes with modules to use the protocol with Apache 1.x or lighttpd, and frameworks for applications in other languages are available (including LISP).

A complete setup for an SCGI application involves these four components:

1. A web server to process incoming http requests--in our case, Apache 2.

2. The Apache SCGI module to delegate requests over the SCGI protocol.

3. An SCGI server process based on the Python scgi_server module.

4. An application to be run in child processes forked off by the server process.

A single SCGI server process serves only one application, but multiple instances of it. Each instance lives in its own child process of the SCGI server. More child processes are created on demand.

Apache module

To install the module by hand, cd into the apache2 subdirectory of the SCGI source tree, and, as root, make install. On Debian-based systems and Gentoo, however, just install the libapache2-mod-scgi package. RPM users can install apache2-mod_scgi.

Next, set up the webserver to load the module and delegate requests (in this example, for the http path "/dynamic") to a separate server process accepting SCGI requests over a TCP socket. That server will typically run on the same machine as the http server, but it doesn't have to. Here's a snippet of Apache 2 config to delegate requests for the http path /dynamic to local TCP port 4000, which is the default:

# (This actually better set up permanently with the command line
# "a2enmod scgi" but shown here for completeness)
LoadModule scgi_module /usr/lib/apache2/modules/mod_scgi.so

# Set up a location to be served by an SCGI server process
SCGIMount /dynamic/ 127.0.0.1:4000

The deprecated way of delegating requests to an SCGI server is as follows:

<Location "/dynamic">
    # Enable SCGI delegation
    SCGIHandler On
    # Delegate requests in the "/dynamic" path to daemon on local
    # server, port 4000
    SCGIServer 127.0.0.1:4000
</Location>

Note that using SCGIMount instead of SCGIServer allows Apache to compute PATH_INFO as most software expects.

Here's a description of the configuration directives accepted by this Apache module. The information is largely derived from the source, so please excuse (or better yet: help improve) poor descriptions:

Server process

To install the Python module for creating SCGI applications, go to the main SCGI source directory and run python setup.py build and then, as root, python setup.py install. Or, on most GNU/Linux systems, just install the python-scgi package.

The hard part is creating that application server process. The SCGI home page provides a server implementation in Python, called scgi_server.py, but this is really just a kind of support module. In order to port an application to use this server, write a main program in Python that imports this module and uses it. There is one, fairly complex, example that makes an application called Quixote run over SCGI.

The main program that you write for yourself creates a handler class describing how requests should be processed. You then tell the SCGI server module to loop forever, handling incoming requests and creating processes running your handler class as needed. It is your own responsibility to ensure that the server process is running.

Once the SCGI server process runs, it can accept requests forwarded by the http server. It will attempt to delegate every request that comes in to an existing child process, invoking its handler. If no child process is free to take the request, an additional child process is spawned (if the configured maximum number of child processes is not exceeded; if it is, the request blocks until a child process becomes free). If a handler "dies" (e.g. exits with an exception), a new child is spawned to replace it.

Writing a handler

The server process must import scgi_server, then derive handler classes from scgi_server.SCGIHandler. Just scgi_server.py can also be run standalone, in which case it return pages displaying the details of your browser's request.

Your handler class should be derived from the SCGIHandler class defined in scgi_server.py, and override its produce() function to process a request. This is new in SCGI 1.12. Before that, the function to override was handle_connection() which involved a lot more work.

Besides self, the produce() function takes several arguments:

1. a dict mapping names of CGI parameters to request details

2. size (in bytes) of the request body

3. an input stream providing the request body

4. an output stream to write a page to.

Alternatively, override SCGIHandler.produce_cgilike() which can read the request body from standard input, write its output to standard output, and receives its request details both as an argument dict and added to its environment variables.

Your output should start out with an http header, so normally you'd want to start out with something like "Content-Type: text/plain\r\n\r\n" followed by page content.

Besides a header containing CGI variables, the request may also contain a body. The length of this body is passed as the CGI parameter CONTENT_LENGTH, but for your convenience is also converted to an integer and passed separately to the produce() function. If your handler needs the request body, it can read this number of bytes from its input socket. Do not rely on EOF; explicitly read the correct number of bytes (or less, if that's what you want).

Writing a server

Your main SCGI server program should create an SCGIServer object (defined in scgi_server.py), passing in your handler class for the handler_class parameter, and then call its serve() method which will loop indefinitely to process requests:

def main():
    SCGIServer(handler_class=MyAppHandler).serve()

if __name__ == "__main__":
    main()

You may want to support command-line options to influence various options of the server's operation. The SCGIServer initialization function takes several arguments:

Links

• SCGI home page

• SCGI download

• Setting up Django with SCGI

• Setting up Quixote with SCGI

• Using Ruby on Rails with SCGI

• bazaar-ng (bzr) repository: http://quixote.python.ca/scgi.dev/