Beginning Pylons - an introduction to using the Pylons web framework
Copyright (C) 2007 Dipl.-Inform. Christoph Haas <
This introductory article was written for Pylons 0.9.6.1 and SQLAlchemy 0.4. Please consider reading the documentation on pylonshq.com instead for more up-to-date information.
- Welcome to Pylons
- Installing Pylons
- pymarks - an example Pylons application
- Create the basic Pylons application: paster create
- Paste(r) - your Pylons toolbox
- paster serve - the built-in web server
- Files and directories in your project
- Remove the static index.html welcome page
- models - accessing a database
- Controllers
- Routes (URL dispatching)
- Templates
- Databases
- Paster's Shell
- Show a list of contacts
- Add an edit form
- Routes
- Templates
- Forms
- Webhelpers
- Form field validation by Formencode
- Sessions
- Databases
- Javascript and AJAX
- Logging and Debugging
- Custom error pages
- Authentication with Authkit
- Internationalization (i18n)
- Deployment
- Testing
- Under the hood
- Glossary
- Resources
- About the author
Welcome to Pylons
"When you do things right people won't be sure you've done anything at all." (Futurama)
You probably came here because you want to create web-based applications in Python and are now looking for an easy way to do that. Perhaps you have even written web applications using PHP or programmed CGIs in Perl. Pylons is a web framework written in Python. It provides a basis for your own web applications. Some knowledge about HTML and Python, a pot of tea and some open-mindedness is all you need.
If you already used other frameworks like Django, Turbogears or Ruby-on-Rails you will feel right at home here. This introduction can help you to quickly get an impression of how developing with Pylons feels. You may skip the next section
This document is supposed to take your hand, show you around and accompany you while you write your first web applications in Pylons. Just do not expect this document to be a complete reference book for any situation. The Pythonweb Wiki is the central documentation storage and provides lots of articles, cookbook recipes, tips and examples. Please look there for help. The goal of this introduction rather is to make you understand and properly use Pylons' components.
What makes Pylons stand out from other frameworks?
The Python wiki has a page on web frameworks and lists over 30 different frameworks. You surely do not have enough time trying them all to find your personal favorite. Let me outline a few features for some appetite wetting:
Comfortable interactive debugger
Programming means making mistakes. And searching for the cause of an error distracts you from the task at hand and is annoying. Especially in web applications you usually do not have a fancy debugger at hand that allows you to view all variables and the piece of code where the error occured. Pylons offers a great online debugger. If your application throws an exception you will get a traceback on the web page, can view local variables and can even enter Python statements. Sometimes people even deliberately throw in a raise Exception statement to make the application stop at that line so they can investigate what is going on. The debugger even works with AJAX requests as it prints the debug URL on the console.
Exploring the world: Paster shell
A Pylons application uses many different Python modules to run. The environment in which your application is running seems pretty opaque. Fortunately you can run the paster shell which is a normal Python shell (or even an ipython shell if you have it installed) but has access to all the global variables and utility functions that Pylons offers. Play with your database models, explore the "Webhelpers" utilities, browse through the available global variables. And if things work as you expect you just copy them into your application without much testing.
Simplifying the development cycle: --reload
Actually the full command for running an application is paster serve --reload development.ini. The reload option makes the Paste(r) web server watch your Python files. If you change the code and save the Python file the paster will detect that change, load the new code and run your application. Your development cycle is saving the file and reloading the page in the browser. It could not be simpler.
WSGI - ready for production use
Pylons does not depend on a certain web server. Other frameworks may have a web server built in - so you depend on its stability and features. If you find that another web server is better suited then Pylons can use it - provided that it knows about WSGI. WSGI is a protocol between web servers and web applications. So any WSGI web application can run with any WSGI web server. And there are many like Paste, FastCGI, SCGI, CGI or mod_python on Apache. And you can even plug in WSGI middleware which is code you can simply plug between the web server and the web application to provide features like authentication.
Smooth upgrades
Pylons is a template for your project. It creates a number of files where your HTML templates, your database models and your controller code goes into. You will surely change that template a lot. But what happens when a new version of Pylons is released? No problem. Pylons can show you the differences between your files and the new template and you are free to adopt any changes. So you are always up-to-date without starting from scratch.
The web developer's toolbox: Webhelpers
The creators of Pylons ported a lot of functions from (Ruby on) Rails and make it available to every Pylons application as the Webhelpers library. It contains functions to deal with HTML tags and HTML forms, they convert numbers into human-readable forms, deal with RSS feeds and split large output (like database tables) into pages by only a few lines of Python code.
Beautiful URLs
While in classical CGIs you cannot really control the looks of the URL (it mostly starts with /cgi-bin/myscript) the Routes component of Pylons allows you to use any URL scheme you like. Configure it so that http://your.web.server/articles/2007/03 will call the articles controller and pass the variables year=2007 and month=03.
Exchangable components
Pylons is open. You can use it with different templating languages and different database toolkits. A certain set of components is already configured so that you can get started quickly. But it is really easy to replace them if you hit some borders.
Seperated models, views and controllers
In Pylons you seperate the models (your database schema), the views (HTML templates) and the controllers (your application code). That is called MVC (model, view, controller). Using MVC you can change database models or the HTML templates without bothering about other Python modules.
Friendly community
Join the Pylons community by subscribing to the mailing list or join us on \#pylons on irc.freenode.net. Meet both users and developers. Get your questions answered and make proposals that the developers may include in the next release. Pylons is still working towards 1.0.
What is a framework?
You read it already: Pylons is a web framework. Generally a framework is a collection of components (like other programs, code libraries or even scripting languages) that work together as a basis for your own application. The application that you write runs inside the context of the framework. When you develop a web application you often have the need for...
- something that prints HTML tags
- something that reads parameters that are sent via HTTP forms
- something that stores information in a database and gets it back
- something that contains your application logic
- something that deals with user accounts
- something that maintains cookie-based sessions
- something that speaks HTTP and sends web pages to your browser
- something that maps URLs to parts of your application
If you have already written a web application (and be it just a simple script that prints "Hello World") you will have done most of the above parts yourself. You probably just used print statements to create the HTML and did not care about complex templating systems. That works well in simple applications. But programs tends to grow larger and without a good concept become less maintainable. You may later have a dozen programs that output HTML pages that look similar. Sure, you can move some common parts into a module that you use from all your programs. But basically you create solutions for typical tasks that almost every web applications have.
Fortunately some developers invested their spare time to create Python modules that fulfil these everyday tasks. For example there is a Python package called Mako that deals with creating HTML output - a so called templating system. Using the Mako syntax you create a template file that contains your HTML page. But you can add Python statements right in that file where you need it and can access variables from other parts of your application. There are if/then/else constructs so that every user of your web site only sees what is needed. And as these templates can even use each other you can easily change the looks of your web site by just changing a master template.
This is just one example of how such components can make your life as a programmer easier. There are a lot of helpful components. And it is a good idea to use them instead of dealing with these boring everyday tasks. Now imagine you take a number of these useful components and get them to work together. A web framework like Pylons is exactly doing that. You may think of Pylons as the "glue" between these components.
What does MVC mean?
You may have heard the abbrevation MVC in the context of web frameworks. MVC is short for Model View Controller and depicts a design pattern used in software engineering. The MVC approach splits your application into:
- Models
- They contain data that your application works with. Models do not contain any information on the meaning of this data. Often the model refers to database tables.
- View
- It is responsible to read data from a model and display it to the user. (While models and controllers are called the same in Pylons you may notice that the equivalent part to a view in Pylons is basically the template.)
- Controller
- Here lies the logic of your application. The controller activates views to display data to the user or gets information from the user and stores them to the models.
You may wonder what is so special about that design pattern. You already wrote programs that communicate with your database, send HTML to the user and contain some kind of intelligence to steer everything. The idea of MVC is that you seperate all these tasks so that you can change one part without breaking another.
For a more complete explanation see Wikipedia's article on MVC.
Pylons' components
What puzzles many beginners is that Pylons works as a collection of components. Pylons is not a framework that contains everything you need. So there is no single handbook describing all the components. Pylons rather connects all the components so you can easily use them together. That means you will still have to learn about each of the components. During this introductory article I will explain the basics of each one. But to fully understand the components you will have to read their respective documentation. Pylons does not even force you to use a fixed set of components. The standard Pylons project uses a good selection though that I would like to show you. It all begins with...
Paste
Paste is a collection of all kinds of tools and programs that help developing and running WSGI applications. It comes with a WSGI web server that you can use to run your application. The central place to store configuration information in your Pylons project is the ini file. By default it is called "development.ini". Later on your deployment server you will likely call it differently - e.g. "myserver.ini". This file is important for Paste so it knows which web applications to load and run. As you can see at http://pythonpaste.org/script/ your development.ini consists of two sections:
- [server:main] This section is meant for Paste itself. It tells Paste which IP address to bind its web server to and which TCP port to use. The use= line refers to the egg that contains Paste's web server. (Eggs are just pieces of Python software.)
- [app:main] This section contains configuration settings for your own web application. It refers to the egg that is actually your application.
You can start the web server in combination with your Pylons application using paster serve development.ini. Unless you specify any app-name as an option Paste will look for an application called main. That is how it finds [app:main]. After some internal Paste logic your config/middleware.py is loaded and its function make_app is called that returns the actual WSGI application that Paste needs. You can even alter that function to add special functionality if you need it.
(Mike Orr has even written an article on the gory details of what happens when your application is run in case you are curious. Nothing for the faint-hearted though.)
Write HTML templates: Mako
This is the templating system. Instead of using print statements to print out HTML that is sent to the user's browser you will use templates. A template can contain pure HTML that is displayed in the browser. If you had a static page you would tell your application to just
render('/welcome.mako')
and Mako will load the file templates/welcome.mako and send it to the browser. Templates are helpful so that you can keep the structure of your document in the template and most of the programming logic in the Python part (called controllers).
One of the more interesting features of templates is that they can be jazzed up by control statements and Python code. A simple example:
<html>
<body>
<h1>Welcome</h1>
<p>Today we serve:</p>
<ul>
% for meal in ['calzone', 'porkpie', 'pizza']:
<li>${ meal }</li>
% endfor
</ul>
</body>
</html>
Templates offer some amazing features. You can use variables that you defined elsewhere in your application. You can include or inherit from other templates. If you know PHP a little then this way to mix HTML with programming logic may appear well-known to you.
Access your database: SQLAlchemy
Most web applications need a database as a persistent storage. You can save your customer information in there, articles of your web shop or any other data that needs to live longer than the duration of an HTTP request. An SQL database can be accessed by the SQL query language. But with SQL toolkits like SQLAlchemy you can make database access more convenient and pythonic. That doesn't mean that you will save time instantly because you need to carefully define your database models but in the long run you will find it easier. And SQL toolskits are not bound to specific database backends. You can easily switch from MySQL to PostgreSQL by just changing the connect string. SQLAlchemy consists of two parts:
- an SQL toolkit
- an object relational mapper
The 'SQL toolkit' part is easy to explain: you tell SQLAlchemy how your database tables look in Python and don't need to write SQL statements any more. Think of SQLAlchemy as an abstraction layer. If you use that abstraction everywhere in your application you do not have to depend on a certain database backend and in many cases don't need to care about subtleties of the different database servers.
An even more interesting part is the object relational mapping that SQLAlchemy delivers. Imagine that you define a trivial empty Python class like:
class Customer(object):
pass
Then you call a mapper function that connects your class to a database table. Now you do not even need to work with tables any more but have some new useful methods on your class. A simple example:
mycustomer = query(Customer).filter_by(name='Jack').first()
This will query the database and get a database record with the name 'Jack' from the table that represents your customers. All the database fields are automatically added to the mycustomer object as class properties. Example:
print "The customer's phone number is", mycustomer.phone
Isn't this more readable than writing SQL queries? And there's more like e.g. many-to-many relationships that are maintained easily or automatic retrieval of rows from other tables that are connected by foreign keys.
SQLAlchemy comes with good documentation but I will explain some basic features of SQLAlchemy later.
Invent your personal URL scheme: Routes
An interesting feature of web frameworks is that you entirely control what you do with the user's HTTP request. That includes the URL itself. Routes' job is to determine what Python code must be run depending on how the URL looks like. Let's take the following URL as an example:
http://example.com/blog/2007/01
You can tell Routes that you want the BlogController to handle this request and pass on the '2007' as a year and the '01' as the month. In Routes syntax this looks like:
m.connect('blog/:year/:month', controller='blog')
The URL routing configuration is found in config/routing.py of your application by the way. By default a URL like
http://example.com/blog/2007/01
is passed to the BlogController (the controller name is indeed computed by taking the string "blog" by adding the word "controller" and using camel-case) with action='2007' and id='01'.
Few wheels to reinvent: Webhelpers
Even with the components described above you will still find yourself writing utility functions for common tasks. The Webhelpers is a package of such helper functions that you can use in your Pylons applications. Some examples of what they do:
- handle AJAX elements
- format text and numbers
- print URLs, links, HTML form elements
- create RSS feeds
There is also a part of Webhelpers that contains functions ported from the Ruby on Rails framework in case you have used Rails before. The Webhelpers are not only helping you to save time on boring tasks but also control the script.aculo.us Javascript library that can be used for AJAX functionality and cool graphical effects to impress your coworkers. To get an idea on how that looks take a journey to https://github.com/madrobby/scriptaculous/wikis But there are a lot of other Javscript frameworks available that work well without the help of Webhelpers.
Caching and cookie-based sessions: Beaker
Beaker is one of the less visible parts of Pylons. It provides cookie-based sessions for example. A session is basically a dictionary where you can store per-user information. To attribute a dictionary to a certain user Beaker sends a randomly created session identification to the user - stored in a cookie. So when the user accesses your web site, their browser will send the cookie along with the HTTP request and Beaker gets the respective session dictionary for you. So you can store temporary sessions per-user like whether they are logged in, what their username is or what language they like to read your web site in. As the sessions are server-based the user can't change the information directly by sending fake cookies. And Beaker also uses a secret string to cryptographically sign the cookie string so the user cannot easily change the cookie and hope to get accidental access to other users' sessions. Beware that sessions are not a permanent storage. They expire after a day by default. Use a database if you need to store information for longer.
Beaker is also used to cache HTML output from the Mako templates. If possible Beaker takes a HTML page that has already been rendered instead of having Mako compute the page again. That saves a few CPU cycles.
Are there other web frameworks?
Yes, a lot of them. See the Pythonweb Wiki for an impressive list. Two other commonly used frameworks are Django and Turbogears. It is hard to write an unbiased comparison because basically all three frameworks try to provide similar functionality. And different programmers may have different expectations. But let's try a comparison anyway.
- Django
- Django targets the web developers who deal mainly with content. The makers of Django are from the newspaper business. They say that they were often asked to implement certain features on their web site with tight deadlines. So they wrote their own framework. Django does not re-use many components. Even their way to access databases (the so called object-relational mapper) is home-made. If you have similar requirements as Django's makers you will find that it is an excellent framework. But it is not meant to have its components replaced easily in case you don't like them. Their templating language is not very "pythonic" - it is a language of its own although an easy one. A nice feature is their admin frontend - it will create HTML forms for you to easily manage the data in your database. And it comes with a nifty way to map URLs to certain functions in your application based on regular expressions. AJAX is basically possible with Django but not very comfortable. They have a web server built in but they suggest you run Django applications on an Apache with mod_python. One thing that can truly be said about Django: they have good marketing. :) Django is well documented, has a large community and there is even a book currently written. All in all Django is a "don't worry - be happy" framework that is tightly integrated and which will help you get things done quickly if you don't care too much which components it uses and don't expect all parts to look perfectly pythonic.
- Turbogears
- Turbogears makes use of existing components. Their web server runs on CherryPy, their HTML templates come from Kid, they access databases through SQLObject and even include a good way to write interactive AJAX applications through the use of a Javascript library called Mochikit that is tightly integrated. They make it easier than Django to replace certain components in case you don't like them. For example you can throw away SQLObject and use SQLAlchemy instead; although many parts of Turbogears still lack full support for SQLAlchemy. Turbogears's URL mapping scheme is less explicit. You can't tell what is supposed to happen for certain URLs. Instead a program of the same name as the first part of the URL is called. So the URLs the user sees may appear less nifty. A nicer feature is their set of web-based utilities. Among them is their WidgetBrowser to browse the library of supplied form elements - or Catwalk - a database administration frontend similar to what Django provides. Turbogears is well documented and has a large community, too. All in all Turbogears has nifty features like its widget library or a good set of utility functions that make life easier. Its documentation is partly incomplete and rather contains recipes and examples than a nice reference. It's obvious that it moves to SQLAlchemy but many features only work well with SQLObject yet. Turbogears may be a nice choice for programmers scared by the magic of Django and like it a bit more explicit. By the way, Turbogears is using Pylons as a base for future releases. Rumors say that Turbogears and Pylons will one day become merged.
- Pylons
- Pylons isn't exactly new but has a smaller community than Django or Turbogears. That does not mean it's technically worse than other frameworks. It is unique in that it is more minimal and flexible than other frameworks. Basically you need to learn the basics of the components it uses. Next you need to understand how these components are controlled from within a Pylons application. So Pylons is the glue between the components with an additional powerful package called "Webhelpers" that aids in many areas from AJAX to RSS feed creation. Pylons' flexibility means that you can exchange the components very easily. You don't like SQLAlchemy? Use SQLObjects. You read that Myghty is superseded by Mako? Use Mako. Pylons may not always hold your hand but it will try to make it easy to use different components than what Pylons uses by default. And you will likely be happy with what Pylons offers by default. The components are carefully chosen by looking at what other web frameworks do wrong. The documentation for Pylons is often lacking and chaotic. It seems like some fans of Pylons tried it because they generally liked the idea of web frameworks but were frustrated with Django or Turbogears. Some say that Pylons is the Ruby-on-Rails written in Python. It is no worse than other frameworks. It just started to focus on the user instead of the developer a bit late.
How is using a web framework different from CGI programming?
Many developers have programmed CGIs in a scripting language like Perl. Mostly because that is what most web hosting services offer. A CGI is a program that is launched by the web server when a user requests a certain URL in the browser. The CGI reads information it gets sent from the browser (and some environment variables that are set by the web server) and prints HTML that is shown in the browser. Your CGIs do not work as standalone applications. You still need a web server like Apache to accept the request and launch the CGI.
Pylons (WSGI) applications are actually similar to CGI applications. While you need a CGI-aware web server to run CGI script you need a WSGI-aware web server to run WSGI applications. Unless you have any special needs you can well use the web server built into Paste. Pylons makes excessive use of the "Paste" tools anyway so you will not have to install any extra software. This approach has several advantages:
- you can develop your application on your workstation easily because you do not need a complex set up with a third-party web server like Apache. Just start up the built-in web server process and try out your application. The server will even check if any of your program parts will change and reload the server automatically. So your development cycle is simple: save your changes in your Python editor and press reload in the browser.
- you can control all aspects of the web server within your application (like returning HTTP error codes) that are otherwise handled by the web server. That enables you to add middleware for authentication or to deal with 404 (page not found) errors exactly in the way you want.
- you can have flexible URL schemes. In CGIs the URLs always reflect the name of the CGI to be run. With Pylons you define what action is run when a certain URL pattern matches (e.g. /articles/2007/01)
- the web server keeps the whole application in memory which boosts the speed of your application. When using CGIs the web server will load and launch your script interpreter everytime the CGI is requested which creates much overhead on busy web sites. Pylons initialises everything it needs upon startup and connects to the database already.
Installing Pylons
"A consultant is someone who in exchange for your watch will tell you the time." (found on a mailing list)
...on Debian/Ubuntu
Debian GNU/Linux - and derived distributions like Ubuntu - provide binary packages for all components that Pylons needs. Unless you have a very good reason and good knowledge about Debian you should stay with the packages. Note that Debian publishes stable releases roughly every two years and that the current release (codename "Etch") does not even contain Pylons. So I suggest you run the 'testing' branch instead. Drawbacks are that you do not get security updates for the installed software any more (you need to take care of that yourself) that every "aptitude upgrade" may upgrade other installed packages on your system, too, and that minor breakages after upgrades are more likely than in "stable".
Your /etc/apt/sources.list should contain a Debian mirror of testing:
deb http://ftp.debian.org/debian/ testing main
Then get the list of packages on the mirror (update) and then install Pylons:
aptitude update aptitude install python-pylons
Upgrading Pylons (and all other installed packages) system) is done by:
aptitude update aptitude upgrade
...through "easy install"
If your operating system does neither have binary packages nor a software management that keeps track of installed software you can install Pylons directly from the Cheeseshop (a central repository for Python modules):
easy_install Pylons
Download the easy_install script from http://peak.telecommunity.com/dist/ez_setup.py if you do not have it yet.
Upgrading the Pylons package once a new version is released is done by:
easy_install -U Pylons
...in a working environment (workingenv)
You may as well install Pylons and all the Python packages it depends on in an isolated directory that is independent from other packages that may be installed on your system. The workingenv <http://cheeseshop.python.org/pypi/workingenv.py/> tool is all you need. An advantage of this approach is that different Pylons applications on your system can use different versions so you can be sure every application works independently.
pymarks - an example Pylons application
"Peer review means that you can feel better because someone else missed the problem, too." (from the Python-user mailing list)
Now that Pylons is installed let us get busy by creating an own Pylons project. In the following chapters we will create a "pymarks" web application that allows users to store their URLs/bookmarks. Think of it as a simple clone of social bookmarking services like 'del.icio.us'. We will start easy by just sending "Hello World" to the browser. But we will increasingly add functionality to get to know the different components that Pylons consists of. The steps of the tutorial are available in a Mercurial repository at http://workaround.org/cgi-bin/hg-pymarks. The repository contains the project at the various stages. After each step I will tell you how to checkout the particular revision from the repository to try it out. So in case you get lost or are too lazy to type everything in yourself you can just use that way.
Create the basic Pylons application: paster create
Let us dive in. Change into a directory where you want to begin with your new project and run:
paster create --template=pylons pymarks
This will create a lot of directories and files that constitute the Pylons application. You will probably see something like this on your screen:
Selected and implied templates:
Pylons#pylons Pylons application template
Variables:
egg: pymarks
package: pymarks
project: pymarks
Creating template pylons
Creating directory ./pymarks
Recursing into +egg+.egg-info
Creating ./pymarks/pymarks.egg-info/
Copying paste_deploy_config.ini_tmpl_tmpl to ./pymarks/pymarks.egg-info/paste_deploy_config.ini_tmpl
Recursing into +package+
Creating ./pymarks/pymarks/
Copying __init__.py_tmpl to ./pymarks/pymarks/__init__.py
Recursing into config
Creating ./pymarks/pymarks/config/
Copying __init__.py_tmpl to ./pymarks/pymarks/config/__init__.py
Copying environment.py_tmpl to ./pymarks/pymarks/config/environment.py
Copying middleware.py_tmpl to ./pymarks/pymarks/config/middleware.py
Copying routing.py_tmpl to ./pymarks/pymarks/config/routing.py
Recursing into controllers
Creating ./pymarks/pymarks/controllers/
Copying __init__.py_tmpl to ./pymarks/pymarks/controllers/__init__.py
Copying error.py_tmpl to ./pymarks/pymarks/controllers/error.py
Copying template.py_tmpl to ./pymarks/pymarks/controllers/template.py
Recursing into lib
Creating ./pymarks/pymarks/lib/
Copying __init__.py_tmpl to ./pymarks/pymarks/lib/__init__.py
Copying app_globals.py_tmpl to ./pymarks/pymarks/lib/app_globals.py
Copying base.py_tmpl to ./pymarks/pymarks/lib/base.py
Copying helpers.py_tmpl to ./pymarks/pymarks/lib/helpers.py
Recursing into model
Creating ./pymarks/pymarks/model/
Copying __init__.py_tmpl to ./pymarks/pymarks/model/__init__.py
Recursing into public
Creating ./pymarks/pymarks/public/
Copying index.html_tmpl to ./pymarks/pymarks/public/index.html
Recursing into templates
Creating ./pymarks/pymarks/templates/
Recursing into tests
Creating ./pymarks/pymarks/tests/
Copying __init__.py_tmpl to ./pymarks/pymarks/tests/__init__.py
Recursing into functional
Creating ./pymarks/pymarks/tests/functional/
Copying __init__.py_tmpl to ./pymarks/pymarks/tests/functional/__init__.py
Copying test_models.py_tmpl to ./pymarks/pymarks/tests/test_models.py
Copying websetup.py_tmpl to ./pymarks/pymarks/websetup.py
Copying MANIFEST.in_tmpl to ./pymarks/MANIFEST.in
Copying README.txt_tmpl to ./pymarks/README.txt
Copying development.ini_tmpl to ./pymarks/development.ini
Recursing into docs
Creating ./pymarks/docs/
Copying index.txt_tmpl to ./pymarks/docs/index.txt
Copying setup.cfg_tmpl to ./pymarks/setup.cfg
Copying setup.py_tmpl to ./pymarks/setup.py
Copying test.ini_tmpl to ./pymarks/test.ini
Running /usr/bin/python setup.py egg_info
Adding Pylons to paster_plugins.txt
Adding WebHelpers to paster_plugins.txt
Would you expect that you already have a working web application now? You do. Try it. Enter the 'pymarks' directory:
cd pymarks
and run your application:
paster serve development.ini
The web server should be starting up:
Starting server in PID 12345. serving on 0.0.0.0:5000 view at http://127.0.0.1:5000
Now point your web browser to http://localhost:5000 and you should see a page titled "Welcome to your Pylons Web Application". That was easy, wasn't it?

Paste(r) - your Pylons toolbox
You will have noticed that the commands started with paster. Paster is the main command of a software part called Paste that consists of several parts that make developing WSGI web applications easier. With Paste you can create, test, upgrade and deploy your web applications and even run them in a simple web server like you just did. Pylons is just a Paste template so when you ran paster create -t pylons pymarks Paste was looking for a template called pylons and laid out your project directory from that template. Such a template consists of directories and files that are copied to your current directory. And as you told Paste that you want your project to be named "pymarks" this string will be written to various places in the files. (That is why renaming your application later requires some searching and replacing.)
paster serve - the built-in web server
The second Paste command you ran was paster serve development.ini. It started Paste's web server component that loaded your pymarks Pylons application and started answering HTTP requests. The development.ini is the control file that tells the web servers what to do. It contains a section [server:main]:
[server:main] use = egg:Paste#http host = 0.0.0.0 port = 5000
This section makes Paster use the Paste#http egg which is Paste's built-in web server. It will listen to HTTP requests on port 5000 of all network interfaces. The Paste webserver then looks for a further section called [app:main] and finds:
[app:main] use = egg:pymarks full_stack = true cache_dir = %(here)s/data beaker.session.key = pymarks beaker.session.secret = somesecret
Paste is mainly interested in the use line which makes it look for an egg called pymarks - your application - and runs it. An egg_ is a container for all files that are needed to run your application. The important file in the egg is pymarks.egg-info/entry_points.txt that gives Paste some information on how to run your Pylons application. Eggs are a nice invention. You can even create a single ZIP file from the egg structure so that you can give your application to other people easily.
Upon deployment the ini file pymarks.egg-info/paste_deploy_config.ini_tmpl is used as a template for the system administrator. It usually has to be customized so that your application knows on which TCP port to run or how to connect to your database server. The name development.ini indicates that this file is used during the development phase. The web server administrators can later name the file anything they want - like "production.ini".
The cache_dir is the directory where Pylons caches HTML pages that are generated from the template system. %(here)s always refers to the root directory of your project (where the INI file lives). The session_key is the name of the cookie that is sent to the browser to maintain cookie-based sessions. The session_secret is an additional security measure that is used to sign the session cookie so that users will not be able to temper the cookies easily. Make sure you set it to something complicated. Use the pwgen tool to create cryptic strings if you do not feel creative.
And of course there is a [DEFAULT] section that the ConfigParser applies to all other sections in the ini file. Make sure you change email_to to the valid email address of the web master. smtp_server should be the name of your mail server that receives emails. And error_email_from is used as the sender's email address when emails are sent. This is used to inform the webmaster in case that your application throws an exception.
Files and directories in your project
The Pylons template has put a lot of files and directories into your project directory. Let us take a look what they mean:
- data/sessions
- Cookie-based sessions are saved into files in this directory. You can store per-user information in a dictionary-like data structure that gets pickled into a file that will be written to this directory.
- data/templates
- Cached HTML files that are rendered from your template files are stored here so they do not have to be computed every time they are requested.
- development.ini
- The startup configuration for Paste. Later when you deploy your application you will have another configuration file. The filename does not really matter. Paste just uses this name as a default.
- docs/
- Put any documentation on the application here. Preferably in rest (restructured text) format.
- pymarks/config/
- Global configuration files of your project. Used to define routes for URL dispatching, middleware (like for adding authentication) or the template system you prefer.
- pymarks/controllers/
- The location of controllers that contain your application logic. This code controls your application, handles arguments and renders templates. By default you start with an error.py controller that handles printing error messages for HTTP errors and a template.py controller that works as a last resort if no other controllers are matching to serve a certain request. But you will create new controllers that will live here.
- pymarks/lib/
- Contains files that set up a number of global variables and objects that you can use in your controllers. For example everything in lib/base.py is made available in every controller. Your own utility modules can go into this directory, too.
- pymarks/model/
- Here go your database models. They define the database schema and set up ORMs (object-relational mappings). The directory does not contain anything by default except for an empty __init__.py file. You can put your own modules here that interface your application with a database backend. Many developers choose SQLAlchemy for that purpose.
- pymarks/public/
- Static files like images, CSS style sheets or Javascript files may go here.
- pymarks/templates/
- Your templates that render the actual HTML output are saved here. The default templating language is Mako.
- pymarks/tests/
- Every controller you create gets a counterpart to implement automated tests here. They are not mandatory but a good idea to verify that your application still does what you want while you develop the application.
- README.txt
- Some basic instructions that you can give to the admininstrator who is supposed to install your application.
- ez_setup, pymarks.egg-info, setup.cfg, setup.py
- Administrative files that are used to create an egg from your project that can be deployed on a web server then.
- test.ini
- Similar to the development.ini. This file is used when you run automated tests on your project.
Remove the static index.html welcome page
After this little excursion into the realms of theory let us get our hands dirty. When you surfed to http://localhost:5000 you received a static welcome page. There is a certain order [1] of things to happen if a user requests a URL. If there is no matching controller to handle the request then Pylons looks for a matching file in your public folder. That is where static files are stored like CSS style sheets, images or Javascripts. If no filename is specified then Pylons will look for a file called index.html. We don't want that welcome page for our project so please remove pymarks/public/index.html and start your web server again:
paster serve --reload development.ini
and reload the page (http://localhost:5000) in your browser.
Pylons now raises a 404 (Not Found) error because no part of your application seems to be responsible for answering your request any more. Pylons raised a Python exception that was intercepted and converted into the HTML error message you just saw.
Oh, and did you notice the --reload parameter I silently introduced? It makes paster monitor your project and all dependent files for changes. If you remove or edit a file then paster will notice that and reload your project. This is very comfortable because all you have to do is change things and then reload the page in your browser. Just don't use the option on a production web server because it costs a few CPU cycles.
| [1] | The search path is defined in your pymarks/config/environment.py file |
models - accessing a database
Before we add controllers and HTML we should first define the models. Models are our interface to the database in the background. Our bookmark application will store a set of bookmarks for every user. So we need two database tables namely users and bookmarks. Every user will have multiple bookmarks so we need a so called one-to-many relationship between the two tables. Since it is not very pythonic to use plain SQL statements in your code we make use of an additional Python component called SQLAlchemy that we use as an object-relational mapper. Such mappers connect Python objects with information in the database. You will love it.
The simplest database backend is SQLite which is essentially a database contained within a single file. You do not need to run any services or daemons on your system. It is a great database system for small applications or during tests.
Installing SQLAlchemy and SQLite
So you will have to install SQLAlchemy and SQLite now.
-
On Debian/Ubuntu:
aptitude install python-sqlalchemy python-sqlite
-
Through easy_install:
easy_install SQLAlchemy easy_install pysqlite
(Note that I am mainly copying the excellent article called SQLAlchemy in a hurry that is the common way to use SQLAlchemy >=0.4 with Pylons.)
Configuring the SQLAlchemy connection string in the development.ini
As I explained already the development.ini is the place for the system administrator who will run your application to set certain application-specific paramaters. So this is where the database is configured that should be used to store data. During the development phase we will be happy with SQLite so please add this line to your development.ini within the [app:main] section:
sqlalchemy.url = sqlite:///%(here)s/pymarks.db
The %(here)s is a special placeholder that refers to your project's root directory - where the development.ini file lives. So SQLAlchemy will create the database file there and call it pymarks.db. When your application is deployed later you can use other databases like MySQL or PostgreSQL by just changing the sqlalchemy configuration variables in your INI file.
Create an SQLAlchemy connection from your environment.py
Next edit the file pymarks/config/environment.py and add this import statement at the top of it:
from sqlalchemy import engine_from_config
And at the end of the load_environment() function (where it says 'CONFIGURATION OPTIONS HERE' you need to add:
config['pylons.g'].sa_engine = engine_from_config(config, 'sqlalchemy.')
When you start your application the engine_from_config function will search your INI file for lines reading sqlalchemy.' and use that information to connect to your database. You can also use other options like sqlalchemy.echo = True which tells SQLAlchemy to echo every request to the console.
Note
If you want to see how the pymarks project would look at this stage you can check it out: hg clone -r model http://workaround.org/cgi-bin/hg-pymarks
# XXXXXXXXXXXXXXXXXX
Controllers
Now it's time to add a little functionality. The actual application logic is contained in controllers. Controllers live in the controllers/ directory. They are used to logically split your application up into pieces of functionality. It really depends on how you like to structure your application. It is possible to throw everything into a single controller but that will not make it easy to keep track of your web application.
Our pymarks application is rather simplistic and just gets one controller that deals with contacts. So let's call the controller contacts. Pylons is a Paste template and comes with a further template called controller. You can use it to create a blank controller:
paster controller contacts
Paste will create two files for you:
Creating .../pymarks/controllers/contacts.py Creating .../pymarks/tests/functional/test_contacts.py
The first file is a controller template that you will be customizing in a minute. The second file is a python class that you can use to run unit tests. Unit tests are used to verify that your application is still doing what it is supposed to do. Change into the controllers directory and edit the contacts.py controller. It will look like this:
from pymarks.lib.base import *
class ContactsController(BaseController):
def index(self):
# Return a rendered template
# return render('/some/template.html')
# or, Return a response object
return 'Hello World'
Just accept the controller like that for a moment and point your browser to http://localhost:5000/contacts/ You will see a friendly "Hello World":

Routes (URL dispatching)
What happened is that your URL was analyzed by the Route rules defined in config/routing.py. The default route is:
map.connect(':controller/:action/:id')
This splits up the path of the URL (the part after the hostname and port) into a controller name, an action name and an id. If the action or id are not present in the URL they are set to None. So the URL http://localhost:5000/contacts/ essentially means controller="contacts". Pylons will look for a controller called ContactsController (that camelcase word is computed automatically) and find it. The methods in your controller refer to the actions that come from the routing. As you didn't specify an action it defaults to index. That's how Pylons finds your def index() and calls it. It then returns the text "Hello World". You had reached the same page if you called http://localhost:5000/contacts/index/42. This would have set id=42 as an argument to your action. def index(self, id) would have passed that argument to your method.
You can add your own routes to call any part of your applications depending on the how the URL looks. A simple case is telling Routes what to do if the user does not specify a certain path. Add this line in the config/routing.py right after where it says Define your routes:
map.connect('', controller='contacts')
Now when you point your browser to http://localhost:5000/ the controller ContactsController will be called that is found in controller/contacts.py.
Templates
So far so good. But the page does not look very nice. It is not even valid HTML. Instead of printing the text directly through return Response(...) you can use templates.
Note
This introduction assumes you are using Mako as the templating system. Pylons 0.9.5 still uses Mygthy per default. To switch to Mako you just need the following changes:
In the config/middleware.py change the config.init_app call to:
config.init_app(global_conf, app_conf, package='pymarks', template_engine='mako')
If you are using UTF8 encoded files you must also add these lines to your config/environment.py file:
tmpl_options = {}
tmpl_options['mako.input_encoding'] = 'UTF-8'
tmpl_options['mako.output_encoding'] = 'UTF-8'
Templates go into your templates/ directory. You can either create a seperate template file for each HTML page you want to output or you use template inheritance. It simply means that you create a kind of master template that other templates use. Go create a template file templates/master.mako and have it contain:
# -*- coding: utf-8 -*-
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head><title></title>
<body>
<h1>My first Pylons application</h1>
${ self.body() }
<p>In case of questions do not hesitate to send us an email.</p>
</body>
</html>
It is just a simple XHTML file. The first line tells Mako that the contents are meant to be UTF8 encoded. You may leave it off if you intend to stay with just ASCII characters. The strange ${ self.body() } is the placeholder where other templates insert their own text.
Now create a template file templates/contacts-index.mako with the follwing content:
# -*- coding: utf-8 -*- <%inherit file="master.mako"/> Good morning, earth!
This template will inherit from the master.mako template and the text "Good morning, earth!" is inserted at the self.body() line of the inherited template. Let us try it. Edit your controllers/contacts.py controller again and replace:
return 'Hello World'
by:
return render('/contacts-index.mako')
Reload the page in your browser and you should see your page that is based on the master template and greets the earth:

Databases
The pymarks application needs a database to store the contacts persistently. To avoid setting up a full-fledges database management system (DBMS) let us just use SQLite which is a relational database capable of speaking SQL. It just stored all the data into a single file so that you do not need to run a database server. Pylons by default uses an object-relational mapper (ORM) called SQLAlchemy. It puts an abstraction layer onto your database and makes your database interaction look almost pythonic. Instead of reading and writing rows by issueing SQL queries you rather toss Python class instances around and SQLAlchemy deals with the SQL in the background. You have to invest more time at the beginning to set up the mapping but in the end working with database rows is a snap.
But first things first. You first need to create a database. It will consist of a table contacts that contains these fields/columns:
- id (a unique primary key)
- name
- phone
Go to your project directory (where the development.ini lives) and create an SQLite database. (Make sure you use the sqlite command for version 3. On Debian the command is sqlite3.). Run sqlite pymarks.db and enter these SQL commands:
sqlite> CREATE TABLE contacts ( ...> id INTEGER PRIMARY KEY, ...> name TEXT, ...> phone TEXT, ...> email TEXT ...> );
While you are in SQLite you may want to insert two dummy rows into that table:
sqlite> INSERT INTO contacts (name, phone, email)
...> VALUES ('John Doe', '1853', 'john@example.com');
sqlite> INSERT INTO contacts (name, phone, email)
...> VALUES ('Ben Bangert', '1875', 'ben@example.com');
sqlite> .exit
Next order of business is telling Pylons where your database can be found. Edit the development.ini and in the [app:main] section add this line:
sqlalchemy.dburi = sqlite:///%(here)s/pymarks.db
Pylons will now know internally where to look for your database. Now you need to define the actual model in the models/__init__.py. A model is a description of your database schema in Python words. Just enter the following code:
import sqlalchemy as sql
from sqlalchemy.ext.assignmapper import assign_mapper
from pylons.database import session_context
meta = sql.BoundMetaData()
contacts_table = sql.Table(
'contacts', meta,
sql.Column('id', sql.Integer, primary_key=True),
sql.Column('name', sql.Unicode),
sql.Column('phone', sql.Unicode),
sql.Column('email', sql.Unicode),
)
class Contact(object):
def __repr__(self):
return "Contact(name=%s, phone=%s, email=%s)" % (
self.name, self.phone, self.email)
assign_mapper(session_context, Contact, contacts_table)
SQLAlchemy is very complex. So don't panic. First a meta data object is created. Think of it as the container for all your database tables. Next you define the contacts_table as an SQLAlchemy Table instance. The first argument it the actual table name in your SQLite database. And of course you define the types of the columns that your table uses. So far there is not much magic.
Now with the object-relational part things get interesting. First you define a very basic class called Contact that does not do much. The __repr__() method is there to pretty-print the fields of a contact entry later. You could as well leave that out and write class Contact(object): pass and it would serve the same purpose. The magic of connecting the contacts table to your Contact class is done by the assign_mapper. The session_context is a way to access your database in a clean multithreaded way. It is set up by Pylons so you better not worry too much over it.
To reiterate: the object-relational part is done in three steps:
- define the database table schema
- define a trivial Python class
- call a mapper to glue these two together
It's been a while since you got your hands dirty. So let's use another nice feature of Pylons to see how the database access works.
Paster's Shell
In addition to the things that Paste has done already (roll-out the Pylons and controller templates and run the web server) it has a nice shell. Please make sure you have ipython installed. The Paste shell will work without it but ipython makes the shell doubly as comfortable. Change into your project directory (where the development.ini file is located and enter paster shell. It will welcome you with it's prompt (In [1]:) and you can enter Python statements right away. The advantage over a regular Python shell (like ipython or idle) is that you are now working in the context of your pylons application. All the parts of your application are accessible to you. So you can try out code before you use it in your controllers.
You should see something like:
Python 2.4.4 (#2, Apr 26 2007, 00:02:45) [...] IPython 0.8.0 -- An enhanced Interactive Python. [...] In [1]:
Make SQLAlchemy fetch all contacts:
In [1]: model.Contact.query().select() Out[1]: [Contact(name=John Doe, phone=1853, email=john@example.com), Contact(name=Ben Bangert, phone=1875, email=ben@example.com)]
Or you can fetch the contact with the index number 2:
In [2]: model.Contact.query().get(2) Out[2]: Contact(name=Ben Bangert, phone=1875, email=ben@example.com)
Or you can search for all contacts who's email addresses start with 'b':
In [3]: model.Contact.query().select(model.Contact.c.email.startswith('b'))
Out[3]: [Contact(name=Ben Bangert, phone=1875, email=ben@example.com)]
Or count how many contacts you have:
In [4]: model.Contact.query().count() Out[4]: 2
Or get the contact that has the phone number '1853':
In [5]: model.Contact.query().get_by(phone='1853') Out[5]: Contact(name=John Doe, phone=1853, email=john@example.com)
Or get the first contact and inspect its attributes:
In [6]: contact = model.Contact.query().get(1) In [7]: contact.name Out[7]: u'John Doe' In [8]: contact.phone Out[8]: u'1853' In [9]: contact.email Out[9]: u'john@example.com'
Go create a new contact:
In [10]: sam = model.Contact() In [11]: sam.name = 'Sam Hocevar' In [12]: sam.phone = '+1 555 1857' In [13]: sam.email = 'leader@debian.org' In [14]: model.session_context.current.flush()
Look at the list of available contacts now:
In [15]: model.Contact.query().select() Out[15]: [Contact(name=John Doe, phone=1853, email=john@example.com), Contact(name=Ben Bangert, phone=1875, email=ben@example.com), Contact(name=Sam Hocevar, phone=+1 555 1857, email=leader@debian.org)]
It might take some getting used to until you know all the methods on your object class. But you probably already got an idea of how working with object-relational mappers feel.
Show a list of contacts
Now that you know how to access the database through SQLAlchemy you can enhance your 'contacts' controller. Edit the controllers/contacts.py file again and make it read:
from pymarks.lib.base import *
class ContactsController(BaseController):
def index(self):
c.contacts = model.Contact.query().select()
return render('/contacts-index.mako')
The c is the namespace where variables live that you want to pass through to the templates. Think of c as content. Edit your template at templates/contacts-index.mako and change it to:
# -*- coding: utf-8 -*-
<%inherit file="master.mako"/>
<h2>List of contacts</h2>
<ul>
% for contact in c.contacts:
<li>${ contact}</li>
% endfor
</ul>
The % for and % endfor is a special command in Mako templates that works exactly like Python for loops. The template just iterates over the list of contacts that your controller passed to the template. The ${ contact } is telling Mako to insert the content of the variable contact.
Reload the page http://localhost:5000/ in your browser. You should see:
Not bad. You already have a web application that shows entries from a database. Let's make the list a bit nicer. The contacts that are printed come from the Contact class's __repr__() function. But every Contact class instance also gets the database fields assigned as properties as you saw above. Change your template to:
# -*- coding: utf-8 -*-
<%inherit file="master.mako"/>
<h2>List of contacts</h2>
% if c.contacts:
<table border="1">
<tr>
<th>Name</th>
<th>Phone</th>
<th>Email</th>
<th>Actions</th>
</tr>
% for contact in c.contacts:
<tr>
<td>${ contact.name }</td>
<td>${ contact.phone }</td>
<td>${ contact.email }</td>
<td>${ h.link_to('Edit', h.url_for(action='edit', id=contact.id)) }</td>
</tr>
% endfor
</table>
% endif
Reload the page in the browser. You now see a nicer HTML table containing the contacts:
There is also a new column with "Edit" links now. These links have been created by means of the Webhelpers module that comes with Pylons. This module is loaded under the name h so you do not have to type much. h.url_for(...) is a function that creates URLs. You just pass arguments that match your routes (:controller/:action/:id - remember?) and the URL is created accordingly. We are currently in the context of the contacts controller. Further arguments are added as HTTP-GET arguments.
Assumed you are currently viewing http://localhost:5000/contacts/edit/42 you would get:
| Example | Result |
| h.url_for(controller='news') | /news/edit/42 |
| h.url_for(controller='/news') | /news |
| h.url_for(controller='contacts') | /contacts/edit/42 |
| h.url_for(controller='/contacts') | /contacts |
| h.url_for(action='new') | /contacts/new/42 |
| h.url_for(action='new', id=None) | /contacts/new |
| h.url_for(day='today') | /contacts/edit/42?day=today |
Hint: a good way to explore the Webhelpers functions is in the paster shell. Enter h. and press the TAB key to see all available functions. Enter h.url_for? to read its docstring or use h.url_for?? for the actual source code of that function.
The URL that h.url_for() created is used in h.link_to() to create the actual <a href="...">link</a> HTML link then. Of course you could have written the line without webhelpers as:
<a href="/contacts/edit/${ contact.id }">Edit</a>
But often the Webhelpers make it clearer.
Add an edit form
Now that your table already contains links that suggest that the user can edit the contacts you should add an edit form. The action that is addressed is called edit so you need a new method def edit() in your ContactsController. Add this code below your def index() in the controllers/contacts.py file:
keks (renderfirst with editsave() action next. then formencode)
controller creates controller plus test
/foobar -> FoobarController
- If the controller is present and begins with '/', no defaults are used
- If the controller is changed, action is set to 'index' unless otherwise specified
controller/action/id
returns: redirect_to(controller='home', action='login') '...' render() abort()
_foobar is a private controller
serving other types than text/html
routes variables are passed in as method arguments: def index(self, a, b, c):
Routes
leading '/' in URLs
Templates
throwing html in templates
render_response()
adding template variables c.foobar
mako style
inherit base
python statements. only "print" actually prints something there.
utf8 in templates
passing variables to included templates is only possible through c.foobar
globals: h.*, c.*, g.*, m.* (availabe in mako?), request.params, request.environ
Webhelpers
forms through webhelpers
which webhelpers get included - which do you need to import (steal from cheatsheet)
own additional utilities using "import myproject.lib.my as my" in lib/base
Sessions
Beaker
saved to disk (will later be able to use database storage for it)
steal from cheatsheet
works with mygthy and mako
Databases
relational databases
normalized databases
echo
relations: - foreign keys - many-to-many - http://docs.pythonweb.org/display/pylonscookbook/Relational+databases+for+people+in+a+hurry
database config in development.ini
setting up the models
assign_mapper - adds methods to own model classes - new objects do not have to be .save()d manually (are automatically part of the session)
http://docs.pythonweb.org/display/pylonscookbook/SQLAlchemy+for+people+in+a+hurry
using Unicode() instead of String() for consistency
session_context and engine
mapped objects are single-object kinds of things. they can be used to pull one thing, alter one thing or create a new thing. they cannot be used to alter multiple things at once.
clearing sessions upon every new request
flushing the session on __after__ unless .flush() is called the transaction is rolled back - no harm done
creating the database schema through websetup.py
specifying the models versus autoloading them
example sessions on "paster shell"
selection = selection.filter(model.DnsRecord.c.type=='A') is shorter (if the mapped class is the same): selection = selection.filter_by(type='A')
select via object classes VERSUS select via table names (joins are only possible on tables - not on objects - because objects have a fixed set of columns)
MyModel.query.select... (from sqlalchemy 0.4)
last_insert_id is added as the id attribute after a .flush() automatically
Javascript and AJAX
script.aculo.us / prototype built in
jQuery a little simpler (show AJAX example with jQuery)
Logging and Debugging
h.log()
debugging is dangerous in production
paster serve --log-file=yourlogfile development.ini
interactive debugger http://pylonshq.com/docs/interactive_debugger.html
(is started on 500 (internal server error) if debug is turned on - otherwise just error 500 is raised)
deliberately: raise Exception
import pdb; pdb.set_trace()
print debug URL on the console for debugging AJAX requests
Under the hood
"Standards are great. Everybody should have one." (Robert Metcalfe)
By now I have just shown you how to use Pylons. And at the beginning of this document I told you to accept the magic of Pylons for a while and concentrate on how to actually use Pylons for good. It's now time to shed some light upon that magic. Your impression may be that Pylons is a huge framework. But actually is it relatively small and just re-uses a lot of other Python packages.
WSGI - The Web Server Gateway Interface
Pylons is a WSGI [3] framework which makes sure you can use it with anything else that speaks WSGI, too. But what is that omninous WGSI thing anyway? Once upon a time there were only web frameworks that provided their web server component (the part that speaks HTTP with the browsers) bundled with the web application (that the web developer programmed). You were not able to just replace the web server part unless you threw away everything you programmed and used another web framework. That was obviously pretty annoying.
Then in 2003 Phillip J. Eby considered making the web application independent from the web server and came up with a proposal (PEP333) for a standard interface between the web server and the web application part. He called the standard WSGI (Web Server Gateway Interface). The idea behing WSGI was that a WSGI-capable application could possibly use any WSGI-capable web server. If the web server you were using so far annoyed you then you would just chose another web server and ran your application with it. Back then WSGI was just theory but today there are several WSGI web servers you can use. So it makes sense to make your web application speak WSGI. Luckily if you are using Pylons then the WSGI magic is already built in. But let me try to give you an idea of what WSGI really is.
If you know about the CGI (Common Gateway Interface) standard then WSGI will sound familiar. Actually it deals with the same problem. A CGI that you program will run on any web server that knows how to run CGIs. Similarly a WSGI application that you program will run on any web server that knows how to run WSGI applications. And the similarities go even further because just like with CGIs you get environment variables like REMOTE_USER or PATH_INFO that gives you additional information on the HTTP request.
Fortunately using WSGI is not hard. Basically the web application is just a callable (a Python function or a Python object that provides a __call__() method). In the simplest form it's just a function that you pass. It will be called with two parameters [2]:
- environ (a dictionary of environment variables)
- start_response (your function that the WSGI webserver will call to send the HTTP status code and a list of HTTP headers)
The classical example in any programming language is to print "Hello World". A WSGI application that does that is:
def my_web_application(environ, start_response):
start_response(
'200 OK',
[('content-type', 'text/html')]
)
return ['Hello world']
The actual web application is the my_web_application function. If you fed this tiny application to a WSGI web server it would:
- call your start_response function and expect the HTTP status code ('200 OK') and a list of HTTP headers to send to the user's browser ('content-type: text/html')
- send the returned list of strings as the actual web page to the browser
On the network it roughly looks like this:
HTTP/1.0 200 OK Server: SomeWSGIWebServer/1.0 Date: Sun, 27 May 2007 06:21:46 GMT content-type: text/html Connection: close Hello world!
As you can see writing WSGI applications is not that hard. But you still need a WSGI web server. Pylons uses the server from the Paste_ package that I will get to in to next chapter.
| [2] | Apparently working with a start_response callback is a bit complicated. It would rather be logical to use a strict input/output approach and pass three paramters to the WSGI application for the HTTP status code, the headers and the body. And actually that is discussed to be used in the next version 2.0 of the WSGI standard. |
| [3] | Some people do not spell out WSGI but rather call it "whisgey". Double-you ess gee eye is preferred though so that people know what you are talking about. |
WSGI middleware - easier than it sounds
Just a word on "middleware" - a concept that is sometimes confusing and an expression that is frequently abused as a buzzword. By definition middleware is something that connects two processes and does something with the data it sees. Middleware is easy to write and useful in WSGI applications. WSGI seperates the web application and the web server. So usually a request goes like:
browser --HTTP--> web server --WSGI--> application
WSGI middleware is a callable that lives between the web server and the web application. It receives a WSGI request (the "environ, start_response" pair) and behaves like a WSGI application so you can plug it in between the server and your application:
browser --HTTP--> web server --WSGI--> middleware --WSGI--> application
As you can imagine you can chains several middlewares together. Writing middleware is useful if you want to do something on every HTTP request. Think of authentication/authorization, changing HTTP headers or adding something to every page. An interesting piece of middleware is Authkit which you can just plug into Pylons to get user authentication and a customizable permissions system. And writing your own middleware isn't hard either. Minimally you write a function that gets WSGI in and throws out WSGI.
Paste - a WSGI treasure chest
Up to now we have just talked about the application side. Of course your application will do nothing if now called by a web server. Pylons' web server component is the httpserver from the Paste package. Let's start with its web server module so that you can run your "Hello World" application. Copy these lines into a Python script and run it:
def my_web_application(environ, start_response):
start_response(
'200 OK',
[('content-type', 'text/html')]
)
return ['Hello world']
import paste.httpserver
paste.httpserver.serve(my_web_application, host='127.0.0.1', port='8080')
Now point your browser at http://localhost:8080 and see your application in action. The paste.httpserver module is the web server that runs your application. It uses the BaseHTTPServer from the Python standard library.
When you first stumble upon Paste's web site you may be confused and not really understand what it does. Paste is a collection of things that make writing WSGI applications easier. You already got to know the httpserver.
For further information on Paste I suggest you start reading Ian Bicking's A Do-It-Yourself Framework <http://pythonpaste.org/do-it-yourself-framework.html> tutorial. He gives a nice introduction on WSGI in conjunction with his Paste package.
What Pylons is actually doing
Mike Orr wrote an interesting article in which he analysed what happens when a Pylons application is called. In this chapter I would like to try an analysis from a different angle. Pylons' principle is that it allows you to change parts of it as you like. To do so you need to understand what is done where. And Pylons really jumps around in several parts of your application and internal modules that you feel lost quickly. Let us therefore take a tour to see what happens when Paste's main job is done by calling the entry point of any Pylons application - the make_app function in the __init__.py file of your project directory. That file imports the make_app function from your config/middleware.py file.
-
config.middleware.make_app
First the configuration from your (development.)ini file is put into a dictionary called "conf". Its 'global_conf' key points to the settings in the ini file's [app:main] section and its 'app_conf' key points to the [DEFAULT] section.
First the load_environment function from your config/environment.py module is called to get a number of global configuration variables.
-
config.environment.load_environment
Now the make_map function from your config/routing.py module is called and sets up the map which is a Routes "Mapper" object that later deals with URL dispatching (which URL runs which controller).
Further options that get set deal with template options (e.g. character sets of the input and output) and paths to controllers, templates and the static files. Finally a Pylons "Config" object is returned (see "pydoc pylons.config.Config") that contains parameters for the template system and the routes.
Then the init_app() method of that very "Config" object is called and passed the global_conf and app_conf configuration as well as your project's name and the template system you want to use.
Now app is created as a PylonsApp object which is actually the main WSGI middleware.
-
PylonsApp
The PylonsApp object is created with some knowledge about the config (from above), the "helpers" (that are got lib/helpers.py) and the global "g" object (that comes from lib/app_globals.py). Next the PylonsBaseWSGIApp is added as another middleware as the actual Pylons WSGI application handler.
-
PylonsBaseWSGIApp
A "Buffet" object is created that gives the application access to the desired template system. When this middleware is called by a request it will also compute the name of the controller for a certain URL (by asking Routes) and call that controller to get a response.
The PylonsApp then plugs in the RoutesMiddleware that sets the wsgiorg.routing_args environment variable as documented in the WSGI routing args specification.
beaker.SessionMiddleware beaker.CacheMiddleware
-
ConfigMiddleware HTTPExceptionHandler ErrorHandler ErrorDocuments
-
Glossary
- Action
- A method of a controller. Methods depict certain actions inside a controller. Usually the second part of the URL's path refers to the action. (That behavior is configurable through the URL routing.) If you call http://localhost:5000/animals/show-dogs/15 then the show-dogs is usually the action's name. Pylons will look for a def show-dogs() in the AnimalsController and run it.
- Controller
- A Python class that contains application logic. Its methods are called actions. Usually the first part of the URL's path refers to the controller's name. (That behavior is configurable through the URL routing.) If you call http://localhost:5000/animals/show-dogs/15 then Pylons looks for a controller called AnimalsController.
- MVC
- Abbreviation of model-view-controller. A concept that divides database models, views (what the users sees) and controllers (the application logic). The goal is to make it easier to change one of these parts without interfering the other.
- Routes
- A software to dispatch URLs to different parts of your application. The default route is :controller/:action/:id which splits up the URL's path into controller, action and id. In case of a URL http://localhost:5000/animals/show_dogs/15 it will look for an AnimalsController and run its method show_dogs() while passing the argument id=15.
- Static files
- Files that are not interactive. Files that do not change while your Pylons application is running. Static files can be e.g. images (JPEG, PNG, GIF), CSS or Javascript. They live in the project's public directory.
Resources
- pylonshq.com
- wiki.pythonweb.org (FAQ, Cookbook, in-a-hurry)
- mailing list (need a google account but can subscribe with other address)
- #pylons (irc.freenode.net)
- pydoc pylons
- ben's talk and slides
About the author
"Never argue with an idiot, they drag you down to their level and beat you with experience." (origin unknown)
If you assumed this document has solely been written by Pylons gurus I need to disappoint you. I am more or less an average Pylons user. My background was programming Perl CGIs for many years. When I discovered Python I was fascinated by the clean code I could write. But writing CGIs is not as easy as in Perl because the 'cgi' class from Python's standard library is less comfortable and complete than Perl's CGI module. Still I developed a complete web application (http://mentors.debian.net) in CGIs with old-school SQL statements to query the database and found myself reinventing a lot of wheels. When I complained about that on the Python mailing list I was pointed at web frameworks - a concept that was completely new to me. But I looked at the Python wiki page about web frameworks and decided I want to try Django and TurboGears. Both appeared very complicated to me and introduced many new concepts so that I was overwhelmed and frustrated. Somehow - even before I wrote more than a few lines in both Django and Turbogears - I hit the borders of what is possible and found myself writing code that did not look "pythonic" at all. I complained about that on IRC and got a hint to try Pylons. The only reason I had not tried Pylons before was that it was not yet available as a Debian package. But after poking a few fellow Debian developers the package was there and I started going through Pylons' QuickWiki tutorial. That was the first time I felt the power and usefulness of a web framework and I even used AJAX for the first time. That was fun and I had something to show my dot-net loving coworker. :) Although Pylons had some rough edges it already had a small but great community that helped a lot. Both when I made mistakes and when Pylons had bugs. It took a while until I could distinguish Pylons components without having to look them up time and again. My Pylons cheatsheet on my workaround.org site was started during that time. But finally I was happily using Pylons and wanted to give something back. My first contribution was an article that this document is based upon: "The concepts of Pylons". Next I was frustrated by the 'pagination' module in the Webhelpers package and after a few weeks came up with my own module which has already made a few people happy and might be included in the Webhelpers in the future. And as I'm spending some time on IRC in the #pylons channel I discovered that others have trouble to get the big picture of Pylons, too. So I finally decided to write this introductory document. I hope you liked it. Have fun with Pylons.
2 Comments
This was the best
Submitted by Kishore (not verified) on
This was the best introduction i found till now on Pylons. It was very useful and really gave an insight of how the pylons framework is framed. Thanks fo your contribution.
Thank you. Although Pylons
Submitted by Christoph Haas on
Thank you. Although Pylons has been declared "stable" (which I translate as: "deprecated") and replaced by the successor project "Pyramid". I haven't yet created a real-works Pyramid application so I can't yet write about it. So this article is a bit outdated.