Archive for the 'Pylons' Category

setuptools versioning - wtf?

Saturday, July 19th, 2008

Nowadays I’m taking Debian’s APT software management system pretty much for granted. It probably represents the most advanced and powerful system. So I’m not instantly turned away if I find other systems that don’t work as well. Observing how Windows users struggle with getting the 50 different installed pieces of software to update that all use their own mechanisms thus wasting computing time and hard disk space while often getting annoyed by “Do you want to update now?” requests or sudden reboots is fun.

Anyway, as an addicted Python programmer I obviously had to deal with the “setuptools” already. It’s Python’s pendant to Perl’s CPAN. It can download Python modules with certain versions, satisfy dependencies automatically etc. The central place for Python modules is the Python Package Index (PyPi). Obviously I usually wouldn’t need to deal with the setuptools often because Debian already provides properly packages Python modules but sometimes it’s nice to deploy applications within a virtualenv environment (similar to a chroot with its own set of module versions not conflicting with other versions installed on the system). Still I think that setuptools is pretty pathetic.

  • You can’t uninstall modules properly. This is a no-go - we aren’t in the Windows 95 ages any more.
  • PyPi often just links to the developers’ home pages which tend to go offline when you urgently need to deploy an application and can’t get a certain version downloaded. (Pylons has kindly collected the needed eggs on their download page for that case.)
  • Sometimes certain versions are requested that are no longer available on PyPi that have been released just weeks ago.
  • Outside of the virtualenv you’d have to install modules system-wide which collides with your software management like APT (unless you have an operating system like Windows that doesn’t care anyway).

But the single most pathetic detail about setuptools is their versioning. Gustavo Noronha Silva pointed me to /usr/share/pyshared/pkg_resources.py which contained an explanation of how version numbers are compared:

Strings like “a”, “b”, “c”, “alpha”, “beta”, “candidate” and so on (that come before “final” alphabetically) are assumed to be pre-release versions, so that the version “2.4″ is considered newer than “2.4a1″.

Finally, to handle miscellaneous cases, the strings “pre”, “preview”, and “rc” are treated as if they were “c”, i.e. as though they were release candidates, and therefore are not as new as a version string that does not contain them.

So the sorting goes something like a, b, c, pre, preview, rc, alpha, beta, candidate, final. And “2.4″ is newer than “2.4a1″ but not newer than “2.4f1″? Please people, lay off the drugs. I feel tempted to send that to The daily wtf.

I thought that setuptools and virtualenv is a good combination if a certain Python module is not yet available in Debian. But I think I’d rather spend 30 minutes to create a proper Debian package with a real version numbering system instead of wasting my time with these antics.

Don’t get me wrong. I appreciate the intention of setuptools and easy_install. But the world needs a 2.0 of this work. From what I heard hardly anyone understands setuptools well enough to provide something better. Bummer.

Our GSoC project has been accepted - yahoo! :)

Monday, April 21st, 2008

This is pretty exciting. I have never bothered about Google’s Summer of Code and have literally been forced (curse you, Kumar) to propose my project I’m working on.

And in the end I have a promising student and Google accepted our proposal. Watching #gsoc while everybody was waiting whether they got accepted caused more traffic than #freenode-newyears. Looking forward to days and nights of my spare time vanish for a fancy T-shirt. :) Nah, seriously. A lot of people have asked for an easy-to-handle web-enabled package repository. It will surely be very helpful as a clean and featureful basis for mentors.debian.net as well as the replacement for PPAs and REVU. And this will be my first large open-source Pylons project, too. I’m excited.

Fun and despair with ExtJS

Tuesday, January 15th, 2008

Since I started playing with Javascript about a year ago I became aware of a pretty large number of Javascript libraries that offer additional functions or even widgets for a nicer user interface. I had started with prototype/script.aculo.us because it was included by Pylons. But when I found out that I’m not really bound to it I started looking around in my usual pointless search for “the perfect tool” ™. Finally I stranded with jQuery and I fell in love with it quickly. Give me results quickly and I’m willing to dig into the more subtle parts of something.

Sure, other people told me that jQuery is lame and Dojo is it or YUI (the Yahoo UI library) or even ExtJS. ExtJS? Yes, I looked at that before. Very impressive demos like the “Web Desktop”. I didn’t want to write anything like that though so I enjoyed it as a proof-of-concept and was done with it. But somehow one of my current projects requires a “grid” - that is a table that acts as a paginator (you get shown chunks of data and can navigate back and forth) but allows you to select rows and run actions. Perhaps I would also need a tree view. ExtJS supported all that so I dumped jQuery for a moment and created a (Mercurial) branch of my project to try out ExtJS.

There were instantly a few things I didn’t like. The whole Javascript was 500 KB large. Hardly something I could use on a production web site. Imagine a >500 KB start page. I also had to throw away my CSS for a moment and use their stylesheet - with 850 not-documented classes. Anyway - I surrendered to that and started playing with ExtJS. Strangely loading the page took 10 seconds in Firefox. After some research I found out that there was a bug in FireBug (a Firefox extension that web developers can hardly live without) with large Javascript files. Yes, 500 KB is large. Finally I found a hacked version of it that disabled a few features just to work better with ExtJS.

So finally my application loaded the Ext-all.js (the documented order and list of files I had to include was wrong) and could copy/paste a grid example into my application. One day later it worked. I had defined Stores, Proxies, ColumnModels, Actions, GridPanels and PagingToolbars and after 150 lines of Javascript I finally loaded JSON data from my web service (running on Pylons) and had the grid fill with it. I better hadn’t looked into the generated HTML - made me dizzy looking at all the nested DIVs and TABLEs. My Pylons code was just 5-10 lines though. It was an adventure of surfing through the API documentation of ExtJS that is complete but unusable to me. It defines all possible parameters and options but those are inherited from one another, some aren’t documented because they are “common” and you hardly get an impression on what they mean or how to use them. There were nice examples and everything they did was well documented. But that didn’t help me do more than that. Even doing server-side sorting of the JSON data meant firing up FireBug and watching the parameters - I had expected that to be documented somewhere.
Finally I decided not to work with ExtJS any more. If my application consists of 90-95% Javascript and 5-10% Python/Pylons then I’m not spending my time as intended. Using ExtJS basically means surrendering to it. Just use their library and style sheet and change as little as possible. 500 KB Javascript and 850 CSS styles are pure bloat. And while their license is basically the LGPL they have some weird additional clauses in their license like you can’t use their theme graphics and styles outside of the ExtJS context. I understand that a library of such a quality can’t be maintained by a handful of volunteers. So I see that they want to make money with it if others start making money with it. But somehow it was too much legal stuff for me. I prefer software that is just free. MIT or BSD. Or even GPL - I don’t mind. (Everytime you complain about the complicated licensing you are pointed to their license consultants to convince you. :) ).

It’s a great library. It has great widgets. It even creates HTML that is strictly valid in all browsers I could find. But I was not ready to surrender my application to ExtJS. I wanted a grid to support my application. Fortunately I didn’t have to reinvent the wheel because Matt Knight has recently published his Ingrid grid plugin for jQuery serving me well. And another (ridiculous) point for jQuery: it has mailing lists. ExtJS has a web forum. I hate web forums. Some people say mailing lists are archaic. But I can delete threads or blacklist people or reply properly in a text format. Morale: the flashiest isn’t always the best - neither for the user nor for the developer.

kate isn’t just a NOTEPAD.EXE for KDE

Sunday, December 2nd, 2007

I’ve been a VIM lover for at least a decade. Nothing beats it at doing complex text editing. Surely it’s not intuitive (although I frequently claim that VI stands for “very intuitive”) but no other text editor can so quickly repeat the last action or reformat a paragraph or insert text at a certain column in every line etc. I am very fast with it and happily ranted at IDEs like SPE, Eric or WingIDE for having such a bad text editor. They were more like a NOTEPAD.EXE that allowed you to navigate with the cursor keys and insert text where you are. But I have to admit that IDEs have a lot of advantages like code completion (vim doesn’t understand Python - not even with omni-completion), templates or dealing with your repository software.

I love cryptic hardcore tools because once I understand them I’m very fast in accomplishing tasks. But even after a decade of using VIM I don’t really understand its configuration or some more advanced features properly. Recently I tried to tell a Python file that I simply like the textwidth to be 100 by adding

:vim:set tw=100:

at the end of the file. No idea why that didn’t work. The command worked well when typed manually.

In addition many basic features like editing multiple files at the same time became a chaos quickly. Multiple buffers? Nah. Tabs are nice but after a certain amount of open files you have to use the arrows to find your file again. Syntax coloring is nice but customizing it is a pain. VIM is great for editing single files and I will always use it to work on DNS zone files for example. But since I develop Pylons (a Python web framework) applications I have to deal with many open files and complex Python code. I tried “pida” which is very promising but is still lacking. For example it doesn’t remember the window panes as I want to have them arranged and find myself moving around things every time I start it up.

I wouldn’t have expected to hear myself saying that vim is not good for all tasks. I had always condemned editors for not being vim and wasn’t open for all the features that I was missing out. Currently I’m trying out “kate”. I had expected it to be a stupid text editor like NOTEPAD.EXE or joe but although it looks similar it is very powerful. Some things I like about it:

  • Handles multiple open files in sessions gracefully (even shows which files have changed)
  • Shows indentation levels in source code (no more guessing where a Python function ends)
  • Text completion (no worse than what vim can do)
  • Closing brackets automatically
  • Split views
  • Syntax highlighting for all types of text files I care about
  • Proper Python indentation handling (vim doesn’t even add an indentation level after a line ending with “:” unless you use dirty tricks)
  • Code folding
  • Snippets (finally I don’t have to copy/paste templates from my knotes any more)
  • Commenting/Uncommenting of lines without :s/^/#/ magic

Kate doesn’t do all things that vim can. But the common tasks are made simpler and more comfortable. No idea if it will be my new editor. But it appears like other text editors besides vim are also worth being tried.