Django: Highlighting with ReST using Pygments

Wednesday, January 13, 2010

In my last post I wrote about code highlighting in HTML formatted text, but since I’m now using reStructuredText for markup things, I want to share how I added synthax highlighting for this.

Within your project directory (the same that contains the settings.py) create a file called rst_directive.py and fill it with the following code:

# -*- coding: utf-8 -*-
"""
    The Pygments reStructuredText directive
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    This fragment is a Docutils_ 0.5 directive that renders source code
    (to HTML only, currently) via Pygments.

    To use it, adjust the options below and copy the code into a module
    that you import on initialization.  The code then automatically
    registers a ``sourcecode`` directive that you can use instead of
    normal code blocks like this::

        .. sourcecode:: python

            My code goes here.

    If you want to have different code styles, e.g. one with line numbers
    and one without, add formatters with their names in the VARIANTS dict
    below.  You can invoke them instead of the DEFAULT one by using a
    directive option::

        .. sourcecode:: python
            :linenos:

            My code goes here.

    Look at the `directive documentation`_ to get all the gory details.

    .. _Docutils: http://docutils.sf.net/
    .. _directive documentation:
       http://docutils.sourceforge.net/docs/howto/rst-directives.html

    :copyright: Copyright 2006-2009 by the Pygments team, see AUTHORS.
    :license: BSD, see LICENSE for details.
"""

# Options
# ~~~~~~~

# Set to True if you want inline CSS styles instead of classes
INLINESTYLES = False

from pygments.formatters import HtmlFormatter

# The default formatter
DEFAULT = HtmlFormatter(noclasses=INLINESTYLES)

# Add name -> formatter pairs for every variant you want to use
VARIANTS = {
    # 'linenos': HtmlFormatter(noclasses=INLINESTYLES, linenos=True),
}


from docutils import nodes
from docutils.parsers.rst import directives, Directive

from pygments import highlight
from pygments.lexers import get_lexer_by_name, TextLexer

class Pygments(Directive):
    """ Source code syntax hightlighting.
    """
    required_arguments = 1
    optional_arguments = 0
    final_argument_whitespace = True
    option_spec = dict([(key, directives.flag) for key in VARIANTS])
    has_content = True

    def run(self):
        self.assert_has_content()
        try:
            lexer = get_lexer_by_name(self.arguments[0])
        except ValueError:
            # no lexer found - use the text one instead of an exception
            lexer = TextLexer()
        # take an arbitrary option if more than one is given
        formatter = self.options and VARIANTS[self.options.keys()[0]] or DEFAULT
        parsed = highlight(u'\n'.join(self.content), lexer, formatter)
        return [nodes.raw('', parsed, format='html')]

directives.register_directive('sourcecode', Pygments)

Open the file __init__.py in the same directory and add the following line:

import rst_directive

Furthermore you might want to create a CSS file containing all the style definitions in your static media directory. The following python script will do the job:

# Call it this way:
# python gen_css.py pygments.css
import sys

from pygments.formatters import HtmlFormatter

f = open(sys.argv[1], 'w')

# You can change style and the html class here:
f.write(HtmlFormatter(style='colorful').get_style_defs('.highlight'))

f.close()

Now you can highlight code using the sourcecode directive:

*Hello World!* in Python is so easy:

.. sourcecode:: python

    print 'Hello World!'

That’s it! If you have any questions, leave a comment.

Django: Highlighting in HTML using Pygments and Beautiful Soup

Wednesday, January 13, 2010

Two months ago I wrote about Django, Pygments and Beautiful Soup.

A few days after the post I switched from HTML markup to reStructuredText and thus didn’t need my filter anymore and forgot to post more about it.

Today I received a comment asking me to publish the source code of the template filter—so here it is:

# encoding: utf-8

"""
A filter to highlight code blocks in html with Pygments and BeautifulSoup.

    {% load highlight_code %}

    {{ var_with_code|highlight|safe }}
"""

from BeautifulSoup  import BeautifulSoup
from django import template
from django.template.defaultfilters import stringfilter
import pygments
import pygments.formatters
import pygments.lexers


register = template.Library()

@register.filter
@stringfilter
def highlight(html):
    soup = BeautifulSoup(html)
    codeblocks = soup.findAll('pre')
    for block in codeblocks:
        if block.has_key('class'):
            try:
                code = ''.join([unicode(item) for item in block.contents])
                lexer = pygments.lexers.get_lexer_by_name(block['class'])
                formatter = pygments.formatters.HtmlFormatter()
                code_hl = pygments.highlight(code, lexer, formatter)
                block.contents = [BeautifulSoup(code_hl)]
                block.name = 'code'
            except:
                raise
    return unicode(soup)

Copy the code into a file called templatetags/highlight_code.py within a new or an existing Django app.

highlight() searches the passed HTML code for <pre>-tags with a class denoting the lexer to be used, e.g.:

<p><em>Hello World!</em> in Python:</p>
<pre class="python">
print 'Hello World'
</pre>

Furthermore you might want to create a CSS file containing all the style definitions in your static media directory. The following python script will do the job:

# Call it this way:
# python gen_css.py pygments.css
import sys

from pygments.formatters import HtmlFormatter

f = open(sys.argv[1], 'w')

# You can change style and the html class here:
f.write(HtmlFormatter(style='colorful').get_style_defs('.highlight'))

f.close()

I hope I explained all this well enough—if not, leave a comment and I’ll update this post. :-)

Django-sphinxdoc 0.2 now with documentation!

Wednesday, December 30, 2009

Just released version 0.2 of django-sphinxdoc. It can now display the documentation itself, the general index and the module index. The source and static views as well as the search functionality is not implemented yet.

I’m still pondering whether to use a Google custom search or the Sphinx JavaScript search. Maybe it would be even better if I used Haystack

You can find a quick guide as well as some other guides in django-sphinxdoc’s documentation section (which is of course powered by django-sphinxdoc ;-)).

The download can be found at bitbucket.org.

Tea Timer 1.6

Tuesday, December 22, 2009

A new version of Tea Timer has just been released. It fixes as a few bugs and has two small new features:

  1. It now displays the current version number on the website.
  2. It can handle multiple instances of the widget, each with different settings. One side effect of this is, that each new instance starts with its built-in default settings. If you don’t close and reopen your widget every hour, this should not be a problem. ;-)

Download Tea Timer 1.6

One feature I did not add, was a way to easily add more background colors. Since each background has its own image, you can’t easily add a box for a hex color code. But if you realy want more or different colors, here is a quick guide to achive this.

  1. Clone the Tea Timer repository or just download the latest version.
  2. Open the bundle TemplateWidget.djproj with Dashcode.
  3. You can find the image unter TemplateWidget/front/frontImg. Adjust its size to 320×120px and color it as you want.
  4. Copy the image to clipboard, pasted in your favourite editor and save it in the Widgets image folder (Tea Timer.wdgt/Images/) as Front_mycolor.png.
  5. Open Tea Timer.wdgt/TeaTimer.html in a text editor. Around line 65 you’ll find the options for all colors. Add your color there:
...
<option value="silver">Silver</option>
<option value="mycolor">My Color</option>
  1. Save and exit the editor.
  2. Install the new version by double clicking on it.
  3. That’s it. Ask if you have any questions. :-)

[Update] Tea Timer does not run under Mac OS X 10.4. I forgot to change this at my last submission to Apple and now I have to wait until I release the next version.

Documentation for django-lastfm with django-sphinxdoc

Sunday, December 20, 2009

Yesterday I finished a first version of django-sphinxdoc that integrates Sphinx documentation into a Django powered website. Its based on Django’s documentation app, but can manage the documenation for more then one app. I’ll post more on this later.

What’s more important is, that I have put the documentation for django-lastfm online with it. :-)

django-lastfm

Sunday, November 22, 2009

On my old Wordpress blog I had a widget that let me display some of my last.fm stats. Since there was no such widget for Django powered sites (or I didn’t search well enough), I created my own as you can see in the right column.

You currently can choose between your recently listened tracks, your weekly artist chart and your top artists. I’ve created a bitbucket project for its further development. So go clone it and give some feedback.

PS: I’ll update/write the documentation within the next few days … ;-)

A BeautifulSoup with Django and Pygments

Saturday, October 24, 2009

Just added syntax highlighting using BeautifulSoup and Pygments. I took the SaltyCrane Blog for inspiration, but in contrast to it, I implemented it as a template filter in a separate application. This is surely not the most performant way, but I’m planning to use memcache, so I think this is ok.

Here an example how to use the filter:

{% load highlight_code %}
{{ my_var_with_code|highlight|safe }}

More Feeds

Tuesday, October 20, 2009

Just added Atom Feeds for post categories and post comments. They are available in the category detail and post detail views.

Comments and an Atom Feed

Sunday, October 18, 2009

Small update for this site: Comments are now working and there is an Atom Feed for the latest posts in this blog.

The behaviour and the style of the comments is not very polished yet, but I’ll work on that later. I’ll also add more feeds, e.g. for post comments, categories and tags.

Magic! Python! Django! Whee!

Wednesday, October 14, 2009

This blog is now run by Django. I didn’t really like Wordpress which I used before. And also I don’t like PHP anymore (I really liked it some years ago, but everything changed, when I learned Python …).

When I told a friend that I wanted to switch from Wordpress to something else, he just said: «Use Django.» So I took a look at the Tutorial and was instantly thrilled.

At first, I wanted to use an existing weblog app, but I also wanted to code an app for my own and since I din’t find a weblog app that I 100% liked, I just decided to write my own. So here it is (far from finished though)!

Features so far:

  • Basic post model (title, slug, pub date, modify date, status, category, HTML body)
  • Post manager for post counts (per year, month, category (and tag))
  • Hierarchical category model – imho, this is the highlight of my app.
  • Archive and Categories
  • Some special template tags.
  • Usage of a recursive template tag for the categories
  • Unit- and doctests for the models and template tags

Features to come:

  • Comments
  • Atom feeds
  • Sidebar
  • Static pages
  • Trac integration for my projects
  • Search
  • Last.fm sidebar widget
  • Tags for posts (via django-tagging?)
  • CKEditor integration
  • Pygments integration
  • Improved image/media support for posts
  • Admin actions for posts (e.g. change status and category)
  • Preview function for new posts

That’s it so far. When I have Trac running, I’ll also publish the source of my applications. :-)

Django pony