Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Celery with RabbitMQ 3.1.0 on Webfaction
A while ago Mark Liu wrote about setting up django-celery on a Webfaction host. I took the tutorial and matched it to the current version of RabbitMQ that is 3.1.0. Please read this article entirely before installing anything! Basically follow Mark's tutorial and install Erlang as well as RabbitMQ but use the current RabbitMQ binary *.tar.gz from the download page. Attention: Depending on what applications you are already running on Webfaction, the 256MB RAM plan may be insufficient to run Celery and RabbitMQ and Erlang. You can give it a try, for me it did not work, I added some more memory to my plan. Monitor your memory usage while installing. I'm using the following script to do so (replace the [username] with your username): # memory_top.sh # lists memory usage for given user every 5 seconds while true do ps -U [username] --no-headers -o rss | (tr '\n' +; echo 0) | bc sleep 5 done Do not configure rabbitmq-server and the rabbitmq-env file as written. Instead do the following (I extracted RabbitMQ to \$HOME/rabbitmq/rabbitmq_server-3.1.0 and will use this paths): Edit \$HOME/rabbitmq/rabbitmq_server-3.1.0/sbin/rabbitmq-defaults: ... # comment these lines: #CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq #LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq #MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia # add these lines: CONFIG_FILE=/home/[username]/rabbitmq/rabbitmq_server-3.1.0/sbin/ LOG_BASE=/home/[username]/logs/user/rabbitmq # create this … -
Celery with RabbitMQ 3.1.0 on Webfaction
A while ago Mark Liu wrote about setting up django-celery on a Webfaction host. I took the tutorial and matched it to the current version of RabbitMQ that is 3.1.0. Please read this article entirely before installing anything! Basically follow Mark's tutorial and install Erlang as well as RabbitMQ but use the current RabbitMQ binary *.tar.gz from the download page. Attention: Depending on what applications you are already running on Webfaction, the 256MB RAM plan may be insufficient to run Celery and RabbitMQ and Erlang. You can give it a try, for me it did not work, I added some more memory to my plan. Monitor your memory usage while installing. I'm using the following script to do so (replace the [username] with your username): # memory_top.sh # lists memory usage for given user every 5 seconds while true do ps -U [username] --no-headers -o rss | (tr '\n' +; echo 0) | bc sleep 5 done Do not configure rabbitmq-server and the rabbitmq-env file as written. Instead do the following (I extracted RabbitMQ to \$HOME/rabbitmq/rabbitmq_server-3.1.0 and will use this paths): Edit \$HOME/rabbitmq/rabbitmq_server-3.1.0/sbin/rabbitmq-defaults: ... # comment these lines: #CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq #LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq #MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia # add these lines: CONFIG_FILE=/home/[username]/rabbitmq/rabbitmq_server-3.1.0/sbin/ LOG_BASE=/home/[username]/logs/user/rabbitmq # create this … -
Celery with RabbitMQ 3.1.0 on Webfaction
A while ago Mark Liu wrote about setting up django-celery on a Webfaction host. I took the tutorial and matched it to the current version of RabbitMQ that is 3.1.0. Please read this article entirely before installing anything! Basically follow Mark's tutorial and install Erlang as well as RabbitMQ but use the current RabbitMQ binary *.tar.gz from the download page. Attention: Depending on what applications you are already running on Webfaction, the 256MB RAM plan may be insufficient to run Celery and RabbitMQ and Erlang. You can give it a try, for me it did not work, I added some more memory to my plan. Monitor your memory usage while installing. I'm using the following script to do so (replace the [username] with your username): # memory_top.sh # lists memory usage for given user every 5 seconds while true do ps -U [username] --no-headers -o rss | (tr '\n' +; echo 0) | bc sleep 5 done Do not configure rabbitmq-server and the rabbitmq-env file as written. Instead do the following (I extracted RabbitMQ to \$HOME/rabbitmq/rabbitmq_server-3.1.0 and will use this paths): Edit \$HOME/rabbitmq/rabbitmq_server-3.1.0/sbin/rabbitmq-defaults: ... # comment these lines: #CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq #LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq #MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia # add these lines: CONFIG_FILE=/home/[username]/rabbitmq/rabbitmq_server-3.1.0/sbin/ LOG_BASE=/home/[username]/logs/user/rabbitmq # create this … -
Water software and python
Weird link I got from a colleague this evening: some Dutch ICT awards were awarded this week. The link is in Dutch, but trust me if I say that the two (to me) relevant awards were awarded to Python (as a fantastic computer language) and to hydrologic (a Dutch water+ict company, for their water software). For me, the weirdness is three-step process: Ok, an award for Python is nice! Almost all our (Nelen & Schuurmans) software is written in Python. And virtually all of it is open source. Just look at github: https://github.com/nens/ and especially https://github.com/lizardsystem/ . Lots of Python and Django. The water software award goes to a closed source Microsoft shop. We (Nelen & Schuurmans) are the perfect combination of water software and open source Python. We're a water consultancy company. And a successful one, too. Lots of great projects. Most of them centered around something called Lizard (http://lizard.net for the mostly-Dutch business language stuff, http://lizard.org for the English IT stuff). Lots of partners working on and with it. It is a real platform with every partner putting in ideas and expertise. Most of the actual programming happens at our place, to be fair, at the moment. But... … -
Django Fieldsets
HTML forms contain a construct called a ``fieldset``. These are generally used to segment a form: splitting a form into groups of fields that are logically grouped. Each fieldset may also have a legend. Django's forms have no concept of a fieldset natively, but with a bit of patching, we can make every django form capable of rendering itself using fieldsets, yet still be backwards compatible with non-fieldset-aware templates. Ideally, we would like to be able to render a form in a way similar to: {% highlight html %} {% raw %} {% for fieldset in form.fieldsets %} {{ fieldset.title }} {% for field in fieldset %} {{ field.label_tag }} {{ field }} {{ field.help_text }} {{ field.errors }} {% endfor %} {% endfor %} {% endraw %} {% endhighlight %} And, it would make sense to be able to declare a form's fieldsets in a manner such as: {% highlight python %} class MyForm(forms.Form): field1 = forms.BooleanField(required=False) field2 = forms.CharField() class Meta: fieldsets = ( ('Fieldset title', { 'fields': ('field1', 'field2') }), ) {% endhighlight %} This is similar to how fieldsets are declared in the django admin. We can't just simply create a subclass of ``forms.Form``, and do … -
Introducing Salmon
Click here to see this post in it's natural habitat and to watch and leave comments. Over the years, I’ve used a lot of server monitoring systems. Big enterprisey ones like Zabbix, Zenoss, and Hyperic, smaller ones like munin and monit, stuff in the middle like Graphite, and hosted solutions like New Relic. Throughout the search, I never found one that hit the sweet spot for me. They were either too complicated and required too much setup upfront or too limited in what they had to offer. Like any good developer frustrated with the tools available, I set off on building my own. The result (still in its infancy) is Salmon. Its aim is to be a simple, yet powerful (enough) server monitoring and alerting system. Salmon itself is a simple project, but takes advantage of some great open source libraries for much of its advanced functionality. Salt Salt is the new kid on the block for server configuration management (think Chef and Puppet). Billing it as a configuration management tool is underselling it though. It’s really a remote execution framework. I hadn’t wrapped my head around this until a chat with a few of the guys from SaltStack at … -
Introducing Salmon
Over the years, I’ve used a lot of server monitoring systems. Big enterprisey ones like Zabbix, Zenoss, and Hyperic, smaller ones like munin and monit, stuff in the middle like Graphite, and hosted solutions like New Relic. Throughout the search, I never found one that hit the sweet spot for me. They were either too complicated and required too much setup upfront or too limited in what they had to offer. Like any good developer frustrated with the tools available, I set off on building my own. The result (still in its infancy) is Salmon. Its aim is to be a simple, yet powerful (enough) server monitoring and alerting system. Salmon itself is a simple project, but takes advantage of some great open source libraries for much of its advanced functionality. Salt Salt is the new kid on the block for server configuration management (think Chef and Puppet). Billing it as a configuration management tool is underselling it though. It’s really a remote execution framework. I hadn’t wrapped my head around this until a chat with a few of the guys from SaltStack at PyCon this year. Salt is a perfect tool for stats collection on remote servers. It offers … -
Simple Job Queues with django_rq
Simple Job Queues with django_rq The de facto solution for job queues with background workers is Celery and RabbitMQ, but it is not the right fit for every project. RQ is an alternative to Celery, and while not as featureful, does provide a lightweight solution that makes it easy to ... -
Introducing Salmon
Over the years, I’ve used a lot of server monitoring systems. Big enterprisey ones like Zabbix, Zenoss, and Hyperic, smaller ones like munin and monit, stuff in the middle like Graphite, and hosted solutions like New Relic. Throughout the search, I never found one that hit the sweet spot for me. They were either too complicated and required too much setup upfront or too limited in what they had to offer. Like any good developer frustrated with the tools available, I set off on building my own. -
Deploying django using docker
In this blog post we'll take a look at Docker and discuss ways to deploy django using docker. About Docker: Docker is a tool that helps you manage lxc containers and images. You can create an image, launch container from the existing image, attach to a running container, and generally play around with the containers. The most important benefit of using docker and containers in general is that you have a clean, hygeinic and portable runtime enivorment for your app. This means you don't have to worry about missing dependecies, packages and other pain points during subsequent deployments. You can take a snapshot of a running container and restore it again when required. Plus, each app runs in it's own isolated container so you can have various versions of libraries and other dependecies for each app without worrying about it's effect on other apps. Coming from the python world where virtualenv is ubiqitous, it's a huge relief to me that there's such a better and cleaner solution. Note: Docker is still under heavy development. It should not yet be used in production. Check the repo for recent progress. To know more about docker and install it, go to the "Getting … -
Core Concepts of Django ModelForms
In my opinion, the concepts behind Django's model forms can be listed in just six (6) bullets. The bullets I've marked in bold at the top are the topic of this blog post, while the two of these that were covered in a previous blog post on Django forms are at bottom of my list. ModelForms render Model fields as HTML. ModelForms select validators based off of Model field definitions. ModelForms don't have to display/change all available fields. ModelForms save dictionaries to SQL tables. Forms are "just" Python constructs. (covered previous) Forms validate Python dictionaries. (covered previous) ModelForms render Model fields as HTML. If I create a Django model: # myapp/models.py from django.db import models class MyModel(models.Model): title = models.CharField(max_length=100) Then attach it to a ModelForm: # myapp/forms.py from django import forms from .models import MyModel class MyModelForm(forms.ModelForm): class Meta: model = MyModel I can render it in a template, or for better clarity in this post, the Python REPL: >>> from myapp.forms import MyModelForm >>> mf = MyModelForm() >>> mf <__main__.MyForm object at 0x1023c8bd0> >>> print(mf) <tr><th><label for="id_title">Title:</label></th> <td><input id="id_title" name="title" maxlength="100" type="text" /></td></tr> ModelForms select validators based off of Model field definitions. One of the nice things about … -
Core Concepts of Django ModelForms
In my opinion, the concepts behind Django's model forms can be listed in just six (6) bullets. The bullets I've marked in bold at the top are the topic of this blog post, while the two of these that were covered in a previous blog post on Django forms are at bottom of my list. ModelForms render Model fields as HTML. ModelForms select validators based off of Model field definitions. ModelForms don't have to display/change all available fields. ModelForms save dictionaries to SQL tables. Forms are "just" Python constructs. (covered previous) Forms validate Python dictionaries. (covered previous) ModelForms render Model fields as HTML. If I create a Django model: # myapp/models.py from django.db import models class MyModel(models.Model): title = models.CharField(max_length=100) Then attach it to a ModelForm: # myapp/forms.py from django import forms from .models import MyModel class MyModelForm(forms.ModelForm): class Meta: model = MyModel I can render it in a template, or for better clarity in this post, the Python REPL: >>> from myapp.forms import MyModelForm >>> mf = MyModelForm() >>> mf <__main__.MyForm object at 0x1023c8bd0> >>> print(mf) <tr><th><label for="id_title">Title:</label></th> <td><input id="id_title" name="title" maxlength="100" type="text" /></td></tr> ModelForms select validators based off of Model field definitions. One of the nice things about … -
Serving files if they exist or fallback to wsgi app
We want to achieve this routing scheme [2]: /anything => static file, if it exists ! / => the wsgi app For some reason this isn't as straightforward as in Nginx. Example [1]. With Apache you have to employ mod_rewrite, which is not straightforward if you're not writing rewrite rules and conditions all day long :) The first this is to alias the wsgi app to a different location than the root: WSGIScriptAlias /wsgi /path/to/the/app.wsgi We need to make the location of the static files the DocumentRoot: DocumentRoot /path/to/static/files <Directory /path/to/static/files> Order deny,allow Allow from all </Directory> And now the mod_rewrite dance: RewriteEngine on Set some conditions: if the request path is not a filename: RewriteCond /path/to/static/files%{REQUEST_FILENAME} !-f And not a directory: RewriteCond /path/to/static/files%{REQUEST_FILENAME} !-d Optional, exempt any other special locations (I have an Alias /media): RewriteCond %{REQUEST_URI} !^/media And finally the rewrite rule: RewriteRule ^(.*)$ /wsgi$1 [PT,L] PT - Take the request to go through the URL mapping machinery (so Alias, WSGIScriptAlias etc are applied again for the resulting URL) L - Stops processing any other RewriteRule for this request. This is implied by PT but it put it there in case I forget this :). The full configuration: … -
Serving files if they exist or fallback to wsgi app
We want to achieve this routing scheme [2]: /anything => static file, if it exists ! / => the wsgi app For some reason this isn't as straightforward as in Nginx. Example [1]. With Apache you have to employ mod_rewrite, which is not straightforward if you're not writing rewrite rules and conditions all day long :) The first this is to alias the wsgi app to a different location than the root: WSGIScriptAlias /wsgi /path/to/the/app.wsgi We need to make the location of the static files the DocumentRoot: DocumentRoot /path/to/static/files <Directory /path/to/static/files> Order deny,allow Allow from all </Directory> And now the mod_rewrite dance: RewriteEngine on Set some conditions: if the request path is not a filename: RewriteCond /path/to/static/files%{REQUEST_FILENAME} !-f And not a directory: RewriteCond /path/to/static/files%{REQUEST_FILENAME} !-d Optional, exempt any other special locations (I have an Alias /media): RewriteCond %{REQUEST_URI} !^/media And finally the rewrite rule: RewriteRule ^(.*)$ /wsgi$1 [PT,L] PT - Take the request to go through the URL mapping machinery (so Alias, WSGIScriptAlias etc are applied again for the resulting URL) L - Stops processing any other RewriteRule for this request. This is implied by PT but it put it there in case I forget this :). The full configuration: … -
Django Proxy Model State Machine
Finite State Machines (fsm) are a great way to model something that has, well, a finite number of known states. You can easily specify the different states, and the transitions between them. Some time ago, I came across a great way of doing this in python: [Dynamic State Machines](http://harkablog.com/dynamic-state-machines.html). This maps well onto an idea I have been toying with lately, replacing a series of linked models representing different phases in a process with one model type. Initially, I had thought to just use a `type` flag, but actually changing the class seems like a better idea. One aspect of django's models that makes it easy to do this is the concept of a Proxy Model. These are models that share the database table, but have different class definitions. However, usually a model instance will be of the type that was used to fetch it: {% highlight python %} class ModelOne(models.Model): field = models.CharField() class ModelOneProxy(ModelOne): class Meta: proxy = True ModelOneProxy.objects.get(pk=1) # Returns a ModelOneProxy object. ModelOne.objects.all() # Returns all ModelOne objects. {% endhighlight %} However, by using a type field, we can, at the time it is fetched from the database, turn it into the correct type. {% … -
Neat and tidy read-only fields
I have a recurring pattern I'm seeing, where I have a field in a model that needs to be read-only. It usually is a ``Company`` to which an object belongs, but it also occurs in the case where an object belongs to some collection, and isn't permitted to be moved to a different collection. Whilst there are some workarounds that apply the field's value to the instance after creating, it's nicer to be able to apply the read-only nature declaratively, and not have to remember to do something in the form itself. Unfortunately, in django, normal field subclasses don't have access to the ``initial`` argument that was used to construct it. But ``forms.FileField`` objects do. So we can abuse that a little. We also need a widget, that will always return ``False`` for questions about if the value has been changed, and re-render with the initial value at all times. {% highlight python %} from django import forms class ReadOnlyWidget(forms.HiddenInput): def render(self, name, value, attrs): value = getattr(self, 'initial', value) return super(ReadOnlyWidget, self).render(name, value, attrs) def _has_changed(self, initial, data): return False class ReadOnlyField(forms.FileField): widget = forms.HiddenInput def __init__(self, *args, **kwargs): forms.Field.__init__(self, *args, **kwargs) def clean(self, value, initial): self.widget.initial = initial … -
Django AJAX Forms
I think the more [Django](http://www.djangoproject.com) code I write, the more I like one particular feature. Forms. Simple as that. Forms are the reason I keep coming back to django, and discard other web frameworks in other languages, even though I really want to try them. One pattern I have been using a fair bit, which was touched on in another post, is using AJAX to handle form submission, and displaying the response. Before we continue, a quick recap on what Django's forms offer us. * A declarative approach to defining the fields a form has, including validation functions. * Will render themselves to HTML input elements, as appropriate. * Handle validation of incoming form-encoded (or otherwise provided) data. * Fields can validate themselves, and can include validation error messages as part of the HTML output. * (Model forms) handle instantiation of and updating of model instances. A normal form-submission cycle contains a POST or GET request to the server, which responds with a fresh HTML page, which the browser renders. The normal pattern for successful POST requests is to redirect to a GET afterwards, to prevent duplicate submission of forms. By doing an ajax request instead of a full-page request … -
Using Sentry to log exceptions and logging messages from Django projects
sentry is something like an logs aggregation application. It can be used to log exceptions and other log messages from Django or other framework/programming languages (including JavaScript). Instead of sending a lot of emails to a project admin Django will send them to Sentry which will aggregate them and present in a readable form. Sentry is actually a Django application that can be launched setup on your local computer - perfect for testing. In this article I'll show you how to run Sentry on your local computer (basic setup) and how to configure Django and other parts of the Python code for optimal Sentry logging results. -
Making your Django app more pluggable
This blog post is about ways of doing things in your Django app, and pitfalls to avoid, so your app is easier to plug in to other sites. Introduction One of the most enticing promises of Django is that you’ll be able to add features to your site by just downloading apps and plugging them in. That’s one of the many reasons we use Django here at Caktus: we can build useful web sites for our clients more quickly by not having to re-invent the same building blocks all the time, and focusing on the unique value-add for each client. This is also one of the attractions of building and releasing open-source Django apps: they’re easy for other people to use, and having other people use your code is very satisfying. But Django apps don’t become easily pluggable automatically, and it’s easy to do things in a way that make it much harder for other sites to use an app. A book could be written about this topic. This post just gives examples of some areas that can cause problems, and might stimulate some thought when designing your app. Not everything needs to be an app Does your package have … -
Using mysql load data infile with django
Using mysql load data infile with django -
Using Fabric to update a remote svn checkout with ssh public key authentication
Using Fabric to update a remote svn checkout with ssh public key authentication -
Allow squid/mod_wsgi to pass the HTTP_AUTHORIZATION header to Apache
Allow squid/mod_wsgi to pass the HTTP_AUTHORIZATION header to Apache -
Django "view-permissions" for related objects
Django "view-permissions" for related objects -
Nested resources in Tastypie
Nested resources in Tastypie -
Custom choices in Django admin
Custom choices in Django admin