Django community: RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
A rich python console and more in Kate editor
I have done some improvements in the plugins: python_console_ipython, python_autocomplete, python_utils, js_utils, xml_pretty and django_utils. These plugins I added a month and a half ago (except python_console_ipython) to the kate repository. I have done two improvements and a new feature: Now they work with Python2 and Python3 (except python_autocomplete, this only works with Python2, due pysmell dependence) Now they have a configuration page (except python_autocomplete, this should not have configuration parameters) Page Config Plugins The new feature is the integration of the python_autocomplete and python_console_ipython plugins with the project plugin. The behaviour of these plugins depends of the loaded project. That is to say, the python_autocomplete plugin autocompletes with our project modules. Currently the kate community is working to add a new python autocomplete using jedi (this will work with Python2 and Python3). Python Autocomplete (with our modules) And the python_console_ipython plugin has something that we can name the project console. A ipython shell with the python path of our project and with the environments variables that the project plugin needs. IPython Console (converted in a django shell) To use this we need to add in the project file (.kateproject) a new attribute named “python”, with this structure: { "name": … -
What an Operation
This week sees django.db.migrations gain Operations, an Executor, and new command plans. Much of the work on migrations so far has been laying a good bit of groundwork, but now everything is starting to come together into a working whole. The most important thing to land this week is Operations - the things which migrations will be structured around. Here's an example of what a migration would look like: from django.db import migrations, models class Migration(migrations.Migration): dependencies = [("myapp", "0001_initial")] operations = [ migrations.AddField("Author", "rating", models.IntegerField(default=0)), migrations.CreateModel( "Book", [ ("id", models.AutoField(primary_key=True)), ("author", models.ForeignKey("migrations.Author", null=True)), ], ) ] As you can see, the Operations define what the migration does, and serve as the top-level element that allows more declarative migrations - as opposed to those in South, which were an opaque block of procedural code. What's in a name? But what exactly do Operations do? That's a relatively easy question to answer - they have an interface of exactly three methods. The first, state_forwards, takes a project state (that's an in-memory representation of all of your models and fields at a point in time) and modifies it to reflect the type of change it makes - AddField, for example, would add … -
Basic Ajax
Ajax is essential these days, and fortunately Django provides some good mechanism for doing AJAX well. This video goes over some of the basics you need to do and know when doing AJAX in Django.Watch Now... -
Basic Ajax
Ajax is essential these days, and fortunately Django provides some good mechanism for doing AJAX well. This video goes over some of the basics you need to do and know when doing AJAX in Django.Watch Now... -
Mysql too slow in tests ? ramdisk it !
Suppose you're doing it wrong: you're using MySQL and your test suite is slow very slow. MySQL is very slow at DDL statements so creating/clearing/loading that test database all the time is going to be very slow. People have worked out some solutions like: Using TRUNCATE instead of DROP/CREATE and/or other tricks [1]. Fine tunning mysql ... just for development ?!? [2], [3]. Use TransactionTestCase Use sqlite in :memory: - very bad idea, lots of subtle bugs won't be catched in the test suite. And finally there's one solution that doesn't have the code trade-offs but it's a pain to get it set up: MySQL running on ramdisk (known as tmpfs). I've seen some solutions [4], [5], [6], none worked on Ubuntu (12.04) very well for me thus I've cooked up my own version. It's largely based on [6] but with cleanup code in case you already ran it and added the missing database setup part (you might want to edit the passwords and database names). #!/bin/bash -xEe i=$1 if [ -z "$i" ]; then echo "Missing argument: instance number." exit 1 fi port=$[3306+$i] pid=`cat "/var/run/mysqld/mysqld$i.pid" || true` if [ -n "$pid" ]; then kill -9 $pid while kill -0 … -
Mysql too slow in tests ? ramdisk it !
Suppose you're doing it wrong: you're using MySQL and your test suite is slow very slow. MySQL is very slow at DDL statements so creating/clearing/loading that test database all the time is going to be very slow. People have worked out some solutions like: Using TRUNCATE instead of DROP/CREATE and/or other tricks [1]. Fine tunning mysql ... just for development ?!? [2], [3]. Use TransactionTestCase Use sqlite in :memory: - very bad idea, lots of subtle bugs won't be catched in the test suite. And finally there's one solution that doesn't have the code trade-offs but it's a pain to get it set up: MySQL running on ramdisk (known as tmpfs). I've seen some solutions [4], [5], [6], none worked on Ubuntu (12.04) very well for me thus I've cooked up my own version. It's largely based on [6] but with cleanup code in case you already ran it and added the missing database setup part (you might want to edit the passwords and database names). #!/bin/bash -xEe i=$1 if [ -z "$i" ]; then echo "Missing argument: instance number." exit 1 fi port=$[3306+$i] pid=`cat "/var/run/mysqld/mysqld$i.pid" || true` if [ -n "$pid" ]; then kill -9 $pid while kill -0 … -
A Short Summary on Sybase SQL Anywhere and Python
As some of my older rage-filled articles indicated, we’re still running some services on Sybase’s SQL Anywhere. Since it cost me many hours and sanity wrangling, I think it may be helpful to others to summarize the current situation for Python developers. Abstract It hurts a little bit less now. We’re using the official sqlanydb driver in production. Options There are four ways known to me to access a SQL Anywhere database in Python: Their official sqlanydb driver that requires their client libraries. Used by sqlany-django which gives you Django’s ORM with SQL Anywhere. Their official ODBC drivers via pyodbc. The python-sybase module which also needs a pretty complete SQL Anywhere installation to build. Supported by SQLAlchemy. FreeTDS via pyodbc. Problems Historically, there have been problems with all of them: sqlanydb crashed. Their ODBC drivers leaked semaphores which led to a denial of service eventually. sqlanydb seems to leak prepared statements on exceptions which leads to Resource Governor errors. I never got python-sybase working and it seems abandonware. FreeTDS randomly hangs in queries, doesn’t get a lot of attention either. In our experience so far, it seems like the first two problems have been remedied as of sqlanydb 1.0.5 (which … -
Non-trusted mercurial .hgrc on a Vagrant virtualbox
I use Vagrant and virtualbox for developing inside ubuntu on OSX. I have a couple of mercurial (hg) checkouts and they were giving me grief: $ hg up Not trusting file /some-repo/.hg/hgrc from untrusted user 501, group dialout The .hg/hgrc file contains the remote repo URL, so pulling even won't work. The problem is apparently the interaction between virtualbox and OSX if you use NFS to mount part of your OSX's harddisk. The solution/workaround is quite simple. Tell mercurial to trust that 501 user in your ~/.hgrc: .... [trusted] users = 501 Now it works! -
Continuous integration of webapps with Buildbot
Buildbot, the Continuous Integration Framework Buildbot is an excellent tool used by many well known products like Firefox and Google Chrome. Although it is used to build complex projects from code compilation to packaging it can also be used to do continuous integration of smaller projects and apps. Buildbot can get notifications from services like GitHub and BitBucket, and from in-house hosted repositories. It can be set up to trigger builds on results of other builds and allows full service control through IRC among many other features. In this story I show how to set up Buildbot to do continuous integration of a sample web app and a sample dependent web project. I focus on using Git repositories, hosted in both Github and your own location. -
Continuous integration of webapps with Buildbot
Buildbot is a powerful and flexible tool used by many sophisticated products like Python, Firefox, Webkit, Google Chrome, Twisted, Node.js and [many more](http://trac.buildbot.net/wiki/SuccessStories). Buildbot, the Continuous Integration Framework Buildbot is primarily used to build software for different architectures covering the process from code compilation to packaging, but it can also be used to do continuous integration of less sophisticated projects where tools and services like [Jenkins](http://jenkins-ci.org/) and [Travis-CI](https://travis-ci.org/) are more popular but less flexible. In this story I introduce Buildbot to run test suites for two products, a sample web application and a sample dependent web project. The web app is a fully functional open source example of a Django pluggable application. While the project is an implementation of the Django [tutorial](https://docs.djangoproject.com/en/1.5/intro/tutorial01/) plus a few simple test cases. Source code for the [app](https://github.com/danirus/django-sample-app) and the [project](https://github.com/danirus/django-sample-project) as long as the configuration files for Buildbot are available in GitHub. #### 1. The scenario The configuration will face the following final scenario: 1. Test suites for both the app and the project will run under several versions of Django and Python in order to be sure that the app is usable for every supported Django release and that the project might be … -
Continuous integration of webapps with Buildbot
Buildbot is a powerful and flexible tool used by many sophisticated products like Python, Firefox, Webkit, Google Chrome, Twisted, Node.js and [many more](http://trac.buildbot.net/wiki/SuccessStories). Buildbot, the Continuous Integration Framework Buildbot is primarily used to build software for different architectures covering the process from code compilation to packaging, but it can also be used to do continuous integration of less sophisticated projects where tools and services like [Jenkins](http://jenkins-ci.org/) and [Travis-CI](https://travis-ci.org/) are more popular but less flexible. In this story I introduce Buildbot to run test suites for two products, a sample web application and a sample dependent web project. The web app is a fully functional open source example of a Django pluggable application. While the project is an implementation of the Django [tutorial](https://docs.djangoproject.com/en/1.5/intro/tutorial01/) plus a few simple test cases. Source code for the [app](https://github.com/danirus/django-sample-app) and the [project](https://github.com/danirus/django-sample-project) as long as the configuration files for Buildbot are available in GitHub. #### 1. The scenario The configuration will face the following final scenario: 1. Test suites for both the app and the project will run under several versions of Django and Python in order to be sure that the app is usable for every supported Django release and that the project might be … -
Kruskal’s Algorithm
I planned to write a serious of posts on working and analysis of some of the basic algorithms. I also planned to give link to a JAVA implementation of the same. I will start off this series by first describing the Kruskal’s Algorithm for finding the cost of the Minimum Spanning Tree in a graph. Minimum Spanning Tree: A minimum cost spanning tree is a subgraph induced on the original graph such that the induced sub-graph is a tree on all the vertices on the original graph and the sum of weights of the edges on this graph is the minimum among all the possible spanning trees of the graph. The basic intuition behind the Kruskal’s algorithm is that the least weighted edge of the original graph will appear in the minimum spanning tree. Hence, sorting the edges based on their weight might be a first step in building the spanning tree. Does that mean, the first n-1 least weight edges are sufficient to build the spanning tree (If n is the number of vertices in the graph) ? We also need to ensure that the edges picked for the spanning tree do not form a cycle. Hence, that is … -
Video and image sitemaps in Django
I recently added a sitemap to a site with many images and videos. Having the media files in the sitemap helps search engines to index them. I hadn't done this with Django before, so here are some notes. If you're not familiar with sitemaps in Django check out the official documentation and create a sitemap for a model with media files. Adding Videos To customize the output and to add videos to the default sitemap you'll want to create a custom template as described in the documentation. I always copy the default template from contrib/sitemaps/templates/sitemap.xml You can access your model instances through url.item. Here's an example for a video model that's connected to an item through foreign keys: {% load thumbnail %} {% for video in url.item.video_set.all %} <video:video> {% thumbnail video.image "662x372" crop="center" as thumb %} <video:thumbnail_loc>{{ MEDIA_URL }}{{ thumb }}</video:thumbnail_loc> {% endthumbnail %} <video:title>{{ url.item.title }}</video:title> {% if url.item.description %} <video:description>{{ url.item.description|striptags }}</video:description> {% endif %} <video:content_loc>{{ MEDIA_URL }}{{ video.file }}</video:content_loc> <video:publication_date>{{ url.item.pub_date|date:'c' }}</video:publication_date> <video:family_friendly>yes</video:family_friendly> <video:live>no</video:live> </video:video> {% endfor %} Include something simliar to this inside the <url> tag. I think this site uses sorl-thumbnail, so you'll probably want to modify the thumbnail section at least. One problem … -
Video and image sitemaps in Django
I recently added a sitemap to a site with many images and videos. Having the media files in the sitemap helps search engines to index them. I hadn't done this with Django before, so here are some notes. If you're not familiar with sitemaps in Django check out the official documentation and create a sitemap for a model with media files. Adding Videos To customize the output and to add videos to the default sitemap you'll want to create a custom template as described in the documentation. You can access your model instance through url.item. Here's an example for videos: {% load thumbnail %} {% for video in url.item.video_set.all %} <video:video> {% thumbnail video.image "662x372" crop="center" as thumb %} <video:thumbnail_loc>{{ MEDIA_URL }}{{ thumb }}</video:thumbnail_loc> {% endthumbnail %} <video:title>{{ url.item.title }}</video:title> {% if url.item.description %} <video:description>{{ url.item.description|striptags }}</video:description> {% endif %} <video:content_loc>{{ MEDIA_URL }}{{ video.file }}</video:content_loc> <video:publication_date>{{ url.item.pub_date|date:'c' }}</video:publication_date> <video:family_friendly>yes</video:family_friendly> <video:live>no</video:live> </video:video> {% endfor %} Include something simliar to this inside the <url> tag. I think this site uses sorl-thumbnail, so you'll probably want to modify the thumbnail section at least. One problem I had were invalid timestamps. The sitemap specs are very specific, either YYYY-MM-DD or YYYY-MM-DDThh:mm:ss+TZD. The first format … -
Video and image sitemaps in Django
I recently added a sitemap to a site with many images and videos. Having the media files in the sitemap helps search engines to index them. I hadn't done this with Django before, so here are some notes. If you're not familiar with sitemaps in Django check out the official documentation and create a sitemap for a model with media files. Adding Videos To customize the output and to add videos to the default sitemap you'll want to create a custom template as described in the documentation. I always copy the default template from contrib/sitemaps/templates/sitemap.xml You can access your model instances through url.item. Here's an example for a video model that's connected to an item through foreign keys: {% load thumbnail %} {% for video in url.item.video_set.all %} <video:video> {% thumbnail video.image "662x372" crop="center" as thumb %} <video:thumbnail_loc>{{ MEDIA_URL }}{{ thumb }}</video:thumbnail_loc> {% endthumbnail %} <video:title>{{ url.item.title }}</video:title> {% if url.item.description %} <video:description>{{ url.item.description|striptags }}</video:description> {% endif %} <video:content_loc>{{ MEDIA_URL }}{{ video.file }}</video:content_loc> <video:publication_date>{{ url.item.pub_date|date:'c' }}</video:publication_date> <video:family_friendly>yes</video:family_friendly> <video:live>no</video:live> </video:video> {% endfor %} Include something simliar to this inside the <url> tag. I think this site uses sorl-thumbnail, so you'll probably want to modify the thumbnail section at least. One problem … -
Getting features into Django
Getting new features into Django isn’t easy. It’s that way for a reason — I spoke recently about why conservatism is a virtue — but it does happen. I’d like to do a better job explaining how we decide what goes in and what goes out, so here’s a lightly adapted version of something I posted on the mailing list this evening. It’s three things I look for when I’m trying to determine whether something is “right” for Django or not: -
South 0.8, Migrations and DjangoCon
A new release of an old friend, and more news on django.db.migrations. I've wanted to get a new release of South out for ages, so I'm delighted that I've finally done so. South 0.8 is now available on PyPI - there's not a great many new changes, the most notable (and the reason for the major version bump) being Python 3 support. Aymeric Augustin was instrumental in getting that support implemented, so I'd like to thank him for his work on it. On a related note, support for Python 2.5 is being dropped - if you still need that, you'll need to stick with the 0.7.x series. The other notable change is support for index_together, one of the new improvements in Django 1.5 and something that should have been released a while ago. There's still no first-party support for AUTH_USER_MODEL - it'll work fine as long as you're not distributing third-party apps with migrations. The overall solution to that is something that will have to be implemented in the rewrites that are underway. db.migrations Those rewrites are coming along well, however. Last week I was at DjangoCon EU, in Warsaw, Poland, and I had a fantastic time, as you can … -
A Rhapsody In Warsaw
A field, a tent, and a large amount of Polish food - the makings of a great conference. DjangoCon and I have a long history. The very first DjangoCon, back in 2008, was also my very first conference - and I've achieved the slightly dubious honour of having attended every single one. They are not, of course, the only conferences that I go to; these days I try to speak at a variety of events. I've seen a lot of venues and they're all variations on a theme. That theme, of course, is large rooms full of chairs. DjangoCon EU 2013, hosted last week in Warsaw, bucked that trend and was probably the best yet - and that's not something I say lightly. Ola Sitarska and the rest of her team went for an inspired gamble that really paid off. The stage, and Craig Kersteins. From flickr.com/photos/patrick91 When I first heard of the plans to host this year's DjangoCon EU in a circus tent, I was a little sceptical - after all, conference venues have evolved over many decades to serve the many needs of a large-scale event. Seating, airflow, power, networking, A/V, catering and toilets are all needs of … -
Django: Stop Writing Settings files
At this year's DjangoCon Europe I gave a Lightning talk titled "Stop Writing Settings Files". You can see the slides on SpeakerDeck. After the talk I got some interesting feedback. Some people disagree on some points for very valid reasons so I tought I'd write in more details what I was talking about. Settings across multiple environments, how do they work? A historically famous pattern is the "local_settings trick": it simply consists in adding the following lines at the end of the project's settings: try: from local_settings import * except ImportError: pass This works combined with a local_settings.py that's kept out of source control and managed manually for development-specific or production-specific settings. One issue with this technique is that it's hard to extend the base settings. If you want to add something to the value defined in the base settings, you need to copy the initial value completely. There is no way to add something to INSTALLED_APPS or MIDDLEWARE_CLASSES without redefining the value completely. To solve this problem, another pattern has emerged. Coined by Jacob Kaplan-Moss as The One True Way, it consists in reversing the import flow. Instead of importing local settings from the base settings, just import the … -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Django 1.5 Cheat Sheet
At Mercurytide, we know all too well the difficulties of memorizing shortcuts when you work in different frameworks. Our skilled developers have created a solid, quick-start cheat sheet with an easy to reference layout. -
Keynote - Daniel Greenfeld
Django conferences have a tradition: there's an external luminary that gets to give a critical talk on Django. His talk won't be that. He's not external either: he wrote two scoops of Django together with Audrey (see also my review). Being critical is sometimes easy. Just bash class based views, for instance. Bashing is easy. A rant like Zed Shaw's is fun, but he's not asked because of his rants, but because of his contributions (like books). Similarly, Django delivers working stuff and that working stuff makes a lot of our work possible. So here are some good points about Django: Django is everywhere. So many people and companies use it. Django is powered by Python. Pep8, python is beautiful. And there's the import this zen of python that we use all the time to steer others and ourselves in the right direction. Django's API wins. It is understandable. No weird names: templates, views, logging, sessions. Django projects also have understandable structures. If there's no views.py or models.py or templates/ directory, you know someone messed something up. Fat models are great. Just put your business logic all on your models. They do get big this way, however. You can make … -
Lightning talks day 3 - Djangocon.eu
html5lib Browsers are terribly forgiving. Python's parsers don't deal with everything, even valid html5 docs. html5lib was a problem. Google code and so and not python 3 compatible. The new html5lib supports python 3. Github, readthedocs, works fine! See https://pypi.python.org/pypi/html5lib Real time web - Aymeric Augustin He looked at web sockets in django. He played with tulip, Guido's library for async python. He had 1000 processes calculating a 'game of life' screen and django connected with them just fine and pushed the result to the browser. PyWaw PyWaw is a python community in Warsaw. They have now had 24 meetings with about 55 attendees. At the last meeting they even had 100 people attending. They are not alone in Poland, there are other user groups. So... go back to your cities and start user groups! Scrapy Screen scraping is when you need to get structured information from the web, quickly and with no hassle. Scrapy takes the hassle out of screen scaping. It takes away the pain of parsing horrible html. It has perfect documentation and a helpful community. You can even scrape from amazon, even including logging in. What can you do? Convert SVG to VML. Stock checker for … -
Does your stuff scale? - Steven Holmes
They grew from a two-person company to a 70-person one in two years. Central to that growth were Django and google app engine. Scalability means both load scalability and functional scalability. You also have to deal with organizational scalability and geographical scalability if you want to grow your organization. 1: Running Django on app engine It is easy to get confused. Is app engine real? Is it a joke? How to run your django stuff on it? Their reasons to use it: Auto-scaling. They build high-profile stuff and it needs to scale. They had a valentine day site that got a lot of attention on that day and it automatically scaled up without a change in the app. The day after it scaled down automatically, too. Services and APIs. No sysadmin needed. Some caveats with app engine: it is a sandbox. You you cannot do "pip install". The filesystem isn't there in the traditional sense; there is a blob storage instead. And it is lock-in, mostly; portability is an issue. They could work arounds these issues and ended up with a better application as a result. There are three ways (that they use) of running Django on app engine: Django …
