Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Middleware post-processing in Django: a gotcha
One of the requirements for the new Heart website we've just launched was to allow users to personalise their location to one of 33 radio stations across the country. For various reasons, this meant rewriting all the links on the page, dynamically, depending on the user's location setting. The easiest place to do this sort of post-processing in Django is in response middleware. So I wrote a quick class that used regexes to grab all the href and action attributes (for a and form elements respectively - images didn't need localising) and add the relevant locations. Because it was dynamic, I used the ability of re.sub to call a function to determine the replacement value; and to save on multiple database queries, I saved various things in the instance. So it looked a bit like this: href = re.compile(r'(href|action)=["\'](.+?)["\']') class LocalisationMiddleware(object): def process_response(self, request, response): self.current_station = get_station(request) self.stations = Station.objects.values_list('slug', flat=True) content = href.sub(self.re_replace, response.content.decode('utf8')) response.content = unicode(content) return response def re_replace(self, matchobj): current_station = self.current_station url = "/%s%s" % (current_station.slug, matchobj.group(2)) return "%s=%s" % (matchobj.group(1), url) But then, during testing, we started getting some rather odd bug reports. Someone would be happily browsing the London pages, and would … -
Ressenya de creant Bits, el déjà vu
El segon creant bits dedicat a la introducció de Python i Django d'ahir va a tornar aplegar un bon nombre de gent interessada per la programació i per Python i Django. Cares conegudes i gent que vaig poder desvirtualitzar. Em va fer especial il·lusió poder desvirtualitzar Guillem, ja que per un motiu o altre mai ens havíem pogut trobar personalment. Aquesta vegegada preferírem no allargar molt la jornada i no es donà la xerrada damunt la posada en producció d'aplicacions Django. La passada jornada En Bernat va tenir molt poc temps i acabarem molt tard, així que ho hem deixat per a una millor ocasió. Aquest pic el consum de gominolas per part dels assistents va ser menor, lluny del rècord de kilo i busques de l'altra vegada. :-P La idea és que si algú s'avorreix al manco se n'endurà un sabor dolç de boca. Entre l'anecdotari comentar el mal cos que se'ns quedà a tots quan un tassó d'aigua va vessar damunt un portàtil Macbook Pro nou de trinca. Després de netejar-lo va tornar a la vida i esper que segueixi així. Hi va haver un segon intent de tragèdia, quan el que es va vessar va ser cafè … -
Django Meetup slides and little update
Due to a busy week, we can only provide a short update this time. Andy Smith from Google (and a Jaiku developer) has done a little presentation on Django on App Engine at the San Francisco Django Meetup on January 27th. Unfortunately, there is no video, but you can at least take a look at his slides. The talk was mostly about App Engine and the last third was about Django on App Engine. He also talked about the Django non-relational project, but nothing in depth and there were no demos. There were also a few other Google developers at the meetup, so hopefully we could get some more attention from Google and the Django community. We'd love to have some more impressions from people who attended the meetup. Were there any interesting questions? Another little update is that recently I added "startswith" query support to the App Engine backend. It's based on the unicode trick described in the App Engine documentation. This means you'll need a composite index for this kind of query if you combine it with other filters and you can't order by a field other than the "startswith" filter's field. Still, this should help with getting … -
Returning an actual proper real life HTTP code from a Django error page
Go to a non existent page on a Django site and you will (hopefully) be met with a friendly error page telling you not to panic, everything is OK and all you’ve done is mistyped the URL or something. If it’s your thing, you may be interested enough to look see what the actual HTTP code for the page is in the header; chances are that it will be a 200 rather than a 404 as the default handler just passes the dealings onto the HttpResponse class. Generally speaking this is fine, but there are situations where an accurate code would be very handy, as I found out the other day when I was trying to detect whether a file had been uploaded to a remote server. Scraping the resultant HTML for “Page not found” is not my idea of a robust solution. So, instead, pass the error page’s HTML into the respective class by putting something like this in urls.py: handler404 = 'urls.return_404' handler500 = 'urls.return_500' def return_404(request): return HttpResponseNotFound( render_to_string("errors/404.html")) def return_500(request): return HttpResponseServerError( render_to_string("errors/500.html")) Fullest of props to PiotrLegnica at Stack Overflow for this most elegant of solutions. Edit: After further examination (see the comments) the default … -
World Government Data
World Government Data. Launched last week, this is the Guardian’s meta-search engine for searching and browsing through data from four different government data sites (with more sites planned). Under the hood it’s Django, Solr, Haystack and the Scrapy crawling library. The application was built by Ben Firshman during an internship over Christmas. -
Little Tweaks
We're super-pleased with the new deployment setup. Many things just got easier for us and there's been very little fallout. As a result of the short amount of time since that rolled out, most of what we rolled out tonight we slight interface refinements. Latest news styling now resembles other portions of the site. Deleted recipes that have been forked should no longer display in any lists. Links in inbox messages should now be clickable. Error pages now match the rest of the site. We've also got two new features nearly ready and are in the process of ripping out some old tech & migrating to something that will make browsing the site better. -
Developing Reusable Django Apps: App Settings
Eventually your app will need some sort of configuration. Supplying many parameters to customise your views, template tags and filters to allow template authors easily harness the power of your app and implementing registration pattern are all sensible things to do. But at some point you will need configuration for your app at project level. [...] -
More Work Under The Hood
We've made some big changes under the hood tonight, especially to how we deploy new code to the server (so very much faster and consistent!). This shouldn't impact anyone, but if you notice some portion of the site isn't working for you, please give us a heads up. This is a pretty big change and there's a chance we overlooked something in the process. Additionally, we've got a couple bugfixes that have made their way out. The initial snapshot of a recipe will now include all ingredients/instructions. If something isn't filled out in a snapshot (i.e. no source information), we no longer clutter the screen with an empty field. There's a new button on the "Add recipe" screen, which allows you to add the recipe and return to the add screen to enter another. Very useful when entering a lot of recipes. We'll have more coming on the typical "New Stuff Tuesday" but we wanted to get this out the door during a slow time. -
Pinax cheat sheets
I have a few friends that I’m bringing up to speed pretty quickly on Django. The basics are going well, and I’m providing the foundations for the effort. To take advantage of all the good stuff out there, I’ve started … Continue reading → -
Using fabric to deploy a Pinax project
I spent the better part of this morning screwing around with Fabric to get even a basic deployment working correctly with Pinax. I may be doing this horrifically backwards… Since I finally nailed something that worked for me and I … Continue reading → -
Django History Tables
I've uploaded a new version of django-history-tables [1] to the mercurial repository. It now should run on Django 1.0 + 1.1 and finally supports (although basically) the Admin interface. For those that don't know django-history-tables (guessing that's 99.9% of everybody reading this), it is a pluggable Django application to create addition tables inside the database recording the history of changes and deletion of Django models. This in itself is not unique there are many more project out there doing basically the same but most implement this by serializing data (such as pickling, json, xml, etc). Big drawback of this approach is that when schema changes your serialized history does not change with it. Somewhere down the line when you have forgotten all the schema changes it will be most difficult to use this history data. django-history-tables works by using one additional 'normal' database table to store history information; major benefit of this approach is that there's no pickling, no serializing, and it should be able to support anything that your database supports as well. Schema changes in this approach needs to be applied to both tables. When you change your model, you will need to change the schema of the … -
Django Smuggler 0.1.1 released!
Yesterday I published the first stable version of Django Smuggler. An pluggable application for easily export/import fixtures via the administration interface. Especially useful for transporting data in production for the development project and vice versa, but can also be used as a backup tool. Smuggler is in the Python Package Index (PyPI) and you can easily install it using the tools pip or easy_install. Try: pip install django-smuggler or: easy_install django-smuggler Feedback (with or without code) is always welcome! -
Django Smuggler 0.1.1 lançado!
Ontem publiquei a primeira versão estável do Django Smuggler. Uma aplicação plugável para, de maneira simples, exportar/importar fixtures via a interface de administração. Útil principalmente para transportar dados do projeto em produção para o de desenvolvimento e vice-versa, mas também pode ser utilizado como ferramenta de backup. O Smuggler está disponível no Python Package Index (PyPI) e você pode instalá-lo facilmente via pip or easy_install. Experimente: pip install django-smuggler ou: easy_install django-smuggler Feedback (com ou sem código) é sempre muito bem-vindo! -
django-couchdb не взлетел
django-couchdb это ДБ бекэнд для Джанги с поддержкой CouchDB, разработанный командой 42cc. Начиная новый проект, я очень рассчитывал на это решение, но увы оно совсем не оправдало надежд. Основная цель была, используя CouchDB не отказываться от стандартной джанговской админки и попробовать её заставить работать при помощи это бекэнда. Любовь к админке обусловлена тем, что она уже написана и её не надо изобретать с квадратными колесами нам самим, то что наши контент-менеджеры уже отчасти привыкли к ней и не хочется сбивать им workflow. Авторы django-couchdb не скрывают что эта их реализация больше академическая и какого-то реального юзкейса у них не было. Что и проявилось в итоге. Сразу надо сказать, что на данный момент основная гитхабовская ветка не работает с текущей Джангой и свежим CouchDB. Это связано с тем, что в Джанге в последнее время серьезно переработали внутренний API бекэндов баз дынных, а новый CouchDB чуть более строг с структуре документов чем раньше. Но ладно, завести этот бекэнд вполне можно, что силами Кости Меренкова мы и сделали. Как и предполагалось, заработал базовый ORM, админка, а следовательно и приложения из contrib. Пожив какое-то время с таким решением, мы поняли, что во многих местах для нашей задачи оно избыточно, а в других просто неудобно … -
Pingdjack
Понадобился мне недавно pingback-сервер, и я, посмотрев на django-pingback, в приступе NIH-синдрома написал своё приложение -- pingdjack. За название спасибо Михаилу Лукьянченко. Отмазка Более-менее реальные претензии к django-pingback у меня тоже есть. Хотя я сразу признаюсь, что не пробовал во что бы то ни стало с ней ужиться, просто код почитал. Претензии такие: Зависит от django-xmlrpc, хотя на мой взгляд XML-RPC в Питоне и так достаточно простая вещь, чтобы не городить для неё Джанго-специфичного слоя. Использование обобщённого XML-RPC-слоя тянет за собой необходимость делать в нём регистрацию pingback-хендлеров или регистрировать middleware, которая будет делать это за вас. В качестве цели пинга регистрируются объекты моделей, а не любой произвольный URL сервера. У приложения есть свои модели, и как я понимаю, пингбеки оно хранит как само хочет, а не так, как мне надо. Библиотека не ищет цитату из поста и автора так, как мне давно хотелось. Pingdjack Получилась библиотечка из двух модулей: клиента и сервера. Единственная внешняя зависимость -- html5lib, без неё нынче никуда :-). Клиента я просто выдрал из Cicero. Он состоит из трёх функций: external_urls, которая ищет внешние ссылки в куске HTML'а ping, которая пингует один URL от имени другого ping_external_urls, которая соединяет одно с другим, пингуя все внешние ссылки из … -
Testing your first Django app
This is my unofficial part five for the Django tutorial. I've attempted to write this in a similar style to the Django documentation and hopefully this will be useful for those looking for the next step after the tutorial or trying out testing with django for the first time. Testing is something I think we can all do better, I certainly know I could do better testing my code sometimes. If you don't have the polls app from the end of part four and don't want to do it again you can grab it from my github. The code added in this tutorial is also available in the master branch of the repository. This tutorial begins where Tutorial 4 left off. We're continuing the Web-poll application and will focus on testing our application and proving that it works as expected. The testing environment Let's first look at how you run the tests, make sure you are in the mysite directory and run the command python manage.py test. You will see output similar to this: Creating test database... Creating table auth_permission Creating table auth_group Creating table auth_user Creating table auth_message Creating table django_admin_log Creating table django_content_type Creating table django_session Creating table … -
Testing your first Django app
This is my unofficial part five for the Django tutorial. I've attempted to write this in a similar style to the Django documentation and hopefully this will be useful for those looking for the next step after the tutorial or trying out testing with django for the first time. Testing is something I think we can all do better, I certainly know I could do better testing my code sometimes. If you don't have the polls app from the end of part four and don't want to do it again you can grab it from my github. The code added in this tutorial is also available in the master branch of the repository. This tutorial begins where Tutorial 4 left off. We're continuing the Web-poll application and will focus on testing our application and proving that it works as expected. The testing environment Let's first look at how you run the tests, make sure you are in the mysite directory and run the command python manage.py test. You will see output similar to this: Creating test database... Creating table auth_permission Creating table auth_group Creating table auth_user Creating table auth_message Creating table django_admin_log Creating table django_content_type Creating table django_session Creating table … -
Django Doc Wiki
I have been meaning to develop a simple Django-based wiki solution for a while. I prefer writing my notes and documentation in TextMate using Markdown, and keeping them in a public Git repo. However, I also need to share these files with other people every once in a while, and that’s why I wanted to be able to easily display this content in a wiki-like web site, without having to duplicate data. That’s how Django Doc Wiki was born today. I am aware that there are many other Python based wiki solutions out there. MoinMoinWiki seems like the most viable alternative, but it’s still too complex for my needs. Django-wikiapp looks like a useful app as well, but it doesn’t have any hooks for using it with a version control system outside the Django database. Django Doc Wiki is an extremely simple solution, but it works well for me, so I am going to open source it in the hope that other people might have a use for it as well. It comes with two main features: Detect new files in a directory (through a cron job, script included) and updates to existing ones, remove deleted files Support for Markdown … -
Sharding with Django on App Engine
When developing a scalable application for App Engine you need to pay attention to how often a database entity is updated because you can only update any single entity with a maximal frequency of 1-5 times per second. If the frequency of updates for any single entity is higher than this limit you can expect your application to have contention. In order to prevent such situations you can use a technique called sharding. Sharding takes advantage of the datastore's ability to handle many parallel updates on multiple distinct entities efficiently and to handle reads much faster than writes. Let's get to the example of a simple counter for which the frequency of updates is too high for a single entity (such a counter could be used for counting the number of views for a YouTube video): class SimpleCounterShard(models.Model): """Shards for the counter""" count = models.IntegerField(default=0) name = models.CharField(primary_key=True, max_length=500) NUM_SHARDS = 20 @classmethod def get_count(cls): """ Retrieve the value for a given sharded counter. """ total = 0 for counter in SimpleCounterShard.objects.all(): total += counter.count return total @classmethod @commit_locked def increment(cls): """ Increment the value for a given sharded counter. """ index = random.randint(0, SimpleCounterShard.NUM_SHARDS - 1) shard_name = 'shard' … -
Django Training in 2010
The new year's shaping up to be a great one for Django: Django 1.2 is on track to ship this March, and there's no doubt in my mind that it'll be the best release ever. Because of this, we expect to see a lot of new people wanting to learn Django next year, so we're stepping up our training offerings in the new year. This Friday, I'll be holding a one-day Django workshop in New York City. The class is mostly full, but a few spots are still available. This is the second in a series of workshops we've been holding in conjunction with HoldenWeb, and we plan to offer more (see below) In March, I'll be teaching a week-long advanced Django class in Kansas City. This class is perfect for folks who know Django but want to really learn all the ins and outs. We'll look at a whole bunch of advanced uses of Django, learn how to use optional add-ons like GeoDjango, Piston, Haystack, and even dig into the internals of Django a bit. We'll also spend a whole day setting up a real-world Django production environment and tuning it for performance. I'm really looking forward to this … -
OSError while installing Django with buildout and djangorecipe
A corrupted Django tarball can cause mysterious errors from djangorecipe. -
OSError while installing Django with buildout and djangorecipe
Sometimes, I see the following error when trying to run a Django buildout: File "/Users/dan/.eggs/djangorecipe-0.20-py2.6.egg/djangorecipe/recipe.py", line 271, in install_release os.listdir(extraction_dir)[0] OSError: [Errno 2] No such file or directory: '/Users/dan/.downloads/django-archive' After a bit of poking around, I found that this is to do with a corrupted Django tarball. In my case, this is usually because I've interrupted a download with Ctrl-C. Unfortunately it seems that the tarfile module in the Python standard library (at least as invoked by setuptools) treats broken a tar.gz files as an empty archive, without throwing an exception. Since there's no exception, djangorecipe assumes everything was uncompressed without problems, and is therefore rather surprised when the unpacked Django package isn't where it expected it to be. The short term solution is to delete the bad Django archive from your download cache. This will likely be a 'downloads' directory in your buildout, or you may have a global one (as I do). When you next run buildout, the tarball will be freshly downloaded. When I get a moment I'll see if I can modify djangorecipe to notice this condition and not proceed with the build. -
OSError while installing Django with buildout and djangorecipe
Sometimes, I see the following error when trying to run a Django buildout: File "/Users/dan/.eggs/djangorecipe-0.20-py2.6.egg/djangorecipe/recipe.py", line 271, in install_release os.listdir(extraction_dir)[0] OSError: [Errno 2] No such file or directory: '/Users/dan/.downloads/django-archive' After a bit of poking around, I found that this is to do with a corrupted Django tarball. In my case, this is usually because I've interrupted a download with Ctrl-C. Unfortunately it seems that the tarfile module in the Python standard library (at least as invoked by setuptools) treats broken a tar.gz files as an empty archive, without throwing an exception. Since there's no exception, djangorecipe assumes everything was uncompressed without problems, and is therefore rather surprised when the unpacked Django package isn't where it expected it to be. The short term solution is to delete the bad Django archive from your download cache. This will likely be a 'downloads' directory in your buildout, or you may have a global one (as I do). When you next run buildout, the tarball will be freshly downloaded. When I get a moment I'll see if I can modify djangorecipe to notice this condition and not proceed with the build. -
OSError while installing Django with buildout and djangorecipe
Sometimes, I see the following error when trying to run a Django buildout: File "/Users/dan/.eggs/djangorecipe-0.20-py2.6.egg/djangorecipe/recipe.py", line 271, in install_release os.listdir(extraction_dir)[0] OSError: [Errno 2] No such file or directory: '/Users/dan/.downloads/django-archive' After a bit of poking around, I found that this is to do with a corrupted Django tarball. In my case, this is usually because I've interrupted a download with Ctrl-C. Unfortunately it seems that the tarfile module in the Python standard library (at least as invoked by setuptools) treats broken a tar.gz files as an empty archive, without throwing an exception. Since there's no exception, djangorecipe assumes everything was uncompressed without problems, and is therefore rather surprised when the unpacked Django package isn't where it expected it to be. The short term solution is to delete the bad Django archive from your download cache. This will likely be a 'downloads' directory in your buildout, or you may have a global one (as I do). When you next run buildout, the tarball will be freshly downloaded. When I get a moment I'll see if I can modify djangorecipe to notice this condition and not proceed with the build. -
Learn something new every year
As developers, we all make that pledge, don't we? Some of us learn new languages, new frameworks, new APIs, new methodologies. Rarely do we learn a new operating system.30 years ago I was using whatever they called the operating system on the Apple ][ series of computers. 20 years ago I learned to use MS-DOS. 17 years ago I added Windows 3.1 to that mix. 15 years ago I ran into Windows 95 and the subsequent set of 98, 2000, Me, and XP. 10 years ago I started to play with Linux and Unix, mostly learning the simple shell commands needed to shuffle files around. 3 years ago I commenced working on Mac OS X. During my Windows 98 days I was forced in doing help desk and system administration work. I hated it. The pop-up menus were not intuitive. Documenting what you did was hence agony. I begged my job to make me a pure developer but they forced me to continue down the path of Windows administration. Eventually I left that job and became a pure developer.Over the years I got better at Unix and Linux. I even set up two servers for production use, one running Apache …