Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Event sourcing in Django
Django comes with "batteries included" to make CRUD (create, read, update, delete) operations easy. It's nice that the CR part (create and read) of CRUD is so easy, but have you ever paused to think about the UD part (update and delete)? Let's look at delete. All you need to do is this: ReallyImportantModel.objects.get(id=32).delete() # gone from the database forever Just one line, and your data is gone forever. It can be done accidentally. Or you can be do it deliberately, only to later realise that your old data is valuable too. Now what about updating? Updating is deleting in disguise. When you update, you're deleting the old data and replacing it with something new. It's still deletion. important = ReallyImportantModel.object.get(id=32) important.update(data={'new_data': 'This is new data'}) # OLD DATA GONE FOREVER Okay, but why do we care? Let's say we want to know the state of ReallyImportantModel 6 months ago. Oh that's right, you've deleted it, so you can't get it back. Well, that's not exactly true -- you can recreate your data from backups (if you don't backup your database, stop reading right now and fix that immediately). But that's clumsy. So by only storing the current state of … -
Django - Database access optimization
Django Queryset is generally lazy in nature. It will not hit the database until it evaluates the query results. Example: queryset = User.objects.all() # It won't hit the database print (queryset) # Now, ORM turns the query into raw sql and fetches results from the database 1.Caching and QuerySets Generally Django stores the query results in the catche when it fetches the results for the first time. Example: Get all users first names list and emails list from the database. first_names_list = [ user.first_name for user in User.objects.all()] emails_list = [ user.email for user in User.objects.all()] above code hits the database twice. To avoid the extra request to the database we can use the Django's database cache in the following way. users_list = User.objects.all() # No database activity first_names_list = [user.first_name for user in users_list] # Hits the database and stores the results in the cache emails_list = [user.email for user in users_list] # uses the results from the cache. other way is users_list = User.objects.all() db = dict(users_list.values_list("first_name", "email")) first_names_list, emails_list = db.keys(), db.values() Note: Querysets are not cached if query is not evaluated. Eaxmple: If you want to take a subset/part of the query results queryset = Users.objects.all() first_five = queryset[:5] # Hits … -
Webfaction LetsEncrypt Django
Why HTTPS First things first. I do not work for Google nor I have any (social, financial, ethical etc) benefits from this gigantic company. But let be honest. If you are not ranked hign enough in Google's search results, your optimism about your website success is slowly betake to collapse. I think everybody that owns a website, wants his "e-property" to be shown amongst the first results in Google. Of course, you might say, that the search keywords added in the search bar are very important too, but that is not to be discussed here. So, you have a website (maybe you have build it too) and your domain is i.e http://www.ilovewhatido.com/. Then you read this blog post by Google and got terrified about your ranking position. Thinking "Oh man, I have to change the http protocol to https. How painful will this be?" or "how much do I have to pay (monthly or yearly) in order to do that?" or "if I switch to https, will my web app behave the same as it was with http?". Lets give answers to these questions: "Oh man, I have to change the http protocol to https. How painful will this be?" … -
Django custom sitemap (updated 2018)
UPDATE Sep 2018: The old post regarding the template tag had some bugs. As of 2018, I fixed it and now this post is updated and works only for Python 3.6+. So you have done the following: Used your favorite Web Framework (Django) to build your website. Made enough tests to verify that everything is working flawlessly. Translated your whole website in each language and (of course) each page has its translated version. Used i18n_patterns function to prefix your urls with the language code. Hosted your website somewhere and... asked yourself why Google does not index your translated pages of your website. Introduction This excellent article from Google states that there are 2 kinds of "translated" websites: multilingual and multi-regional. You can have none, one of them or both, depending on your needs. Let's assume that you have build a multiregional website: A multilingual website is any website that offers content in more than one language. Examples of multilingual websites might include a Canadian business with an English and a French version of its site, or a blog on Latin American soccer available in both Spanish and Portuguese. It also states that: Keep the content for each language on separate … -
Django Administration: Inlines for Inlines
The default Django model administration comes with a concept of inlines. If you have a one-to-many relationship, you can edit the parent and its children in the same form. However, you are limited in a way that you cannot have inlines under inlines at nested one-to-many relations. For example, you can't show models Painter, Picture, and Review in the same form if one painter may have drawn multiple pictures and each picture may have several reviews. In this article I would like to share a workaround allowing you to quickly access the inlines of an inline model. The idea is that for every inline you can provide a HTML link leading to the separate form where you can edit the related model and its own relations. It's as simple as that. For example, in the form of Painter model, you have the instances of Picture listed with specific links "Edit this Picture separately": When such a link is clicked, the administrator goes to the form of the Picture model which shows the instances of Review model listed underneath: Let's have a look, how to implement this. First of all, I will create a gallery app and define the three models … -
Django is Boring, or Why Tech Startups (Should) Use Django
I recently attended Django Under The Hood in Amsterdam, an annual gathering of Django core team members and developers from around the world. A common theme discussed at the conference this year is that “Django is boring.” While it’s not the first time this has been discussed, it still struck me as odd. Upon further reflection, however, I see Django’s “boringness” as a huge asset to the community and potential adopters of the framework. Caktus first began using Django in late 2007. This was well before the release of Django 1.0, in the days when startups and established companies alike ran production web applications using Subversion “trunk” (akin to the Git “master” branch) rather than using a released version of the software. Using Django was definitely not boring, because it required reading each commit merged to see if it added a new feature you could use and to make sure it wasn’t going to break your project . Although Django kept us on our toes in the early days, it was clear that Django was growing into a robust and stable framework with hope for the future. With the help of thousands of volunteers from around the world, Django’s progressed … -
Using Fanout.io in Django
Earlier this year we started using Fanout.io in Air Mozilla to enhance the experience for users awaiting content updates. Here I hope to flesh out its details a bit to inspire others to deploy a similar solution. What It Is First of all, Fanout.io is basically a service that handles your WebSockets. You put in some of Fanout's JavaScript into your site that handles a persistent WebSocket connection between your site and Fanout.io. And to push messages to your user you basically send them to Fanout.io from the server and they "forward" it to the WebSocket. The HTML page looks like this: <html> <body> <h1>Web Page</h1> <!-- replace the FANOUT_REALM_ID with the ID you get in the Fanout.io admin page --> <script src="https://{{ FANOUT_REALM_ID }}.fanoutcdn.com/bayeux/static/faye-browser-1.1.2-fanout1-min.js" ></script> <script src="fanout.js"></script> </body> </html> And the fanout.js script looks like this: window.onload = function() { // replace the FANOUT_REALM_ID with the ID you get in the Fanout.io admin page var client = new Faye.Client('https://{{ FANOUT_REALM_ID }}.fanoutcdn.com/bayeux') client.subscribe('/mycomments', function(data) { console.log('Incoming updated data from the server:', data); }) }; And in server it looks something like this: from django.conf import settings import fanout fanout.realm = settings.FANOUT_REALM_ID fanout.key = settings.FANOUT_REALM_KEY def post_comment(request): """A django view function … -
Transcoding with AWS- part two
As I have static and media files integrated with AWS now it's time to transcode them. In this post, I will write a short example of how to integrate AWS ElasticTranscoder with Django application. Table of Contents: Basic terms Code Other blog posts in this series Basic terms ElasticTranscoder allows you to transcode files from your S3 bucket to various formats. To set this service up first you have to create a pipeline. What pipeline is? Basically, it's a workflow- how your transcoder should work. You can create a different pipeline for long content and different for short one. In my application I created the following pipeline: As I have my pipeline configured next step is to create jobs. Jobs are tasks for a transcoder that say which file I want to transcode, to what format or codec I want to do this: PresetID is user created or already existing configuration that defines the format of transcoder output: is it mp4 or maybe flac? What resolution should video files have? All of this is set up in present. As we know basic terms used in AWS Elastic Transcoder let's jump into the code. Code AWS has very good python API … -
Transcoding with AWS- part two
As I have static and media files integrated with AWS now it's time to transcode them. In this post, I will write a short example of how to integrate AWS ElasticTranscoder with Django application. Table of Contents: Basic terms Code Other blog posts in this series Basic terms ElasticTranscoder allows … -
Django, custom SMTP server for logging
One of my client uses google apps, so I use that account to send out mails with Django, which works just fine. However, google silently drops all error mails to the admins, for whatever reason you can't debug and will never hear anything about anyway. As those mails only go to the dev team it doesn't really matter which mailserver is used, so I decided to write my own email handler. To use it add it to the LOGGING settings like you would the normal AdminEmailHandler and add the necessary settings to your configuration. Raw from django.conf import settings from django.core.mail import get_connection from django.utils.log import AdminEmailHandler class MyAdminEmailHandler(AdminEmailHandler): """ Use a custom handler because gmail sucks... """ def connection(self): return get_connection( host=settings.LOG_EMAIL_HOST, port=settings.LOG_EMAIL_PORT, username=settings.LOG_EMAIL_USER, password=settings.LOG_EMAIL_PASSWORD, use_tls=settings.LOG_EMAIL_USE_TLS, ) -
Django, custom SMTP server for logging
One of my client uses google apps, so I use that account to send out mails with Django, which works just fine. However, google silently drops all error mails to the admins, for whatever reason you can't debug and will never hear anything about anyway. As those mails only go to the dev team it doesn't really matter which mailserver is used, so I decided to write my own email handler. To use it add it to the LOGGING settings like you would the normal AdminEmailHandler and add the necessary settings to your configuration. Raw from django.conf import settings from django.core.mail import get_connection from django.utils.log import AdminEmailHandler class MyAdminEmailHandler(AdminEmailHandler): """ Use a custom handler because gmail sucks... """ def connection(self): return get_connection( host=settings.LOG_EMAIL_HOST, port=settings.LOG_EMAIL_PORT, username=settings.LOG_EMAIL_USER, password=settings.LOG_EMAIL_PASSWORD, use_tls=settings.LOG_EMAIL_USE_TLS, ) -
A Quick Look at the Kubernetes Python Client
For those of you that don't know there is a new Python API client in the kubernetes-incubator project: client-python. There has been some high quality Python clients like pykube, but client-python can serve as the official Python client. ## The Structure of the Client client-python is a client that is mostly generated based on the swagger spec (UI). While pykube has the benefit of being totally idiomatic, client-python can support practically all of the endpoints and react quickly to changes in the API. client-python supports Python 3 and is currently tested against Python 3.4. ## Usin[...] -
How to Create Group By Queries With Django ORM
This tutorial is about how to implement SQL-like group by queries using the Django ORM. It’s a fairly common operation, specially for those who are familiar with SQL. The Django ORM is actually an abstraction layer, that let us play with the database as it was object-oriented but in the end it’s just a relational database and all the operations are translated into SQL statements. Most of the work can be done retrieving the raw data from the database, and playing with it in the Python side, grouping the data in dictionaries, iterating through it, making sums, averages and what not. But the database is a very powerful tool and do much more than simply storing the data, and often you can do the work much faster directly in the database. Generally speaking, when you start doing group by queries, you are no longer interested in each model instances (or in a table row) details, but you want extract new information from your dataset, based on some common aspects shared between the model instances. Let’s have a look in an example: class Country(models.Model): name = models.CharField(max_length=30) class City(models.Model): name = models.CharField(max_length=30) country = models.ForeignKey(Country) population = models.PositiveIntegerField() And the raw … -
Transcoding with AWS- part one
Nowadays moving everything to the cloud becomes more and more popular. A lot of software companies move their technology stack to such infrastructure. One of the biggest players in this field is Amazon Web Services - AWS. That's why I decided decided to adapt existing code from my previous project and move transcoding to write blog posts about that. In this series I process to cloud. Table of Contents: Overview of series Moving static and media files to AWS Overview of series I decided to adapt code from my previous blog series about celery and rabbit-mq. I did that because code from this django application actually transcodes mp3 files to other formats. This series will be divided into these parts: Moving static and media files to AWS Transcoding files inside AWS transcoder Notifying user that transcode is complete User downloads transcoded file Moving static and media files to AWS AWS transcoder operates only on files that are inside S3 bucket so first I need to change how these files are served in django. Let's say that I already had my account on AWS. Next step is to generate specific account using IAM. While creating a user I want to give … -
Transcoding with AWS- part one
Nowadays moving everything to the cloud becomes more and more popular. A lot of software companies move their technology stack to such infrastructure. One of the biggest players in this field is Amazon Web Services - AWS. That's why I decided decided to adapt existing code from my previous project and … -
Birthday retrospective
So, recently Evennia celebrated its ten-year anniversary. That is, it was on Nov 20, 2006, Greg Taylor made the first repo commit to what would eventually become the Evennia of today. Greg has said that Evennia started out as a "weird experiment" of building a MUD/MUX using Django. The strange name he got from a cheesy NPC in the Guild Wars MMORPG and Greg's first post to the mailing list also echoes the experimental intention of the codebase. The merger with Twisted came pretty early too, replacing the early asyncore hack he used and immediately seeing a massive speedup. Evennia got attention from the MUD community - clearly a Python-based MUD system sounded attractive.When I first joined the project I had been looking at doing something MUD-like in Python for a good while. I had looked over the various existing Python code bases at the time and found them all to be either abandoned or very limited. I had a few week's stunt working with pymoo before asking myself why I was going through the trouble of parsing a custom script language ... in Python ... Why not use Python throughout? This is when I came upon Evennia. I started … -
Optimizing the construction of Django QuerySets
Django’s ORM is normally fast enough as-is, but if you’ve ever profiled a high traffic view with a fairly complicated query, you might have found that constructing QuerySet can take a noticeable portion of your request time. For example, I once found a query on the front page of the site I was working on that took 1ms to construct and 1ms for the database to answer. With a performance budget of 100ms, that was 1% gone on computing the exactly same SQL. Thankfully we don’t need to instantly drop down to raw SQL to optimize such cases, as Django’s QuerySet API naturally lends itself to caching the intermediate objects. Since each operation on a QuerySet returns a new object with the change applied, they’re always lazy as to executing the SQL, and operations can (normally) be chained in any order, you can build the non-specific part of your QuerySet up as a cached object and then apply final, specific filtering required at request time. Just a note before we dive in: this should be one of the least reached for tools in your optimization toolbox - normally it’s enough to fix the basics such as avoiding N+1 queries with … -
How to reset migrations in Django 1.7 - 1.8 - 1.9 and above
Migrations help you propagate models changes to your database schema,they are particularly helpful in the situation when you need to change your database structure and you don't want or you can't drop a database table and recreate it or when you have a production database with tables which has millions of rows .Any developer has experienced situations where he has to change the structure of an existing table such as adding,deleting or renaming a field.In some cases droping the table and recreate it solve the problem and release the developer from the headache related to and resulted by the process but just imagine a scenario where your application is already in production with millions of database rows ,droping your old tables is not a choice.You can't even dear to think about it so migrations are here to present you a more acceptable and professional solution. Simply migrations lets you change your database schema while keeping your data. How to get started with migrations ? Getting started with migrations is easy especially with the latest versions of Django,starting with Django 1.7 .In fact from Django 1.7 migrations become obligatory since they are integrated within your django workflow. To work with migrations … -
Django Under the Hood 2016 Recap
Caktus was a proud sponsor of Django Under the Hood (DUTH) 2016 in Amsterdam this year. Organized by Django core developers and community members, DUTH is a highly technical conference that delves deep into Django. Django core developers and Caktus Technical Manager Karen Tracey and CEO/Co-founder Tobias McNulty both flew to Amsterdam to spend time with fellow Djangonauts. Since not all of us could go, we wanted to ask them what Django Under the Hood was like. Can you tell us more about Django Under the Hood? Tobias: This was my first Django Under the Hood. The venue was packed. It’s an in-depth, curated talk series by invite-only speakers. It was impeccably organized. Everything is thought through. They even have little spots where you can pick up toothbrush and toothpaste. Karen: I’ve been to all three. They sell out very quickly. Core developers are all invited, get tickets, and some funding depending on sponsorship. This is the only event where some costs are covered for core developers. DjangoCon EU and US have core devs going, but they attend it however they manage to get funds for it. What was your favorite part of Django Under the Hood? Tobias: The talks: … -
Introduction to Django Models
In this part of the tutorial, I will show you some basic model definition by creating a simple blog. As discussed earlier, database tables are translated from the models.py files. If you haven't followed along with earlier tutorials, you can do the setup here. Choose branch exercise4. As usual we start by creating a new branch: git checkout -b blog Activate virtualenv, if you haven't already: source ../virtualenv/bin/activate A is an application by itself, so we create one inside Django with the following command. python3 manage.py startapp blog You should see these directories and files in your source dir: blog/ main/ manage.py MyTutorial/ requirements.txt In order for django to discover the new application, you need to include it in the installed apps in settings.py ***MyTutorial/settings.py*** ... INSTALLED_APPS = [ 'main', 'blog', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', ] … As usual we write some tests first. If you have a look at the other test file main/tests.py you can see, that we have some very similar goal here. It might be tempting to import those tests and tweak it a little, but that wouldn't be right. As for Unit Tests, it needs to be completely independent from each other. So … -
How to break Python
Don’t worry, this isn’t another piece about Python 3. I’m fully in favor of Python 3, and on record as to why. And if you’re still not convinced, I suggest this thoroughly comprehensive article on the topic, which goes over not just the bits people get angry about but also the frankly massive amount of cool stuff that only works in Python 3, and that you’re missing out on if you still only use Python 2 ... Read full entry -
How to break Python
Don’t worry, this isn’t another piece about Python 3. I’m fully in favor of Python 3, and on record as to why. And if you’re still not convinced, I suggest this thoroughly comprehensive article on the topic, which goes over not just the bits people get angry about but also the frankly massive amount of cool stuff that only works in Python 3, and that you’re missing out on if you still only use Python 2 ... Read full entry -
How to Filter QuerySets Dynamically
Filtering QuerySets dynamically is a fairly common use case. Sure thing there is a pluggable app to make your life easier. This tutorial is about how to use the django-filter app to add a hassle-free filtering to your views. To illustrate this tutorial I will implement a view to search for users. As usual the code used in this tutorial is available on GitHub. You can find the link in the end of this post. Installation Easiest way is to install it with pip: pip install django-filter That’s it. It’s ready to be used. Make sure you update your requirements.txt. The default language of the app is English. It already come with some localization and language support. Currently the supported languages are: de, es_AR, fr, pl, ru, zh_CN. Unless you want to use any of those languages in your project, you don’t need to add django_filter to the INSTALLED_APPS. Usage Create a file named filters.py inside your app folder: filters.py from django.contrib.auth.models import User import django_filters class UserFilter(django_filters.FilterSet): class Meta: model = User fields = ['username', 'first_name', 'last_name', ] The view is as simple as: views.py from django.contrib.auth.models import User from django.shortcuts import render from .filters import UserFilter def search(request): … -
Getting started with pytest
Pytest is my preferred Python testing library. It makes simple tests incredibly easy to write, and is full of advanced features (and tons of plugins) that help with more advanced testing scenarios. To demonstrate the basics, I’m going to walk through how I’d solve the first couple cryptopals challenges in a test-driven style, using py.test. Spoiler alert: I’m going to spoil the first challenge, and maybe a bit of the second, below. -
Django Channels: Using Custom Channels
In my earlier blog post - Introduction to Django Channels, I mentioned that we can create our own channels for various purposes. In this blog post, we would discuss where custom channels can be useful, what could be the challenges and of course we would see some code examples. But before we begin, please make sure you are familiar with the concepts of Django Channels. I would recommend going through the above mentioned post and the official docs to familiarize yourself with the basics. Our Use Case Channels is just a queue which has consumers (workers) listenning to it. With that concept in mind, we might be able to think of many innovative use cases a queue could have. But in our example, we will keep the idea simple. We are going to use Channels as a means of background task processing. We will create our own channels for different tasks. There will be consumers waiting for messages on these channels. When we want to do something in the background, we would pass it on the appropriate channels & the workers will take care of the tasks. For example, we want to create a thumbnail of an user uploaded photo? …