Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
The Almighty Pause Container
When checking out the nodes of your Kubernetes cluster, you may have noticed some containers called "pause" running when you do a `docker ps` on the node. $ docker ps CONTAINER ID IMAGE COMMAND ... ... 3b45e983c859 gcr.io/google_containers/pause-amd64:3.0 "/pause" ... ... dbfc35b00062 gcr.io/google_containers/pause-amd64:3.0 "/pause" ... ... c4e998ec4d5d gcr.io/google_containers/pause-amd64:3.0 "/pause" ... ... 508102acf1e7 gcr.io/google_containers/pause-amd64:3.0 "/[...] -
A Complete Beginner's Guide to Django - Part 6
Introduction Welcome to the sixth part of the tutorial series! In this tutorial, we are going to explore in great detail the Class-Based Views. We are also going to refactor some of the existing views so to take advantage of the built-in Generic Class-Based Views. There are many other topics that we are going to touch with this tutorial, such as how to work with pagination, how to work with Markdown and how to add a simple editor. We are also going to explore a built-in package called Humanize, which is used to give a “human touch” to the data. Alright, folks! Let’s implement some code. We have plenty of work to do today! Views Strategies At the end of the day, all Django views are functions. Even class-based views (CBV). Behind the scenes, it does all the magic and end-up returning a view function. Class-based views were introduced so to make it easier for developers to reuse and extend views. There are many benefits of using them, such as the extendability, the ability to use O.O. techniques such as multiple inheritances, the handling of HTTP methods are done in separate methods, rather than using conditional branching, and there are … -
Implement search with Django-haystack and Elasticsearch Part-1
Haystack works as a search plugin for Django. You can use different backends Elastic-search, Whose, Sorl, Xapian to search objects. All backends work with the same code. In this post, I am using elastic search as backend. Installation: pip install django-haystack Configuration: add haystack to installed apps INSTALLED_APPS=[ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', #add haystack here 'haystack', 'books' ] Settings: Add back-end settings for the haystack. HAYSTACK_CONNECTIONS = { 'default': { 'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine', 'URL': 'http://127.0.0.1:9200/', 'INDEX_NAME': 'haystack_books', }, } Above settings for elastic search. Add signal processor for the haystack. This signal will update objects in … -
How to Create a Custom Django User Model
Django's built-in User model a... -
The Simple Power of Django Validators
# The Simple Power of Django V... -
Understanding Django Permissions And Groups
Permissions: Actually permissions are of 2 types: 1.Model level permissions 2.object level permissions If you want to give permissions on all cars, then Model-level is appropriate, but if you Django to give permissions on a per-car basis you want Object-level. You may need both, and this isn't a problem as we'll see. For Model permissions, Django will create permissions in the form 'appname.permissionname_modelname' for each model. If you have an app called 'drivers' with the Car model then one permission would be 'drivers.delete_car'. The permissions that Django automatically creates will be create, change, and delete.Read permission is not included in CRUD operation.Django decided to change CRUD's 'update' to 'change' for some reason. You can use the metaclass to add more permissions to a model.: class Book( models.Model ): # model stuff here class Meta: permissions = ( ( "read_book", "Can read book" ), ) Permissions is a set of tuples, where the tuple items are the permission as described above and a description of that permission. Finally, to check permissions, you can use has_perm: obj.has_perm( 'drivers.read_car' ) Where obj … -
Overriding Django Model behaviour with Proxy Model
The main Usage of a proxy model is to override the main functionality of existing Model. It is a type of model inheritance without creating a new table in Database. It always query on original model with overridden methods or managers. You can query, delete, edit, create etc on Proxy model but the effects will be on the original model. You can define models as a proxy just by adding proxy = True in Meta class of model. # Original Model class Employee(models.Model): EmployeeType = ( ("D", "Developer"), ("M", "Manager"), ) name = models.CharField(max_length=55) type = models.CharField(choices=EmployeeType, max_length=1, default='D') def __str__(self): return self.name # Model manager to use in Proxy model class ManagerEmpManager(models.Manager): def get_queryset(self): return super(ManagerEmpManager, self).get_queryset().filter( type='M') def create(self, **kwargs): kwargs.update({'type': 'M'}) return super(ManagerEmpManager, self).create(**kwargs) # Proxy Model class ManagerEmployee(Employee): objects = ManagerEmpManager() class Meta: proxy = True In ManagerEmployee I have overridden create method to set type … -
how to pass extra context data to serializers in django-rest-framework ?
Serializers are used to validate the data in Django rest framework. Generally, serializers provide basic validations based on the type of field. Model serializers use model validations(primary key, foreign key, unique, etc). We not only use these basic validations but also custom validations to some of the fields. For writing custom validations we may require some extra data in validation methods of serializers. Django Rest Framework Serializer takes the following parameters while creating the serializer object. read_only, write_only, required, default, initial, source, label, help_text, style, error_messages, allow_empty, instance, data, partial, context, allow_null Django serializers designed in such a way that makes an application more powerful and simplified the development process. To pass extra context data to the serializer we use "context" parameter. "context" parameter takes the data in the form of a dictionary. we access the context data inside serializer with statement "self.context" Let's take an example to understand it more clearly. Case: Send an email to any user with given content but, do not send email to the specific list of emails. For this case, we need to validate the email and raise an error if it is in excluded emails list. To do this we need to have excluded emails list … -
Upgrade All Pip & Python Packages & Store an Archive
This guide is still being deve... -
Understanding Routers in Django-Rest-Framework
We can use function-based views(FBV) and generic views(class-based views[CBV]) to develop rest API by using the Django-REST-Framework. It's a good approach to use FBV or CBV with defined URL configurations. Mapping views with the URL's is a good idea but, probably not the best. We can better organize this by using Routers and ViewSets. Advantages using ViewSets and Routers over traditional views We can avoid configuring the URL's with views. Routers generate automatic URL patterns and maps every URL to its respective method based on a type of the request. It deals with a little abstraction but, it can speed up the development process. We can also speed up the debugging process as well with a little practice. Router generates standardized url patterns. We can expect consistent behaviour from viewsets and routers We can avoid repetitive code in views. For example, in traditional views we need to have two api views for detail and list. But, we can achive it with a single ViewSet class. If we develop a large api and if we don't use viewset and routers then it will result in more views and urls. It will definitely affect our application(api) maintainablity and development time. Lets see … -
Docker-compose in Jenkins pipelines: keep builds separate
We use Jenkins with the pipelines plugin as our continuous integration server to test our software. The pipelines plugin means all the steps are in a Jenkinsfile inside your repo instead of as manually entered commands in a web interface (like it used to be). Nice and clean. Also nice and clean: using dockers (with docker-compose) to run the tests in. The Jenkins machine used to be one big huge mess of installed software. Now it is clean and tidy: everything you need to run tests, you'd better install it inside your docker. Recent problem. Recently Jenkins changed the "workspace" location: the place where the project is checked out and where your docker-compose is run. It used to be /var/lib/jenkins/workspaces/longdirectoryname. "longdirectoryname" would include (abbreviated) your project's name, your branch/PR's name and some hash. Any networks/volumes/etc created by docker-compose would include that unique name. Recently, that location became /var/lib/jenkins/jobs/ORG_NAME/jobs/PROJECT_NAME/... and some more, ending with .../workspace. So suddenly every build is inside a directory called workspace so docker-compose will by default start prefixing networks/volumes with workspace instead of with a string that's unique per project/branch! So if you run a branch and pull request at the same time, both docker-compose runs will … -
A Complete Beginner's Guide to Django - Part 5
Introduction Welcome to the 5th part of the tutorial series! In this tutorial, we are going to learn more about protecting views against unauthorized users and how to access the authenticated user in the views and forms. We are also going to implement the topic posts listing view and the reply view. Finally, we are going to explore some features of Django ORM and have a brief introduction to migrations. Protecting Views We have to start protecting our views against non-authorized users. So far we have the following view to start new posts: In the picture above the user is not logged in, and even though they can see the page and the form. Django has a built-in view decorator to avoid that issue: boards/views.py (view complete file contents) from django.contrib.auth.decorators import login_required @login_required def new_topic(request, pk): # ... From now on, if the user is not authenticated they will be redirected to the login page: Notice the query string ?next=/boards/1/new/. We can improve the log in template to make use of the next variable and improve the user experience. Configuring Login Next Redirect templates/login.html (view complete file contents) <form method="post" novalidate> {% csrf_token %} <input type="hidden" name="next" value="{{ next … -
Evennia in Hacktoberfest 2017
Evennia, the Python MUD/MUSH/MU* creation library participates in the Hacktoberfest 2017 (sign up on that page)! Hacktoberfest is open for all open-source projects like ours. After registering, if you make at least four Pull Requests to a public repo on Github during October (need not just be to Evennia), you win a limited-edition T-shirt! The help Evennia out and get your T-Shirt, look at our Issue Tracker. I have marked some issues with "Hacktoberfest" but you could take on any issue you want. Take a look in particular at the Unit test issue if you are looking to get into contributing on a smaller scale while helping us tremendously.If you have any questions on contributing (or it's your first time making a Pull Request), don't be shy to drop into #evennia on irc.freenode.net or ask in our forum/mailing list. Have fun! -
Numbers in Translatable Strings
Sentences in websites like "You've got 1 messages." or "Messages you've got: 5" sound unnatural and not human-friendly. But the GNU gettext tool used with Django for translations has an option to define different pluralization depending on the number which goes together with the counted noun. Things get even more interesting with certain languages which have not just singular and plural like English, German, French, or Spanish, but more plural forms or just a single one. Tell me the background Let's talk about grammar. Most languages have two plural forms for counted elements: one for singular, like "1 thing", and one for plural, like "n things". However, certain languages have either just one form for singular and plural, or multiple plural forms depending on the number of elements that go with them. For example, my mother tongue Lithuanian is a Baltic language coming from Indo-European language family keeping archaic features from ancient Sanskrit. Lithuanian has 3 plural forms. When one counts apples in Lithuanian, they say "1 obuolys", "2-9 obuoliai", "10-20 obuolių", "21 obuolys", "22-29 obuoliai", "30 obuolių", "31 obuolys", "32-39 obuoliai", etc. The second most widespread language on the web after English is Russian. Russian is an Eastern Slavic … -
How to Create Django Data Migrations
Data Migration is a very convenient way to change the data in the database in conjunction with changes in the schema. They work like a regular schema migration. Django keep track of dependencies, order of execution and if the application already applied a given data migration or not. A common use case of data migrations is when we need to introduce new fields that are non-nullable. Or when we are creating a new field to store a cached count of something, so we can create the new field and add the initial count. In this post we are going to explore a simple example that you can very easily extend and modify for your needs. Data Migrations Let’s suppose we have an app named blog, which is installed in our project’s INSTALLED_APPS. The blog have the following model definition: blog/models.py from django.db import models class Post(models.Model): title = models.CharField(max_length=255) date = models.DateTimeField(auto_now_add=True) content = models.TextField() def __str__(self): return self.title The application is already using this Post model; it’s already in production and there are plenty of data stored in the database. id title date content 1 How to Render Django Form Manually 2017-09-26 11:01:20.547000 […] 2 How to Use Celery … -
A Complete Beginner's Guide to Django - Part 4
Introduction This tutorial is going to be all about Django’s authentication system. We are going to implement the whole thing: registration, login, logout, password reset, and password change. You are also going to get a brief introduction on how to protect some views from non-authorized users and how to access the information of the logged in user. In the next section, you will find a few wireframes of authentication-related pages that we are going to implement in this tutorial. After that, you will find an initial setup of a new Django app. So far we have been working on an app named boards. But all the authentication related stuff can live in a different app, so to achieve a better organization of the code. Wireframes We have to update the wireframes of the application. First, we are going to add new options for the top menu. If the user is not authenticated, we should have two buttons: sign up and log in. Figure 1: Top menu for not authenticated users. If the user is authenticated, we should instead display their names along with a dropdown menu with three options: my account, change password and log out. Figure 2: Top menu … -
Evennia 0.7 released
As of today Evennia 0.7 is officially out! A big thank you to all collaborators that have helped with code and testing along the way!Here is the hefty forum post that details how you migrate to Evennia 0.7. Evennia 0.7 comes with a range changes and updates (these are just the ones merged from the latest devel branch, a lot more has happened since 0.6 that were already in master): EvMenu formatting functions now part of class - EvMenu no longer accepts formatting functions as inputs, these are now part of the EvMenu class. To override the formatting of EvMenu nodes you should now override EvMenu and replace the format method you want. This brings EvMenu more in line with other services in Evennia, all of which are built around overriding.Scripts are now valid message senders - Also Scripts can now act as "sender" of a Msg, for example to a Channel or a player.Prefix-ignoring - All default commands are now renamed without their @-prefixes. Both @examine, +examine or examine will now all point to the same command. You can customize which prefixes Evennia simply ignores when searching for a command. The mechanic is clever though - if you create … -
A Complete Beginner's Guide to Django - Part 3
Introduction In this tutorial, we are going to dive deep into two fundamental concepts: URLs and Forms. In the process, we are going to explore many other concepts like creating reusable templates and installing third-party libraries. We are also going to write plenty of unit tests. If you are following this tutorial series since the first part, coding your project and following the tutorial step by step, you may need to update your models.py before starting: boards/models.py class Topic(models.Model): # other fields... # Add `auto_now_add=True` to the `last_updated` field last_updated = models.DateTimeField(auto_now_add=True) class Post(models.Model): # other fields... # Add `null=True` to the `updated_by` field updated_by = models.ForeignKey(User, null=True, related_name='+') Now run the commands with the virtualenv activated: python manage.py makemigrations python manage.py migrate Or if you prefer to use my source code as a starting point, you can grab it on GitHub. The current state of the project can be found under the release tag v0.2-lw. The link below will take you to the right place: https://github.com/sibtc/django-beginners-guide/tree/v0.2-lw The development will follow from here. URLs Proceeding with the development of our application, now we have to implement a new page to list all the topics that belong to a given Board. … -
Blue/Green Deployments on Kubernetes
> For those that want to dive right in, I have put up a tutorial and some sample manifests on github. Check it out at https://github.com/IanLewis /kubernetes-bluegreen-deployment-tutorial Kubernetes has a really awesome built-in feature called Deployments. Deployments come with the ability to do rolling updates of containers when you update your application to a new version. Rolling updates are a great way to update applications because your app uses about the same amount of resources during an update as it does when not updating, all with minimal impact to performance and availability. [...] -
Blue/Green Deployments on Kubernetes
> For those that want to dive right in, I have put up a tutorial and some > sample manifests on github. Check it out at > https://github.com/IanLewis/kubernetes-bluegreen-deployment-tutorial Kubernetes has a really awesome built-in feature called Deployments. Deployments come with the ability to do rolling updates of containers when you update your application to a new version. Rolling updates are a great way to update applications because your app uses about the same amount of resources during an update as it does when not updating, all with minimal impact to performance and availabilit[...] -
Multitenancy: juggling customer data in Django
Suppose you want to build a new SaaS (Software as a Service) application. Suppose your application will store sensitive data from your customers. What is the best way to guarantee the isolation of the data and make sure information from one client does not leak to the other? The answer to that is: it depends. It depends on the number of customers y -
cache_memoize - a pretty decent cache decorator for Django
This is something that's grown up organically when working on Mozilla Symbol Server. It has served me very well and perhaps it's worth extracting into its own lib. Usage Basically, you are probably used to this in Django: from django.core.cache import cache def compute_something(user, special=False): cache_key = 'meatycomputation:{}:special={}'.format(user.id, special) value = cache.get(cache_key) if value is None: value = _call_the_meat(user.id, special) # some really slow function cache.set(cache_key, value, 60 * 5) return value Here's instead how you can do exactly the same with cache_memoize: from wherever.decorators import cache_memoize @cache_memoize(60 * 5) def compute_something(user, special=False): return _call_the_meat(user.id, special) # some really slow function Cache invalidation If you ever need to do non-trivial caching you know it's important to be able to invalidate the cache. Usually, to be able to do that you need to involved in how the cache key was created. Consider our two examples above, here's first the common thing to do: def save_user(user): do_something_that_will_need_to_cache_invalidate(user) cache_key = 'meatycomputation:{}:special={}'.format(user.id, False) cache.delete(cache_key) # And when it was special=True cache_key = 'meatycomputation:{}:special={}'.format(user.id, True) cache.delete(cache_key) This works but it involves repeating the code that generates the cache key. You could extract that into its own function of course. Here's how you do it with … -
A Complete Beginner's Guide to Django - Part 2
Introduction Welcome to the second part of our Django Tutorial! In the previous lesson, we installed everything that we needed. Hopefully, you are all setup with Python 3.6 installed and Django 1.11 running inside a Virtual Environment. We already created the project we are going to play around. In this lesson, we are going to keep writing code in the same project. In the next section, we are going to discuss a little bit about the project we are going to develop, so to give you some context. Then after that, you are going to learn all the Django fundamentals: models, admin, views, templates, and URLs. Hands on! Web Board Project I don’t know about you, but personally, I learn much more by seeing practical examples and code snippets. For me, it’s difficult to process a concept where in the examples you read Class A and Class B, or when I see the classical foo(bar) examples. I don’t want to do that with you. So, before we get into the fun part, playing with models, views, and everything. Let’s just take a moment and discuss very briefly about this project we are going to develop. If you already have experience … -
Tecken loadtesting
My project over the summer, here at Mozilla, has been a project called Mozilla Symbol Server. It's a web service to upload C++ symbol files, downloading C++ symbol files and symbolicating C++ crash stacktraces using said uploaded C++ symbol files. It went into production last week which was fun but there's still lots of work to do on adding beyond-parity features and more optimizations. What Is Mozilla Symbol Server? The code name for this project is Tecken and it's written in Python (Django, Gunicorn) and uses PostgreSQL, Redis and Celery. The frontend is entirely static and developed (almost) as a separate project within. The frontend is written in React (using create-react-app and react-router). Everything is run as Docker containers. And if you ask me more details about how it's configured/deployed I'm afraid I have to defer to the awesome Mozilla CloudOps team. One the challenges is that the downloading part is that it needs to be fast and handle high traffic. Today I did some load testing and managed to start 14 concurrent clients that bombarded our staging server with realistic HTTPS GET queries based on log files. It's actually 8 + 4 + 2 concurrent clients. 7 of them … -
Mozilla Symbol Server (aka. Tecken) load testing
(Thanks Miles Crabil not only for being an awesome Ops person but also for reviewing this blog post!) My project over the summer, here at Mozilla, has been a project called Mozilla Symbol Server. It's a web service that uploads C++ symbol files, downloads C++ symbol files and symbolicates C++ crash stacktraces. It went into production last week which was fun but there's still lots of work to do on adding beyond-parity features and more optimizations. What Is Mozilla Symbol Server? The code name for this project is Tecken and it's written in Python (Django, Gunicorn) and uses PostgreSQL, Redis and Celery. The frontend is entirely static and developed (almost) as a separate project within. The frontend is written in React (using create-react-app and react-router). Everything is run as Docker containers. And if you ask me more details about how it's configured/deployed I'm afraid I have to defer to the awesome Mozilla CloudOps team. One the challenges I faces developing Tecken is that symbol downloads need to be fast to handle high volumes of traffic. Today I did some load testing on our stage deployment and managed to start 14 concurrent clients that bombarded our staging server with realistic HTTPS …