Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Djangocon: automated spell checking in django projects - Jakob Schnell
(One of my summaries of a talk at the 2018 European djangocon.) We are humans and we make typos. So there are typos in our code. The two common places for typos are documentation and the user interface. Documentation is normally only provided in a single language and it are large text files, so spell checking is relatively easy. Django documentation is often build with Sphinx. For that, there is a sphinx extension: sphinxcontrib-spelling. You can even integrate it in your CI as a post-build check. There will be words that are correct for your project but that aren't in the regular dictionary: for that there's a local "wordlist" you can use. For code (and our GUI), it gets more complicated. You would have to read python code, css code, html code, javascript code... hard. But there is a solution! Translations. Once your project gets bigger, you probably want to start translating it. Once you have set up your translation mechanism (gettext), you have all your strings gathered into one place: the .po files. Hurray, now we can do spell checking. Gettext is the standard mechanism in Django to deal with translations. You'll see lines like from ... import ugettext … -
Djangocon: strategies to edit production data - Julie Qiu
(One of my summaries of a talk at the 2018 european djangocon.) She works at the catalog team of "spring", a clothing website. Internal tools are often not available. There are always edge cases. Time-sensitive changes are sometimes needed ("right now"). You could just do a quick SQL query in the database. Normally, a colleague will look over your shoulder and double-check what you're about to type. But when it is friday afternoon and your highest-paying client wants a last-minute change.... They do have a collections of horror stories... Here are some strategies: Develop a review process for manual edits. What they've done is to create a spreadsheet. You'd have to write in there your name, the sql code, what you want it to do, who you want to review it. Only after the review, you are allowed to run it on the server. The advantage is that it is easy to implement and that you get an audit trail. You also teach engineers what is the right thing to do. A disadvantage is that you still can get mistakes. And it is fine for smaller changes, but not really for elaborate SQL and long-running queries. Write scripts and run … -
Djangocon: slow food digest better (maintain an old project) - Christopher Grebs
(One of my summaries of a talk at the 2018 european djangocon.) Full title: "slow food digest better - or how to maintain an 8.5 year old python project without getting lost". Christopher had to maintain such a project - and actually liked it. It was https://addons.mozilla.org, actually. It started out as a quickly-hacked-together php project. Now it an almost modern django project. The transition from PHP to the django version took almost 16 months. During that time there were bugs, translation errors, downtime: irritating. The site went fully live in january 2010. The big advantage of the move to django was that lots of tests were added at that time. The site wasn't anything special. Mostly django. Still quite some raw SQL from the old system. Celery for some tasks. Mozilla at one time had the "Firefox OS" for mobile phones. For that, they build the "firefox marketplace". The work was based on the addons.mozilla.org code, but with some weird hacks based on which site it was running... During that time the addons.mozilla.org website itself was pretty much left alone. In 2015 they had to re-animate the old codebase. At that time it was running django 1.6, barely, as … -
Djangocon: thursday lightning talks
(One of my summaries of a talk at the 2018 european djangocon.) Recording this conference with python and gstreamer - Peter Körner He's part of the CCC video team. Most of it is explained on https://c3voc.de/wiki He showed diagrams of their wonderful setup, but I cannot possibly write that down :-) The code is at https://github.com/voc/voctomix You're welcome to ask them questions! Cosmic ray research - Arne de Laat Arne used to work in astronomy on a project to do research on particles that result from supernova explosions. These particles can reach our athmosphere ("cosmic rays"). HiSPARC, high school project on astrophysics research and cosmics. Simple measurement stations, build by the schools, to detect those cosmic rays. One particle that hits the athmosphere hits other particles, so it results in multiple speed-of-light other particles. So if you detect a bunch of them at almost the same time at several nearby locations, you can more or less work out the direction of the explosion. See http://data.hisparc.nl/ for a sample. How to propose a talk - Lacy Williams Henschel The first thing you need is an idea. What have you learned recently? What do you have a strong opinion on. What used … -
Djangocon: writing code.... evolve it instead - Emma Gordon
(One of my summaries of a talk at the 2018 european djangocon.) Emma is member of the Cambridge programmers' study group. One of the topic there at one time was machine learning. Cool. But she started thinking about computers learning to write code theirselves. What would that mean for us, programmers, regarding job security? Automation/mechanisation has eliminated lots of jobs already. Self-checkouts at supermarkets. Agricultural mechanisation. Will the same happen to programmers? IBM created a huge computer called "Watson" that could win at some game show (jeopardy). Machine learning, natural language processing, etc. What is interesting about that, for Emma, is that "Watson" could be used for "reading" medical journal papers. The computer could amass knowledge (especially of edge cases) which a human doctor could not do: how many papers can you read per week? Same with legal stuff. Human doctors and lawyers won't be replaced, but parts of their jobs could be done/assisted by computers. So what about us, programmers? Code is text. You could look at genetic algorithms. Random search, guided random search. It takes its inspiration from biology. You could generate random strings and hope they'll match this talk's title (for instance). You need a "fitness function" … -
Djangocon: a different form of navigation - Chaim Kirby
(One of my summaries of a talk at the 2018 european djangocon.) We use forms a lot. He's not going to talk about regular forms, but about using forms for navigation. Many people have lots of data nowadays. Or have to work with it. Or have to program code for people that work with lots of data. Chaim works in healthcare. He showed a demo of riskscape: 150GB, 130 million records, 47 fields. The demo showed a form where you could select a few criteria ("only give me people with type 2 diabetes") and pick an action (show number of cases on a map). Then it started to get interesting. Click on an area and then create a graph according to age group. You could also click several items on the homepage that resulted in a map or graph with a pre-populated selection. Technically: A form's data propagates through pages. You can pre-populate a form so that you can use it as an URL (from a dashboard, for instance). This is done with django-modelqueryform. It generates forms that act as a power search based on a model's Meta. To carry the form context between pages you have to render the … -
Djangocon: it's not a bug, it's a bias - Anna-Livia Gomart
(One of my summaries of a talk at the 2018 european djangocon.) If you have a bad experience with a program, is it a bug of can there be a bias in the software? Apple's Siri, at first, had answers for questions like "where do I hide a body?", but it didn't have an answer for "where can I get an abortion?"... Are algorithms neutral or can they be biased? You can have biased programs, biased users and biased data. Programs are made by programmers. Programmers are humans. Humans have experiences. Experience is often colored... We, as programmers, can read the algorithms. So it is our role (or our duty) to critique them. Look at computer games. Often there is "procedural rhetoric" in it: some worldview or agenda that is embedded in the game's enviroment or rules. Facebook has a definition of "friend". So in fact they're re-defining "friend"... A problem you often have: normalcy. As designer/programmer/manager, you have certain assumptions. These might not be true for the user. "Everyone lives in a EU city and has 4G". "No-one has a last name called Null". "Everybody reads English perfectly". An important bias: the negativity bias. Negative reviews are notice much … -
Djangocon: building real-time apps with django - Iacopo Spalletti
(One of my summaries of a talk at the 2018 european djangocon.) The web isn't a pure request/response web anymore. There's real-time web now. There are lots of tools (yeah!), but you also get lots of complexity (booo!). Django channels is a framework to use django in the non-http world. Websockets, but not only those. Channels (version 1) is out now for three years, version 2 came out a few months ago. The talk assumes version 2. Some concepts: Asynchronous. For us, this means event-driven. A problem is that asynchronous code is normally harder to understand because of its non-linear nature. But... channels hides most of the complexity for you. ASGI. A protocol spec (like WSGI), independent of channels. It is ASGI all the way down. You can nest them. Protocol server. This implements the ASGI spec for a specific protocol. It interacts with the connection. It bridges the network and the application. For channels, this often is 'daphne'. Routing. Maps incoming messages to consumers. It is nestable and composable. Scope. A "scope" is created for every connection. It holds the data of the connection. Channel. IPC mechanism ("Inter Process Communication"). FIFO "at-most-once" queue. Consumers. The core abstraction to build … -
Djangocon: can packaging improve django deployments? - Markus Zapke-Gründemann
(One of my summaries of a talk at the 2018 european djangocon.) Markus started created websites at the end of the 1990s. He calls himself an open source software developers. Open source is how he learned programming, so he likes giving back. How can packaging make deployments easier, faster and more reliable? He showed a django project structure, modified for packaging it with python. Extra files are MANIFEST.in, setup.cfg, setup.py. And the actual project code is in a subdirectory, as that's what setup.py likes. If you have apps, he suggests to put them in an yourproject/apps/ subdirectory. Many people use requirements.txt files to list their dependencies. They only use setup.py for libraries or apps that they publish. But why not use the setup.py also for projects? There's an install_requires key you can use for mentioning your dependencies. Note that if your setuptools is new enough, you can put everything into setup.cfg instead of having it all in a weird function in setup.py. This includes the dependencies. Your setup.py will now look like this: from setuptools import setup setup() As we're deploying a project (instead of a library), we can even pin the requirements to specific versions. He mentioned bumpversion to … -
Djangocon: representing hierarchies in relational databases - Jacob Rief
(One of my summaries of a talk at the 2018 european djangocon.) Jacob works with django since 2011 and he created and maintains a couple of django apps. One of the projects he took over maintenance of is django-treebeard. django-treebeard can be used to create hierarchical structures in your relational database. A tree structure. Like the structure of a company. Or nested categories in a shopping website. There are four common solutions to get a hierarchical relation in your database: Adjecent list. Basically you add a parent foreign key to your model. Traversing the tree has bad performance, as you have to recurse through every "level" to find all parents or children of a node. It gets worse if you also want to filter. You can Materialized path tree. You give every node a special "path" field. The first element is A, its children are AA, AB, AC. One level down AAA, AAB, ABA, ABB, ABC, etc. Now if you want to find the children of 'AAB', you search for all items with a path that starts with 'AAB'. (You have to exclude the current node, though). As long as you have a good index, the performance is good. If … -
Djangocon: creating solid APIs - Rivo Laks
(One of my summaries of a talk at the 2018 european djangocon.) Our applications are increasingly also used by other computers. But the APIs they use are ironically aimed at humans: at the programmers that have to program on your API. "API" means "application programming interface", but it might be better to say "application programmer interface" as you really have to focus on the programmer that has to do the programming! What makes an API good? Documentation. Familiarity. Lack of friction. Documentation This is often overlooked. But it is the first point of contact of your API, so it gives the first impression. An important deciding factor. It takes lots of work to write good documentation, but the effort is worth it. Look at your documentation as a "sales page" for your API. What do you want on your sales page? Here are some suggestions: How do I access it? Do I need authentication? How? What is the root URL? General info on encodings and formats. Make it explicit. Pagination. Document the common errors (error codes) you can get. Code for getting started. It gives your users a warm feeling if their first try works right away. When you document … -
Djangocon: making smarter queries with advanced ORM resources - Mariana Bedran Lesche
(One of my summaries of a talk at the 2018 european djangocon.) There are details in django's ORM/model/query documentation that you often skip over. But there are very useful things in there. Recently she had to work on an old PHP project where she had to write SQL by hand. And some of those raw SQL statements, she couldn't get working right away with django. So she went back to the harder sections of the documentation. When should you use the advanced methods? For instance when the regular optimizations don't give enough results: Good indexes on your tables. Paginated requests. select_related, prefetch_related are there. You limit the output with values, values_list, only and defer. You use a database cache. You use the assertNumQueries on tests to figure out if too many queries are made. Mariana comes from Brazil, where they have a law that says that all non-secret goverment information should be made available. There's the https://brasil.io project that tries to make the data available in useful datasets (as the source data isn't always in a useful format). That's where she got her sample data from: info on companies and their owners. Owners can be regular persons, other companies or … -
Djangocon: accessibility matters: building a better web - Lindsey Dragun
(One of my summaries of a talk at the 2018 european djangocon.) Lindsey runs the DisabledInTech slack channel. For Web accessibility, see https://www.w3.org/WAI/fundamentals/accessibility-intro/ as a good intro. There are standards to help with accessibility and for many government instances, they're mandatory. Disabilities are often grouped in: visual, auditory, motor/physical, cognitive/neurological, language/speech. There are other (more inclusive!) ways to group: permanent (born blind), temporary (like just being sick!), acquired (like ageing), societal (like lefthandedness). Notice how this comes much closer? There are alternative input/output tools. Braille displays, eye trackers, foot pedals instead of mouse buttons, etc. Important: think deeper. Categories can be desceptively simple. Users don't always fall into just one category. Examples. The web is sometimes like the letters at the eye doctor: "read the line with the smallest letters". Color contrast. il1: what is the I, what the L, what the number? Dislexia? Bad font? Make changes noticable. Feedback in forms, for instance. Which field contained the error? The above list might make it look like accessibility is easy. It often isn't. How do I implement accessibility? Avoid gimmicks. What's popular on the web isn't necessarily handy. Infinite scrolling in combination with a sidebar. When you scroll, you'll never … -
Djangocon: lightning talks wednesday
(One of my summaries of a talk at the 2018 european djangocon.) Lightning talks are quick 5 minute talks, so I won't have every detail and speaker name correctly :-) Adding multi-factor authentication to your django project - Justin Mayer Passwords are ubiquitous. Passwords are terrible. Solutions: sms (bad), TOTP, one-time passwords. U2F key. That one is great. It will also be a standard in browsers. There's django-u2f to help you with it. He managed to get a demo working within 5 minutes! Instant feedback - Johannes Hopper Making reviews (on pull requests) great again with the new "github checks". What is tedious as a maintainer are the small tasks like "could you remove the trailing spaces". With "github checks", you can let a linter do that job and give the feedback to the user. Such feedback of a program is much less irritating to the person that created the pull requests than when you do it yourself. "Github checks" is a new (beta) feature. It is made for computer feedback. With a colleague, he set up a collection of linters, ready for integration with github. Hosted on AWS. If you're interested, contact him. (It is all open source, of … -
Official Django REST Framework Tutorial - A Beginners Guide
If you have struggled to complete the official tutorial on your own, consider this guide a good place to start instead. -
I'm in Heidelberg for the djangocon 2018 (plus other info)
Yes, I'm in Heidelberg (Germany) for the annual European djangocon. That's not a particularly shocking discovery, as I've been to quite a number :-) There are however some reasons for me to write this entry. The most important reason is that I've set up my old blog software on a brand new laptop. "Old software" means "python 2" and it is "python 3" now. And I've modernized the setup on my server a bit. So: this entry is my first test if my setup also works when writing a blog entry. That's not something I want to test when I'm making summaries tomorrow. New laptop? Two months ago I asked advice: mac/linux. Both had advantages. What I type this on is a linux laptop. A powerful Dell precision 5520! I'll probably write about my new setup later :-) New laptop? Yes, because of a new job. The company is also quite new (from February this year), it doesn't have a website yet: "Triple S transformations". I'd describe my job as senior developer with a big focus on python and open source. Triple S is part of VolkerWessels, the second biggest Dutch construction firm. Our aim is to replace expensive and … -
Using Braze so that marketers are not the developers’ nightmare
Braze is an external platform to send messages like emails, push notifications, in-app, sms, etc. The platform has two ways to send them: through an API or offers an SDK to integrate.You can store in Braze the user and products data that you need and use it in the messages to be sent. The platform offers the possibility to send messages at scale.Marketers can manage messages and their scheduling without relying on programmers.CampaignsMessages are sent through campaigns. Braze provides multiple scheduling options for campaigns. These alternatives allow you to send messages to users at the right time according to the needs.The first option is the Scheduled Delivery. Messages will be sent as soon as the campaign is launched, sending at a designated time (it allows to specify the days and hours the campaign will send the messages) or using the “Intelligent Delivery” (once, daily, weekly, monthly).The second option is Action-Based Delivery. Messages are sent when users perform actions. The trigger events can be starting a session, performing a custom event, performing a campaign’s conversion event, making a purchase, interacting with other campaigns, among others. For this option you can select how long to wait before sending the campaign. The campaign’s … -
Taming Irreversibility with Feature Flags (in python)
Feature Flags are a very simple technique to make features of your application quickly toggleable. The way it works is, everytime we change some behavior in our software, a logical branch is created and this new behavior is only accessible if some specific configuration variable is set or, in certain cases, if the application context respects some -
Best Django Books 2018
List of current Django 2.0 books. -
How To Deploy Django Channels To Production
In this article, we will see how to deploy django channels to production and how we can scale it to handle more load. We will be using nginx as proxy server, daphne as ASGI server, gunicorn as WSGI server and redis for channel back-end. Daphne can serve HTTP requests as well as WebSocket requests. For stability and performance, we will use uwsgi/gunicorn to serve HTTP requests and daphne to serve websocket requests. We will be using systemd to create and manage processes instead of depending on third party process managers like supervisor or circus. We will be using ansible for managing deployments. If you don't want to use ansible, you can just replace template variables in the following files with actual values. Nginx Setup Nginx will be routing requests to WSGI server and ASGI server based on URL. Here is nginx configuration for server. server { listen {{ server_name }}:80; server_name {{ server_name }} www.{{ server_name }}; return 301 https://avilpage.com$request_uri; } server { listen {{ server_name }}:443 ssl; server_name {{ server_name }} www.{{ server_name }}; ssl_certificate /root/certs/avilpage.com.chain.crt; ssl_certificate_key /root/certs/avilpage.com.key; access_log /var/log/nginx/avilpage.com.access.log; error_log /var/log/nginx/avilpage.com.error.log; location / { proxy_pass http://0.0.0.0:8000; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_redirect off; } … -
How To Deploy Django Channels To Production
In this article, we will see how to deploy django channels to production and how we can scale it to handle more load. We will be using nginx as proxy server, daphne as ASGI server, gunicorn as WSGI server and redis for channel back-end. Daphne can serve HTTP requests as well as WebSocket requests. For stability and performance, we will use uwsgi/gunicorn to serve HTTP requests and daphne to serve websocket requests. We will be using systemd to create and manage processes instead of depending on third party process managers like supervisor or circus. We will be using ansible for managing deployments. If you don't want to use ansible, you can just replace template variables in the following files with actual values. Nginx Setup Nginx will be routing requests to WSGI server and ASGI server based on URL. Here is nginx configuration for server. server { listen {{ server_name }}:80; server_name {{ server_name }} www.{{ server_name }}; return 301 https://avilpage.com$request_uri; } server { listen {{ server_name }}:443 ssl; server_name {{ server_name }} www.{{ server_name }}; ssl_certificate /root/certs/avilpage.com.chain.crt; ssl_certificate_key /root/certs/avilpage.com.key; access_log /var/log/nginx/avilpage.com.access.log; error_log /var/log/nginx/avilpage.com.error.log; location / { proxy_pass http://0.0.0.0:8000; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_redirect off; } … -
Debugging Python Applications with pdb
Debugging isn’t a new trick – most developers actively use it in their work. Of course, everyone has their own approach to debugging, but I’ve seen too many specialists try to spot bugs using basic things like print instead of actual debugging tools. Or even if they did use a debugging tool, they only used a small set of features and didn’t dig deeper into the wide range of opportunities good debuggers offer. And which could have saved those specialists a lot of time. Debugging Python with the Print Command As mentioned above, some of us use ‘print’ to display information that reveals what’s going on inside the code. Some of us use a logger for the same purpose – but please don’t confuse this with the logger for the production code, as I’m referring to developers that only add the logger during the problem searching period, i.e. just until the developing process ends. But the truth is, print has a lot of loopholes. Probably, its biggest drawback is that you have to add changes to your code, then re-run the application to see the different variable you’re writing, or the current variable type, for example. So how is a … -
Django Channels 2.0 to Production Environment
Learn how to deploy a Django C... -
New Course: Creating and Distributing Python Packages
Myself and Audrey Roy Greenfeld have released a course, Creating and Distributing Python Packages. Features it includes: Covers Python packaging details including setup.py, not to mention automating things like testing, versioning, documentation, dependency management, and checking for security issues. Available in English or Spanish. Demonstrates Cookiecutter and Cookiecutter PyPackage. Will help fund our open source work in a way that doesn't involve us directly asking people for money (because we suck at that). Who is this course good for? Do you have a script, function, class, or snippet of code you copy/paste from project to project? If that's the case, this course is for you. We'll teach you how to package that up and add automated tests, documentation, and so much more. This especially applies to organizations that share code between projects. If you are copy/pasting common code that means updating it across projects is challenging and error prone, introducing the risk of bugs and security concerns. Plus, odds are it's not tested reliably. By packaging your copy/pasting code and adding automation, it becomes just another dependency, easily updated across multiple projects. Or what if you have an API that needs a Python client SDK? If that's the case, then … -
Always return namespaces in Django REST Framework
By default, when you hook up a model to Django REST Framework and run a query in JSON format, what you get is a list. E.g. For GET localhost:8000/api/mymodel/ [ {"id": 1, "name": "Foo"}, {"id": 2, "name": "Bar"}, {"id": 3, "name": "Baz"} ] This isn't great because there's no good way to include other auxiliary data points that are relevant to this query. In Elasticsearch you get something like this: { "took": 106, "timed_out": false, "_shards": {}, "hits": { "total": 0, "hits": [], "max_score": 1 } } Another key is that perhaps today you can't think of any immediate reason why you want to include some additonal meta data about the query, but perhaps some day you will. The way to solve this in Django REST Framework is to override the list function in your Viewset classes. Before # views.py # views.py from rest_framework import viewsets class BlogpostViewSet(viewsets.ModelViewSet): queryset = Blogpost.objects.all().order_by('date') serializer_class = serializers.BlogpostSerializer After # views.py from rest_framework import viewsets class BlogpostViewSet(viewsets.ModelViewSet): queryset = Blogpost.objects.all().order_by('date') serializer_class = serializers.BlogpostSerializer def list(self, request, *args, **kwargs): response = super().list(request, *args, **kwargs) # Where the magic happens! return response Now, to re-wrap that, the response.data is a OrderedDict which you can change. Here's …