Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
For hire
Since a slightly-wider circle of people know this now, it’s time to just go public with the news: my last day at now-former employer was toward the end of January. At the time I told a few friends, but wasn’t in a huge rush to find something else immediately; I’d been getting ready to depart for a little while, and there were some things (a bit of travel, catching up on some open-source projects, and ... Read full entry -
Efficient many-to-many field lookup in Django REST Framework
The basic setup Suppose you have these models: from django.db import models class Category(models.Model): name = models.CharField(max_length=100) class Blogpost(models.Model): title = models.CharField(max_length=100) categories = models.ManyToManyField(Category) Suppose you hook these up Django REST Framework and list all Blogpost items. Something like this: # urls.py from rest_framework import routers from . import views router = routers.DefaultRouter() router.register(r'blogposts', views.BlogpostViewSet) # views.py from rest_framework import viewsets class BlogpostViewSet(viewsets.ModelViewSet): queryset = Blogpost.objects.all().order_by('date') serializer_class = serializers.BlogpostSerializer What's the problem? Then, if you execute this list (e.g. curl http://localhost:8000/api/blogposts/) what will happen, on the database, is something like this: SELECT "app_blogpost"."id", "app_blogpost"."title" FROM "app_blogpost"; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 1025; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 193; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 757; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 853; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 1116; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN "app_blogpost_categories" ON ("app_category"."id" = "app_blogpost_categories"."category_id") WHERE "app_blogpost_categories"."blogpost_id" = 1126; SELECT "app_category"."id", "app_category"."name" FROM "app_category" INNER JOIN … -
Django, Sitemaps and alternates
Django, Sitemaps and alternates Django’s own sitemaps module is great for quickly generating sitemaps, but it unfortunately does not support more than the bare minimum of attributes. Also, it uses the template engine to create XML and this makes me sad. We had to add a sitemap.xml file to a customers’ website. The site supports several languages and runs on several domains, so we had to carefully specify alternate language pages and canonical URLs so that the website would be punished for duplicate content. Therefore, a year ago I set out to build a sitemaps app for Django which supports Django’s sitemaps, but also allows adding entries with additional attributes. The result of this work was django-sitemaps. from django_sitemaps import Sitemap def sitemap(request): sitemap = Sitemap(build_absolute_uri=request.build_absolute_uri) sitemap.add_django_sitemap(SomeSitemap, request=request) sitemap.add( url, changefreq='weekly', priority=0.5, lastmod=datetime.now(), alternates={ 'en': '...', 'en-ch': '...', 'en-gb': '...', 'de': '...', ... }, ) return sitemap.response(pretty_print=True) Today, I also added support for generating the most simple robots.txt files possible: All user agents, and only Sitemap: <absolute url> entries. The recommended usage is now (still using url() instead of path() because I’m old and rusty): from django_sitemaps import robots_txt from app.views import sitemap urlpatterns = [ url(r'^sitemap\.xml$', sitemap), url(r'^robots\.txt$', robots_txt(timeout=86400)), … -
Our approach to configuring Django, Webpack and ManifestStaticFilesStorage
Our approach to configuring Django, Webpack and ManifestStaticFilesStorage I spent some time finding out how Django and webpack should be configured so that they work well together both in development and in production. We are now using this setup on dozens of websites, so now is a good time to write down the approach. Requirements for development were: Hot module reloading (especially of CSS, but also of JavaScript code). Ability to use SCSS, ES6 and ES7. PostCSS support (although I currently only use the autoprefixer, nothing else). Requirements for production: Filenames depending on the content of assets, so that we can add far future expiry headers without worrying about cache busting, and not only for webpack-generated assets but also for everything else, so ManifestStaticFilesStorage is a must. Separate delivery of CSS and JavaScript code – we want to add CSS to the <head>, and JavaScript at the end of the <body>. Working webpack code splitting is a plus if possible. There are a few problems that have to be solved: Django has to know about asset URLs generated by webpack. Some assets are used both in HTML and in CSS/JavaScript code, e.g. a logo image. The obvious way to be … -
First Rotterdam (NL) Python meetup: my summaries
Welcome to the first Python Rotterdam meetup - Thijs Damsma He searched for python meetups in Rotterdam, but didn't find any. So he clicked on the "do you want to start one?" button. Today's meetup is at Van Oord (a big family owned marine contractor), but anywhere else in Rotterdam is fine. And you can help out. Everything is fine, from talks to workshops, as long as it is python-related. The goal is to have a few meetups per year. Jupyter lab - Joost Dobken Joost is a data engineer. He uses python daily, mostly jupyter notebooks. Jupyter notebooks are ideal for getting to know python, btw. Jupyter lab is the next iteration of notebooks. It is in beta, but usable. Normally, you still work in notebooks, but you can also start a regular python prompt or a terminal. It starts to look like an IDE: file browser, generic editor, etc. And very nice for data analysis, for instance with a fast CSV browser. He demoed the markdown integration. A source file with an (live updating) output window side by side. And a terminal window at the bottom. Very fancy: there is an extension that works with google docs. So … -
How to Record Video in OpenCV & Python
OpenCV makes it simple to reco... -
Open source activity (April 2018 edition)
It’s time for a post about recently released Django and Python packages by yours truly. Open source activity (April 2018 edition) django-imagefield A more opinionated version of django-versatileimagefield, which keeps the amount of code in production at a minumum has a strong preference towards generating images in advance, not on demand (which means it stays fast whatever the storage backend may be) fails early when invalid images are uploaded instead of crashing the website later speckenv speckenv helps keep configuration and secrets in the environment, not in the code. It knows how to parse .env files, and how to read structured values from the environment (not only strings, but also bools, lists, dictionaries – in short, Python literals) django-curtains It is useful to give clients protected access to a website in development. django-curtains allows keeping the work in progress secret by only allowing authenticated access (using either Django’s authentication system, or HTTP Basic authorization) Of course, activity on older projects hasn’t ceased either. New releases of xlsxdocument, django-user-messages, django-http-fallback-storage, html-sanitizer, feincms3, django-cabinet, django-authlib, etc. are available on PyPI. -
OpenCV & Python: How to Change Resolution or Rescale Frame
Let's assume you're working of... -
Reliable Way To Test External APIs Without Mocking
Let us write a function which retrieves user information from GitHub API. import requests def get_github_user_info(username): url = f'https://api.github.com/users/{username}' response = requests.get(url) if response.ok: return response.json() else: return None To test this function, we can write a test case to call the external API and check if it is returning valid data. def test_get_github_user_info(): username = 'ChillarAnand' info = get_github_user_info(username) assert info is not None assert username == info['login'] Even though this test case is reliable, this won't be efficient when we have many APIs to test as it sends unwanted requests to external API and makes tests slower due to I/O. A widely used solution to avoid external API calls is mocking. Instead of getting the response from external API, use a mock object which returns similar data. from unittest import mock def test_get_github_user_info_with_mock(): with mock.patch('requests.get') as mock_get: username = 'ChillarAnand' mock_get.return_value.ok = True json_response = {"login": username} mock_get.return_value.json.return_value = json_response info = get_github_user_info(username) assert info is not None assert username == info['login'] This solves above problems but creates additional problems. Unreliable. Even though test cases pass, we are not sure if API is up and is returning a valid response. Maintenance. We need to ensure mock responses are … -
Reliable Way To Test External APIs Without Mocking
Let us write a function which retrieves user information from GitHub API. import requests def get_github_user_info(username): url = f'https://api.github.com/users/{username}' response = requests.get(url) if response.ok: return response.json() else: return None To test this function, we can write a test case to call the external API and check if it is returning valid data. def test_get_github_user_info(): username = 'ChillarAnand' info = get_github_user_info(username) assert info is not None assert username == info['login'] Even though this test case is reliable, this won't be efficient when we have many APIs to test as it sends unwanted requests to external API and makes tests slower due to I/O. A widely used solution to avoid external API calls is mocking. Instead of getting the response from external API, use a mock object which returns similar data. from unittest import mock def test_get_github_user_info_with_mock(): with mock.patch('requests.get') as mock_get: username = 'ChillarAnand' mock_get.return_value.ok = True json_response = {"login": username} mock_get.return_value.json.return_value = json_response info = get_github_user_info(username) assert info is not None assert username == info['login'] This solves above problems but creates additional problems. Unreliable. Even though test cases pass, we are not sure if API is up and is returning a valid response. Maintenance. We need to ensure mock responses are … -
Resetting Django Migrations
Resetting Django Migrations Sometimes you’ll need to reset your Django migrations or simply make a clean up. This process can be performed very easily in many situations but can also become complex if you have a big number of migration files and database tables .In this tutorial I’ll show you a few options to enable you to effectively delete or reset your Django database migrations without affecting the working of your project. Case 1: Dropping the whole database is not a problem In this case we’ll describe how to rest database migrations in a situation where you don’t actually have a problem deleting or dropping your whole database. Deleting migration files Start by deleting each migration file inside your project apps except for the Python module file init.py You can use a bash script in Unix based operating systems to do this quickly: find . -path "*/migrations/*.py" -not -name "__init__.py" -delete find . -path "*/migrations/*.pyc" -delete The first line looks for Python files (migration files) inside migrations folder in each project’s app except for init.py then delete them. The second line looks for the Python compiled version of the previous migration files and delete them. Dropping the database The next … -
Adding dynamic dns hostnames to Django INTERNAL_IPS
While some of us are blessed with fixed IP addresses, some ISPs don’t offer this option unfortunately. If you’re using the Django INTERNAL_IPS setting to show or hide the debug toolbar this can be a problem. Luckily the Django settings files are pure Python and it’s easy enough to work around this issue with a few lines of code: import sys INTERNAL_IPS = [ # Any fixed internal IPs ] INTERNAL_HOSTS = [ # Some dyndns host # Some no-ip host # etc... # For example: 'wol.ph', ] # Only enable when running through the runserver command if 'runserver' in sys.path: import dns.resolver resolver = dns.resolver.Resolver() # Set the timeout to 1 so we don't wait too long resolver.timeout = 1 resolver.lifetime = 1 for host in INTERNAL_HOSTS: try: # For ipv6 you can use AAAA instead of A here ip = resolver.query(host, 'A')[0].address INTERNAL_IPS.append(ip) except dns.exception.Timeout: print('Unable to resolve %r, skipping' % host) Note that the code above uses dnspython which can be installed through pip. Link to this post! -
Single Page Apps with Flask and Angular 4|5 Tutorial Series
In this tutorial series we'll be using Python, Flask, SQLAlchemy and Angular 5 to build a modern RESTful web application with an architecture that consists of a front-end application with Angular 5 and a back-end REST API using Flask. The application we'll be building is a simple CRUD (Create, Read, Update and Delete) system for managing customers. This can be further extended to build a fully featured Customer Management System by implementing more use cases. The first tutorial will cover how to set up both the back-end and front-end applications and how to install the necessary dependencies. Series Tutorials Introduction and Setting Up Flask and Angular (current tutorial) Using the SQLAlchemy ORM to Create Database Models Routing and Navigation with The Angular Router Angular State Management with ngrx Integrating Angular with Flask Building a REST API with Flask Consuming The REST API with Angular HttpClient Adding JWT Authentication Getting Ready for Production and Deployment Before we dive into the practical steps let's briefly go over the technologies we are going to use in this tutorial: Flask: a Python micro-framework for building web applications that's know to be light and scalable. Flask has many features such as: It's easy to setup … -
How to Integrate Highcharts.js with Django
Highcharts is, in my opinion, one of the best JavaScript libraries to work with data visualization and charts out there. Even though Highcharts is open source, it’s a commercial library. It’s free for use in non-commercial applications though. In this tutorial we are going to explore how to integrate it with a Django project to render dynamically generated charts. In relation to drawing the charts and rendering it to the client, all the hard work is done by Highcharts at the client side. The configuration and setup is pure JavaScript. The main challenge here is on how to translate the data from your backend to a format that Highcharts will understand. This data may come from a database or an external API, and probably is represented as Python objects (like in a QuerySet), or simply be represented as Python dictionaries or lists. Generally speaking, there are two ways to do it: Via the request/response cycle, using Python and the Django template engine to write the JavaScript code directly in the template; Via an async request using AJAX, returning the data in JSON format. The first option is like a brute force and in many cases the easiest way. The second … -
The Moment
When social media was small, it was a bubble: a quiet conversation among friends. It was nice, but not particularly challenging. Pleasant, but maybe not useful. Then there was a moment. A short one. Social media was perfect. The bubble popped, and suddenly there were voices from outside the bubble. But it was still small, still manageable, not yet the all-consuming force it is today. I felt comfortable sharing all sorts of things. -
Testing Django applications in 2018
I spend a lot of time writing Django applications. At each of my last three jobs I worked with Django, and I’m the primary maintainer of quite a few open-source Django applications. Which means I’ve written a lot of tests for code that uses Django. And although Django provides a lot of useful tools for testing, there are areas where it doesn’t prescribe or even suggest how you should do things, and over the years ... Read full entry -
Django Rest Framework User Authentication Tutorial
Build a Django API with login, logout, and signup token-based endpoints. -
Writing A New Blog Engine
Since around February of 2012, I've been publishing this blog as a static HTML site using Pelican. The experience was pretty good, but over time I ran into a few problems with the fact that I never upgraded the site to match current versions of Pelican. Which meant the following: My RSS feed didn't follow the modern W3C RSS/Atom specifications. So I haven't been published in Planet Python in years. As time went by, upgrading to modern Pelican became harder and harder. And trying to get it to work wasn't much fun either. So I started looking at other options. My requirements: All my old page links needed to work. I didn't want to have to cook up some kind of redirect system. I wanted to be able to make customizations without fighting through a complex extension system. Theming needed to be easy. Markdown needed to be supported. While I like RestructuredText, the honest truth is that I can pour out my thoughts faster with Markdown. With those requirements in mind, I got started reviewing other tools. I tried a bunch of options (Hugo, Lektor, Pelican again, etc) but none of them met my requirements to the degree I wanted. … -
Python Asyncio Web Scraping
A lot of Python programs are S... -
What to use at my new job: mac or linux?
First things first: "new job"? Yes, I'm switching jobs. I'll give more details once my new job actually has a website. They're quite a new company :-) Question: "new job" means "new laptop". And I'm wondering if you could give me some input on whether to ask for a mac or a linux laptop. What is currently the best choice? (Yes, windows is totally out). For the last 15 years I've only used macbooks, apart from the first year at my current job. After that first year I managed to switch back to apple. Before those 15 years I used linux desktops (slackware, suse, debian, mandrake) for 10 years. And before that windows 3.1. Some of my thoughts: The three year old macbook pro 15" I have now is such a wonderful machine. Well made. Great screen. If someone looks over your shoulder, they actually see what is on your screen. On many laptops, you have to sit right in front of it, because the screen starts to darken if you look at it from an angle. Not so on my macbook. Oh, and the sharpness of the characters.... I'm someone who really enjoys such beauty. The best trackpad there … -
Facebook Messenger Bot with Django
We still have to do a live tes... -
A Multiple Model Django Search Engine
The Django [ORM](https://en.wi... -
Making mistakes
A couple weeks ago when I was writing what became pwned-passwords-django, I tweeted about a weird issue I was seeing when running the tests for part of it. As it turned out, I’d overlooked something important, and the fix ended up being a one-line change. But that kicked off a little side discussion about the importance of being open about these kinds of “trivial” mistakes; it’s easy for newer or less-confident programmers to do something ... Read full entry -
Thoughts on Publishing a Technical Book (Part 2)
What I've learned writing and self-publishing a book on web development with Django. -
Tutorial: Django REST with React (Django 3 and a sprinkle of testing)
I gave a talk: "Decoupling Django with Django REST and React" at Pycon Italy X in Florence. Slides here! Django REST with React: what you will learn In the following tutorial you’ll learn: how to build a simple Django REST API how to structure a Django project with React Django REST with React: requirements To follow along with the tutorial you should have: a basic understanding of Python and Django. a basic understanding of JavaScript (ECMAScript 2015) and React. a newer version of Node.js installed on your system Ready? Let's get started! Django REST with React: setting up a Python virtual environment, and the project First things first make sure to have a Python virtual environment in place. Create a new folder and move into it: mkdir django-react && cd $_ Once done create and activate the new Python environment: python3 -m venv venv source venv/bin/activate NOTE: from now on make sure to be always in the django-react folder and to have the Python environment active. Now let's pull in the dependencies: pip install django djangorestframework When the installation ends you're ready to create a new Django project: django-admin startproject django_react . Now we can start building our first Django …