Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Storing Passwords in a Highly Parallelized World
Why “Use bcrypt.” is not the best recommendation (anymore). -
Modified Preorder Tree Traversal in Django
Hierarchical data are everywhere, from product catalogs to blog post comments. A classic example is the tree of life, where kingdoms are subdivided into a hierarchy of phylum and class down to genus and species. What if you wish to store this data in a database table, which is inherently flat? Databases do not natively store hierarchies, so you need to work around that. MPTT, or modified preorder tree traversal, is an efficient way to store hierarchical data in a flat structure. It is an alternative to the adjacency list model, which is inefficient. This post will cover the MPTT approach, examine its tradeoffs, then explore django-mptt, which is a package for adding MPTT to your Django models. MPTT Structure The MPTT approach adds a lft and a rght attribute to a model, which lets you easily determine parent-child relationships. See the example tree with lft and rght values below (GlobalCorp, for example, has a lft value of 1 and a rght value of 20). The dotted line in the image above shows the path taken to calculate child relationships within the tree. The model attributes allow you to do SQL queries such as the one below to get all … -
Laying the Channels Groundwork
Progress on Django Channels, and working out how the different parts will talk to each other. I've been working on Django Channels over the last few weeks, mainly on settling things down into more of a final shape than they've been up to this point - and I'd like to discuss some of the progress that's happening, and the reasoning behind it. The first major step was deciding on the plan of how to distribute the Channels code. We could have just put it all in 1.10, and left other Django versions out in the dark, but that would leave a massive gap, and a long time until many existing sites could start using it. Of course, I would only have considered bringing it to earlier versions of Django if it wasn't prohibitively difficult to do; thankfully, the Channels design fits cleanly around the existing Django abstraction, which is part of why I'm so fond of it. The plan, then, is to ship Channels as two main separate parts: django.channels, which runs Django consumers and view code when messages arrive on routed channels. Will ship as a pluggable app called channels for earlier Django versions. daphne, a HTTP/WebSocket protocol server … -
Chained Queries in Django
Writing clean code is as important as getting things working. In this article I'll show you three different ways to handle queries in Django; from worst to best. -
GeoDjango and Leaflet.js- part one
Today is the day, when I will start my blog. I would like to publish new posts once a week. The comments and feedback will be pretty much appreciated as I have just started blogging in English. I assume that the Reader knows how to run basic Django application. However, if this is not the case, I encourage you to start here. So, let's start: First of all, what is GeoDjango? From official geodjango documentation: GeoDjango intends to be a world-class geographic Web framework. Its goal is to make it as easy as possible to build GIS Web applications and harness the power of spatially enabled data. To run GeoDjango you have to install some additional packages. In ubuntu you can easily do this way: $ sudo apt-get install binutils libproj-dev gdal-bin Gdal-bin package is not necessary but it's very helpful so I encourage to install it. Then you need to choose what database you will use with your GeoDjango project. I personally choosen PostgreSQL with PostGIS because it support all of the GeoDjango features according to this table. After starting project, and making new application add few lines to your settings.py: DATABASES = { 'default': { 'ENGINE': 'django.contrib.gis.db.backends.postgis', 'NAME': … -
GeoDjango and Leaflet.js- part one
Today is the day, when I will start my blog. I would like to publish new posts once a week. The comments and feedback will be pretty much appreciated as I have just started blogging in English. I assume that the Reader knows how to run basic Django application. However, if this is not the case, I encourage you to start here. So, let's start: First of all, what is GeoDjango? From official geodjango documentation: GeoDjango intends to be a world-class geographic Web framework. Its goal is to make it as easy as possible to build GIS Web applications and harness the power of spatially enabled data. To run GeoDjango you have to install some additional packages. In ubuntu you can easily do this way: $ sudo apt-get install binutils libproj-dev gdal-bin Gdal-bin package is not necessary but it's very helpful so I encourage to install it. Then you need to choose what database you will use with your GeoDjango project. I personally choosen PostgreSQL with PostGIS because it support all of the GeoDjango features according to this table. After starting project, and making new application add few lines to your settings.py: DATABASES = { 'default': { 'ENGINE': 'django.contrib.gis.db.backends.postgis', 'NAME': … -
GeoDjango and Leaflet.js- part one
How to setup basic GeoDjango application first part. -
What We Open Sourced in 2015: A New Year's Retrospective
This year we had the pleasure of building a number of unique solutions for several organizations. In addition, we had the support of these clients to open source the tools we built. By open sourcing our work, we enable others to use, replicate, and even improve upon the tools we’ve created. With the year coming to a close, we thought we would take a moment to reflect on the projects we’ve had the opportunity to open source this year. Commodity Tracking System, International Rescue Committee (IRC) On behalf of the IRC, Caktus developed significant upgrades to a previous iteration of the Commodity Tracking System (CTS). The fedex-style systemove enables IRC employees to reliably track the shipment of humanitarian aid to Syrian refugees across Jordan, Turkey, and Syria. To help handle increasing volume of shipments the IRC oversees, we created a cohesive web-based application that links information from an array of technologies to track and verify delivery of aid shipments. The enhanced application enables the IRC to quickly create and customize large shipments, generate custom barcodes for each shipment for scanning purposes, and pinpoint these shipments through a mobile data collection system. Finally, the system enables the IRC to easily report … -
Preliminary Django 1.9 support
Work is underway to make django-fluent compatible with Django 1.9. The following packages have been upgraded to support Django 1.9: django-fluent-blogs django-fluent-contents django-fluent-comments django-fluent-dashboard django-any-urlfield django-parler django-polymorphic (with the exception of the admin) django-polymorphic-tree django-slug-preview Work for django-fluent-pages is still underway. A 1.0b1 release is issued for testing. However, it does have issues with the admin and page moving that still need to be fixed. If you can help, please join in! New features A slug preview field, that shows which part of the URL will get the slug. blogs: support for django-categories-i18n, as replacement for django-categories. pages: translations for the textfile pagetype. pages: multiple fallback languages in URLs. pages: draft CSS class in the menu, and "is_child_active" variable for templates. contents: text-filters, to perform post-processing filters on textual content (e.g. apply smartypants, add soft-hyphenation). contents: API for search engine integration. contents: a debug toolbar panel for locating content. -
Hosting Django Application with Nginx and UWSGI
Setting up Django: Django is a python based web- application development framework. Setting up a sample app and running it as easy as pie. In here we will use 1. VIRTUALENV: a tool to create isolated Python environments. If it is not previously installed you can install it using sudo apt-get install python-virtualenv 2. PIP: a tool for installing python packages which if not previously available can be installed using sudo apt-get install python-pip Since we now have all the required packages for creating a Django Application. we need to Create Virtualenv virtualenv [virtualenv-name] Activate the virtualenv and install Django in it source /bin/activate pip install django Start a Django project django-admin.py startproject Deploying your sample Project: navigate into the project and you can find manage.py file. There you need to execute python manage.py runserver IP-Address and Port are not mandatory and by default, it runs on 127.0.0.1:8000 if you open http://127.0.0.1:8000 in a browser you will have Django welcome page which is enough for our deployment Installing Nginx: Nginx is a web server and like every other web server, it has it Pros and Con's. Nginx was an answer to concurrency issue(handling thousands of concurrent connections) faced in apache and raised to fame. … -
A Python Solution for Making Custom PDFs from HTML
A Python Solution for Making Custom PDFs from HTML -
Reflecting on My Time as Caktus' Open Source Fellow
My name is Ben Phillips and I am Caktus' Open Source Fellow. As my fellowship comes to a close, I wanted to reflect on my time at Caktus and to share my experience and some of what I've learned here. First, however, I should probably share how I ended up here in the first place. Initial Commit Six months ago I was a biologist. That had been the plan all along, after all. It had taken me quite a while. I'd had to work as a barista, stream surveyor, machinist/welder, and mouse husbandry technician and to get a bachelor's degree in biology, but I eventually achieved my goal and found a job as a research technician in a molecular/cell biology lab. Unfortunately, as is often the case with the plans of mice and men (both of which, as a former biologist and mouse technician, I am qualified to speak about), there was a small problem. It slowly became clear to me that this was not what I wanted to do with my life. The research was interesting, but the work was not. I was at something of a loss until one day when, in the middle of a particularly tedious … -
"Complex" and "complicated"
I'm not a native english speaker, so sometimes I need to refresh my memory as to the exact meaning of a word. In this case the exact difference between complex and complicated. I found an english.stackexchange.com page with some nice answers. According to the first answer: Complex is about the number of parts. The more (different) parts, the more complex. Complicated is about how hard/difficult something is. Another answer pointed at the relevant part of the Zen of Python: Simple is better than complex. Complex is better than complicated. So when programming, you can divide up a task or problem into separate parts. The more parts, the more complex. Simple is better than complex, so don't add too many parts if you don't need them. On the other hand, if you want to do everything in one part, it might end up being too difficult, too complicated. Making it more complex (more parts) is better in that case. It is a trade-off. And it takes experience and a bit of Fingerspitzengefühl to make the right trade-off. How to learn it? Talk with more experienced programmers is one way. Another way is to explicitly review your programs yourself with those two … -
Useful commands for PostgreSQL
Some useful PostgreSQL commands that I need from time to time. Execute with postgres user. List all users psql \du Remove a user dropuser USERNAME List all databases psql \list Remove a database dropdb DATABASENAME; Update password of user psql alter user USERNAME with password 'NEWPASSWORD'; Create a database and user, assign user to database createuser -P USERNAME createdb -O USERNAME DATABASENAME Remove all tables of a schema (if all tables in a single schema) drop schema public cascade; create schema public; -
A summary of a year
As 2015 is slowly drawing to an end, I looked back through Evennia's repository to see just what was going on this year. And it turns out it was a lot! I honestly didn't remember some things happened as recently as they did.Note: For those reading this not familiar with Evennia, it's a Python library for creating MUDs (text-based multiplayer games). Making Evennia into a libraryIn February of 2015 we merged what was likely the biggest change happening for a good while in Evennia - the complete refactoring of the Evennia repository into a library. It used to be that when you cloned the Evennia repo, it would come with a pre-made game/ folder where you were supposed to put your custom files. Mixing the stuff you downloaded from us with your own files (which you might want to keep under version control of your own) was not a very clean solution.In the big "library update", we instead created a stand-alone evennia program which you use to create your own "game dir" with pre-created templates. Not only does this allow you to treat the cloned evennia repo as a proper library, you can also use the same evennia install for … -
Caktus CTO Colin Copeland Helps Launch Open Data Policing Website
Today, at Caktus headquarters, CTO and co-founder of Caktus Colin Copeland will stand at a press conference along with activists, police representatives, and elected officials to announce the launch of OpenDataPolicingNC.com. The first site of its kind, OpenDataPolicingNC.com draws on public records to publish up-to-date stop, search, and use-of-force data—broken down by race and ethnicity—for every police department and officer in the state of North Carolina. The volunteer effort, led by The Southern Coalition for Social Justice (SCSJ) and technical leadership by Colin, includes approximately 20 million anonymized data points from 15 years of NC traffic stop data. Colin’s development team for the project included fellow Durham residents, data scientist Andy Shapiro and software engineer Dylan Young. Combined, the team holds over 29 years of software development experience. Working outside their normal, full-time jobs, Colin, Andy, and Dylan worked iteratively to build, test, and open source the new site. In addition, staff attorney of SCSJ, Ian Mance used his understanding of the needs of police agencies, lawyers, and government officials to ensure a user-friendly site. “Traffic stops are the most common way citizens interact with police officers,” said Ian. “This site enables anyone who engages with these issues—whether they be … -
Launching Jazzband
We are all part of the band. Jazzband is a cooperative experiment to reduce the stress of maintaining Open Source software projects. It aims at lowering the barrier of entrance for people willing to step up as maintainers and grants access to GitHub repositories hosted under the Jazzband GitHub organization to everyone who joins. It’s supposed to simplify the process of managing a volunteer project for the original author(s) and makes stepping away easier. Its purpose can most easily be characterized as “relief”: Relief from the exhausting parts of being a maintainer — either of big or small software projects, widely used or only written for one task. Relief of not knowing how to get a fix landed or a feature added as a user. It normalizes the relationship between creators and consumers, between “core committers” and “end users”. There are a few guidelines to follow before deciding to transfer existing projects to the Jazzband or creating a new one like agreeing to the Jazzband’s code of conduct. Some optional features should also be considered to be used by the Jazzband projects to further improve the contribution workflow. I guess some band members will want to take the lead singer … -
slugify() for postgres (almost)
A recent discussion in #django suggested "what we need is a PG slugify function". The actual algorithm in Django for this is fairly simple, and easy to follow. Shouldn't be too hard to write it in SQL. Function `slugify(value, allow_unicode=False)`. * Convert to ASCII if `allow_unicode` is false * Remove characters that aren't alphanum, underscores, hyphens * Strip leading/trailing whitespace * Convert to lowercase * Convert spaces to hyphens * Remove repeated hyphens (As an aside, the comment in the django function is slightly misleading: if you followed the algorithm there, you'd get a different result with respect to leading trailing whitespace. I shall submit a PR). We can write an SQL function that uses the Postgres `unaccent` extension to get pretty close: {% highlight postgresql %} CREATE OR REPLACE FUNCTION slugify("value" TEXT, "allow_unicode" BOOLEAN) RETURNS TEXT AS $$ WITH "normalized" AS ( SELECT CASE WHEN "allow_unicode" THEN "value" ELSE unaccent("value") END AS "value" ), "remove_chars" AS ( SELECT regexp_replace("value", E'[^\w\s-]', '', 'gi') AS "value" FROM "normalized" ), "lowercase" AS ( SELECT lower("value") AS "value" FROM "remove_chars" ), "trimmed" AS ( SELECT trim("value") AS "value" FROM "lowercase" ), "hyphenated" AS ( SELECT regexp_replace("value", E'[-\s]+', '-', 'gi') AS "value" FROM "trimmed" … -
Improving page speed score in Google Page Score test- PART1
About PageSpeed Insights: Google's Page Speed Insights measures the performance of a page for mobile devices and desktop devices. It fetches the url twice, once with a mobile user-agent, and once with a desktop-user agent. The PageSpeed Score ranges from 0 to 100 points. A higher score is better and a score of 85 or above indicates that the page is performing well. Please note that PageSpeed Insights is being continually improved and so the score will change as we add new rules or improve our analysis. PageSpeed Insights measures how the page can improve its performance on: time to above-the-fold load: Elapsed time from the moment a user requests a new page and to the moment the above-the-fold content is rendered by the browser. time to full page load: Elapsed time from the moment a user requests a new page to the moment the page is fully rendered by the browser. 1. time to above-the-fold-load: Above the fold content is some thing that appears on screen's worth, anything that you don't scroll. This is completely related to Designing part, so we'll not concentrate on this in this post. 2. time to full page load: Time to full page load depend upon … -
Celery With Supervisor
Celery: Celery is a task queue with focus on real-time processing,while also supports task scheduling. Task queues are used as mechanisms to distribute work across multiple threads or machines. A task queues input is a unit of work called a task,dedicated worker processes and constantly moniter the queue for new work to perform. Celery communicates via messages using a broker to mediate between workers and clients.To initiate a task client puts a message on the queue, then the broker delivers that message to a worker. Note: You can get more about celery http://celery.readthedocs.org/en/latest/django/first-steps-with-django.html To run celery in virtual environment need to type the following command in your virtual environment export C_FORCE_ROOT="true" Running the worker with supervisor In production you will want to run the worker in the background as a daemon and some times there may be a chance of stopping of celery worker automatically then it should be restarted automatically. To do thes tasks you need to use the tools provided like supervisord. First, you need to install supervisor in your virtualenv and generate a configuration file. $ pip install supervisor $ cd /path/to/your/project $ echo_supervisord_conf > supervisord.conf Next, just add the following section in configuration file: … -
Preserve file names with sorl for better SEO
We use sorl-thumbnail for scaling images by keeping the original one intact. Sorl proven to be a great tool for generating different sized images throughout the website. I presume that you know how to use sorl-thumbnail. Here we see how to preserve filename. Search Engines will give importance to image alt tag content but it definitely has some impact on relevant image file name too. Sorl will generate random names for the thumbnails it will create and as of now, there is no default setting to preserve filename in generating thumbnails. It is possible to have the same name for different sized images as sorl will create thumbnails in random folders, so, there will be no conflict in names. Here is the way to achieve that: We must override the thumbnail backend. So provide THUMBNAIL_BACKEND=’SEOThumbnailBackend’ Now we should create the class that returns our custom filenames. # thumbnailname.py from sorl.thumbnail.base import ThumbnailBackend, EXTENSIONS from sorl.thumbnail.conf import settings from sorl.thumbnail.helpers import tokey, serialize import os.path class SEOThumbnailBackend(ThumbnailBackend): def _get_thumbnail_filename(self, source, geometry_string, options): """ Computes the destination filename. """ key = tokey(source.key, geometry_string, serialize(options)) filename, _ext = os.path.splitext(os.path.basename(source.name)) path = '%s/%s' % (key, filename) return '%s%s.%s' % (settings.THUMBNAIL_PREFIX, path, EXTENSIONS[options['format']]) It’s important … -
Useful commands for MySQL
Some useful MySQL commands that I need from time to time: MySQL List all users select User, Host from mysql.user; Remove a user show grants for 'USERNAME'@'localhost'; revoke all privileges, grant option from 'USERNAME'@'localhost'; drop user 'USERNAME'@'localhost'; List all databases show databases; Remove a database drop database DATABASENAME; Update password of user update mysql.user set password=PASSWORD("NEWPASSWORD") where User = 'USERNAME'; flush privileges; Create a database and user, assign user to database create database DATABASE defaukt character set utf8 collate utf8_general_ci; grant all on DATABASE.* to 'USERNAME'@'localhost' identified by 'PASSWORD'; flush privileges; -
Faster Django Sites with PyPy
This post was created as a collaboration between Lincoln Loop and Maciej Fijalkowski from baroquesoftware.com. In the space of web architecture, the prevalent belief states that "database based applications are I/O bound". While this might have been true in the past, two developments have highly encroached on that. First, memory has been getting bigger and bigger—minimizing the need to access the HD. Second, the database connection layers have been getting increasingly sophisticated—putting more and more pressure on the CPU. For CPU bound Python applications, using PyPy instead of the traditional CPython interpreter can provide massive improvements (5x or more is common). In this post we examine how PyPy can improve the Django ORM's execution times. We built a set of benchmarks using some "simple" queries with the Django ORM. These let us analyze PyPy's performance and warmup stats. Finally, we want to present some future developments to improve the current situation. Let's start with the benchmark description - it's a simple set of queries that were distilled from the BotBot IRC logging system. The original source can be found on GitHub. There is really nothing fancy, just a series of queries to a SQLite database. BotBot uses Postgres in production, … -
Encrypted Postgres Backups
Encrypted Postgres Backups -
Why you should use the Django admin: 9 tips
This writing is inspired by a comment on Reddit concerning my recent post: “The problem is that everyone I speak to seems to think the opposite - that the admin is super-limited, inflexible and hard to customize.” — andybak on Reddit I’m about to break this prejudice right now. The Django admin is a really brilliant piece of software, which can significantly speed up your development. Here are some tips about the Django admin, which I’ve found to be quite useful.