Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Django and nginx file proxy - part two
You have our application up and running but there is a problem. You don't want the user to see that your media files are served from media url. How to fix that? This blog post will tell you one of the solutions. Let's go! How to hide urls from the … -
PyGrunn: creating abstraction between consumer and datastore - Marco Vellinga
(One of my summaries of a talk at the 2017 PyGrunn conference). Marco Vellinga started his talk with train-related analogies and photos, which I of course liked. They have a big monolythic application. Like a big train: move one part (a locomotive) and the rest also moves. In an application you don't want that: if you make a change in one part of the application, you don't want to have to change lots of other parts. Decoupled is better. With abstraction, for instance a user object with username, password and billing information, you can change the backend (splitting user and billing info, for instance) without needing to change anything in the frontend. They created an abstraction layer containing core logic validation error handling data contracts There isn't really support for such a layer in Django, so you have to build it yourself. This was a lot of work and took a couple of tries to get it right. They used Marshmallow for json serialization. For serializing queries (the equivalent of django's 'Q' objects) they build filterql. For (API) versioning they build their own function/class decorator to mark functions as "version 1" or "version 2". They're calling it versionary. It is … -
PyGrunn: django localization ('l10n') - Cees van Wieringen
(One of my summaries of a talk at the 2017 PyGrunn conference). What is L10N (=LocalizatioN)? You have multiple terms like that. Translation (t9n): just translating text. Localization: the adaption of a product or content to a specific language or culture. This includes localization. But also date/time formats. Currency. Units of measurement. Number formatting. i18n (internationalization) is preparing your product for localization. Using unicode. Django translation toolkit. Translation in django. Enable it in your settings.py by adding I18N = True. And use _() around your strings. Where does the underscore function come from? It is a gettext function: from django.utils.translation import gettext as _ There are multiple gettext variants (gettext_lazy, for instance), so he thinks it best to use the full function name instead of the customary underscore. Sometimes you have terms that can be translated in multiple ways. Django has a gettext function that allows adding a "domain". "Spring" can be a "season", but it can also be a "mechanical part". For translating models, there are multiple django apps that handle it and that store the various translations in the database. But there is none that tries to grab the translation from the existing gettext *.po translation files. So … -
PyGrunn: looking at molecules using python - Jonathan Barnoud
(One of my summaries of a talk at the 2017 PyGrunn conference). He researches at fat molecules. He applies simulation to molecules. F = m * a (+ some much more elaborate formulas). With an elaborate simulation, he was able to explain some of the properties of fat (using a "jupyter" notebook). His (python) workflow? First you need to prepare the simulation. He did have (of did build) a simulation engine. The preparation takes text files with the following info: Topology. Initial coordinates. Simulation parameters. Those text files are prepared and fed to the simulation engine. What comes out is a trajectory (a file with the position and direction and speed of every single molecure for all timesteps). The next step is analysis. A problem here is that various simulation engines export different formats.... Similar problem with the input, btw... Luckily we've got python. And for python there are a huge amount of libraries. Including "MDAnalysis" (http://www.mdanalysis.org/) , a library that can use these trajectory files. The output: python numpy arrays. Nice! This way you can use the entire python scientific stack (numpy, scipy, etc) with all its power. Numpy? Made for matrices. So you can work with your entire … -
PyGrunn: deep learning with tensorflow ("Trump tweet generator") - Ede Meijer
(One of my summaries of a talk at the 2017 PyGrunn conference). He originally used a java solution for 'machine learning', but that didn't work very comfortably. He then switch to tensorflow, written in python. Machine learning is learning from data without being explicitly programmed. You feed the computer lots of data and it learns from that. Some examples of the techniques used: linear regression, logistic regression, decision trees, artifical neural networks and much more. Artificial neural networks are what tensorflow is about. A normal neural network has "input layer", a "hidden layer" and "an output layer". The nodes in the three layers are connected. The neural net tries to learn the "weights" of the connections. Deep learning means you have neural networks with multiple hidden layers. Often it deals with features at different levels of abstractions. Images that have to be recognized can be cut in to several pieces of different sizes and fed to the net as those parts, but also as the full image. Training a model often works wit minimizing error by using the "gradual descent" method. Tensor flow? What are tensors? Well, a 0D tensor is a scalar, 1D a vector, 2D a matrix, etc. … -
PyGrunn: google machine learning APIs for python developers - keynote from Google Cloud
(One of my summaries of a talk at the 2017 PyGrunn conference). Lee Boonstra and Dmitriy Novakovskiy gave the keynote. Python at google. Python is widely used at google, it is one of its official languages. It is integral to many of the google projects, for instance youtube and 'app engine'. And lots of open source libraries. Every API has its own python client. Google for python developers. What can you do as a python programmer on google? Google cloud platform. It consists of many parts and services that you can use. You can embed machine learning services like tensorflow, speach API, the translation API, etc. Serverless data processing and analytics. Pub/Sub, bigquery (map/reduce without needing to run your own hadoop cluster), etc. Server stuff like kubernetes, container services. Machine learning. There have been quite some presentation on this already. Look at it like this: how do you teach things to your kids? You show them! "That is a car", "that is a bike". After a while they will start learning the words for the concepts. Machine learning is not the same as AI (artificial intelligence). AI is the process of building smarter computers. Machine learning is getting the computer … -
PyCon tutorials, day one
11:39: This morning’s tutorial was on the OpenAPI specification, née Swagger. I wasn’t satisfied with this one. Much of it was a series of, “Look at X. Now make the changes in your file that are like X,” exercises. At times it was almost pattern-matching in nature. I wish it had spent more time delving under the covers. However, I extracted some useful nuggets of information. In previous jobs, my teams have used Django Rest Framework for the API implementation and django-rest-swagger to generate live documentation. Time-consuming, “now you have two problems,” issues were always encountered. Django Rest Framework is a framework in and of itself, and so it’s a framework wrapping another framework. Add in django-rest-swagger and you almost have three frameworks, with multiple points of impedance mismatch to each other. (This is understandable. It’s amazing it all works as well as it does.) The process of keeping it all happy took time, and the end results were problematic. In at least two instances, my team chose to lower the API’s quality in order to keep using DRF. One pain point was nested parameters; they (particularly modifiable ones) were extraordinarily painful. So, back to the tutorial. Coffee Meets Bagel’s … -
Using Python and Google Docs to Build Books
When I started my latest fiction book, The Darkest Autumn, I wrote out the chapters as individual files. I did it in a text editor (Sublime) and saved the files to a git repo. The names of the files determined their order, chapters being named in this pattern: the-darkest-autumn $ tree . ├── 01_Beginnings.md ├── 02_Town_of_Ravenna.md ├── 03_Walls_of_Ravenna.md As the book developed I thought about moving it to Scrivener. If you don't know, Scrivener is an excellent tool for writing. It encourages you to break up your work into chapters and scenes. The downside is that Scrivener is complex (I want to write, not figure out software) and Scrivener isn't designed for simultaneous collaboration. The latter issue is a very serious problem, as I like to have others review and comment on my writing as I go. What I really wanted to do is combine the chapter breaking of Scrivener with the simplicity and collaboration of Google Docs. Preferably, I would put the book chapters into Google Docs as individual files and then send invites to my editor, wife, and my beta readers. By using Google Docs I could ensure anyone could access the work without having to create a … -
Using Python and Google Docs to Build Books
When I started my latest fiction book, The Darkest Autumn, I wrote out the chapters as individual files. I did it in a text editor (Sublime) and saved the files to a git repo. The names of the files determined their order, chapters being named in this pattern: the-darkest-autumn $ tree . ├── 01_Beginnings.md ├── 02_Town_of_Ravenna.md ├── 03_Walls_of_Ravenna.md As the book developed I thought about moving it to Scrivener. If you don't know, Scrivener is an excellent tool for writing. It encourages you to break up your work into chapters and scenes. The downside is that Scrivener is complex (I want to write, not figure out software) and Scrivener isn't designed for simultaneous collaboration. The latter issue is a very serious problem, as I like to have others review and comment on my writing as I go. What I really wanted to do is combine the chapter breaking of Scrivener with the simplicity and collaboration of Google Docs. Preferably, I would put the book chapters into Google Docs as individual files and then send invites to my editor, wife, and my beta readers. By using Google Docs I could ensure anyone could access the work without having to create a … -
Runtime Permissions in Android: The What, Why, and How
It’s great if you are dealing with an application from a famous and well-known developer whom you can trust. But imagine that you want to install a music player that requires, let’s say, information about your location. It looks very suspicious, doesn’t it? The post Runtime Permissions in Android: The What, Why, and How appeared first on Distillery. -
Kubernetes Health Checks in Django
Kubernetes Health Checks in Django -
Django and nginx file proxy - part one
In this blog post series, I will show you how to use Nginx for hiding download urls. Django will serve us as a backend. Let's go! In this series I will build simple web application - user upload a file via api and then she/he wants to download it. But … -
Fastest Redis configuration for Django
I have an app that does a lot of Redis queries. It all runs in AWS with ElastiCache Redis. Due to the nature of the app, it stores really large hash tables in Redis. The application then depends on querying Redis for these. The question is; What is the best configuration possible for the fastest service possible? Note! Last month I wrote Fastest cache backend possible for Django which looked at comparing Redis against Memcache. Might be an interesting read too if you're not sold on Redis. Options All options are variations on the compressor, serializer and parser which are things you can override in django-redis. All have an effect on the performance. Even compression, for if the number of bytes between Redis and the application is smaller, then it should have better network throughput. Without further ado, here are the variations: CACHES = { "default": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/0', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", } }, "json": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/1', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", "SERIALIZER": "django_redis.serializers.json.JSONSerializer", } }, "ujson": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', 'redis://127.0.0.1:6379') + '/2', "OPTIONS": { "CLIENT_CLASS": "django_redis.client.DefaultClient", "SERIALIZER": "fastestcache.ujson_serializer.UJSONSerializer", } }, "msgpack": { "BACKEND": "django_redis.cache.RedisCache", "LOCATION": config('REDIS_LOCATION', … -
Distillery QA Team Makes the Finals in the 2nd Open EU Championship as They Prepare for the Grand Final in Poland
The Distillery Quality Assurance team competed in the 2nd Open European Championship for Web and Mobile Specialists and made it to the finals! Anna Varetsa, QA Manager, and Ekaterina Golubeva, QA Specialist, excelled at solving the different challenges presented by Jury Board, a selection of highly qualified and experienced industry professionals. The results were revealed on April 28th, after two weeks of nervously waiting for the outcome. The post Distillery QA Team Makes the Finals in the 2nd Open EU Championship as They Prepare for the Grand Final in Poland appeared first on Distillery. -
Django Version Viewer Announcement
Presenting a new open source plugin: The Django Version Viewer! Imaginary Landscape is pleased to present a helpful new plugin for the Django Web Framework - The Django Version Viewer! If you find yourself frequently switching between projects or perhaps looking at a codebase you haven’t seen in some time, it’s often useful to know the contents of the project’s virtual environment. Typically this information is only accessible either by examining your project’s requirements.txt (which may or may not accurately reflect what is installed in your environment) or by accessing your app’s server directly to examine the app’s virtual environment. The Django Version Viewer is a handy tool that exposes this information in a handful of ways: Through the Django Admin panel The plugin conveniently exposes a link at the top of your admin index pane to expose a simple drop down listing all of your project dependencies. Through a DjangoCMS Toolbar Menu Item If Django CMS is installed, a new menu item will be added to the CMS Toolbar Page Menu that will allow opening the version viewer popup. Through an API call Alternatively, you can make a request to a URL at www.yourAppsDomain/django_version_viewer to retrieve a list of … -
Django and working with large database tables
The slides from my presentation for the Django Stockholm Meetup group. Contains small comparison between MySQL and PostgreSQL in terms of performance when modigying the tables structure. Django and working with large database tables from Ilian Iliev -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. In a production environment you can setup supervisor to launch your simple SMTP server. With a simple configuration like this: [program:smtp_server] command = /home/ubuntu/.virtualenvs/venv/bin/python /home/ubuntu/mail_gateway/smtp_server.py user = root Then try to send an email to your server, if the server public IP has been associated to a DNS record yourserver.example.com you can use something like test@yourserver.example.com as a recipient, and you should see the incoming email in the logs. Have fun with emails! -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. https://gist.github.com/baxeico/d5d6f855774ba9f72e11909648a46cae In a production environment you can setup supervisor to launch your simple SMTP server. With a simple configuration like this: [program:smtp_server] command = /home/ubuntu/.virtualenvs/venv/bin/python /home/ubuntu/mail_gateway/smtp_server.py user = root Then try to send an email to your server, if the server public IP has been associated to a DNS record yourserver.example.com you can use something like test@yourserver.example.com as a recipient, and you should see the incoming email in the logs. Have fun with emails! The post A simple Python email gateway appeared first on Augusto Destrero - Freelance developer and sysadmin. -
A simple Python email gateway
Say you have a Django web application that you want to integrate with emails to make it possibile to send data and files to your web application over SMTP. The good news is that Python has a simple SMTP daemon in the standard library, together with modules to parse emails. Let’s see how to create a simple email gateway on top of these tools. Using the following Python module you can setup a simple pure Python email server on port 25, parse the incoming emails and make a multipart/form-data POST HTTP request to your web application. You’ll need the awesome Requests library for the HTTP request to work. import smtpd import asyncore import logging import email from email.header import decode_header import requests logger = logging.getLogger(__name__) class CustomSMTPServer(smtpd.SMTPServer): def process_message(self, peer, mailfrom, rcpttos, data): try: logger.debug('Receiving message from: %s:%d' % peer) logger.debug('Message addressed from: %s' % mailfrom) logger.debug('Message addressed to: %s' % str(rcpttos)) msg = email.message_from_string(data) subject = '' for encoded_string, charset in decode_header(msg.get('Subject')): try: if charset is not None: subject += encoded_string.decode(charset) else: subject += encoded_string except: logger.exception('Error reading part of subject: %s charset %s' % (encoded_string, charset)) logger.debug('Subject: %s' % subject) text_parts = [] attachments = {} logger.debug('Headers: %s' … -
Django webpacker - A compression tool to bundles css, js files
In production, it's challenging to developers to improve the performance of an application. One of the technique to improve performance is compressing the CSS, js files into a single file. Django webpacker provides you a single compressed CSS, js file to the page. Instead of generating compressed files for every request in production, we're generating compressed css, js files, updates to html file in the development only. Django webpacker really helps you to generate these compressed files with webpack Django webpacker is a django compressor tool which bundles css, js files to a single css, js file with webpack and updates your html files with respective css, js file path with a single management command.. Django webpacker uses webpack, npm, aws s3 services to provide its features in an effective way. Webpack is a system for bundling the front-end assets of a web application into a few static resources that represent the bulk of the application's CSS, Javascript, and images. npm makes it easy for JavaScript developers to share the code that they've created to develop applications. With the help of packages(A reusable code) will generate compressed css , js files. Django webpacker also supports django-storages to load compressed static files from AWS … -
How to use nested formsets in django
Django Formsets manage the complexity of multiple copies of a form in a view. By using formsets you can know how many forms were their initially, which ones have been changed, and which ones should be deleted. Similar to Forms and Model Forms, Django offers Model Formsets, which simplify the task of creating a formset for a form that handles multiple instances of a model. Django also provides inline formsets which can be used to handle set of objects belong to common foreign key. In the below example models, we can write a inline-formset for handling all children for a parent or all addresses of a child. # models.py class Parent(models.Model): name = models.CharField(max_length=255) class Child(models.Model): parent = models.ForeignKey(Parent) name = models.CharField(max_length=255) class Address(models.Model): child = models.ForeignKey(Child) country = models.CharField(max_length=255) state = models.CharField(max_length=255) address = models.CharField(max_length=255) # forms.py from django.forms.models import inlineformset_factory ChildrenFormset = inlineformset_factory(models.Parent, models.Child, extra=1) AddressFormset = inlineformset_factory(models.Child, models.Address, extra=1) By using above formsets you can handle all children for a parent in one page and can handle all addresses of a child in other page. But if you want to allow the users to add/edit all children along with addresses, all in a single page, then … -
What's great about Django girls to inspire women into programming
Now a days programming is a hot topic across the globe. Django girls are inspiring more women into programming. Which is an international community to teach programming stuff and to getting more women into tech.They run free programming workshops all over the world and offering several opportunities for women to develop coding skills and to make the industry more balanced. It's important to encourage beginners/non-programmers that programming is not hard as they think. Where does it start? It started in 2014 by two women, Ola Sitarska and Ola Sendecka. Their main aim is to fall women into the programming world. Django girls workshops are free and till now they have conducted the events Over 67 countries around the world, where women can learn extensive programming stuff and they will get an awesome programming experience at the first time. It was not only a programming session but something as exciting. What They Do? They will guide all the participants to learn how to code using Django. They will start from the ground level. So, no one left behind. There are so many mentors & volunteers to guide everyone in each step. A participant can move as fast or as slow as … -
How to convert xml content into json using xmltodict
xmltodict is a python package which is used to convert XML data into JSON and vice versa. Within a less span of time, we can also process large amounts of data and get the desired results. Installation: pip install xmltodict It supports both python2.7, 3.4 versions Advantages over other packages: 1. No need to process each node in XML document to get the results 2. Easy to convert complex XML data into JSON 3. It's user-friendly Converting XML to a dictionary: >>> import xmltodict >>> import json >>> json.loads(json.dumps(xmltodict.parse(''' <root> <persons city="hyderabad"> <person name="abc"> <name age="50" mobile="789" /> </person> </persons> <persons city="vizag"> <username></username> <person name="xyz"> <name age="70" mobile="123" /> </person> </persons> </root> '''))) {u'root': {u'persons': [{u'@city': u'hyderabad', u'person': {u'@name': u'abc', u'name': {u'@mobile': u'789', u'@age': u'50'}}}, {u'@city': u'vizag', u'person': {u'@name': u'xyz', u'name': {u'@mobile': u'123', u'@age': u'70'}}}]}} Converting dictionary to XML: >>> xmltodict.unparse({u'root': {u'persons': [{u'@city': u'hyderabad', u'person': {u'@name': u'abc', … -
Django deployments without downtime
This post describes a deployment and maintenance method for Django projects that was designed with scalability in mind. The goal is to push new releases into production at any moment with minimal or no downtime. Upgrades can be performed with unprivileged access to the production server, and rollbacks are possible. I use Gunicorn, Fabric and Supervisord in the examples. -
Stable Django deployments without downtime
This post describes a deployment and maintenance method for Django projects that was designed with scalability in mind. The goal is to push new releases into production at any moment with minimal or no downtime. Upgrades can be performed with unprivileged access to the production server, and rollbacks are possible. I use Gunicorn, Fabric and Supervisord in the examples.