Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Djangocon: open data on the sea shore - Richard Moch
(One of the summaries of a talk at the 2014 djangocon.eu). Richard Moch shows us open sea map, open street map for the sea! Light houses, shipping langes, etc. They're also trying to build up a sea depth map consisting of open data. See http://depth.openseamap.org/ . There's more: http://openweathermap.org/ . Yes, open weather data for current weather and forecasts. There is an open meteo foundation that tries to get it off the ground. Based on this open weather data, you can build apps, for instance http://meteo-parapente.com for showing good locations for doing paragliding. Open weather data? Then you need to measure it. They are working on an open source autonomous weather station. Modern French diesel unit in Kruth in 2007 -
Djangocon: The whys and hows of using django formsets - Shena Priscilla Makini
(One of the summaries of a talk at the 2014 djangocon.eu). Many people don't use or know about django formsets. She google for articles and found "formsets: the worst thing ever" and "formset gotcha: or how to lose your sanity". Ouch. What are form sets for? When you want to use the same form multiple times. For instance for a data grid. A table with one form per row, for instance. Django formsets help you put several of those forms on one page. The example she showed was of changing your "mailman" settings for a couple of mailinglists on one page. It is used in postorius, a new web UI (django) for mailman. It works something like this: SomeFormset = formsets.formset_factory(SomeForm, extra=5) formset = SomeFormset() And in the template it is just: {% formset %} This gives it to you as a table. You could also iterate over it and get the individual forms: {% for form in formset %} ... {% endfor %} (Personal note: oh blast, I didn't know about this feature! I know just the spot where I desperately need to use it. That one app where I do some manual field generation and mess about with … -
Djangocon: purist unit tests are a waste of time - Harry Percival
(One of the summaries of a talk at the 2014 djangocon.eu). Harry Percival, despite what you might guess due to the talk's title is very testing-oriented. Just look at his website http://www.obeythetestinggoat.com/ :-) He demoed a quick addition to a small TODO list app. Using outside-in TDD (test driven development). Outside-in? Just start at the outside (functional test that hits the TODO page). You try to click a not-yet-existing link. You add the link with a dummy url. You adjust the test. And so on. And slowly you dive into more detail. And slowly you get to write more detailed tests. And slowly you descent from the functional test level to the unit test level. A funny term for this is "programming with wishful thinking". "I wished I had a list of TODO items". So then you first mock it and then build it for real. "I wished I had a nice 'List' object". Repeat. Main "vallée de la Doller" French railway station Burnhaupt in 2007. -
Djangocon: the power of natural language - Ilja Bauer
(One of the summaries of a talk at the 2014 djangocon.eu.) Ilja Bauer's talk has as subtitle "from ATDD to lean modeling". ATDD means Acceptance Test Driven Development. Constantine the African was an ancient physician and one of the first ones to use test driven development: he tested people's morning urine manually and could check whether they had diabetes or not. This is however a very manual process. In our test driven development, we can luckily automate a lot. Software development is a communication problem. He showed a picture of the biblical Tower of Babel where people couldn't understand each other anymore because they suddenly spoke different languages. It sometimes seems the same with the way our customers' requirements somehow doesn't quite end up in the software in the right way. Do we really understand each other? A solution can be acceptance test driven development: you write the acceptance tests together with the programmer. The client has a request. You write down the requirements as textual scenarios. You need to somehow convert the text into an automated acceptance test. He currently uses natspec, a plugin for Eclipse that is free for open source projects. You write the software and continuously … -
Djangocon: introduction to user experience design - Meghan Reilly
(One of the summaries of a talk at the 2014 djangocon.eu.) Meghan Reilly is a user experience strategist. In the USA, they're slowly starting to cycle a bit more. And they're still learning how to share the road as cyclists and car drivers. When she's cycling, everybody else seems to be a bad car driver: they're all over the place. When she drives a car, the cyclists seem to be novices that are all over the road. (Personal comment: if you want to have more cyclists on your road, the cycling infrastructure needs to be good. Look for instance at Dutch junction design) She asked "who has been frustrated by bad websites". And afterwards "who ever implemented something in a website in the simplest way possible, thinking 'the user will figure it out or read the documentation'"... What is missing is empathy. Empathy is the ability to understand and share the feelings of the other. In user experience design we're putting the customer at the center. Not ourselves. For many, user experience design will be a totally new territory (just like she's now in new territory, never having visited Europe before). What is user experience? The experience you have, good … -
Djangocon: gamers do REST - Angel Ramboi
(One of the summaries of a talk at the 2014 djangocon.eu.) Angel Ramboi works for a company, demonware that provides services for games, like leaderboards and so. They mostly use REST for communication. Interoperability and scalability are great. The whole web is one big REST API in a way, actually, so using REST is almost a guarantee that you can scale your service and that you can get it robust. Their stack: django 1.6, python 2.7, mysql 5.6 (sharded), CentOS and apache+mod_wsgi. So nothing really fancy and exotic. They take reliability very seriously. REST API design? Read Roy Fielding's thesis. Afterwards you'll be surprised at the elegance of it all. GET/POST/PUT/DELETE verbs for a CRUD API. HTTP for the communication protocol. JSON is handy for representation (they even use some light-weight schema validation). They use a pragmatic approach: "good enough" is better than "perfect". For development, they use github. Development is on branches. When everything is ready, they package it up as an RPM and deploy it. They do app configuration with YAML. It works better for them than python settings files as you can also use it for other applications. And in their case they can now also run … -
Djangocon: django minus django - Jacob Burch and Jacob Kaplan-Moss
(One of the summaries of a talk at the 2014 djangocon.eu.) Jacob Burch and Jacob Kaplan-Moss say Django is awesome. It comes with basically everything you need to get your site started. But it doesn't come with everything. And sometimes you might want to rip something out and replace it with something else. You could rip out the template engine and put jinja2 in. Or remove the views and only use REST and lots and lots of javascript. And you add redis or so to make communication between the frontend and backend. And slowly it isn't so recognizable as a Django project anymore. Django's batteries-included nature is great, but sometimes those batteries hem us in. So they're going to talk about ripping things out of django. You can see what goes well and in which places django fights back. Templates Something that is often replaced is django's template language. You might want something more powerful or you might want a completely different template language. Jinja2 is a common candidate. More expressive and it is often quicker. You do need to pass a regular python dictionary as context to jinja, though, instead of a django context. You can use https://github.com/jbalogh/jingo to … -
Djangocon: wednesday lightning talks
(One of the summaries of a talk at the 2014 djangocon.eu.) Lazy devops' guide to SSO with Kerberos - Aymeric Augustin The goal is to provide single sign on (SSO) to an internal application within your company network. You can use ksktutil to create a keytab. You then need to configure apache to look at that keytab. And you need to tell it to require a valid user. Django has support for such an externally set user that's send as a REMOTE_USER in the header. See https://docs.djangoproject.com/en/1.7/howto/auth-remote-user/ You next need to check if your browser works out-of-the-box or whether you need to do some more configuration There's more data in his notes: https://gist.github.com/aaugustin/10715655 A pycon in an African nation - Daniele Procida He hopes to get a pycon off the ground in sub-saharan Africa (not South Africa, as there's already a conference there). The goal: sustainability. They have to be able to build up skills locally. Skills shouldn't have to be imported. The only way forward is to use open source products. Through his university, he has some contacts. Nothing is fixed. There is much work to be done. And you cannot rush it: if you blow it, you could … -
Djangocon keynote: devops, it is much more than just automation - Kevin van Wilder
(One of the summaries of a talk at the 2014 djangocon.eu.) Kevin van Wilder talks about devops. (Note: I changed the title from "your product is more than the application" to "devops, it is much more than just automation" as that covers the main message much better). Nothing good happens after 14:00 on friday. So don't do a friday deploy. He remembered a quick fix they needed to deploy one friday evening. It actually went OK and it even worked during the press conference demo. 10 days later the site died, though. Who's problem is it? "It worked in development". So, "it is an operations problem now?". Is devops the solution? You cannot really be a "devops person". It is more a movement, a philosophy: lets work together as dev and ops. Partially it is just a name for something that people have been doing for a long time. A useful acronym is CLAMS: Culture. How can we influence co-workers? You cannot directly change culture. But you can change behavior. And behaviour becomes culture. If you do it right, you'll start influencing others to do right, too. We've been influenced ourselves at this conference, right? Haven't we been shown The … -
Djangocon: typography and the web, the state of the art - Idan Gazit
(One of the summaries of a talk at the 2014 djangocon.eu.) Idan Gazit is someone I really like to hear at django conferences. I'll simply point at two earlier talks. Design for developers (2010) and compass and less (2011). There's a nice book "thinking fast and slow". A general principle is that anything you can to to reduce cognitive strain is worthwhile. One of them is making your text readable. This is mentioned explicitly. The printing press was invented 600 years ago and we did a lot of writing beforehand, so we have a lot of experience. There are two kinds of factors: Micro. Typeface, kerning. Macro: measure ("width"), leading ("spacing"), flow of type on a page. There's a lot of common wisdom for books. How to lay it out, the ideal size of margins. But on the web we have a huge problem: we do not have control. There's no fixed page size, we have many screen sizes and screen resolutions and screen quality. But still... there's a lot that we can learn from the existing body of knowledge. Type size. 16 pixels is the default size in browsers. In books it is often smaller. But it is 16 … -
Djangocon: really, really fast django - Christophe Pettus
(One of the summaries of a talk at the 2014 djangocon.eu.) Christophe Pettus works for http://pgexperts.com . So... postgresql expert! I've got two other talks of him postgresql when it is not your job and advanced postgresql in django. How fast django anyway? You hear things like "the orm is slow" or "the template engine is old". So... when in doubt, measure. How high-overhead are django's components? You'll need to have a good test. Just returning an empty response is nonsense. He did a bunch of tests and django holds up pretty well. The overhead is very low. Most ORM operations are O(N) on number of fields. So on average, a 1-field model will be 10 times faster than one with 10 fields. Updating an update? Use the ORM update method instead of looping over items. Much faster. Using manual SQL can be a little bit faster than the ORM, but it is neglegible. Django's basic request loop is plenty fast. Request/response cycles to the database generally swamp everything else. Always do bulk and batch operations instead of iterating. Don't use components you don't need. Often you load a lot of libraries. Do you really need a full REST library … -
Djangocon: from icontains to search - Honza Kral
(One of the summaries of a talk at the 2014 djangocon.eu.) Honza Kral talks about searching, which is a big topic. He'll focus on unstructured search: fulltext search. Search is an interface to your data. Textual search: $ grep -ri django * or: >>> Documents.objects.filter( ... Q(title__icontains='django') | ... Q(body__icontains='django')) Don't do this. None of them are going to scale. And you're going to miss things. If you search for "running" you'll miss "runs", for instance. The first index in world history was the 1230 bible concordance. Just a simple list of biblical words with pointers to the passages in which the words occurs. We can do the same. Technically it is an inverted index. We can do more than just looking up individual words. We can also look at phrases. In those cases you look both at the files the words occur in, but also at their place in those documents. We can do more things when storing words in our index: Leave out common words like 'a' and 'the'. Lowercase everything. Normalize words (normalize various verb variants to just one, normalize single/plural). Look for synonyms. (Fast is almost the same as rapid, for instance). All this happens at … -
Djangocon: taming complexity with django - Ustun Ozgur
(One of the summaries of a talk at the 2014 djangocon.eu.) Ustun Ozgur tells about someone's talk about simplicity, simpleness and complexity. What does "complexity" tell us as django developers? He quotes Richard Gabriel who talks about "habitability" as a characteristic of source code that enables developers that see a piece of code to understand it and to work on it, even if they haven't seen the code before. This is so for django. Complexity should be fought head-on. Look at domain driven design. Keep your architecture close to the actual domain. That helps. Django has a couple of layers of defense against complexity. URLs. Django routes various parts of the url to various applications. The division into seperate applications helps a lot to fight complexity. Models and managers. Django uses several well-known design patters that help structure your code in the right way. But... watch out for "fat" classes that do too much. You could look at mixins and compositions. Or you can move some of the fat class's methods to separate functions. Sometimes no code is the best code. You could put state machine state in a variable (a dict or so) and use that. Data is easier … -
High Performance Django Kickstarter Project
I'm really excited to (finally) announce that we are writing a book! We've been working with Django professionally for a long time (over 7 years now). During that time, we've learned a lot about how to use the framework to build fast and scalable websites. We're bundling up all that knowledge into an e-book called High Performance Django and it is up on Kickstarter today. We're already well into the writing process, so we can tell you a little about what you'll get. The book is heavy on Django tips, but also reaches far beyond Python, showing you how to architect and tune the rest of your stack. It gives you an opinionated battle-tested blueprint utilizing many of the same techniques as high-profile Django sites like Disqus, Instagram, and Pintrest. We hope to get the first edition out in July, but we need your help to make that happen. Writing and editing the book is going to be a massive undertaking and you can help support our effort. Give us your vote of confidence by backing the project today! -
High Performance Django Kickstarter Project
I'm really excited to (finally) announce that we are writing a book! We've been working with Django professionally for a long time (over 7 years now). During that time, we've learned a lot about how to use the framework to build fast and scalable websites. We're bundling up all that knowledge into an e-book called High Performance Django and it is up on Kickstarter today. We're already well into the writing process, so we can tell you a little about what you'll get. The book is heavy on Django tips, but also reaches far beyond Python, showing you how to architect and tune the rest of your stack. It gives you an opinionated battle-tested blueprint utilizing many of the same techniques as high-profile Django sites like Disqus, Instagram, and Pintrest. We hope to get the first edition out in July, but we need your help to make that happen. Writing and editing the book is going to be a massive undertaking and you can help support our effort. Give us your vote of confidence by backing the project today! -
A break from the tutorial: Intuitive stuff
Hey guys! Let's take a break from the bombarding of information, and let's do something interesting. Let's talk about some of the features that are taken for granted in all the giant websites. You may already have noticed them. If not, double back and check them out. We'll also see how to implement them in your website.The "Keep me logged in":Many websites like Facebook and Gmail offer this option at the time of login. What it does is even if you close the browser/tab/window, the next time you open it it opens on your homepage. Even more fascinating, if you try to log in from some other device it you're not logged in, and you even get an alert that someone tried to access your account from an unknown location. Let's see how this is done in steps:When you log in, cookies with your username are stored in your browser. If you close it and reopen it, and access that site again, it checks for that-site-specific cookies. If they are there, they take you to your homepage. Else, you're redirected to the login page.To prevent multiple people accessing the account at the same type, the session is logged in to … -
Djangocon: Django class based views - Leonardo Giordani
(One of the summaries of a talk at 2014 djangocon.eu) Leonardo Giordani gives a survival guide for novices on django's class based views: what they are and how they work. What are they? Django views based on python classes. For many this translates into some django stuff based on some python magic. If you followed the tutorial but don't exactly understand what a view is, class based views can be quite hard to understand. A view is the part of django that gets the request and returns a response. Views used to be monolythic functions. A request comes in and a response goes out. It is hard to replace one specific part of it. If you could split it up, you could replace the individual parts. Object oriented programming (so: classes) is a good way to split up something that's monolythic into several separate methods on an object. That's what django's class based views do. In the case of class based views, it is worth your time to actually read django's source code. Get familiar with the details, it helps you understand what happens. First the as_view() method you see in the urls.py: url(... SomeView.as_view()). It makes sure your view … -
Djangocon: the future of postgresql in django - Marc Tamlyn
(One of the summaries of a talk at 2014 djangocon.eu) Marc Tamlyn is well-known for his succesful kickstarter campaign to improve postgresql support in django. He asked for a show of hands: some 90% of the attendees use postgresql. It is the most popular database for django projects. Most of the core team favours it. It has a wide feature set and it is a proper active (non-oracle-owned) open source project. Wide feature set... but not all of the features are supported by django. You can use several add-ons, but they're not that nicely integrated. So he proposed a kickstarter project for creating django.contrib.postgres. 14k (in UK pounds) was raised by the kickstarter campaign! Wow. He thanked everyone that contributed (a couple of big contributions by companies by name). The core of the project is to add support for a couple of data types, like array, json and hstore. Array can list just about any type. Nested structure is OK. No foreign keys, though. This one is almost ready for django 1.8. HStore is a key-value store. The most requested feature! It is quite similar to django-hstore, which is already a very good add-on. They'll help him with getting HStore … -
Djangocon: web components in django - Xavier Dutreilh
(One of the summaries of a talk at 2014 djangocon.eu) Xavier Dutreilh talks about web development. You probably use libraries like jquery and backbone. And css frameworks like bootrap. And tools like bower, grunt, yeoman and jshint. One of the things you should care about is performance. Especially for mobile devices, you don't want to send over too much css/js. You don't want too exotic and hard technologies, as you as a web developer probably will lack the specialized knowledge. Accessibility is important. Cross-browser functionality, too. But... we often disregard the wishes we enumerated above and simply look for ready-to-use framework and pile them on top of each other. And we hope everything works as we hope and expect. Which it doesn't and we spend lots of time debugging. We inherit technical dept by piling everything up like this. What can we do? Well, we first have to stop making such a mess in the first place and have to start looking at the requirements we figured out. The best tip is to think about individual elements. We could also try to do it lean and mean: custom code. Start by looking at a web page. What does it consist … -
Djangocon: pair up django and web mapping - Mathieu Leplatre
(One of the summaries of a talk at 2014 djangocon.eu) Mathieu Leplatre (who works on makina corpus) says web mapping should be simple and google maps should become unusual. Fundamentals of cartography Projections and postgis. Cartography is based on location: longitude (x) and latitude (y). Normally you use decimal degrees: -180 to +180 longitude, -90 to +90 latitude. Well known from GPS tools. Problem: the earth isn't perfectly round. GPS uses the "WGS 84" reference ellipsoid to standardize the latitude and longitude. Problem 2: you often have to show it on a flat map or flat display. So you have to project the 3D data on a 2D surface. Naturally this means compromises. Projections are for instance the mercator projection that google uses. For more background, examples and especially illustrations, look at mapschool.io Transformation often happens between "WGS 84" (srid=4326, that is the most-used one for storing it in databases) and google mercator for display (srid=3857). Those scary-looking "srid" numbers? Spatial reference ID. But don't worry, those two numbers are almost always the default value for databases and tools. Another fundamental concept is the data format. Basically you have either vector data (points, lines, polygons) or rasters (bitmap data). Vector … -
Djangocon: two short talks about django-oscar and about performance gains
(One of the summaries of a talk at 2014 djangocon.eu) An introduction to django-oscar - David Winterbottom David Winterbottom is, which I just now noticed, the originator of the very useful www.commandlinefu.com website that I've had in in my RSS reader for the past few years. Recommended: you get a useful tip out of it every month or so. More related to django: he's the author of the e-commerce framework 'django-oscar'. In e-commerce, especially B2B (business to business), you get loads and loads of exceptions. He originally worked on a PHP system that basically broke down under all those exceptions to the standard rules. He now works on django-oscar, which is used by a number of very large customers. Ecommerce: products, baskets, orders. Overridable apps. It could be added to your site without modifying anything: it won't take over your whole site. It doesn't use the django admin though, but a custom edit site. Philosophically, oscar isn't an out-of-the box product. It is a generic product, so it cannot know how your tax system works. Which payment provider you use, How you've handled shipping. So you'll have to add that yourself. Oscar provides a base platform you can build upon. … -
Djangocon: distributed systems - Raphael Barrois
(One of the summaries of a talk at 2014 djangocon.eu) Raphael Barrois has this definition for distributed systems: they are a set of autonomous, interconnected services. Possibly a different codebase. Each service with its own database. Why? When? For elaborate systems you can make a nice tidy elaborate architecture. But after two years of bugfixing and feature-adding and changes it inevitably becomes a huge sprawling mess. So... split it up! Into separate components. When would you start doing something distributed-like? When a new feature has to be added that's not an incremental improvement. A feature that is strongly disconnected from the core features. Another reason can be that you install a new version of your main project somewhere, possibly in another datasenter or with a separate database. It still has to be operated by the same ops team, though. Or there could be technical reasons like scaling/sharding or geographic expansion or debundling/upgrading. Building it Where should you start? A good point is to extract generic code. misc, util, tools. Everything that's not business-specific. Perhaps also some basic business components: common models, UI, etc. Put this kind of code into separate models and adapt your deployment process for fast-moving internal dependencies … -
Djangocon: two talks, ecology and healthchecks
(One of the summaries of a talk at 2014 djangocon.eu. Two (short) talks actually.) Django powered ecological data - Jakub Witold Bubnicki Jakub Witold Bubnicki works on data-intensive science. For instance for biological sciences, more and more data is becoming available. Globally. In data intensive research, you often have to integrate and adapt multiple sources. Smaller teams (let anone individual PhD students) can often not create the infrastructure needed. He collects photos from his "camera traps" for two years now. He has to integrate this with geographical data and, for instance, remote temperature datasets. Often, the data includes both a space and a time component (location and date). The collected data needs to be easily accessible, discoverable, shareable and reusable. This is often not the case. So he started designing a django architecture to fix this. He also uses lots of existing applications like metacat (catalogs), geoserver (geographical data) and rasdaman (rasters). Django provides the authentication, filtering, browsing, searching, permissions. It connects to the external applications. The django site itself also has an API so that others can connect to it. Note: they're still working on it, it is not finished. But the generic data browsing and a bunch of … -
Djangocon: visibility for web developers - Bruno Renie
(One of the summaries of a talk at 2014 djangocon.eu) Note beforehand: Bruno Renie had several useful things to say at previous djangocons. I'd like to draw specific attention to his lightning talk last year about settings. He later wrote a more elaborate version on his own blog. Something I'm gonna take a deeper look at in the coming weeks as I'll have to fix something in many of our internal sites :-) He works in a diverse team and in a quite complex infrastructure, so they need to make their infrastructure visible: errors, events, metrics. Errors: easy: use sentry. Events: basically, a log call. The errors and sentry mentioned above are great for developers, but you often need info on the requests that don't fail, too. For this, you need centralized logging: searching on several machines in logfiles doesn't cut it. As an aggregator, they use logstash + eleasticsearch. They use kibana as a frontend. rsyslog/syslog-ng can work with logstash-forwarder ("lumberjack") to forward everything to logstash. In python, you can use a sysloghandler to send everything to the syslog. You have to use structured logging. A formatted string is nice for the logfile, but not if you want to … -
Djangocon keynote: where the wild things are - Aymeric Augustin
(One of the summaries of a talk at 2014 djangocon.eu) Aymeric Augustin talks about app loading. App loading is a step in django's initialization sequence: import all models and populate a cache. It is also a project that improves this step. The trac ticket for this feature is seven years old... Many many people have worked on it and provided patches and branches. The final implementation looks like this: from django.apps import AppConfig class MyConfig(AppConfig): name = ... label = ... verbose_name = ... # and in the settings file: INSTALLED_APPS = ( 'some_app', # Old style. 'yourapp.apps.MyConfig', # Points to the config class. ... All nice, but in practice there were crashes when using the implementation in production with gunicorn. The root cause was that django doesn't have a proper signal that says that it is fully configured and ready. So he started digging into the code. And found out that actually reading all the models and setting everything up occurs at various points in time, depending on how you call it. "runserver" is different from "shell" which is different from "running it from wsgi". At the core is the AppCache that django uses behind the scenes. This looks …