Django community: Community blog posts RSS
This page, updated regularly, aggregates Community blog posts from the Django community.
-
Currencies
I think that in e-commerce it is vital to bill customers in their own currency. Certainly for us, with the UK accounting for such a tiny proportion of the Nespresso capsule market then export will be very important for us. I’ve finally round to deploying the completed infrastructure for euro billing. All it takes is a simple change in the config file and the site will start using euros. As soon as Streamline get themselves in gear we will be able to launch an Irish website in euros. It will also be much easier to add other currencies in time. I do really wish that Django had support for currencies rather than leaving it up to the developer and a quick Google shows that I am not alone in wanting it. There is at least a decimal type but that's not quite the same. The big advantage of currency being built in is that it would make it easier for the different e-commerce projects using Django to share code. -
Test Suite
I've spent the past few days putting together a proper test suite. It's been a lot of work, 813 lines of code and 159 specific tests. It covers the bug that caused the site to fail I blogged about and from now on whenever there is a bug I will add a test for it to the suite to ensure it never happens again. I'm also using coverage.py which is a great way of spotting what's still to be done. The tests are a mix of unit tests on very specific parts of the codebase and also functional tests that go completely through the process of signing up for the site and placing an order and also an existing customer placing an order. These test go right through to charging a card on the SagePay test servers. In a commercial environment it's very hard to get time to spend on quality. There's always a big to-do list and to say "Let's just pause for a while to improve quality" it a hard thing to say. The benefits are that I can now go a bit faster with development and make changes with more confidence as I can be more sure … -
First Euro Transaction and a Bug
Today was a bit of a milestone as we launched our Irish Nespresso capsule site using the sites framework and processed our first euro transaction. Everything seemed fine. Unfortunately things then went wrong and a lot of our UK customers ended up presented with euro pricing! Up to this point I'd only had one development server but I quickly setup two different dev servers, one for each country. After a while I was able to reproduce the error on the dev servers with the currencies appearing wrongly on the different sites. After a bit of Googling I came across this post but the wonderful Graham Dumpleton (is it wrong to love another man?). It turns out I actually stumbled across my first bug in Django but fortunately there is a simple fix. -
Porting to Zope
After much thought I've decided to port my application away from Django to Zope. I've just been finding things too straightforward with Django and I miss the days of struggling with problems only to discover they were bugs in the framework or the challenge of getting the ZODB to scale. I'm also really missing XML. The only thing that makes me happier than coding in Python is working in XML. Update: This was of course a very poor April Fools joke! -
Heroku
I hurt my back yesterday and as a result I'm stuck in bed. So I thought I would make use of the time by evaluating Heroku. I am overall very impressed and next time I do a project I will deploy it on Heroku from the start. However I've come across a few problems. The first one is that Heroku requires the use of a CNAME. Unfortunately CNAME's can only be used if the domain is in the form http://www.finecoffeeclub.co.uk/ rather than http://finecoffeeclub.co.uk/ Changing the address is out as I would need to re-do several extended validation SSL certificates. Certain DNS providers have their own method of doing a CNAME at the root but unfortunately Freeparking doesn't. I would also have to do some work around getting a fixed IP endpoint for Sagepay, use of the temporary file system and setting up storage for the static assets. On the other hand: I have no scaling issues with the present infrastructure, indeed I forecast a 50 to 100 times more traffic on the present single server and to date there hasn't been a problem with the infrastructure. I do though have quite a lot of other things to do overall migrating … -
MailChimp
One way or another e-commerce tends to involve quite a bit of email! I had thought about doing it myself and I did run a site a long time ago which fired out a lot of email and you can get it to work but it really is jolly hard work. You need to implement SPF and DomainKeys and also get in contact with Microsoft and Yahoo! if you want your mail delivered (Google, as usually, just works perfectly without any hassle at all). Anyway, I've got too many other important tasks so I am using MailChimp. They have a brilliant API which allows really good reporting and very detailed (and easy!) segmentation. It is so nice for non-technical people to be able to mail a segment without requiring custom SQL. I've written a cronjob which calls a Django management command every day to upload my new customers and checkout abandonments. I'm also use MailChimp web hooks to call back to Django when customers unsubscribe. I've added MailChimp's e-com 360 tracking to Django so we've got reports of revenue from each campaign. I've used some incredibly expensive enterprise mail packages before but MailChimp is quite simply streets ahead and tens … -
The American Dream
Americans are beginning to get the Nespresso bug so I think the time has come for us to launch a USA Nespresso capsule site. The good news is that the currency work is already done so all we need to do is fix the English. The Americans really are much more sensible than us when it comes to writing English, they do cut out a whole lot of unnecessary vowels and generally spell words closer to their pronunciation. Django has great i18n and I'm translating from en-gb to en-us. The only issue is that out of the box the LocaleMiddleware sets the language based on the web browser. However I want the language set based on which URL the user is on. Fortunately it's a trivial thing to fix. All I've done is put this into my custom middleware.py: request.LANGUAGE_CODE = settings.LANGUAGE_CODE And then I can set LANGUAGE_CODE in the relevant settings.py One thing that slightly got me is that locale codes are different to language codes. A locale code looks like en_US while a language code is en-us. It does matter which you use. -
Encoding Problem
It was pointed out today that the euro sign was not displaying properly on the news page. I had a look and I saw that the problem is that the database encoding is set to Latin-1 instead of UTF-8. It turns out that was the default on the version Debian I started with and as I've dumped and restored the database the encoding has remained the same. The solution is relatively simple which is to dump the database and reload with the correct encoding. This could be done during a short maintenance window in the middle of the night. But no! As the sun no longer sets on the Fine Coffee Club empire there is no particularly good time when we can shut down for maintenance. Update: It turned out to only require a couple of minutes of outage. However the point remains. -
Currency Refactor
This is technical notes just for me on the currency refactor. The background is that currency is presently on the presentation layer in the templates. However as I'm expanding currencies this is getting more complex and there is also a problem that invoices are generated as a PDF so I have to do the currency code twice: once for the templates and once for the PDF which break the DRY (Do not Repeat Yourself) rule. The code has to deal with the following: Changing currency symbol, e.g. £1.00 and $1.00.Changing decimal separator, e.g. €1.00 and €1,00Changing currency symbol position, e.g. €1.00 and 1,00€Different currencies with the same symbol e.g. Canadian dollars (CAD) and US dollars (USD). An additional requirement is that I want to be able to add currencies relatively frequently without changing too much code. Pushing this down into the model layer should make things a bit easier but there is quite a lot of complexity that my design has to cope with. Products and Shipment Methods have multiple prices, one for each currency, chosen by settings.pyBaskets have one currency for multiple totals depending on settings.pyShipment and Order have the currency in the model and multiple totalsAdmin interface reports … -
Testing Update
I'm off on holiday on Friday for a week without much in the way of internet access so I'm going to have a code freeze as of now. The final code I committed was i18n for the French launch. Unfortunately there was a bug in one of the live site settings files which caused some problems on the Irish site. I fixed it and wrote a test which checks all the major settings across all the countries. I also ran coverage.py on the codebase which shows the percent of the code covered by tests and can generate a nice HTML report. At the moment it is 61%. During the code freeze I am going to focus on adding unit tests to try get the coverage over 70% this week. In other news, outsourcing of fulfilment is going well with Seko chosen who have warehousing in the USA, UK, Europe, Australia and other locations giving us a complete global presence. They are really a bit big for us but that means we can stick with them long term. There's going to be quite a lot of work in integrating into their API but it is very well documented. It also means … -
Django / ZenDesk API
Anna was off and I was left doing all of the email support which we do through the excellent Zendesk. After about 10 minutes I has deeply frustrated with the process of finding customer orders on our system. So I've written a simple application which links straight from a ticket to the relevant orders with just a couple of clicks. It has saved an enormous amount of time. I'm going to generalise this slightly by getting it to bring up the Django user and then I'll publish it as the first bit of code to be released. -
Django Errors Going to Spam
Like many sites I've setup Django to email me when there are errors. Unfortunately the errors were all ending up in my spam. I had a look at the mail system and there were no problems there. The first issue was very simple. It'd not set SERVER_EMAIL in settings.py which meant that the emails were being sent from 'root@localhost' which spam filters are not going to like. The second problem was the very large number of emails which was also a problem for me in that I stopped taking the emails seriously. The Django ALLOWED_HOSTS setting improves security but generates a lot of errors (this has been fixed in the development version of Django but for now it fills up the log). I found an excellent article Prevent email notification on SuspiciousOperation with detailed code which I've implemented and it's made a huge difference. -
Blog Move
I've decided to move my blog from WordPress.com onto Django on the Fine Coffee Club site. I put together a simple blogging app in about 100 lines of code. I'm struggling to write a proper justification for this, WordPress worked perfectly well and the last thing the world needed was another blogging product. The best I can come up with is that I wanted to play with the syndication framework and it gave me the chance to play around with some other odds and ends in the framework that I don't need in my day to day e-commerce. It is also easier to customise when it is on your own server and I would rather not install WordPress as that would also required PHP and MySQL to be installed which are a little heavy weight to be on my main server. It's a very weak justification I know! R6J9WMEMPRC4 -
Reverse Is A Good Idea
Up to know the Fine Coffee Club websites have each had their domain like this: finecoffeeclub.comfinecoffeeclub.co.ukfinecoffeeclub.iefinecoffeeclub.ft However for Australia and Canada we are not going to give them their own domain names for various good reasons, so these addresses will be: finecoffeeclub.com/aufinecoffeeclub.com/ca I thought that would be an easy change to make the change but it turned out to be quite tricky as all of the links in the templates and more than 100 redirects in the Python code will all have to change. It would also be a bit of a pain to work out exactly what the URL's as some of them will be prefixed with the country code and some of them not. I am now porting everything over to using reverse resolution and I wish I had done this from the start. Up till now I hadn't really appreciated how useful an idea it is. With the url tag I can now leave the whole thing up to Django. -
Converting our multi-page Django app to use AMD
Introduction The application which powers 2degrees has, to date, mainly been driven by a Django-powered back-end. Django is agnostic about any front-end stack and this has certainly contributed to a fairly ad-hoc approach to javascript. With an increasing demand for a more interactive experience, we have been adding more and more javascript, experimenting with AngularJS to power our pinboard, and looking at ways to use/build frameworks to play nicely with the back-end. As a result of this work, we have found issues of manageability of the javascript code. Manageability issues The issues we had fell into the following categories: Risk of missing dependencies when inheriting complex pages with lots of javascript. Dependencies loading out-of-order in some cases. The code becoming increasingly complex to manage (poor separation of concerns in the code, etc.) AMD to the rescue? We hoped that a modular approach to javascript would address all of these issues and more. After some research about various module patterns and loaders, we decided to use RequireJS. This decision was based mainly on API features and how well-maintained and well-documented the library is. Porting the codebase to using RequireJS After performing a spike to asses the complexity of the task, we … -
使用 uncss 找出未使用的 CSS 样式
我们知道, 保持代码的经凑干净对于代码的可阅读性至关重要, 之前的博文中也讲过这点. 但在修改CSS时, 我们经常会因为大量的CSS代码和CSS自身的可覆盖性而只增加不删减. 本篇中我们就介绍一下uncss, 一个可以用于找出未使用的CSS代码的工具, 方便我们精简代码. uncss最基本的用法是命令行: uncss http://example.com > styles.css 输出的结果就是在使用的CSS代码, 而未使用的CSS代码已经被自动删除掉. uncss是如何工作的呢? HTML文件通过PhantomJS载入, 相关JavaScript代码被运行 在使用的CSS代码从HTML文件中抽离 CSS代码通过css-parse重新组合起来 document.querySelector将HTML中未找到的selector分离出去 剩余的CSS代码重新加入组合 就像每个NodeJS工具一样, 我们能使用其JavaScript API: var uncss = require('uncss'); var files = ['my', 'array', 'of', 'HTML', 'files'], options = { ignore : ['#added_at_runtime', /test\-[0-9]+/], media : ['(min-width: 700px) handheld and (orientation: landscape)'], csspath : '../public/css/', raw : 'h1 { color: green }', stylesheets : ['lib/bootstrap/dist/css/bootstrap.css', 'src/public/css/main.css'], ignoreSheets : [/fonts.googleapis/], urls : ['http://localhost:3000/mypage', '...'], // Deprecated timeout : 1000, htmlroot : 'public' }; uncss(files, options, function (error, output) { console.log(output); }); /* Look Ma, no options! */ uncss(files, function (error, output) { console.log(output); }); /* Specifying raw HTML */ var raw_html = '...'; uncss(raw_html, options, function (error, output) { console.log(output); }); -
Extending Django's QuerySet to return approximate COUNTs
UPDATE: I've re-written and open-sourced a better way of doing the below as part of my library django-mysql. The docs there on approximate counting are just as good a read as the below, and you can pip install the solution. I was looking through the MySQL slow_log for YPlan and discovered that there were a lot of SELECT COUNT(*) queries going on, which take a long time because they require a full table scan. These were coming from the Django admin, which displays the total count on every page. "Why is SELECT COUNT(*) such a slow query?" you might think, "surely MySQL could just keep a number in the table metadata and update it on INSERT/DELETE." Aha! You are totally right - for the MyISAM storage engine. But Innodb, which you shoudl be using, provides transacational support and other niceties, at the cost of making such a metadata count impossible. Each transaction must be isolated from the others until it commits or rolls back, so a single 'accurate' COUNT(*) value per table is impossible. It would also be a point of contention from locking, which MyISAM doesn't care about anyway because it locks the whole table for any write. Hence, … -
Extending Django's QuerySet to return approximate COUNTs
UPDATE: I've re-written and open-sourced a better way of doing the below as part of my library django-mysql. The docs there on approximate counting are just as good a read as the below, and you can pip install the solution. I was looking through the MySQL slow_log for YPlan and discovered that there were a lot of SELECT COUNT(*) queries going on, which take a long time because they require a full table scan. These were coming from the Django admin, which displays the total count on every page. "Why is SELECT COUNT(*) such a slow query?" you might think, "surely MySQL could just keep a number in the table metadata and update it on INSERT/DELETE." Aha! You are totally right - for the MyISAM storage engine. But Innodb, which you shoudl be using, provides transacational support and other niceties, at the cost of making such a metadata count impossible. Each transaction must be isolated from the others until it commits or rolls back, so a single 'accurate' COUNT(*) value per table is impossible. It would also be a point of contention from locking, which MyISAM doesn't care about anyway because it locks the whole table for any write. Hence, … -
Extending Django's QuerySet to return approximate COUNTs
UPDATE: I’ve re-written and open-sourced a better way of doing the below as part of my library django-mysql. The docs there on approximate counting are just as good a read as the below, and you can pip install the solution. I was looking through the MySQL slow_log for YPlan and discovered that there were a lot of SELECT COUNT(*) queries going on, which take a long time because they require a full table scan. These were coming from the Django admin, which displays the total count on every page. “Why is SELECT COUNT(*) such a slow query?” you might think, “surely MySQL could just keep a number in the table metadata and update it on INSERT/DELETE.” Aha! You are totally right - for the MyISAM storage engine. But Innodb, which you shoudl be using, provides transacational support and other niceties, at the cost of making such a metadata count impossible. Each transaction must be isolated from the others until it commits or rolls back, so a single ‘accurate’ COUNT(*) value per table is impossible. It would also be a point of contention from locking, which MyISAM doesn’t care about anyway because it locks the whole table for any write. Hence, … -
Python中使用Faker创建虚拟数据
在测试数据库时, 我们经常会需要用到假数据来支持代码的运行. 本篇就介绍一下Faker, Faker的唯一功能就是生成半随机的虚假数据, 例如名字, 地址, 域名, 段落等. 创建virtualenv, 并安装Faker: mkvirtualenv test pip install fake-factory 创建假名字: from faker import Factory #---------------------------------------------------------------------- def create_names(fake): """""" for i in range(10): print fake.name() if __name__ == "__main__": fake = Factory.create() create_names(fake) 得到不同的名字, (你得到的应该与我不同): Mrs. Terese Walter MD Jess Mayert Ms. Katerina Fisher PhD Mrs. Senora Purdy PhD Gretchen Tromp Winnie Goodwin Yuridia McGlynn MD Betty Kub Nolen Koelpin Adilene Jerde 不希望名字中含有铅坠河后缀: from faker import Factory #---------------------------------------------------------------------- def create_names2(fake): """""" for i in range(10): name = "%s %s" % (fake.first_name(), fake.last_name()) print name if __name__ == "__main__": fake = Factory.create() create_names2(fake) 接下来我们看如何生成其他虚假数据: from faker import Factory #---------------------------------------------------------------------- def create_fake_stuff(fake): """""" stuff = ["email", "bs", "address", "city", "state", "paragraph"] for item in stuff: print "%s = %s" % (item, getattr(fake, item)()) if __name__ == "__main__": fake = Factory.create() create_fake_stuff(fake) 你可能会得到以下信息: email = pacocha.aria@kris.com bs = reinvent collaborative systems address = 57188 Leuschke Mission Lake Jaceystad, KY 46291 city = West Luvinialand state = Oregon paragraph = Possimus nostrum exercitationem harum eum in. Dicta aut off Faker中还有许多其他方法没有在这里提及到, 你可以自行查看其文档. -
Tinkering with Django and Ember.js
Hi everyone. Recently I have been looking around at the various JavaScript frameworks that have been blooming out recently. Three came in front of the others, namely BackboneJs, AngularJs and EmberJs. After looking at all 3 of them, EmberJs is the one which appeals to me the most. But if you hadn't guessed from the tutorials on this blog, I also like Django a lot and most tutorials out there talking about EmberJs and actually using a data backend are using Ruby on Rails and the few that I could find that were talking about Django were based on TastyPie while I like Django-REST-Framework better... So I decided to dive in and go full speed ahead and learn the hard way. The result of which is Djember-CMS. -
Python中如何重新引入被覆盖的自带function
最近在写python应用时遇到一个问题: 引入某个模块时会自动引入自定义的int到python的namespace中, 从而覆盖了python自带的int function. 因为我们需要使用python的int, 所以不得不找到重新引入这int的方法: 幸运的是, 这一问题还是很容易解决的, 我们只需要使用__builtins__: from __builtins__ import int as py_int 这样一来我们又可以重新使用python的int了, 但在此时叫做py_int. 一个function或变量的被覆盖最常见的原因是在引用时使用了"*": from something import * 当这样使用import时, 我们无法明确的指导究竟引入了哪些变量或function, 也无法知道这些变量或function是否会覆盖原来的变量或function. 所以这也是在使用import时不推荐使用"*"的主要原因之一. 在python 3中, 可以使用builtins代替__builtins__. -
July 2014 ShipIt Day Recap
This past Friday we celebrated another ShipIt day at Caktus. There was a lot of open source contribution, exploring, and learning happening in the office. The projects ranged from native mobile Firefox OS apps, to development on our automated server provisioning templates via Salt, to front-end apps aimed at using web technology to create interfaces where composing new music or performing Frozen’s Let It Go is so easy a anyone can do it. Here is a quick summary of the projects that folks worked on: Calvin worked on updating our own minimal CMS component for editing content on a site, django-pagelets, to work nicely with Django 1.7. He also is interested in adding TinyMCE support and making it easy to upload images and reference them in the block. If you have any other suggestions for pagelets, get in touch with Calvin. Philip worked on a code to tag words in a text with basic information about their etymologies. He was interested in exploring words with dual French and Anglo-Saxon variations eg “Uncouth” and “Rude”. These words have evolved from different origins to have similar meanings in modern English and it turns out that people often perceive the French or Latin … -
Django访问多个PostgreSQL Schema
django缺少对PostgreSQL的多schema支持, 之前我们尝试了多种方法访问除public schema之外的schemas, 但这些方式都难以维护. 然而, 最近我们发现这一问题可以使用PostgreSQL的search_path参数轻松地解决. 一个简单的例子 假设一个django项目中所有的表都创建在django schema中, 并且我们的项目需要用到legacy schema中几个表. 我们可以通过PostgreSQL数据库映射, 然后在django中使用DATABASE设置实现, 让我们展示一下其他两种不同的设置方法: 在连接时设置search_path 假设django和legacy schema已经存在, 且django项目具有数据库的访问权. 在django设置中, 我们在options中设置search_path: # settings.py DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'OPTIONS': { 'options': '-c search_path=django,public' }, 'NAME': 'multi_schema_db', 'USER': 'appuser', 'PASSWORD': 'secret', }, 'legacy': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'OPTIONS': { 'options': '-c search_path=legacy,public' }, 'NAME': 'multi_schema_db', 'USER': 'appuser', 'PASSWORD': 'secret', }, } 这一设置方式对于已有的项目而言需要修改的地方最少. 设置不同的数据库用户 对于以上设置而言, 最大的问题可能是set search_path参数在每次django项目与PostgreSQL数据库进行连接时都需要传输. 为了节约这一时间, 我们可以事先为每个用户设定search_path: 用postgres用户登入psql shell: -- user accessing django schema... CREATE USER django_user LOGIN PASSWORD 'secret'; GRANT appuser TO django_user; ALTER ROLE django_user SET search_path TO django, public; -- user accessing legacy schema... CREATE USER legacy_user LOGIN PASSWORD 'secret'; GRANT appuser TO legacy_user; ALTER ROLE legacy_user SET search_path TO legacy, public; django项目中settings.py: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'multi_schema_db', 'USER': 'django_user', 'PASSWORD': 'secret', }, 'legacy': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'multi_schema_db', 'USER': 'legacy_user', 'PASSWORD': 'secret', }, } 以上便是两种不同的使用search_path的方式. 自定义django database router也许也能做到自动选择正确的schema. -
Deferred Tasks and Scheduled Jobs with Celery 3.1, Django 1.7 and Redis
Setting up celery with Django can be a pain, but it doesn't have to be. In this video learn what it takes to setup Celery for deferred tasks, and as your cron replacement. We will use Celery 3.1 and Django 1.7 both introduce changes you need to be aware of.Watch Now...