*Pycon.au Trip Report*
A bunch of CPUG members recently attended pycon.au - I usually try to write a trip report after attending these types of conferences, thought it might be of interest to the CPUG community.
Authors were John Weng (JW), Lames Li (JL) and Gavin Jackson (GJ).
Please find our trip report below. As you can see lots of interesting content that will feed in to future initiatives - also validates many of our current technology selection choices - for example, the use of ansible, postgresql etc.
Talks are available to watch online via https://www.youtube.com/user/PyConAU
Pycon.au 2015 trip report (Gavin Jackson, John Weng, James Li) 31 July - 3 August
Wow, pycon.au was one of the best conferences I have attended! Great location, food, presenters and attendees!*Day 1 (Django miniconf)*What Django can learn from Twisted by Amber "Hawkie" Brown
GJ: Django miniconf keynote, looked at what Twisted does well and what Django does well, and identifies places in both projects where cross pollination could occur. For example, twisted could make use of Django forms if it was refactored into a more modular architecture (like deform), whilst Django is (currently) missing a twisted like asynchronous callback mechanism.
JL: Hendrix is a Twisted Django. If we choose to stick with Django and want really high performance, we can dig into this product.
JL: It seems like Django form handling is good, extremely well documented. I will look more into this in Django investigation project.
JW: As Amber mentioned Twisted has its base functionality support Python 3 (40%). And one thing I found pretty interesting is the "Twisted Django" architecture, which sounds have great performance benefits. One of the video she highly recommended,DjangoCon 2008 by Cal Henderson. Also it is worth to sit down and study Twisted's testing code. It's on my plan.The one true way of doing Django deployments... by Tom Eastman
GJ: Interesting look at the challenges that all devops face when packaging and deploying python webapps to dev, test and prod environmentsMoby and the Beanstalk by Sebastian Vetter
GJ: Discussed docker deployment of django webapps to AWS
JL: A side note of a good deployment process:
As automatic as feasible
predictable and repeatable
is able to be rolled back if necessary
minimise third-party infrastructure dependence
doesn’t clobber your settings
JW: Docker is one of the big topics among the 3 day conference. It's a light weight deployment container with the applications and its dependencies and getting ride of the giant virtual machines. This sounds quite sexy. Currently a few of its supporters run Docker in production, but I think it's just a matter of time. Don't be afraid to search by Honza Král
GJ: This is an interesting presentation on how to integrate Django with Elastic for search integration.
JW: Elastic search is a document originated search engine. It extensively employs JSON. Actually this search engine under the hood is based on the Apache Solr.PostgreSQL in Django 1.8 by Christophe Pettus
GJ: A great talk on how Django is supporting more advanced postgres features and how to use GIN indexes to speed up searching within Arrays, HSTORE, Range
JL: By applying range, we can deal with problems related to detect time overlaps more easily
JW: An absolutely great talk. PostgreSql has a number of features, such as array, hstore and range field, we haven't discovered, and they come with some low-level support which can significantly simplify the work in some situations. The other thing I just realised is that postgresql is fully UTF-8. Recreating the Pycon 2015 site in under 30 minutes by Tim Heap
GJ: This talk looked at using the Wagtail django CMS extensionRealtime websites with Django and Meteor by Tyson Clugg
GJ: This was an unreal talk, Meteor looks really cool (I don't think I would use this in prod though!)
Core Team Panel, featuring: Chris Beaven, Markus Holterman, Russell Keith-Magee, Curtis Maloney
JW: Realtime does sounds really cool! Django DDP is still in progress. Gevent is a Python 3 library providing fast event loop.Integrating Django with Identity Management Systems by Fraser Tweedale
GJ: This was presented by a Redhat employee, really just focussed on internal SSO (kerberos, dogtail)
JW: An SSO implementation based on FreeIPA.Stop and smell the Djangos by Richard Shea
GJ: A good wrap up of some of the django features - the I8N support was particularly interesting*Day 2*Keynote: Designed for education: a Python solution by Carrie Anne Philbin
GJ: A solid keynote discussing how to teach programming to children (and teachers!)Slow Down, Compose Yourself - How Composition Can Help You Write Modular, Testable Code by Amber "Hawkie" Brown
GJ: A good discussion on how to structure your code for easier (and faster) unit tests.
JW: Good to know such a testing strategy, highly utilising the OOP.Fang: Pythonic dependency injection by Nathan Craike
GJ: Nathan built his own dependency injection framework
JW: Good idea, but I am still worried about its overhead.Ansible, Simplicity, and the Zen of Python by Todd Owen
GJ: This was my favourite talk of the day, did a great job of digging in to the design decisions related to Ansible and looking at how they relate to the Zen of python (import this)Mondas, in my Python? by Xuanyi Chew
JW: good introductory talk to Mondas, much better than my uni lecturer.Docker + Python by Tim Butler
JW: Type hints is an interesting idea which is good enough without losing Python's flexibility. PEP 484 will come with python 3.5.b1.
JW: Current Python does support the Haskell-like def a(b: str) -> str feature.
JW: Besides, MyPy project is worthy taking a look which provides some type check without running your program.Using benchmarks to understand how WSGI servers work. by Graham Dumpleton
GJ: This was quite a complex talk, my take home was that WSGI performance drops when there are more threads, so we should decrease the number of threads (to 3-4) and increase the number of processes (depending on the amount of memory available)Python's Transaction to Multilingual Programming, by Nick Coghlan
JW: OSX comes default with UTF-8! YEAH! JSON Standards for the Web by Jamie Lennox
GJ: Another presentation from Redhat looking at the lay of the land - in a nutshell certain JSON standards are turning restful json into an API standard that is comparable to SOAP (jsonapi, jsonhome, jsonschema)Cooking with Cryptography by Fraser Tweedale
GJ: A new python cryptography library (called cryptography) now reigns supreme - this now supersedes pycrypto, designed to simplify crypto algorithm selection (and make use of base OS crypto libraries).Build a Better Hat Rack: All Contributions Welcome by Katie McLaughlin
GJ: Discussed how to reward and acknowledge non-code contribution to pythonFour non-cryptographic hash functions enter. One leaves, by Adam Harvey
JW: This talk introduces some hash functions which designed for less secure concerns but more performance.Lightning talks
JL: Python can be used as function language such as Haskell.
l = [1,2,3,4,5]
x, *xs = l
x = 1
xs = [2,3,4,5]
Another trick I learned is that "_" will return the last evaluated value.*Day 3*Keynote: Consequences of an Insightful Algorithm by Carina C. Zona
GJ: Interesting keynote that examined the social implications of automation - great speaker.Learn You a Flask by Lachlan Blackhall
GJ: It was great seeing a non-django talk! Flask is a microframework (similar to pyramid), I would say that it is even more lightweight than pyramid. There are hundreds of extensions that bring the functionality up to parity however.
This looks interesting: http://flask-sso.readthedocs.org/en/latest/Are they ready? A quick review of graph databases and Python, by Nic Crouch
JW: very good introduction of graph database and a general evaluation of several most popular graph databases. One conclusion from the speaker on when to use the graph database is that we should always consider using PostgreSQL first... Arrested Development - surviving the awkward adolescence of a microservices-based application by Scott Triglia
GJ: This was a great presentation, looked at how yelp migrated to a microservices based architecture and took a look at the swagger definition format for Restful JSON API creation.Interactive visualization for the curious by Edward Schofield
GJ: A very high level look at the graphing and visualisation libraries that are available on the python platform.Just Use Postgres.by Rhys Elsmore
GJ: Another excellent postgresql presentation, looked at some of the advanced postgres features used by the security team at Heroku - shows how it can replace Mongodb and Redis.
JL: View concept is introduced. A view is a schema we slice from a table. It can add flexibility to us.If view is updated, the relevant table will also be updated. Another good trick that I learned is we can lock a row in the table by doing the following:
select balance from accounts where user_id = 1 for update;
JW: Table inherent!!! CREAT table a () inherits (ptb);Make your logs work for you by Honza Král
GJ: This is the talk that I was hanging out for - did a deep dive into the ELK stack (Elastic, Logstash and Kibana) for log file collection, analysis and visualisation (great presentation!).
JW: Can't wait to give it a goCython
JL: This can be used when we need to perform massive calculations. Cython is super efficient compared with Python.
pip intall cython (install Cython)
apt-get install build-essential
pip install easycython (install Cython compiler tool)
We then do:
Its syntax is 80% similar to python and can be invoked by python code. It is extremely powerful to support threads.
However, use Cython sparingly, because it is difficult to debug and adds complexity to the code base.*Interesting links/projects*
Worth Reading (general)http://12factor.net/
- The new Spolsky ruleshttp://engineeringblog.yelp.com/2015/03/using-services-to-break-down-monoliths.html
- Introduction to microservices based architecture
- Bootstrap/angular django forms (look pretty sweet)https://www.meteor.com/
- Cool looking async/websocket based GUI frameworkhttps://github.com/commoncode/django-ddp
- Django implementation of DPP protocolhttp://tysonclugg.github.io/#/
- Tysons excellent presentation on Meteorhttps://docs.djangoproject.com/en/1.8/topics/forms/modelforms/
- Build forms automatically from django models (similar to deform)http://stackoverflow.com/questions/1179469/is-it-posible-to-generate-django-models-from-the-database
- Build django models from an existing database schemahttp://flask.pocoo.org/
- Flask websitehttp://flask.pocoo.org/extensions/
- Flask extensions (to make it do something useful)https://github.com/mleibman/SlickGrid
- a really sweet looking html table widget
- a package used to build debian packages (not sure if we currently use this?)
- Documents the extended postgres features that are supported by sqlalchemyhttp://pgxn.org/
- Postgresql Extension
- A framework used to teach young kids programminghttps://groklearning.com/
- An online service used to tech older children how to do real programminghttp://sonic-pi.net/
- An application that allows kids to play around with programming sound/synthesizers
- The docker project
Various tools and frameworks that allow you to use restful json properly (like SOAP)http://bcwaldon.cc/2012/08/19/using-warlock-and-json-schemas.htmlhttp://jsonpatch.com/http://jsonapi.org/http://swagger.io/https://github.com/Yelp/bravadohttps://github.com/striglia/pyramid_swaggerhttp://json-schema.org/
Various libraries used for visualisationhttp://matplotlib.org/http://bokeh.pydata.org/en/latest/http://d3js.org/
Elastic/Logstash/Kibana - great collection of tools used for log file collection, analysis, search and real time visualisationhttps://www.elastic.co/productshttps://www.elastic.co/products/beats/packetbeat