Larry also gave a (very good) talk on the GIL, and the barriers to removing it in current CPython.
The problem of course is reference counting, which is inherently thread unsafe. Adding locking to reference counting gives a performance hit to single threaded code of about 30%.
A "better" approach would be to move CPython away from reference counting to a more modern garbage collection scheme (something like a generational collector). The difficulty is that existing C extensions would be incompatible with this new CPython as C extension objects would be opaque to the new collector until they're modified (so the new collector is able to find references they hold to other objects). So you would break compatibility with every existing C extension.
The experience of pypy shows that breaking compatibility with C extensions is a deal breaker for many people, and it's a very hard road to substantial adoption with that barrier in place.
Larry's conclusion is that the performance hit for single threaded code (locking around the reference count) is probably the only viable option if we want free threading in CPython.
I worked for a while at a company called Resolver Systems building a spreadsheet application in IronPython. A lot of people wanted to use numpy in their spreadsheets, but IronPython didn't have the Python C API - so numpy didn't work (and a lot of other Python code that relied on C extensions). We (well, mostly ) implemented something called IronClad. A combination of C#, IronPython, C and assembly code that implemented the Python C API for IronPython. It had binary compatibility with compiled windows extensions (existing C extensions didn't need to be recompiled to work with IronClad). Quite an amazing, and terrifying, piece of engineering.
Because it worked with IronPython, built on .NET, it needed to work without a GIL and with the generational collector. We just pretended that the GIL not being there wasn't a problem (the acquire and release GIL C API methods were empty stubs). This never was a problem for us. For C extensions to still be able to use reference counting, and .NET garbage collection to also work, we needed a hybrid approach.
I've sketched out a potential hybrid approach for CPython that would (if it is sound) allow CPython to move to a more modern garbage collection scheme whilst remaining compatible with C extensions that haven't been updated to be compatible with the new collector. It would require extensions to be recompiled, so it breaks ABI compatibility.
Objects that don't support "new garbage collection" (ngc) are opaque to the garbage collector, they are considered to hold no references which are then tracked externally. So reference counting becomes slower (!), but it's only "old" extensions that are using it.
The reference count field remains for all objects, but is unused by normal Python objects.
Access to the ref count field is protected by a lock to be thread safe.
IncRef and DecRef macros become very different (but are, again, not used by standard objects).
Objects incompatible with "new garbage collection" need a new field. This is the tricky bit if it's to be done without requiring the objects to change - the field might need to be created and tracked by the interpreter rather than the extension. (That's the first hand-wavy bit.) The field is a "reference pool", a list of weak references to objects referenced by the opaque object.
When incrementing a reference count, if the count goes from zero to one a reference is also added to a global pool. This keeps the object safe from "normal garbage collection". A new weak ref to the object referenced is added to the reference pool for the referer. (This is the really tricky bit IncRef needs to know which object is doing the incrementing. So effectively IncRef needs to become an instance method. IronClad didn't have this problem because it only ever used "shadow objects" - a kind of proxy to the real Python object because IronPython objects have a very different memory layout to the one that Python C extensions expect - so we only needed to track the reference count on the shadow objects.)
When decrementing a reference count, if the count drops to zero the reference from the global pool is removed, once again making the object available for normal collection.
When an opaque object is collected by ngc, all the objects held in the "reference pool" for that object need to have their reference count decremented.
So, performance for reference counting extensions will be worse until those extensions are updated to support ngc - but the hybrid system should work.
Another issue is probably that a moving collector would also be incompatible with C extensions that use direct memory access (possible for things like lists). We had that problem theoretically in IronClad but never had any actual problems due to this. Well behaved C code ought to be going through API methods to access members rather than direct memory access anyway.
Crossing the Soul Gap: A Rational Faith
We're using something called MAAS "Metal As A Service" - a Canonical open source product for managing servers. The combination of MAAS, Juju and OpenStack is what many of our big customers are using. They're paying us to create and manage big deployments of OpenStack on many servers. The servers are managed by MAAS and OpenStack is put on the servers with Juju. So ensuring that Juju works well with MAAS is very important to us.
The servers I'm using are these nice, and cheap, HP Proliant boxes. In combination with FreeNAS they make great NAS devices too.
It's about devops and the awesome devops tool we work on, Juju. To the clouds:
tenho 3 checbox and 1 button , ao clicar no botão executa o programa selecionado no checkbox ???
segue meu código :
import wx, os, sys
def _init_(self, *args, **kw):
super(executa, self).__init__(*args, **kw)
pnl = wx.Panel(self)
cb1 = wx.CheckBox(pnl, label='CALCULADORA', pos=(15, 50))
cb2 = wx.CheckBox(pnl, label='BLOCO DE NOTAS', pos=(15, 70))
cb3 = wx.CheckBox(pnl, label='PAINT BRUSH', pos=(15, 90))
btn = wx.Button(pnl, label='ABRIR', pos=(80, 150), size=(60, -1))
self.SetTitle('abre programa selecionado')
def check_value(self, e):
os.system("start /wait calc.exe")
os.system("start /wait NOTEPAD.exe")
os.system("start /wait PBRUSH.exe")
ex = wx.App()
if _name_ == '__main__':
- Canonical Ltd.Software Engineer, 2010 - present
- Comsulting.deSoftware Engineer, 2009 - 2010
- Resolver SystemsSenior Software Engineer, 2006 - 2010
Michael writes and blogs about Python far more than is healthy, and in the real world lives in Northampton, UK, with his wife Delia.
BTGuard Review | BTGuard Reviews | Hide IP Tools Reviews
Initial check of new torrents and acqusition of seeds is a bit slower than normal, but afterwards the transfer speeds match watch I was gett
Troy Hunt: Lessons in website security anti-patterns by Tesco
Troy Hunt on observations, musings and conjecture about the world of software and technology
Forcing web API consumers to be robust | tartley.com
I have a web API that returns object attributes as JSON, i.e. name=value pairs. We're discovering that adding new fields breaks a lot of
I Voted for New Zealand Lamb Shanks from Westin Gourmet!
Vote for you favourite at www.westingourmet.co.uk/chooseyourgiveaway, or why not try the Lamb Shanks for yourself www.westingourmet.co.uk/ne