Yet another awesome use of the Synergy app

I’ve mentioned before that the Synergy app is awesome.

The image to the right is a configuration at my friend’s house where my laptop (bottom) is controlling his dual-monitor computer setup (one regular monitor and an HD TV on top).

It works great as a remote and, yes, you can watch Hulu on it without being charged for watching Hulu on a TV device.

There are a lot of new internet TV solutions, but with just a little bit of extra work a very easy-to-control TV setup works that’s far more flexible than what’s on the market today.

That being said, I do own an Apple TV and what’s nice about these types of devices is that they are small and focus on a simple user interface to get access to content.

Speed up your Python: Unladen vs. Shedskin vs. PyPy vs. Cython vs. C

Lately I’ve found that prototyping code in a higher-level language like Python is more enjoyable, readable, and time-efficient than directly coding in C (against my usual instinct when coding something that I think needs to go fast).  Mostly this is because of the simplicity in syntax of the higher level language as well as the fact that I’m not caught up in mundane aspects of making my code more optimized/efficient. That being said, it is still desirable to make portions of the code run efficiently and so creating a C/ctypes module is called for.

I recently created an application (I won’t go into the details now) that had a portion of it that could be significantly sped up if compiled as a C module.  This spawned a whole exploration into speeding up my Python code (ideally while making minimal modifications to it).

I created a C module directly, used the Shedskin compiler to compile my Python code into C++, and tried the JIT solutions PyPy and Unladen Swallow.  The time results for running the first few iteration for this application were surprising to me:

cpython:        59.174s
shedskin: 1m18.428s
c-stl:             12.515s
pypy:           10.316s
unladen:       44.050s
cython:         39.824

While this is not an exhaustive test, PyPy consistently beats a handwritten module using C++ and STL!  Moreover, PyPy required little modification to my source (itertools had some issues) [1].  I’m surprised that Uladen and Shedskin took so long (all code was compiled at O3 optimization and run on multiple systems to make sure the performance numbers were relatively consistent).

Apparently out-of-the-box solutions these days can offer nearly a 10x improvement over default Python for a particular app. and I wonder what aspects of PyPy’s system accounts for this large performance improvement (their JIT implementation?).

[1] Uladen required no modifications to my program to run properly and Shedskin required quite a few to get going.  Of course, creating a C-based version took a moment :-).

Update 1: Thanks for the comments below.  I added Cython, re-ran the analysis, and emailed off the source to those who were interested.

Update 2: The main meat of the code is a nested for loop that does string slicing and comparisons and it turns out that it’s in the slicing and comparisons that was the bottleneck for Shedskin.  The new numbers are below with a faster matching function for all tests (note that this kind of addition requires call ‘code twiddling’, where we find ourselves fiddling with a very straightforward, readable set of statements to gain efficiency).

cpython:       59.593s
shedskin0.6:   8.602s
shedskin0.7:   3.332s
c-stl:              1.423s
pypy:             8.947s
unladen:       29.163s
cython:         26.486s (3.5s after adding a few types)


So C comes out the winner here, but Shedskin and Cython are quite competitive.  PyPy’s JIT performance is impressive and I’ve been scrolling through some of the blog entries on their website to learn more about why this could be. Thanks to Mark (Shedskin) and Maciej (PyPy) for their comments in general and and to Mark for profling the various Shedskin versions himself and providing a matching function. It would be interesting to see if the developers of Unladen and Cython have some suggestions for improvement.

I also think it’s important not to look at this comparison as a ‘bake-off’ to see which one is better.  PyPy is doing some very different things than Shedskin, for example.  Which one you use at this point will likely be highly dependent on the application and your urge to create more optimized code.  I think in general hand-writing C code and code-twiddling it will almost always get faster results, but this comes at the cost of time and headache.  In the meanwhile, the folks behind these tools are making it more feasible to take our Python code and optimize it right out of the box.

Update 3: I also added (per request below :-)) just a few basic ‘cdef’s and types to my Cython version.  It does a lot better, getting about 3.5s on average per run!

A couple of recent Python tidbits

Five years ago the two most practical and used computer languages for me were C and Perl.

While I still use C for the nitty-gritty stuff that needs to be fast, I’m finding that a lot of stuff can get done, and get done very fast in a scripting language.  Over a year ago I learned the wonders of Ruby (which, to me, is basically a superior replacement to Perl from a ‘it’s fun to code in’ perspective  and is easy to transition to from Perl).

But overwhelmingly, I’ve found myself enjoying and using Python.  The biggest selling point from my perspective is the high quality scientific computing and plotting support (which in many cases has replaced my use of R-project for these types of things).

Here are three little tidbits that I’ve recently found handy and take virtually no effort to begin using:

(1) First,to speed up the things that need the speed, calling C functions from Python is super-handy.  I really like the ctypes module because in many cases, as long as your C functions take the default types as inputs, you can simply expose your function via a dynamic library with no real extra effort.

(2) List comprehensions are a fun and very useful feature in Python.  But many times, you don’t want to create the whole list before iterating through it.  Here’s where generator expressions come in.  Instead of brackets around your comprehension, just put parens, and suddenly you can iterate over these elements without creating the entire list in advance (especially useful with a module like itertools).

(3) Finally, one thing that had bugged me a little in Python, especially since I’m used to C, is that there really is no real scanf equivalent [1].  What I end up doing is parsing out my variables from a string and on separate lines explicitly setting their type.  This just takes too many lines (and I could often do it in fewer lines in C)!   After some thought, a lab mate and I converged on:

>>> types = [float, str, int]

>>> fields = [‘3.14159’, ‘Hello’, 10]

>>> pi, hi, ten = [f(x) for f,x in zip(types,fields)]

[1] Note that while there is a suggestion in the Python docs on how to do this, it just suggests how to extract different types with regular expressions, not concisely convert them.