Galcon Games
Phil Hassey - game dev blog
Phil Hassey as Syndrome
"Look, stores don't sell
costumes like this to just anyone."

Restricted tinypy to C++ compiler

elephant1I’ve spent the last week working on a tinypy to C++ converter.  It works!  See the screenshot to the right – I’ve managed to port a pygame game over to C++.

Here’s how (and some of the catches):

  • I require type annotation of all the functions and methods.  ”def test(x:str)->str: return x”
  • I do two passes on each file, the first pass to catch all the function types and class members, and the second pass to generate the code.
  • I generate C++ code that has automagic reference counting.  So you have to code your script so it won’t have any cyclic references if you want garbage collected for you.

How is this different from shedskin (really cool project!)?

  • Built-in reference counting, instead of using libgc.
  • I require the user to type annotate everything.
  • It only supports a subset of the tinypy subset of python.  Shedskin supports a much larger subset of python.

So what’s the point?

  • Well, I learned a lot about STL and C++.
  • I know it will produce iPhone friendly code, I’m pretty sure libgc isn’t iPhone friendly?  (At least, I haven’t found anything via a few searches…)
  • Way less magic.  Since everything is annotated, there are no surprises.
  • Implementing C++ modules is pretty easy – the code can be inlined within the python code and it just works.
  • This will make it easier for me to develop C++ games.

Anyway, if you’re super brave, you can check out svn://www.imitationpickles.org/tinypy/branches/tinypy2 .. I don’t have the elephants example in there, but the pygame.py that I include gives you a pretty good example of a complex module.

I’m going to chat with the tinypy folks to see if we’ll merge this into the tinypy trunk or have it as a separate project.  I’m not quite sure what makes sense to everyone else :)  The nice bit about merging this in is that I could unify the test suites nicely.  And tinypy would still function as normal, just better tested, and with function annotation parsing supported.  All that said, I should make a tinypy module for tinypyC++ so that I can do some code evals!

14 Responses to “Restricted tinypy to C++ compiler”

  1. Alex B Says:

    How much does the generated code depend on C++ features? Any chance for translation to plain C?

  2. Donny Viszneki Says:

    I’ve been thinking of doing something like this for Javascript. Thanks for publishing your work! If you have any advice about what was difficult or what you could have improved, please post!

  3. philhassey Says:

    @Alex: it is very dependent on C++. However, take a look at encode2.py and builtins.h .. There’s no reason why you couldn’t comb through those two files and change them for outputting C code.

    @Donny: No problem :) The thing that “worked” was having a test suite for tinypy already, I really just had to comment out the tests that were “dynamic” and just grind through them one at a time until it all worked. And then I had to add more tests. Really, just a long string of tests that sort of forced me to coax the code into working.

    The 2 pass type inference I do is pretty dodgy, and I think it could use some tightening up, but it seems to work in a pinch.

    I’m not sure how practical all of what I did was, but I learned quite a bit of C++ which I’m sure will be useful.

  4. Rene Dudfield Says:

    nice work Mr H!

  5. Ginta Says:

    @Donny: You might tae a look at pyjs.org. It does something like this for javascript.

  6. rockins Says:

    Owesome, Phil.

  7. GWF Says:

    This looks very fun and interesting. Do you have any sense of the performance between cpython, tinypy, and the generated C++?

  8. philhassey Says:

    @GWF: C++ is like 4-40x faster than Cpython which is about 2x faster than tinypy.

  9. mark dufour Says:

    looking at the shedskin test set, I’d say the performance difference is more like 1-100x, and depends very much on what you’re doing. the cpython builtins and memory subsytem are rather efficient. so if you’re using strings a lot, or sets, and/or many other small objects, chances are a (straightforward) translation to C++ is actually not that much faster and sometimes even slower. but of course usually there is a good speedup, especially if you avoid things that really slow C++ down (usually memory allocations).

  10. philhassey Says:

    @mark – that’s quite true about memory allocations. With the julia.py example (included in svn right now) you can see that I create a single “pos” object for the pixel position to be passed to the set_at() method.

    I did this because I found that creating a new list for each pixel was really slowing things down. By re-using the same list, I got a major speedup.

  11. Ryan Schneider Says:

    Does it support creating C++ classes consumable within C++? That is, can I define a Python class and then refer to the C++ized class from outside of Python, or does it just do functions?

    I’ve been thinking about prototyping an iPhone game I’d like to build in pygame, and it’d be really cool if I could reuse my game logic module in the actual game.

  12. phil hassey Says:

    @Ryan – yeah, it generates classes that can be used in C++ :) Check out the svn and go into examples and run ./build .. Check out the pygame.hh and pygame.cc that are produced.

    (You might need to do an ln ../builtins.h builtins.h to get it to work)

  13. Shared Items – December 6, 2009 | booyaa dot org Says:

    [...] Restricted tinypy to C++ compiler [...]

  14. gautier Says:

    OMG OMG I was thinking about doing exactly this !!


Galcon   Watermelons   Dynamite   The Hairy Chestival
All content of imitation pickles (c) 1999-2008 - Phil Hassey  "we care"