Phil Hassey - game dev blog
Phil Hassey as Rambo
".. I've been there,
I know what it's like,
and I'll do it again if I have to."

64k tinypy – now with list comprehensions and fancy arguments

Well, I got two of my favorite python features added in – list comprehensions [x*x for x in range(1,5)] and fancy arguments test(1,2,a=3,*c,**d).

Adding list comprehensions was painless, took only a few minutes. The arguments change was rather difficult because I had to add more rules to the parser, change how the bytecode was outputted, and rework all the internal calling stuff in the VM. Now every function takes a single argument of type ‘params’ which can contain all the details of a call.

I’ve also added in better error handling. Errors print out a backtrace of the function calls and lines of code. This is making debugging everything much easier.

I’m also finding that the more features I add, well, the slower things get. I’m also coming up against my 64k limit, so I’ve moved my tests out of the core code into a separate tests.py file to ensure that I don’t “trim down the tests” to give myself more space 🙂 The tests I’ve made have been invaluable in making it possible to do big changes fairly quickly.

Next on the agenda – either bootstrapping, or making a game with it 🙂

For the brave: svn://www.imitationpickles.org/tinypy/trunk ; python tests.py ; ./run_julia_o3 (not very fast)

12 Responses to “64k tinypy – now with list comprehensions and fancy arguments”

  1. Anonymous Says:

    I really like what you’re doing here. The build system is a bit overtly dirty (but because you’re one guy and focusing on the vm, that’s nothing to scoff at).

    I’m a game developer, and things like this are going to be motivational when writing a CPython implementation for Xbox360 or PS3.

  2. philhassey Says:

    Heh, yah, it’s all in the “just hacking it together” stage, of course 🙂

    The main use-case I have for what I’m doing is games, so stuff like that is part of what I’m looking at doing, maybe.

    Thanks!
    Phil

  3. Chuck Says:

    Question, though. Why did you just not use the py_compile package to convert your python scripts into bytecode?

  4. philhassey Says:

    Chuck – I’m hoping to be able to bootstrap this project so that in a game you could compile and run scripts on the fly. Maybe 🙂

  5. Nymius Says:

    Have you seen pyvm?

  6. philhassey Says:

    No I hadn’t, that’s really cool! I’ll be looking into that stuff a bit more. I wonder if I could run Galcon with it …

  7. philhassey Says:

    Hmn .. just tried to compile pyvm and got this:

    ../lwc/objdir/lwc seg-malloc.c+ > cdir/seg-malloc.c
    /usr/include/string.h:9 [Token 7283]: invalid declaration separator : `__asm__’
    Somewhere in ### __errnum , char * __buf , size_t __buflen ) __asm__ ( “__xpg_strerror_r” ) __attribute__ ( ( __nothrow__ ) ) __attribute__ ###
    make[1]: *** [cdir/seg-malloc.c] Error 1

    Hmn ..

  8. Michael Foord Says:

    Congratulations. This is a *really* interesting project. Good luck and keep hacking. 🙂

    Fuzzyman

  9. Rene Dudfield Says:

    Very cool!

    Does this version have very fast image manipulation? Like with integers and stuff?

    Maybe you could fast path the basic function call somehow? like f(x) ?

  10. philhassey Says:

    Rene – no such luck at this point, but if you roll back to my earlier versions, the python->C code does some interesting stuff. Now I’m targetting more just doing a good VM setup. However, it would be possible to write a JIT function that took a function and generated fast asm functions for doing image manipulation.

  11. Carl Friedrich Bolz Says:

    Hi Phil,

    you just discovered rule number one of python reimplementations: everything is fast as long as you don’t have all the features :-). It doesn’t completely apply to you, since your goal is not to reach complete CPython compatibility. But for several Python reimplementation projects it’s some sort of recurring pattern. When the new implementation is started you get extreme speed claims, which are then silently retracted as more features are implemented which slows everything else down.

    Cheers,

    Carl Friedrich

  12. philhassey Says:

    Carl – Yeah, I’m certainly learning (first hand) what each bit of syntax costs 🙂 At present I’m just trying to keep the speed “reasonable”. I think the only unique thing this project might bring to the table is the small code footprint of 64k of readable code. For me that’s important, because working on much larger things just isn’t all that much fun!