Phil Hassey - game dev blog
Phil Hassey as Snidely Whiplash
"You can't buy awesomeness.
You're born that way."

tinypy 64k – nearing 1.0 – and it really does fit in 64k!

Updates – I’ve got it all fitting in 64k*. It’s amazing how many functions that don’t do anything you can come across if you look around long enough. Not to mention how many little things you can trim out that don’t actually do anything. I have no idea where all this cruft comes from, but having a nice suite of tests sure is helpful for re-working stuff. I also cut out a bunch of stupid features nobody would ever use**. I was able to reduce the number of native types from 9 to 7.

One of the challenges I faced was trying to fix up the incremental garbage collection. My initial implementation was rather inefficient and caused some odd problems with how I wanted to code things. I was using a dict to store all the “white” items, which caused loads of dict hash lookups.

So in my mind I crafted a grand vision on how to accomplish this goal. I would adjust all objects (sans numbers) to contain a pointer to some data which would have some header data for the GC to do some bookkeeping in. Great! However, when I implemented this, I found that a number of problems presented themselves: I had to perform a malloc for each and every string that I used, which killed performance, actually making things 2-3x slower. I also noticed that the weird struct I defined was maybe a bit less standards compliant. This attempt was a wash.

So I re-crafted my grand vision. This time I would do the same thing. Brilliant aren’t I? Anyway, the results were basically the same. Who’d’ve thunk? It was slower again, this time I was quite confused by it, since I had worked around some of the string issues. I also found that the API for creating new strings wasn’t quite as “clean” as my original simple one. This caused some issues in the exception handling mechanism. I had to toss this try as well.

At this point, having re-mangled the code twice and having poor results, I suspected something else might be wrong. My brain was turning into mush. Each time I had completely edited my “tp.h” with all my struct changes in one go. I decided to make a final attempt at reworking tinypy, this time *one* data type at a time. After each data type I added I was able to see if my changes caused any performance issues. I found that my function data type was the culprit. My hashing function (borrowed from lua) wasn’t getting enough entropy and was generating massive collision cases! A few tweaks later, this was resolved. I was able to also craft the string interface to be backwards compatible with the original string interface while also working with the new garbage collection. This “step by step” approach got me to my goal. All said and done, with a bit more tweaking, I was able to *double* the speed of tinypy 🙂

Lesson learned – even if it’s only 64k, it’s better to do changes step by step instead of in one big go. valgrind and callgrind are your friends. (Although I found that tinypy doesn’t entirely agree with callgrind … ideas anyone?)

To wrap up this excessively long post about me trying to get code to work — this weekend I’m hosting a Ludum Dare warmup compo. I’m going to give tinypy a run in the “real world”. Here’s to hoping! Next week I plan on releasing the 1.0 version of tinypy.

I’m also thinking about renaming some of my files. And although pylang, dumbparse, and dump2vm have a certain rustic charm, I wonder if I’d do better with names like goat, gorilla, and sausage. Or maybe more descriptive names like tokenator, parsalizer, and bytecodatron.

svn:// for the brave. If you want a zip or an exe, check back in a week. I’ll have all those and more (a game!) Note that I’ve split the SDL dependency out of the main tinypy code. tinypy-sdl.c lets you run my example. The bootstrapping process also has a final step of compiling with -O3, which I think might not work for everyone. It gives pretty good speed gains on my system, so if it works for you, great!

*python will do a bit of search-n-replace to cut it down to size. I’ve resisted doing anything really ghastly, the code is still indented and readable. See README.txt for more disgusting details on how I cheat to pretend this is 64k.

**Okay, I’ve used some of those features. But hey, this is a 64k implementation, I’ve got to trim the fat.

8 Responses to “tinypy 64k – nearing 1.0 – and it really does fit in 64k!”

  1. Ali Afshar Says:

    I have been following from the sidelines (via planet python). It’s great to see you are making progress on this bizarre and interesting project. Keep it up!

  2. Michael Dillon Says:

    How does this compare to PyMite which also claims to be able to run on 8-bit microcontrollers with as little as 64kb of program memory and 4k of RAM?

    Have you done a feature comparison of the two?

    Is it possible that you could use some of the techniques from PyMite to make a microPython that is closer to standard Python and more generally usable?

    I ask these questions because it seems that PyMite and tinypy are on the threshold of making Python a viable language for embedded systems development.

  3. James Tauber Says:

    At some point, I’d love to tie this in to Cleese.

  4. philhassey Says:

    pymite looks pretty neat. I hadn’t heard of it .. it doesn’t seem to compile though 🙁 Anyone got any ideas?
    global.c:112: warning: dereferencing type-punned pointer will break strict-aliasing rules

    I don’t think tinypy would do so well with minimal amount of RAM right now, but I bet someone could create a branch of it that did.

  5. Paul Boddie Says:

    PyMite doesn’t support classes and a certain amount of other stuff:

  6. philhassey Says:

    note: valgrind issues resolved by downloading the source and compiling the latest release. I haven’t updated my distro in a while, so most of my packages are out of date.

  7. Truy Says:

    Please port this to the iphone / ipod touch ! Pretty please?

  8. philhassey Says:

    Truy – heh, maybe if you buy me an iphone 🙂 I’ve actually no idea if that would be hard to do or not. After the 1.0 release maybe you should give it a go!