I've been following PyPy for awhile. It's an incredibly ambitious project, and the fact that it is now close to being a competitive Python implementation for real-world use is amazing.
A brief summary of why the project is so cool: the actual Python interpreter is written in a subset of Python known as RPython. The actual Python interpreter can by run sandboxed within another Python process, for example (albeit slowly). That also means the actual language implementation can be clean, concise, and beautiful like a relatively high-level language allows.
They also wrote a JIT generator that takes an interpreter written in RPython and makes a JIT out of it. Right now, the JIT generated from their Python interpreter could fairly be called faster than CPython and will probably only improve.
The neatest part is that it is feasible that someone could write an interpreter for another language, like Ruby or Javascript, in RPython or a language semantically equivalent to it. Conceivably the JIT generator could then compile it down to a very fast language implementation, with little optimization work required by the language implementor.
This kind of separation of concerns is something that could produce absolutely incredibly practical value in the near future. Even if PyPy doesn't work out (it is way behind their anticipated schedule, but I think it will) the developers behind this deserve a lot of respect for breaking so much ground.
It looks like they also have a very neat and clear way to measure and visualize the speed of PyPy across version: http://speed.pypy.org/overview/ -- didn't someone say that what you can measure you improve?
It's complicated :) PyPy's objects are significantly smaller than CPython ones, however the the assembly the JIT generates, plus the bookeeping and profiling data it stores take up space.
Pretty impressive the benchmarks, though they are getting close, having to re implement all the C extensions is definitely going to suck -- you can for example use twisted, but nothing that uses SSL, because twisted normally calls into pycrypto for them :(
Twisted uses PyOpenSSL for SSL. Twisted Conch (the SSH server/client) uses PyCrypto. Also, the CPython C extension API is being implemented in the pypy branch cpython-extension.
Even if PyOpenSSL doesn't work right now, servers can use SSL proxies (either user-level, or in-kernel, such as OpenSolaris' KSSL)
sorry, I forgot about the split of the dependencies, we use both Conch and ssl sockets at work. Using SSL proxies is a complication because you normally end up killing the client IP addresses without more creative proxies.
Some of the PyPy developers have started working on a branch that will provide a bridge API, allowing CPython extension modules to be used inside of PyPy. As of today the regular expression engine from CPython can be successfully compiled and imported (though isn't totally functional). IronPython has a similar project named IronClad.
Using native libraries using CTypes should probably be the way forward from now on. This allows one to use pure Python to wrap C libraries, in a manner not specific to the Python implementation.
In the Ruby world, the FFI library has accomplished the same thing. Extensions written using FFI work in Ruby 1.8, 1.9, JRuby, and Rubinius. They also avoid the annoying compilation phase.
There is generally a performance hit compared to native extensions, though.
A brief summary of why the project is so cool: the actual Python interpreter is written in a subset of Python known as RPython. The actual Python interpreter can by run sandboxed within another Python process, for example (albeit slowly). That also means the actual language implementation can be clean, concise, and beautiful like a relatively high-level language allows.
They also wrote a JIT generator that takes an interpreter written in RPython and makes a JIT out of it. Right now, the JIT generated from their Python interpreter could fairly be called faster than CPython and will probably only improve.
The neatest part is that it is feasible that someone could write an interpreter for another language, like Ruby or Javascript, in RPython or a language semantically equivalent to it. Conceivably the JIT generator could then compile it down to a very fast language implementation, with little optimization work required by the language implementor.
This kind of separation of concerns is something that could produce absolutely incredibly practical value in the near future. Even if PyPy doesn't work out (it is way behind their anticipated schedule, but I think it will) the developers behind this deserve a lot of respect for breaking so much ground.