[Shedskin-list] shed skin pre-0.0.26 update

[Mark Dufour]

hi all,

just a quick update on some recent improvements in shed skin:

-I added support for most of os.path (nt and posix versions), by
running SS over the CPython implementation and modifying the result a
bit (e.g. a varargs hack for join)

-paul boddie created a nice debian package, including man page, which
I hope to use starting from the next release

-pavel added some os functionality, as a google ghop task:
os.{environ, readlink, fstat, getuid, getgid}
(if there are any high-school students on this list, there are more
tasks!! :-))

-pavel also contributed a nice .html version of the README, generated
from an .rst file.

-another ghop student, robert, is working on basic socket support. I
think he will get this working soon.

-addition of several strings, in the form of 'a+b+c' is really fast
now with shed skin, as it gets compiled to a call to a special
string-addition function.

-overloading builtins, as in 'str = 4' now work, be it that the
respective variable is always assumed to be local ;) (so no real
overloading then)

-someone mentioned it might be possible to use PCRE (perl-compatible
regular expressions) to support the 're' module. it looks like a
really good library, but I'm not sure yet how compatible it is with
're'? well, if anyone would like to look into this, I'd be happy to
help out :-)


thanks,
mark.

[b***@mailas.com]

overloading builtins, as in 'str = 4' now work, be it that the respective variable is always assumed to be local ;) (so no real overloading then)<

I think this is an anti-feature. I'd like CPython to give me a syntactic
error when I "overload" builtins :-) I know that sometimes it's useful,
but it often a source of bugs too.

-someone mentioned it might be possible to use PCRE (perl-compatible regular expressions) to support the 're' module.<

That someone was me, lot of time ago. I think it's not 100% equal, but
often close enough, and probably better than nothing.


When you talk about giving SS "Escape analysis" I presume you are
talking about something like this:
http://en.wikipedia.org/wiki/Java_performance#Escape_analysis_and_lock_coarsening
That is allocating objects on the stack if the compiler can demonstrate
that they are used only inside the scope. The D language has the "scope"
attribute to 'suggest' (not compulsive) the compiler when a class can be
allocated on the stack like that.

Bye,
bearophile

[Kay Hayen]

Hello everbody,

Post by b***@mailas.com

Post by Mark Dufour
-someone mentioned it might be possible to use PCRE (perl-compatible
regular expressions) to support the 're' module.<

That someone was me, lot of time ago. I think it's not 100% equal, but
often close enough, and probably better than nothing.

Just to mention, there are excellent pure python parser generators, but with
C++ as the backend language (as opposed to C as in Cython), I would strongly
suggest to use things like:

http://boost-sandbox.sourceforge.net/libs/xpressive/doc/html/index.html

Quote:
----
xpressive is an object-oriented regular expression library. Regular
expressions (regexes) can be written as strings that are parsed dynamically
at runtime (dynamic regexes), or as expression templates that are parsed at
compile-time (static regexes). Dynamic regexes have the advantage that they
can be accepted from the user as input at runtime or read from an
initialization file. Static regexes have several advantages. Since they are
C++ expressions instead of strings, they can be syntax-checked at
compile-time. Also, they can refer to other regexes and to themselves, giving
static regexes the power of context-free grammars. Finally, since they are
statically bound, the compiler can generate faster code for static regexes.
----

Most people use re module for the simple things in life, most expressions are
static. If Shed Skin was trying to out-perform CPython on parsing, that is
the best chance. The optimizing C++ compiler can perform operations much
faster than PCRE ever could.

That said, I have used PCRE in the past and enjoyed it - verily. It just seems
the wrong approach for a compiler to enforce dynamic translation when there
is no need to.

Best regards,
Kay Hayen

[Mark Dufour]

hi kay,

Post by Kay Hayen
xpressive is an object-oriented regular expression library. Regular
expressions (regexes) can be written as strings that are parsed dynamically
at runtime (dynamic regexes), or as expression templates that are parsed at
compile-time (static regexes). Dynamic regexes have the advantage that they

thanks for the suggestion. static compilation sounds like an
interesting option! do you happen to have any pointers to benchmarks
comparing the resulting matching speed? and I'm guessing there is a
tool to translate string-based RE's to expression-template form?

Post by Kay Hayen
That said, I have used PCRE in the past and enjoyed it - verily. It just seems
the wrong approach for a compiler to enforce dynamic translation when there
is no need to.

if RE usage really becomes much more efficient, static compilation
would probably fit better with SS's philosophy..

anyone willing to look into this further..?


thanks again,
mark.

[Kay Hayen]

Hello Mark,

Post by Mark Dufour

Post by Kay Hayen
xpressive is an object-oriented regular expression library. Regular
expressions (regexes) can be written as strings that are parsed
dynamically at runtime (dynamic regexes), or as expression templates that
are parsed at compile-time (static regexes). Dynamic regexes have the
advantage that they

thanks for the suggestion. static compilation sounds like an
interesting option! do you happen to have any pointers to benchmarks
comparing the resulting matching speed? and I'm guessing there is a
tool to translate string-based RE's to expression-template form?

Like I said, you are using way too little Boost libraries. Many of them can do
things during compile time that were previously thought impossible.

No idea about such a tool though. But it should be straightforward. For a unit
test, you will need it.

Yours,
Kay

[Mark Dufour]

hiya bearo,

Post by b***@mailas.com

overloading builtins, as in 'str = 4' now work, be it that the respective variable is always assumed to be local ;) (so no real overloading then)<

I think this is an anti-feature. I'd like CPython to give me a syntactic
error when I "overload" builtins :-) I know that sometimes it's useful,
but it often a source of bugs too.

often str and such are only locally overloaded ('for str in
file(..):'), so SS can still compile things fine. but I agree it's
rather ugly programming. I think I will add a compiler warning.

Post by b***@mailas.com

-someone mentioned it might be possible to use PCRE (perl-compatible regular expressions) to support the 're' module.<

That someone was me, lot of time ago. I think it's not 100% equal, but
often close enough, and probably better than nothing.

you probably suggested it too, yes.. :-)

Post by b***@mailas.com
When you talk about giving SS "Escape analysis" I presume you are
http://en.wikipedia.org/wiki/Java_performance#Escape_analysis_and_lock_coarsening
That is allocating objects on the stack if the compiler can demonstrate
that they are used only inside the scope. The D language has the "scope"
attribute to 'suggest' (not compulsive) the compiler when a class can be
allocated on the stack like that.

yes. this is a more specific entry:
http://en.wikipedia.org/wiki/Escape_analysis

for my thesis, I implemented a really simple escape analysis, and
already got a 15% performance improvement or similar on average. I
think it's one of those things that are much easier to add to a static
compiler than to a specializing JIT, but I might be wrong. note that
(see my thesis) you can also translate heap allocation into static
preallocation in some cases. all really cool stuff, but enough
material for a separate thesis.. :-) (if anyone has seen a (PhD)
thesis on this topic, I'd be very interested in reading it.)


mark.