Things you would like to see in a tool or the class library.

Method annotations

Postby jan_vrany » 10. Dec 2008, 16:25

Method annotations or pragmas (VW terminology) is very powerful feature, that can be used
in many cases - or VW style menu extension
methods are just two of them.

St/X support for user-defined annotations (I prefer the term "annotation" rather than "pragma") is very limited.
One can use just "<resource: #symbol #(array of symbols)>". Other annotation-like beasts like "<cdecl: ...>"
are handled specially in the parser. Moreover, resources are returned just like symbols, not instances of
particular class. And, resources are parsed from the source whenever a method is asked for them. Obviously, this
is really slow - I tried to use special resource to implement menu extension methods and finally I gave up because
of poor performance.

My idea is to extend/hack Smalltalk/X to allow method annotations to be any object (just like in SELF).
For instance:
Code: Select all
     <MenuExtension extends: #classMenuSpec insertAfter: #Repository>
     "menu spec follows..."

Should create an annotation of class MenuExtension with extends and insertAfter instvars filled
(object should be created in compile-time) This object should be associated with its method.
In other words, querying method for annotations should be fast. Ideally as fast as instvar return :-)

Implementation issues:
- a Parser class mush be hacked a bit.
- Ideally, a new instvar, say annotations should be added to class Method. Alternatively, this can be
workarounded using WeakIdentityDictionary mapping instances of method to collection of its annotations.
- hasResources should be renamed to hasAnnotations :-)

Ideally, one day, method package or category will become just another kind of annotation (just like in SELF).
Yes, in this case eXept guys will have to change stc compiler. That might not be so easy, I know.

So, does it make sense for you? Any suggestions or comments?
Posts: 28
Joined: 10. Dec 2008, 12:37

Re: Method annotations

Postby cgittinger » 11. Dec 2008, 17:24

Oh yes - you are absolutely right in all of your points.
The big obstacle in the way is our old friend, the stc compiler. That one is (still) written using yacc and lex, in C. Therefore, it has very limited chances of doing any real work at compile time - especially, and for this issue relevant: it cannot evaluate any smalltalk. I must admit, that I do have a love/hate relationship to stc; on one hand, its an inflexible beast, written in C. On the other hand, its the one tool which allows for most of ST/X's portability, helps to bootstrap to other architectures and also generates the standAlone dll and exe objects.
We could enhance stc to pass some of the parsed code unevaluated and have the initialize code handle this later, when smalltalk has control. That would mean, that the annotation-contents is passed untranslated by stc to some later stage during the startUp and then compiled by smalltalk (possibly lazily).
However, we have actually stopped further enhancements of stc because we think that its too hard to maintain and needs a replacement:
For quite some time, I have started to work on a new compiler, which uses the same front-end as the bytecode compiler but takes the generated bytecode and generates C from those. It is written in Smalltalk and will need the old stc for bootstrapping. Obviously, it will understand Smalltalk and will be a good basis to implement all of the features you aked for. Right now, I cannot tell you when that will be ready for production - it is in a releatively early stage, and definitely not stable enough for publication. It will also need a different librun, as I plan to generate continuation-capapable C-code (for all those seaside fans). I will keep you informed. Is anyone willing to help working on that ? I could also think about some bachelor/master/diploma themes based on that...

Posts: 61
Joined: 12. Nov 2008, 17:05

Re: Method annotations

Postby jan_vrany » 11. Dec 2008, 18:35

Yeah, stc is a problem. Passing untranslated annotations in class-library *_Init(phase, pIF) routine
is good idea. As a consequence, we cannot have compile-time checked annotations like Java ones
(I'm not saying that we need this, just stating that we cannot have it).

cgittinger wrote:Is anyone willing to help working on that ? I could also think about some bachelor/master/diploma themes based on that...

Me, definitely.
BTW: Just before an hour, one student asked me for a project/master thesis :-)

So, where to start? At least, Parser has to be hacked...
Posts: 28
Joined: 10. Dec 2008, 12:37

Re: Method annotations

Postby cgittinger » 11. Dec 2008, 23:44

I do have some frontEnds already; so the parser is not the biggest issue at the moment.

The idea is to have multiple frontEnds, generating intermediate code for various backends. I have frontends for ST, JavaScript (and it should also work for things like PHP, Python etc).
The ST frontend's testSuite parses and translates all of the existing smalltalk code.
I wanted to have a Java and C# frontend as well, to be able to suck in lots of existing code. It has alerady been proven (although not finished to product maturity) that Java code can co-exist within a smalltalk image; both by the VisualAge-for-Java guys, and by my own ST-VM integration. The same is true for C#.

I have backends for byteCode (almost complete), direct-C-Code (some) and tree-code (lisp-like).
My far-end goal for the in-image translation actually was to get rid of bytecode, and try tree-code, because I dont see any reason for the jitter to reanalyse the bytecode to find loops and other obvious constructs. Bytecode is a yesterdays solution to a yesterdays problem (small memory). If all of the bytecodes where replaced by treecode, the system would only be a few megabytes larger - who cares these days :P

I also have a bytecode to C-code translator which is almost finished. It can read all of the existing smalltalk code and generates valid C-code for methods and blocks. I am working on the Init-function code generation and the rest of the infrastructure interfacing to librun. My plan was to generate compatible code first, but migrate to a new librun later. I have also started some little experiments in the jitter area (doing it in smalltalk). Maybe we should merge in the work of the squeak guys (Eliots work and the Exhubery things).

The bytecode-to-c-code translator generates continuation-passing c-code. That means, that it will support seaside, prolog-like backtracking and othr fancy control when finished (Its the same kind of code which is generated by some Lisp and ML systems, which target for C). However, as of today, I do not have runtime support for it. As part of my lectures, some of my students and me developed a self-bootstrapping minimalist lisp engine, which is also able to compile itself and also works with continuation passing. My plan is to bring this (lisp-encoded) VM underneath and use it for both lisp and ST (with a flexible object model which supports both). The big problem with the students is that I got freshmen every other semester, so each time I had to start from scratch with them :lol:
The situation will probably be better next year, when I might get both bachelor and master students (its not yet fixed, but likely). I'd really love to continue with that work as a dualistic scheme/ST engine and IDE would definitely be great.

All of this is and has been an idle-cpu-time task of myself, and is not making progress in the speed I would like. So help would really be welcome at many places. I would also get rid of the existing VM and the existing object model, and trade memory usage for flexibility.

So, where do you want to work on ? :lol:

Posts: 61
Joined: 12. Nov 2008, 17:05

Re: Method annotations

Postby jan_vrany » 12. Dec 2008, 00:42

Huh, you did a lot of work already! Nice.

The system you are describing is something I'm dreaming about
(to say truth, many of my ideas are influenced by your post in VW wiki @
Actually, I'm trying to do my PhD in this area.

I did some initial experiments, using SELF as a bootstrap language. I've some
hardcodes AST-based SELF interpreter, initial implementation ObjectMemoryBackend.
(all implemented in St/X cos I don't like to prototype in C :-)
I also created a library for creating AST interpreters with debugging facilities
and created unified debugger, allowing one to debug application consisting of
byte-code based code written in St/X and AST-interpreted DSL code in single environment.

So I definitely would like to cooperate on topics you outlined. It will help me a lot,
since I'm the only one in Czech Rep working in VM area...
Is there any concrete task I can work on?
Posts: 28
Joined: 10. Dec 2008, 12:37

Return to Wishlist

Users browsing this forum: No registered users and 1 guest

Design by GB