S-Lang scripting for MC

[psprint]

Hi
I'm sending a patch that is a proposition for:

• having a scripting language support in MC,
• having a plugin API support in MC.

The language/engine is S-Lang from the library that's already linked with MC – libslang.so. The syntax is very friendly as it's basically almost the same as syntax of C. Usage of the scripting is also very friendly, thanks to the tool ​SLIRP which can be used to add C-backed functions to the interpreter (basically, it's binding process). The tool is very mature and well coded, it has many features (look at the file slirprc in the patch to see how the number of char array elements is automatically passed to the C function and omitted from the interpreter function).

I've exported a few functions, such as get_byte or cursor_offset, etc. The task isn't yet finished, as many other functions have to be exported, such as display moving function. Also, a few mechanisms have to be came up with.

There's a proof of concept plugin grow_shrink_integer.plugin.sl implemented (see the patch). Plugins are automatically sourced from ~/.config/mc/plugin dir.

What do you think? Does this light and very nice scripting mechanism have a chance of merging?

PS. I attach also a PDF with description of the current scripting API.

[psprint]

On this occasion I would like to ask when will be any (i.e.: first) of my patches merged? Because it's little strange that even #4171 isn't yet added to master… Am I doing something wrong?

[ossi]

gee, look - a 3rd contender for a scripting integration ...
and the 2nd from you, heh. i guess you're retracting #4136 then?

how does this compare in size with moofie's lua stuff? that's what the integration question mostly hinges on, i think.

according to trac's diff view, the indentation in the patch is somewhat random. how did you even manage to do that ...? :}

Am I doing something wrong?

yes - not doing enough meditation. :D
no, seriously, you need patience. a lot of it.

[psprint]

Replying to ossi:

gee, look - a 3rd contender for a scripting integration ...
and the 2nd from you, heh. i guess you're retracting #4136 then?

Yes. Python was a bad idea… libpython weights 2.7 MB.

how does this compare in size with moofie's lua stuff? that's what the integration question mostly hinges on, i think.

-rwxrwxr-x 1 guest guest 957K Jan 25 19:55 'src/mc^2'*
-rwxrwxr-x 1 guest guest 910K Jan 25 19:55 src/mc*

As for the maintainer effort required to keep things up, S-Lang is much much smaller/easier to maintain:

du -shc {modules,library} *.[ch]
1.1M    modules
1.2M    library
4.0K    modules.c
4.0K    modules.h
4.0K    pre-init.h
2.3M    total

du -shc src/slang*.[ch]
16K     src/slang_api_functions.c
24K     src/slang_api_functions_glue.c
4.0K    src/slang_api_functions.h
12K     src/slang_engine.c
4.0K    src/slang_engine.h
60K     total

The API isn't complete, so one could say that it will grow such too, however I cannot see this happening… S-Lang is explicitly light, small overhead.

according to trac's diff view, the indentation in the patch is somewhat random. how did you even manage to do that ...? :}

But I didn't do anything and I've ran make indent…

Am I doing something wrong?

yes - not doing enough meditation. :D
no, seriously, you need patience. a lot of it.

Ahso, OK, I'll try to, being | this close to setting up a fork sometimes…

[psprint]

An example plugin that reads a number from the buffer, alters it and writes back

[andrew_b]

Sorry, there only 24 hours in a day an night. And there is not only mc in my life. I not enough time to implement my own ideas because I must to work, I need to sleep, and make a lot of other things IRL.

Replying to psprint:

Replying to ossi:

gee, look - a 3rd contender for a scripting integration ...
and the 2nd from you, heh. i guess you're retracting #4136 then?

Yes. Python was a bad idea… libpython weights 2.7 MB.

Ok, #4136 closed.

S-Lang is explicitly light, small overhead.

You propose to make S-Lang a mandatory dependency regardless of --with-screen value. If --with-screen=ncurses, mc will be linked with both screen libraries. It's overkill, and I'm afraid this solution has no chance being approved.

[psprint]

I was rather thinking about making the scripting disabled when ncurses would be chosen as the screen library. Does this make it OK for such compact, small scripting to be added?

[psprint]

An example plugin which comments the current line with /* … */

[psprint]

A simple plugin that capitalizes the first letter of the word under cursor

[zaytsev]

I have already expressed me opinion on the subject, but I guess I need to document this in this ticket as well: I would be very much against S-Lang scripting.

For once, it's a language nobody uses, and it's tied to the screen library. In the end I would very much like supporting one screen library instead of two (see related conversations with Egmont), and if scripting is going to be integrated in a way that we can start loading things out of the core into scripts, this will basically make it mandatory for most users, and in turn binds us to S-Lang, whereas if one were to pick one library only, this should actually be ncurses in my opinion.

For all Lua faults, it's a very widespread language, which has an active community, libraries, several virtual machines including JIT compiler, halfway reasonable support for most paradigms, editors and IDEs, an so on and so forth. It's not tied to a screen library and can be made optional. It's lightweight and battle-tested, and runs on tiniest MIPS machines without any problem.

Now, the size comparison of this prototype and mc^2 makes absolutely no sense. Look at how much excellent documentation and examples, including implementation of many tickets here come with it. There is much more prose and Lua in the patch, than the actual glue code, and glue code is also done very well.

On top of that, mc^2 shows a strategy for universal scripting that allows to customise any area of mc, using the same language and API. If I understood correctly the plan here is to bolt S-Lang scripting quickly to the editor "because we can". If we ever go for scripting, this should be done the way mc^2 was done - we get all areas covered and can concentrate on C-core and leave everything else up to users/distributors or write ourself, but in an interpreted language with managed memory and not low-level language like C.

If you want S-Lang scripting, fine by me, but I'm out.

[zaytsev]

Ahso, OK, I'll try to, being | this close to setting up a fork sometimes…

As I said on the mailing list, this might actually be good for you, depending on what your goals are, and nobody here is going to be upset about it at all. We could even link to it somewhere if you want. Forks don't have to be enemies. It's just that if our understanding of where mc should go and how it should be done can't be reconciled, it doesn't help to argue endlessly - everyone can just do their thing, and in the end, users will only have more choice. It's good for everybody.

[psprint]

Replying to zaytsev:

I have already expressed me opinion on the subject, but I guess I need to document this in this ticket as well: I would be very much against S-Lang scripting.

For once, it's a language nobody uses

From what I've seen it is used in academic environments, as far I can tell it's because of its easy vectorizing support, e.g.: one can write:

x = [PI, PI/2, PI/3];
y = sin(x);

and it's tied to the screen library. In the end I would very much like supporting one screen library instead of two (see related conversations with Egmont), and if scripting is going to be integrated in a way that we can start loading things out of the core into scripts, this will basically make it mandatory for most users

IMO it would be possible to compile-out scripting code when ncurses would be selected, and it would be just an option that the user would have.

For all Lua faults, it's a very widespread language, which has an active community, libraries, several virtual machines including JIT compiler, halfway reasonable support for most paradigms, editors and IDEs, an so on and so forth. It's not tied to a screen library and can be made optional. It's lightweight and battle-tested, and runs on tiniest MIPS machines without any problem.

OK, if it runs on tiny machines then my impression of it as being a significant size is incorrect. However, I would feel wrong to add it to mc, as it's a foreign language that is added to "take over" the original project… Like: "hey, mc now has Lua support, great, I can now do some Lua stuff in it (instead of regular MC stuff)". Slang would be different, because its universe is smaller (although it does have some modules:

13K csv-module.so
7.0K fcntl-module.so
12K fork-module.so
9.0K iconv-module.so
24K rand-module.so
9.0K select-module.so
18K slsmg-module.so
27K socket-module.so
26K sysconf-module.so
18K termios-module.so
7.5K varray-module.so
16K zlib-module.so
184K total

).

Now, the size comparison of this prototype and mc^2 makes absolutely no sense. Look at how much excellent documentation and examples, including implementation of many tickets here come with it. There is much more prose and Lua in the patch, than the actual glue code, and glue code is also done very well.

Yes I've seen and got surprised how much of lua code has been done.

On top of that, mc^2 shows a strategy for universal scripting that allows to customise any area of mc, using the same language and API. If I understood correctly the plan here is to bolt S-Lang scripting quickly to the editor "because we can".

No, I've provided a proof of concept solution and API to draw some people into the project, to not repeat the (apparent) mistake of moofie where everything is done and ready in a … patch bomb…

If we ever go for scripting, this should be done the way mc^2 was done - we get all areas covered and can concentrate on C-core and leave everything else up to users/distributors or write ourself, but in an interpreted language with managed memory and not low-level language like C.

Yes, I'm hoping for such outcome also with S-Lang scripting.

If you want S-Lang scripting, fine by me, but I'm out.

OK, so I'm going to set up a fork: ​https://github.com/neo-mc/neo-mc, as I see your and Andrew skepticism for S-Lang light scripting, meaning that it'll probably never be accepted. That's a miserable fate of mine of trying to break to some wider audience, which can take a year or so with a miserable result anyway… Too bad, I was REALLY HOPING for uplifted MC by my work, not some solitary fate… :(

[andrew_b]

Replying to psprint:

IMO it would be possible to compile-out scripting code when ncurses would be selected, and it would be just an option that the user would have.

Functionality of mc should be the same in both cases, regardless of screen library.

[ossi]

It's good for everybody.

not really.

more choice is actually bad for users; that's established psychology.

from a practical perspective, users will typically miss out on something, because the forks don't diverge in only one way. you can't expect the fork to feed mainline; that never goes well for long (guess what happened with neomutt, for example ...).

and it's bad for mc, too, because it misses out the potential contributors that choose to join the fork instead of mainline.

i'm not really sure what to do at this point. i agree that s-lang is a rather terrible choice, and would prefer to get rid of the library altogether as well.

if you wonder what other choices exist, prepare to be overwhelmed: ​https://github.com/dbohdan/embedded-scripting-languages :o
of these, i personally like the idea of micropython best, but it's not stable yet.

mc² was done very professionally, and its choice of lua was probably about as close to optimal for the purpose as it gets. the only thing that kept it out was the lack of commitment. so ideally, someone would adopt the project and rope it into mainline.

i'm not sure what psprint's role in that might be. he is apparently highly motivated and productive, but also kinda careless - the typical innovator type, while mc needs (mostly) maintainer types at this stage.

[psprint]

Replying to ossi:

i'm not really sure what to do at this point. i agree that s-lang is a rather terrible choice, and would prefer to get rid of the library altogether as well.

I just want to say, that I was skeptical too, until I've realized how simple and light the language is, to then become surprised by the SLIRP utility, which when compared to the SWIG (that I've used when trying out python) behaved like: "you want your C functions in interpreter? no problem, no need to perform any work, just here you go!", and then looking at the kitchensink Slirp example to notice that it's syntax and feature is virtually identical to Swig which is heavy and produces heavier glue code… To then come to a conclusion that S-Lang is a firm choice, everything just works.

i'm not sure what psprint's role in that might be. he is apparently highly motivated and productive, but also kinda careless - the typical innovator type, while mc needs (mostly) maintainer types at this stage.

Thanks for the description, I could agree on it, in ~90% :)

[zaytsev]

hi ossi, I agree with you on most accounts, but I just want to clarify what I meant by "good for everybody": let's reformulate it by saying "given the resource and ability constraints of everybody involved at this point, letting psprint pursue his friendly S-Lang scripting fork seems to be the least bad solution for all of us".

The constraints as I see them:

1. We are not able to commit more time than we currently donate to move forward with scripting
2. We do not agree with technical choices that psprint thinks are good
3. psprint produces some great work, but also stuff that's just so extremely painful to work with as a maintainer which puts a lot of load on us - see the latest patches on Trac, because we want to get it done properly before it's integrated and psprint wants to get fast feedback and more forward quickly
4. For us it is important that mc continues to work well for everybody, and we don't get huge shit storms after every release like "everything is broken" as it used to happen a few years ago, and then be put under pressure to fix at least the most important stuff quickly (so we can't lower the bar for 3.)

I'm sure that psprint is a great guy and he just wants to achieve his goals, make things better and have fun. Why offend and demotivate him by constant criticism and slow feedback?

If he sets up his fork as a clean patchset that tracks our master branch and gets rebased every release, he will profit from our work, and have his goals achieved as well without having to deal with us all the time.

If he sets up his fork so, that it diverges from us, and he will have to cherry-pick our stuff only when needed - also fine. Either he will manage to build a community around it, and in this process he will have to understand what problems we have to deal with as mainline and successfully solves them better than we do, then we will become irrelevant - great result. I will be only happy if someone makes me obsolete.

Or maybe he will understand the problems and will not want to pursue it in the end - also ok. Then maybe he can come back to working with us so that we can integrate his valuable improvements, but his expectations will be different...