Does coroutine stacks grow in Lua, Python, Ruby or any other languages?

Question

There are some languages which support deterministic lightweight concurrency - coroutine.

1. Lua - coroutine
2. Stack-less Python - tasklet
3. Ruby - fiber
4. should be many more... but currently I don't have much idea.

Anyway as far as I know, it needs many of separated stacks, so I want to know how these languages handle the stack growth. This because I read some mention about Ruby Fiber which comes with 4KB - obviously big overhead - and they are advertising this as a feature that prevents stack overflow. But I don't understand why they're just saying the stacks will grow automatically. It doesn't make sense the VM - which is not restricted to C stack - can't handle stack growth, but I can't confirm this because I don't know about internals well.

How do they handle stack growth on these kind of micro-threads? Is there any explicit/implicit limitations? Or just will be handled clearly and automatically?

Answer 1

For ruby:

As per this google tech talk the ruby vm uses a slightly hacky system involving having a copy of the C stack for each thread and then copying that stack on to the main stack every time it switches between fibers. This means that Ruby is still restricts each fibre from having more than a 4KB stack but the interpreter does not overflow if you switch between deeply nested fibres.

For python:

task-lets are only available in the stackless variant. Each thread gets its own heap based stack as the stackless python vm uses heap based stacks. This mess they are inherently only limited to the size of the heap in stack growth. This means that for 32 bit systems there is still an effective limit of 1-4 GB.

For Lua:

Lua uses a heap based stack so are inherently only limited to the size of the heap in stack growth. Each coroutine gets its own stack in the memory. This means that for 32 bit systems there is still an effective limit of 1-4 GB.

To add a couple more to your list C# and VB.Net both now support async/await. This is a system that allows the program to preform a time consuming operation and have the rest of that function continue afterwards. This is implemented by creating an object to represent the method with a single method that is called to advance to the next step in the method which is called when you attempt to get a result and various other internal locations. The original method is replaced with one that creates the object. This means that the recursion depth is not affected as the method is never more than a few steps further down the stack than you would expect.