WIP: Add compiler deoptimization infrastructure for live editing

[LuYifei2011]

Related Discussion

#16 (comment)

Proposed Changes

This PR explores allowing compiled scripts to deopt back to the interpreter without restarting them.

Implemented:

• Preserve source block IDs in compiler IR
• Track deoptimization frames during JS generation
• Emit deoptimization snapshots at yield points
• Restore interpreter stack state from snapshots
• Support repeat and while loops

Not implemented:

• Procedures
• Reporter stack restoration
• Promise/async blocks
• Compatibility layer blocks
• Full support for all control structures

Reason for Changes

Currently, compiled scripts continue executing old generated code after the project is edited. This is a prototype for switching compiled scripts back to the interpreter at safe points instead of restarting them.

Test Coverage

• Tested deoptimizing repeat loops
• Tested deoptimizing while loops

[SharkPool-SP]

This might affect performance for people who don't want this. Maybe this should be a runtime option?

[GarboMuffin]

Write a design document (with a lot of human words, not ai slop) before you waste a billion tokens on something that may not be interesting to merge

[LuYifei2011]

Write a design document (with a lot of human words, not ai slop) before you waste a billion tokens on something that may not be interesting to merge

It might be poorly written. 🙏

1. a script is compiled to js and a thread starts
2. check if the blocks change when the thread comes to a yield
3. if not changed, it will do yield with nothing
4. if the blocks changed, the thread will yield with a deoptFrame snapshot like this:

   {
        frames: [
            {
                blockId: "abc",
                isLoop: true,
                warpMode: false,
                executionContext: {
                    loopCounter: 12
                }
            }
        ]
   }

5. the sequencer uses the snapshot to restore/build stack and stackFrames
6. mark this thread as not compiled
7. the next time the script starts, the new thread will compile again with new blocks

[LuYifei2011]

Write a design document (with a lot of human words, not ai slop) before you waste a billion tokens on something that may not be interesting to merge

Design Document

Background / Problem Statement

Currently, TurboWarp clears its cache after a block is modified, ensuring newly started threads use the latest blocks, while already running threads continue to use the old blocks.

This solution aims to make already running compiled threads fall back to interpreted execution so they can pick up live edits.

Target Design

When is Fallback Triggered?

When a target's Block Container is modified, consistent with the existing cache invalidation behavior.

Will Newly Started Threads Use Compiled or Interpreted Execution After Block Modification?

Compiled. Consistent with the previous implementation.

When Will Running Threads Switch to Interpreted Execution?

When the thread reaches a yield where the following safe switching conditions are met:

• It knows which block it is currently running on (used to build the stack needed by the interpreter).

• It has sufficient information to resume interpreter execution, such as the current loop count (used to build the stack frame needed by the interpreter).

If these conditions are not all met, the old compiled code will continue to run.

Design Boundaries

This solution is only responsible for the switch from compiler execution to interpreter execution, including deciding when to switch and performing state handover.

After the switch, modifications and deletions of blocks are handled by the interpreter and are not within the scope of this solution.

This solution does not guarantee that modifications will take effect immediately; when they take effect depends on when the thread reaches the safe switching point.

---

I used Google Translate to translate most of this design document into English, so there may be some translation errors.

By the way, I understand your concerns about AI-generated content, and I apologize for the overly long AI-generated ​content I posted previously. I did used AI to help me understand parts of scratch-vm more quickly and to discuss some technical questions related to this PR. However, I wrote most of the code, and I have reviewed and understand the proposed design and the code in this PR.

[GarboMuffin]

ok. how do you plan to handle cases like:

• recursive custom blocks
• custom extensions
• deoptimization requirement discovered while evaluating an async reporter; do you re-evaluate those things switching back?
• (this list is highly non-comprehensive - there are lots of compiled thread quirks that are not going to map 1:1 to an interpreter thread)

these are the actually interesting cases, you need to have a plan from the start

do you have a sense what the expected performance hit would be from the bookkeeping and code?

why is falling back to the interpreter preferred over a way to make the compiler handle it appropriately? forkphorus-style continuation-passing in principle can support live script reloading while still being order-of-magnitude speed improvement over the interpreter. there are other ways you can imagine that this is possible too. why not do those?

[GarboMuffin]

will you plan to implement some UI explaining why someone's script becomes 100x slower after making any change in an unrelated part of the sprite that may not even be used by the affected thread?

edge-activated hats are compiled for a while now: where does the evaluation phase part of that fit into this?

[LuYifei2011]

Additional notes:

• My idea is to yield a special marker object when execution reaches a yield point where a safe fallback is possible (I call these "safe points"), signaling a switch to interpreted execution; otherwise, execution continues in compiled mode.
• Therefore, for blocks that haven't been adapted (such as custom extensions or async reporters), execution will proceed normally in compiled mode even if they call yield. The switch to interpreted execution will only occur when reaching blocks that we have explicitly adapted and identified as safe for switching.
• Safe points will be selected from existing yield points and modified; no new yield points will be added.

recursive custom blocks

I think we could maintain a runtime-side procedure call stack to record the information needed to reconstruct interpreter frames (such as parameters) and update this stack when entering or exiting custom blocks.

custom extensions

I would not allow de-optimization when executing custom extension blocks. Most extensions don't have yield, so they wouldn't be candidates for safe points; I also think yields occurring inside asynchronous extension execution would not be treated as safe points.

deoptimization requirement discovered while evaluating an async reporter; do you re-evaluate those things switching back?

I don't think those things should be re-evaluated. My original plan was to avoid de-optimization within reporter blocks and only allow switching back to interpreted execution within supported stack blocks or C-blocks, as this simplifies the implementation.

do you have a sense what the expected performance hit would be from the bookkeeping and code?

I don't have benchmark numbers yet, but I anticipate the main performance impacts will stem from:

• Checking whether to de-optimize at safe points.
• Yielding the de-optimization marker/state object when a de-optimization actually occurs (normally, yield doesn't return a value).
• Managing the call stack for custom blocks (if handling recursion).

why is falling back to the interpreter preferred over a way to make the compiler handle it appropriately? forkphorus-style continuation-passing in principle can support live script reloading while still being order-of-magnitude speed improvement over the interpreter.

• This approach requires fewer modifications; adopting continuation-passing might necessitate extensive changes to the compiler and increase complexity.
• Continuation-passing might introduce additional bookkeeping overhead during normal compiled execution. In contrast, this approach only switches to interpreted execution when a block is modified. Additionally, disabling the de-optimization feature allows the generated code to remain identical to the what the current compiler generates.

will you plan to implement some UI

We could provide a toggle for de-optimization and include some explanatory notes.

making any change in an unrelated part of the sprite that may not even be used by the affected thread

Perhaps we could switch to finer-grained detection to allow for more precise de-optimization.

edge-activated hats are compiled for a while now: where does the evaluation phase part of that fit into this?

• Switching to the interpreter is disallowed while evaluation is in progress.
• If evaluation finishes without activating the edge and the thread retires, no de-optimization is needed.
• When evaluation concludes and execution reaches a yield point prior to the script body, I believe that can serve as a safe point.

_{translated by Google Translate}