Do not return to scheduler while completing {.async.} proc

[etan-status]

Chronos can have an extra implicit return to scheduler when an {.async.} proc completes.

import chronos

proc competitorCb(udata: pointer) =
  echo "competitor"

proc inner {.async.} =
  await sleepAsync(100.milliseconds)

proc middle {.async.} =
  echo "B"
  await inner()
  callSoon(competitorCb)
  echo "C"

proc outer {.async.} =
  echo "A"
  await middle()
  echo "D"

waitFor outer()

Output:

A
B
C
competitor
D

Since inner does not complete synchronously, Chronos adds an extra return to scheduler between C -> D, during which different callbacks / events may be processed. However, the {.async.} transformation macro hides this extra return to scheduler from the user: Visually this looks like a regular return to the outer proc, even though potential side effects similar to those from an explicit await have to be considered.

This PR removes the extra return to scheduler when completing an {.async.} proc, even when the proc did not fully run to completion synchronously. For other, explicitly added callbacks, existing behaviour is unchanged; a new FutureFlag.SyncContinuations can be used to emulate the {.async.} proc behaviour.

⚠️ New behaviour is only activated when the opt-in config chronosSyncContinuations is enabled.

This is in line with other popular languages, where continuations across returns commonly complete without returning to the scheduler. Users more familiar with these other languages as well as LLMs may introduce subtle bugs, because the keywords and control flow look the same, while semantically behaving slightly different in Chronos in certain situations.

Language	Output
Nim (chronos)	A B C competitor D
C#	A B C D competitor
Python (asyncio)	A B C D competitor
JavaScript (setTimeout)	A B C D competitor
JavaScript (queueMicrotask)	A B C competitor D
Rust (tokio)	A B C D competitor
C++ (asio)	A B C D competitor
Swift	A B C D competitor
Kotlin	A B C D competitor

Note the two JavaScript rows, where the queueMicrotask API enables queueing the competitor ahead of the return-continuation. In contrast to Chronos, this still does not allow timers and I/O events to be executed (they are macrotasks).

---

LLM-generated translations.

C#

using System;
using System.Threading;
using System.Threading.Tasks;

ThreadPool.SetMinThreads(1, 1);
ThreadPool.SetMaxThreads(1, 1);

Task competitor = Task.CompletedTask;

await Outer();
await competitor;

async Task Inner() => await Task.Delay(100);

async Task Middle()
{
    Console.WriteLine("B");
    await Inner();
    competitor = Task.Run(() => Console.WriteLine("competitor"));
    Console.WriteLine("C");
}

async Task Outer()
{
    Console.WriteLine("A");
    await Middle();
    Console.WriteLine("D");
}

Python

import asyncio

def competitor_cb():
    print("competitor")

async def inner():
    await asyncio.sleep(0.1)

async def middle():
    print("B")
    await inner()
    asyncio.get_running_loop().call_soon(competitor_cb)
    print("C")

async def outer():
    print("A")
    await middle()
    print("D")

asyncio.run(outer())

JavaScript (setTimeout)

function competitor() { console.log("competitor"); }

async function inner() {
  await new Promise(r => setTimeout(r, 100));
}

async function middle() {
  console.log("B");
  await inner();
  setTimeout(competitor, 0);
  console.log("C");
}

async function outer() {
  console.log("A");
  await middle();
  console.log("D");
}

outer();

JavaScript (queueMicrotask)

function competitor() { console.log("competitor"); }

async function inner() {
  await new Promise(r => setTimeout(r, 100));
}

async function middle() {
  console.log("B");
  await inner();
  queueMicrotask(competitor);
  console.log("C");
}

async function outer() {
  console.log("A");
  await middle();
  console.log("D");
}

outer();

Rust

use tokio::time::{sleep, Duration};

async fn inner() {
    sleep(Duration::from_millis(100)).await;
}
async fn middle() {
    println!("B");
    inner().await;
    tokio::spawn(async { println!("competitor"); });
    println!("C");
}
async fn outer() {
    println!("A");
    middle().await;
    println!("D");
}
#[tokio::main(flavor = "current_thread")]
async fn main() {
    outer().await;
    tokio::task::yield_now().await;
}

C++

#include <asio.hpp>
#include <chrono>
#include <iostream>

using asio::awaitable;
using asio::co_spawn;
using asio::detached;
using asio::use_awaitable;
using namespace std::chrono_literals;

void competitor() { std::cout << "competitor\n"; }

awaitable<void> inner() {
    asio::steady_timer timer(co_await asio::this_coro::executor);
    timer.expires_after(100ms);
    co_await timer.async_wait(use_awaitable);
}

awaitable<void> middle() {
    std::cout << "B\n";
    co_await inner();
    asio::post(co_await asio::this_coro::executor, competitor);
    std::cout << "C\n";
}

awaitable<void> outer() {
    std::cout << "A\n";
    co_await middle();
    std::cout << "D\n";
}

int main() {
    asio::io_context ctx;
    co_spawn(ctx, outer(), detached);
    ctx.run();
}

Swift

func competitor() { print("competitor") }

@MainActor func inner() async {
    try? await Task.sleep(for: .milliseconds(100))
}

@MainActor func middle() async {
    print("B")
    await inner()
    Task { competitor() }
    print("C")
}

@MainActor func outer() async {
    print("A")
    await middle()
    print("D")
}

await outer()
try? await Task.sleep(for: .milliseconds(50))

Kotlin

import kotlinx.coroutines.*

fun competitor() { println("competitor") }

suspend fun inner() {
    delay(100)
}

suspend fun CoroutineScope.middle() {
    println("B")
    inner()
    launch { competitor() }
    println("C")
}

suspend fun CoroutineScope.outer() {
    println("A")
    middle()
    println("D")
}

fun main() = runBlocking {
    outer()
}

[etan-status]

make -j test successful in status-im/nimbus-eth2

[etan-status]

Example bug that is addressed by this PR: #273

Obviously, it is possible to write correct code even without this PR. But when translating from other languages, doing spec copying, or using LLMs, subtle bugs can creep in. These bugs are hard to find in a review because they come from hidden implicit behaviour rather than visibly incorrect lines of code. This PR simply removes this class of footgun.

How much it makes it easier to accidentally stall the event loop, remains to be seen.