ENH: add support for PEP 803 abi3t with Python 3.15.0b2+

[mgorny]

Add tag/extension support and initial tests for abi3t as specified in PEP 803 and implemented in Python 3.15.0b2. This is currently limited to building abi3t extensions from freethreading builds of Python, since Meson does not support forcing abi3t builds explicitly right now. See mesonbuild/meson#15637 for the relevant discussion.

The additional test case utilizes the new API introduced in PEP 793 and PEP 820, and therefore requires Python 3.15.0b2. The relevant test tests wheel correctness both with GIL-enabled and free-threading builds of Python 3.15.0b2.

The behavior for older versions of Python remains unchanged.

[mgorny]

I didn't touch the test matrix, since testing on 3.15 is already being addressed by #853.

[rgommers]

Downstream testing, and releasing after rc1 in early August, is currently blocked on this, so I added a v0.21.0 tag - we should do that next month I think.

[rgommers]

A couple of small issues in the test extension. The rest looks right; I refreshed my memory on Py_TARGET_ABI3T and that should be added automatically in Python.h for a free-threaded interpreter if Py_LIMITED_API is defined - so all good there.

[rgommers]

This is looking pretty good now. Are you expecting it to be squash-merged, or do you want to rewrite history? (we usually do the latter, and rebase-merge)

[mgorny]

Rewrote it now. I've kept (and reworded) the commit changing limited-api test separate.

[rgommers]

LGTM now, thanks @mgorny.

@dnicolodi WDYT about releasing this quickly as 0.21.0? Getting this supported would be nice, makes it more likely package authors will experiment with it before the ABI gets frozen in 5-6 weeks from now. I had a look through the rest of the PR queue, and nothing else is critical, although PR 838 is close and we could aim to get that in as well.

[dnicolodi]

LGTM too, with the minor exception of the test package name: I am sure that I will promptly forget what ft stands for, thus a longer more descriptive name would help there.

I had a look through the rest of the PR queue, and nothing else is critical, although PR 838 is close and we could aim to get that in as well.

I can try to get that in shape this evening. I would still like to get the RPATH handling in order sooner rather than later, but I didn't find the time to finish that work. I guess we can go another release without it.

[dnicolodi]

Sorry, I have a couple more nits before merging this. Let me know if you prefer me to fix these small things up myself.

[mgorny]

Fixed the indentation. Lemme know if you want to me reset the dates too, though the files have more original content from limited-api test than my modifications (or just change them in place).

[dnicolodi]

Thanks! Merged.

[mgorny]

Thanks!

[rgommers]

Downstream testing, and releasing after rc1 in early August, is currently blocked on this, so I added a v0.21.0 tag - we should do that next month I think.

Note that the reason I haven't put up a PR for a next release yet is that there is a still a request to the steering council to change the extension on POSIX platforms from .abi3t.so to .abi3t-SOABI_PLATFORM.so (lots of threads, e.g. this one). That should not require any changes for meson-python, but I'd prefer if this was settled so no one uploads wheels with extension module names that might not be final (even though they should remain working).

[dnicolodi]

Aren't abi3t wheels a Python 3.15 only thing, anyway? That would mean that abi3t wheels should be rejected by PyPI at least until 3.15rc1 is out, and by then the extension module file name extension should be settled. There are other package indexes than PyPI, but PyPI should be the only relevant one here.

Anyhow, I don't see strong downsides in delaying a release for a little while longer.

[rgommers]

They are >=3.15 yes

That would mean that abi3t wheels should be rejected by PyPI at least until 3.15rc1 is out

That's not how it works, you can upload wheels now. And while that seems unhealthy, I have even seen some CPython core encourage that after beta 1 or beta 2 (they like early signal and the ABI is "probably stable, and otherwise just do a new release"). I don't think many package authors will do that, but it's possible.

[dnicolodi]

I stand corrected. Maybe I remember a proposal for blocking wheel updates till rc1, or this is how it worked in the past, or it is how I wish this would be handled. It does not seem a good idea to me to allow wheels target for an ABI that may still change to be uploaded. It opens the possibility for wheels tagged with the wrong ABI to arrive to users, causing very hard to diagnose bugs.