Make tccbox an optional dependency

[MridulS]

Trying to fix conda-forge/sasmodels-feedstock#19

From what I understand tccbox is only an optional dep for windows codepath? This PR exposes it through the Windows-only tinycc extra, and update CI to validate the resulting wheel metadata.

[codescene-access]

No quality gates enabled for this code.

See analysis details in CodeScene

Quality Gate Profile: Custom Configuration
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

[pkienzle]

Can you instead modify pyproject.toml requires to "tccbox; sys_platform == 'win32'" ?

[MridulS]

Hmm, I'm a bit confused as why tinybox is a build-system requirement? It only seems to be used during runtime by kerneldll.py, and if it's in build-system, tinybox is actually never installed when we do pip install sasmodels on a windows machine. It's also declared in build_tools/requirements.txt so the wheel adds that to the metadata as a requirement.

[pkienzle]

The wheel includes the docs which includes a png for each model which is created by running the model. This requires either a compiler or opencl/cuda, hence tccbox is a build requirement.

Doc build is also run as part of CI so tccbox may be needed there as well.

bumps isn't required. I mock the necessary symbols so that bumps_model can be imported during doc build and doc test, but with bumps in the requirements this code path never gets tested. Since sasview independently requires bumps this isn't much of an issue.

Note: For a list of packages that depend on sasmodels, see the sasmodels pypimap

[MridulS]

Thanks for the explanation! I did test building without tccbox in build-requires on a windows runner https://github.com/MridulS/sasmodels-windows-ci/actions/runs/32279909247/job/96155969517 and it seems happy. I'm happy to just put it back in pyproject.toml but I am biased towards removing deps if not absolutely required :)

[pkienzle]

Does modifying the requirements file with "tccbox; sys_platform == 'win32'" to limit tccbox to windows solve the feedstock problem?

Or does the feedstock have its own version of a requirements file that has to be kept in sync with pyproject.toml + build_tools/requirements.txt*?

---

columnize is optional. I have my own implementation that gives identical results.

Check by adding raise ImportError to the sasmodels/compare.py:columnize import statement and running

python -m sasmodels.models -models
python -m sasmodels.list_pars

Feel free to remove the dependency from the code and requirements. It isn't used by sasview.

[jamescrake-merani]

This PR was mentioned at the fortnightly meeting, and I thought I would jump in on the discussion since I've familiar with this because of the work I've done on the Flatpak.

From what I understand tccbox is only an optional dep for windows codepath?

To my understanding, this is not necessarily true. SasView requires a C compiler no matter which platform you run it on because C models need to be compiled during runtime as you might have a plugin model which you've written in C. However, you may already have a C compiler installed on your system, in which case SasView can just use that meaning you wouldn't need to necessarily install tccbox.

Previously, we used to ask Mac users to install the Xcode CLI tools but that might be a fair bit to ask of someone who is not a developer, and likely won't need them for anything else hence why tccbox is now a dependency. tccbox also provides wheels for Linux, so a Linux user without a C compiler can also take advantage of it.

So in that sense, we need it across all three platforms.

But going back to what I said before, its possible the user may already have a C compiler installed. Tinycc is designed to be as small as possible but this does come with the compromise that the binaries it produces aren't very optimised. So if you had something like gcc/clang installed, then thats probably going to produce faster binaries. However at the moment we don't really have a way of choosing which C compiler to use. If I recall correctly, for the the Flatpak we currently use the C compiler that is packaged with the KDE SDK (I think gcc).

I appreciate thats quite a long winded, and open ended reply, and that you need something more concrete for the Conda package. I must admit I don't know much about Conda as I've never had to use it. What exact problem are you having with the Conda package because of tccbox? I've heard that Conda has packages for C compilers but I don't know if thats the issue you're facing?

In the longer term, I think we might need to think about how we want to handle the C compiler. If we make tccbox optional, then SasView may not be functional on systems where no C compiler is available when installed straight from pip. For things like Flatpak, and PyInstaller distributions, we have better control as to what we put into them, and as such this is not necessarily a problem as long as we handle it there. But if someone is installing SasView through pip, we ideally want to make sure they have everything they need to run the program.

I also think we should make a decision about whether we want to insist that all SasView users use tcc. I guess an argument for this is that we know that every model will be compiled by the same C compiler. But on the other hand, the user might want to choose which C compiler they use.

[pkienzle]

tccbox is only useful on windows. It ships with stripped down headers from mingw gcc so that code can compile and run against Microsoft's libc.

It is easy to install a compiler on linux, so we don't need tccbox for that.

tccbox now supports macho builds and can run on darwin, but it does not ship with C headers needed to compile. Those are available with the xcode command line tools, but if we install those then we might as well use clang. We could borrow headers from a lightweight posix library such as uClibc, but we are still left with some architecture specific headers that need to match the interfaces in libSystem* in OS/X. Or maybe we could modify the mingw headers.

---

Update Using the tinycc git distribution (current head), I hacked sasmodels kerneldll, adding -Itinycc/include -Itinycc/win32/include to the tinycc compile command. I shut down caching to force c-model recompile. With these changes all the model tests pass. An OS/X version of tccbox is now feasible.

[jamescrake-merani]

It is easy to install a compiler on linux, so we don't need tccbox for that.

It is indeed easy, but my main concern is that we then wouldn't be able to have someone on Linux run uvx sasview, and immediately be able to run SasView without having a C compiler installed. They'll instead get an error, and it might not be immediately obvious to them how to fix the problem.

On the other hand, perhaps we'd intentionally want a Linux user to bring their own C compiler, and keep the Flatpak as a 'one click' installation method for SasView. The issue in this case is that haven't said this explicitly. In fact, the README for SasView on Github suggests installing the application directly through pip using uv.

Regarding macOS: I thought one of the reasons we included tccbox was so that we didn't need users to install the Xcode tools? Though I had a look back at previous issues, and I noticed you mentioned this in a comment (this was after my previous contract at ISIS so I hadn't seen it):
SasView/sasview#3290

It might be best to make an issue about this on the main repo. I'm just going to have a quick check on a macOS VM without Xcode to see if its able to build models.

[jamescrake-merani]

So I ran SasView on a fresh macOS VM (both through the installer, and pip). I could not even get to the fitting widget but I suspect it may have been due to OpenCL (I'm running a VM on an ARM Mac but I don't know what kind of GPU access it has). Running the model test on sasmodels, I get the error that xcode not installed.

If we could get tinycc to work on the macOS distribution that would be amazing. If not, and its absolutely necessary that we use the xcode tools, we should probably add a warning in SasView to prompt the user to install them, as its not obvious unless they already knew this (we do document this in the FAQ but its easy to miss).

[pkienzle]

tccbox tcc for arm64 is broken—models with e.g., -D somewhere in the expression fail.

Clone the repo https://repo.or.cz/tinycc.git and run ./configure; make; make test.

Copy the resulting tinycc/tcc to your VM, along with tinycc/include and tinycc/win32/include.

Apply the following sasmodels patch to /Applications/SasView6.app/Contents/Resources/sasmodels:

git diff
diff --git a/sasmodels/kerneldll.py b/sasmodels/kerneldll.py
index 216b668d..39dcd8e4 100644
--- a/sasmodels/kerneldll.py
+++ b/sasmodels/kerneldll.py
@@ -169,12 +169,15 @@ elif COMPILER == "msvc":
 elif COMPILER == "tinycc":
     # TinyCC compiler.
     CC = [
-        tccbox.tcc_bin_path(),
+        #tccbox.tcc_bin_path(),
+        "/path/to/tinycc/tcc",
         "-nostdinc",
         "-std=c99",
         f"-L{tccbox.tcc_lib_dir()}",
         f"-L{joinpath(tccbox.tcc_lib_dir(), 'tcc')}",
-        f"-I{tccbox.tcc_dist_dir()}/include",
+        #f"-I{tccbox.tcc_dist_dir()}/include",
+        "-I/path/to/tinycc/include",
+        "-I/path/to/tinycc/win32/include",
         "-shared",
         "-rdynamic",
         "-Wall",
@@ -204,6 +207,7 @@ def compile_model(source, output):
     command = compile_command(source=source, output=output)
     command_str = " ".join('"%s"'%p if ' ' in p else p for p in command)
     logging.info(command_str)
+    print(command_str)
     try:
         # Need shell=True on windows to keep console box from popping up.
         shell = (os.name == 'nt')
@@ -284,7 +288,7 @@ def make_dll(source, model_info, dtype=F64, system=False):
     logging.debug("make_dll: dll located %s as %s in %s",
                   model_info.id, model_file, dll)
 
-    if not os.path.exists(dll):
+    if True or not os.path.exists(dll): # Turns off cache
         # Make sure the DLL path exists. Use abspath since python docs warn
         # that makedirs is not robust against '..' in path.
         os.makedirs(os.path.abspath(SAS_DLL_PATH), exist_ok=True)

While this is good enough for us, to push upstream we need a set of headers that can pass a libc test suite such as https://git.musl-libc.org/git/libc-testsuite.

Handy tools for debugging include files on Darwin:

# Jump to the SDK directory
cd $(xcrun --show-sdk-path)

# Show symbols available in the mac equivalent to libc
less usr/lib/libSystem.B.tbd

# Show defines from a particular header using clang
echo "#include <stdlib.h>" | clang -E -dM -x c - | less

# Show defines from a particular header using tcc
echo "#include <stdlib.h>" | (cd /path/to/tinycc && ./tcc -nostdinc -E -x c -Iinclude -Iwin32/include -dD -) | less