Drastically Improve Variable Loading Speed

[erenkarakal]

Disclaimer

To anyone looking to test this build. Make a backup of your variables file!

Problem

Skript's variable loading can be very slow as you get more variables. There are 3 main bottlenecks during variable loading.

1. Regex is used to read the variables.csv file lines. It makes the code readable, but around 17% of the time is spent matching the line to the regex pattern.

2. Inside DefaultYggdrasilInputStream, throw new EOFException() is used as a breakpoint. When this is called millions of times, it ends up taking up more than 20% of the tick.

3. The variable name comparator in VariablesMap is taking 15% of the tick. I looked into it, but it's not addressed in this PR. I am not sure if it can be optimized.

Solution

1) Added a new FastEOFException exception. This exception overrides fillInStackTrace() to have an empty implementation, which makes variable loading around 30% faster. (33.8s -> 24.2s when loading 15 mil variables)

2) Rewrote the splitCSV method to not use regex, which is an additional 25%~ boost (24.2s -> 19.0s when loading 15 mil variables)

The rest of the changes are code cleanup, just IJ suggestions.

Testing Completed

The speed tests were done with a single list of 15 million global variables, each variable having a different number. This PR still needs heavy testing with different variable types and varying variable names to make sure no breaking changes were added.

This python script was used to generate the variables:

import csv

num_entries = 15_000_000
output_file = "variables.csv"

with open(output_file, "w", newline="") as f:
    writer = csv.writer(f)

    for i in range(num_entries):
        decimal = str(i)
        hex_value = f"{i:016X}"

        writer.writerow([
            f"a::{decimal}",
            "long",
            hex_value
        ])

print(f"Generated {num_entries:,} entries in {output_file}")

---

Completes: none
Related: none
AI assistance: Claude Sonnet 5 Medium was used in splitCSV rewrite

[sovdeeth]

As for the comparator, I've also looked into optimizing it, and it's pretty close to as good as it can be without changing behavior afaik. Not an easy source of performance at least.

[sovdeeth]

I would also like to see junit tests for the csv parser with a focus on edge cases

[sovdeeth]

this might be allowed (quotes in things like var names?) Please double-check.

[sovdeeth]

javadocs for why this exists would be good