Skip to content

fix: Restrict using slot for strings #2700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aleksanderkatan wants to merge 52 commits into
base: main
Choose a base branch
from
Open

fix: Restrict using slot for strings #2700

Show file tree
Hide file tree
Changes from 49 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
669e47f
Remove setName from mergeExternals
aleksanderkatan Jun 16, 2026
692ac90
Name unnamed externals in propAccess and getById
aleksanderkatan Jun 16, 2026
a944c60
Limit tgpuDeclare to one external
aleksanderkatan Jun 16, 2026
754eb92
Remove unnecessary merge from tgpuResolve
aleksanderkatan Jun 16, 2026
a217940
Limit rawCodeSnippet to one $uses
aleksanderkatan Jun 16, 2026
1f8bccb
Add FnExternals, and throw if $uses is called multiple times
aleksanderkatan Jun 16, 2026
b0ec447
Prepare string-implemented functions for external swap
aleksanderkatan Jun 16, 2026
236075c
Finish replacing old externals with new externals && cleanup
aleksanderkatan Jun 19, 2026
7c7faad
More cleanup
aleksanderkatan Jun 19, 2026
a03b83f
Detect $uses on function with plugin metadata earlier
aleksanderkatan Jun 19, 2026
1a16a8b
Add FnExternals docs
aleksanderkatan Jun 19, 2026
9f33691
Add external name collision tests
aleksanderkatan Jun 19, 2026
052f36b
Add more tests & fix fragment out external
aleksanderkatan Jun 19, 2026
5ec7052
More tests && allow redundant external
aleksanderkatan Jun 19, 2026
54b8b27
Update docs
aleksanderkatan Jun 19, 2026
d7d48ae
Add compute tests
aleksanderkatan Jun 22, 2026
fc8b147
Move external tests to external.test.ts
aleksanderkatan Jun 22, 2026
cfb074a
Fix old external tests by tweaking the api
aleksanderkatan Jun 22, 2026
5b26ee6
nr fix
aleksanderkatan Jun 22, 2026
312f78d
Merge branch 'main' into fix/forbid-multiple-uses
aleksanderkatan Jun 22, 2026
29b0354
self-review fixes
aleksanderkatan Jun 22, 2026
f9a4385
Fix multiple .$uses error
aleksanderkatan Jun 22, 2026
741af47
Merge branch 'main' into fix/forbid-multiple-uses
aleksanderkatan Jun 23, 2026
a7abca9
Initial rewrite
aleksanderkatan Jun 24, 2026
b0b5472
Add an end of chain test
aleksanderkatan Jun 24, 2026
ca3e9e1
Simplify end of chain tracking
aleksanderkatan Jun 24, 2026
48727bb
Update snapshots
aleksanderkatan Jun 24, 2026
e346c8e
Update declare tests
aleksanderkatan Jun 24, 2026
4974305
Remove warn test
aleksanderkatan Jun 24, 2026
65fcedf
Simplify regex
aleksanderkatan Jun 24, 2026
38a0643
Update packages/typegpu/src/core/resolve/externals.ts
aleksanderkatan Jun 24, 2026
960994b
Merge remote-tracking branch 'origin/main' into fix/external-traversa…
aleksanderkatan Jun 26, 2026
f969d37
Simplify naming
aleksanderkatan Jun 26, 2026
73dc847
Remove cache
aleksanderkatan Jul 1, 2026
4573395
Merge remote-tracking branch 'origin/main' into fix/external-traversa…
aleksanderkatan Jul 1, 2026
a5d2b0a
Copilot review fixes
aleksanderkatan Jul 2, 2026
a9da9a7
Merge remote-tracking branch 'origin/main' into fix/external-traversa…
aleksanderkatan Jul 2, 2026
1acd191
Merge branch 'main' into fix/external-traversal-for-raw-wgsl-implemen…
aleksanderkatan Jul 2, 2026
cb6c68f
Merge remote-tracking branch 'origin/main' into fix/external-traversa…
aleksanderkatan Jul 6, 2026
10e93da
Add some regex tests
aleksanderkatan Jul 6, 2026
c6bc427
Add tests
aleksanderkatan Jul 9, 2026
7ee8f2d
Throw in raw-wgsl case
aleksanderkatan Jul 9, 2026
f09f85c
fix: Don't treat strings as resolvable
iwoplaza Jul 9, 2026
049e477
Fix the resolvable check
aleksanderkatan Jul 9, 2026
737c910
Update tgsl tests
aleksanderkatan Jul 9, 2026
44bab90
Move rawfn tests
aleksanderkatan Jul 9, 2026
a63b41f
Add ResolvableString and forbid strings in raw fn
aleksanderkatan Jul 9, 2026
e4c4757
lint & types
aleksanderkatan Jul 9, 2026
c0d3132
lint
aleksanderkatan Jul 9, 2026
34ae441
Fix import
aleksanderkatan Jul 9, 2026
3d934c9
Review fixes
aleksanderkatan Jul 10, 2026
83271c4
Merge remote-tracking branch 'origin/main' into fix/restrict-using-sl…
aleksanderkatan Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 28 additions & 28 deletions apps/typegpu-docs/tests/individual-example-tests/log-test.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ describe('console log example', () => {
expect(shaderCodes).toMatchInlineSnapshot(`
"@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -48,8 +50,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -90,6 +90,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -99,8 +101,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -150,6 +150,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -159,8 +161,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -203,6 +203,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -212,8 +214,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn log1serializer() {
Expand Down Expand Up @@ -834,6 +834,8 @@ describe('console log example', () => {
num: u32,
}

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -843,8 +845,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -995,6 +995,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1004,8 +1006,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn log1serializer() {
Expand Down Expand Up @@ -1052,6 +1052,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1061,8 +1063,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1103,6 +1103,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1112,8 +1114,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1158,6 +1158,8 @@ describe('console log example', () => {

@group(0) @binding(1) var<uniform> logCountUniform: u32;

var<private> dataBlockIndex: u32;

@group(0) @binding(2) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1167,8 +1169,6 @@ describe('console log example', () => {

@group(0) @binding(3) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1212,6 +1212,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1221,8 +1223,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1299,6 +1299,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1308,8 +1310,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn log1serializer() {
Expand Down Expand Up @@ -1453,6 +1453,8 @@ describe('console log example', () => {
return mainVertex_Output(vec4f(positions[vertexIndex], 0f, 1f));
}

var<private> dataBlockIndex: u32;

@group(0) @binding(0) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1462,8 +1464,6 @@ describe('console log example', () => {

@group(0) @binding(1) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1506,6 +1506,8 @@ describe('console log example', () => {
return mainVertex_Output(vec4f(positions[vertexIndex], 0f, 1f));
}

var<private> dataBlockIndex: u32;

@group(0) @binding(0) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1515,8 +1517,6 @@ describe('console log example', () => {

@group(0) @binding(1) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down Expand Up @@ -1552,6 +1552,8 @@ describe('console log example', () => {

@group(0) @binding(0) var<uniform> sizeUniform: vec3u;

var<private> dataBlockIndex: u32;

@group(0) @binding(1) var<storage, read_write> indexBuffer: atomic<u32>;

struct SerializedLogData {
Expand All @@ -1561,8 +1563,6 @@ describe('console log example', () => {

@group(0) @binding(2) var<storage, read_write> dataBuffer: array<SerializedLogData, 40>;

var<private> dataBlockIndex: u32;

var<private> dataByteIndex: u32;

fn nextByteIndex() -> u32 {
Expand Down
6 changes: 5 additions & 1 deletion packages/typegpu/src/core/function/fnCore.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
replaceExternalsInWgsl,
mergeFunctionExternals,
} from '../resolve/externals.ts';
import { ResolvableString } from '../resolve/resolvableString.ts';
import { extractArgs } from './extractArgs.ts';
import type { Implementation, SeparatedEntryArgs } from './fnTypes.ts';

Expand Down Expand Up @@ -127,7 +128,10 @@ export function createFnCore(

this.setExternals('args', {
in: Object.fromEntries(
entryInput.positionalArgs.map((a) => [a.schemaKey, a.schemaKey]),
entryInput.positionalArgs.map((a) => [
a.schemaKey,
new ResolvableString(a.schemaKey),

@iwoplaza iwoplaza Jul 12, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we could use a raw code snippet here, instead of creating a new ResolvableString type

]),
),
});
}
Expand Down
96 changes: 47 additions & 49 deletions packages/typegpu/src/core/resolve/externals.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,18 +66,18 @@ export function addReturnTypeToExternals(
}
}

function identifierRegex(name: string) {
return new RegExp(
`(?<![\\w\\$_.])${name.replaceAll('.', '\\.').replaceAll('$', '\\$')}(?![\\w\\$_])`,
'g',
);
}
export const anyIdent = /([$_\p{XID_Start}][$\p{XID_Continue}]*)/u; // WGSL ident, modified to include $
const anyPropChain = new RegExp(`(${anyIdent.source})(\\.${anyIdent.source})*`, 'ug');
export const boundedPropChain = new RegExp(
`(?<![\\p{XID_Continue}\\$.])${anyPropChain.source}(?![\\p{XID_Continue}\\$])`,
'ug',
);

/**
* Replaces all occurrences of external names in WGSL code with their resolved values.
* It adds all necessary definitions to the resolution context.
* @param ctx - The resolution context.
* @param externalMap - The external map.
* @param externalMap - The external map. Assumes that keys don't contain dots.
* @param wgsl - The WGSL code.
*
* @returns The WGSL code with all external names replaced with their resolved values.
Expand All @@ -87,52 +87,50 @@ export function replaceExternalsInWgsl(
externalMap: ExternalMap,
wgsl: string,
): string {
return Object.entries(externalMap).reduce((acc, [externalName, external]) => {
const externalRegex = identifierRegex(externalName);
if (wgsl && externalName !== 'Out' && externalName !== 'in' && !externalRegex.test(wgsl)) {
console.warn(`The external '${externalName}' wasn't used in the resolved template.`);
// continue anyway, we still might need to resolve the external
const keys = Object.keys(externalMap);
if (keys.length === 0) {
return wgsl;
}

const maybeInvalidKey = keys.find((key) => key.includes('.'));
if (maybeInvalidKey) {
throw new Error(`External key '${maybeInvalidKey}' contains invalid character '.'`);
}

return wgsl.replaceAll(boundedPropChain, (match) => {
const chain = match.split('.');

if (!Object.hasOwn(externalMap, chain.at(0) as string)) {
// this prop access does not start with an external
return match;
}

if (isResolvable(external)) {
if (isNamable(external) && getName(external) === undefined) {
setName(external, externalName.split('.').at(-1) as string);
let currentItem: unknown = externalMap;
let suffix = '';

for (const [i, elem] of chain.entries()) {
currentItem = (currentItem as ExternalMap)[elem];
if (isResolvable(currentItem) || typeof currentItem === 'string') {
suffix = chain
.slice(i + 1)
.map((s) => `.${s}`)
.join('');

if (isNamable(currentItem) && getName(currentItem) === undefined) {
setName(currentItem, elem);
}

break;
}
return acc.replaceAll(externalRegex, ctx.resolve(external).value);
}

if (external !== null && typeof external === 'object') {
const foundProperties = [
...wgsl.matchAll(
new RegExp(
`${externalName
.replaceAll('.', '\\.')
.replaceAll('$', '\\$')}\\.(?<prop>.*?)(?![\\w\\$_])`,
'g',
),
),
].map((found) => found[1]);
const uniqueProperties = [...new Set(foundProperties)];

return uniqueProperties.reduce(
(innerAcc: string, prop) =>
prop && prop in external
? replaceExternalsInWgsl(
ctx,
{
[`${externalName}.${prop}`]: external[prop as keyof typeof external],
},
innerAcc,
)
: innerAcc,
acc,
);
if (typeof currentItem !== 'object' || currentItem === null || i === chain.length - 1) {
console.warn(
`During resolution, the external '${chain.slice(0, i + 1).join('.')}' has been omitted. Only TGPU resources, 'use gpu' functions, primitives, and plain JS objects can be used as externals.`,
);
return match;
}
}

console.warn(
`During resolution, the external '${externalName}' has been omitted. Only TGPU resources, 'use gpu' functions, primitives, and plain JS objects can be used as externals.`,
);

return acc;
}, wgsl);
return ctx.resolve(currentItem).value + suffix;
});
}
Loading
Loading