diff --git a/STYLE.md b/STYLE.md index f930fccb0..6514394d0 100644 --- a/STYLE.md +++ b/STYLE.md @@ -298,6 +298,7 @@ These are proper nouns used in Hardhat 3 that need to be capitalized in the docu - Network Connection - Network Config - Build Profile +- Test Profile - Isolated Build - Ignition Module - Ignition diff --git a/src/content/docs/docs/guides/testing/code-coverage.mdx b/src/content/docs/docs/guides/testing/code-coverage.mdx index b67c7ec9b..4485edee5 100644 --- a/src/content/docs/docs/guides/testing/code-coverage.mdx +++ b/src/content/docs/docs/guides/testing/code-coverage.mdx @@ -3,7 +3,7 @@ title: Computing code coverage of your tests description: How to compute the code coverage of your tests sidebar: label: Code coverage - order: 7 + order: 8 --- import Run from "@hh/Run.astro"; diff --git a/src/content/docs/docs/guides/testing/eip712-types.mdx b/src/content/docs/docs/guides/testing/eip712-types.mdx index 841786e61..9f23a5f46 100644 --- a/src/content/docs/docs/guides/testing/eip712-types.mdx +++ b/src/content/docs/docs/guides/testing/eip712-types.mdx @@ -3,7 +3,7 @@ title: EIP-712 type hashing in Solidity tests description: How to use EIP-712 type hashing cheatcodes in Solidity tests sidebar: label: EIP-712 type hashing - order: 6 + order: 7 --- Hardhat provides three cheatcodes for EIP-712 hashing in Solidity tests: diff --git a/src/content/docs/docs/guides/testing/gas-snapshots.mdx b/src/content/docs/docs/guides/testing/gas-snapshots.mdx index 2364e895c..aa57fe74a 100644 --- a/src/content/docs/docs/guides/testing/gas-snapshots.mdx +++ b/src/content/docs/docs/guides/testing/gas-snapshots.mdx @@ -2,7 +2,7 @@ title: Gas snapshots description: How to track and compare gas usage across test runs sidebar: - order: 9 + order: 10 --- import Run from "@hh/Run.astro"; diff --git a/src/content/docs/docs/guides/testing/gas-statistics.mdx b/src/content/docs/docs/guides/testing/gas-statistics.mdx index 658b8e7be..8d904147e 100644 --- a/src/content/docs/docs/guides/testing/gas-statistics.mdx +++ b/src/content/docs/docs/guides/testing/gas-statistics.mdx @@ -2,7 +2,7 @@ title: Gas statistics for your test runs description: How to get gas statistics from your test runs sidebar: - order: 8 + order: 9 --- import Run from "@hh/Run.astro"; diff --git a/src/content/docs/docs/guides/testing/index.mdx b/src/content/docs/docs/guides/testing/index.mdx index b72cdaaae..b180db204 100644 --- a/src/content/docs/docs/guides/testing/index.mdx +++ b/src/content/docs/docs/guides/testing/index.mdx @@ -86,3 +86,4 @@ Hardhat 3 includes built-in support for [code coverage](/docs/guides/testing/cod - To learn how to write tests in TypeScript using [`node:test`](https://nodejs.org/api/test.html) and [viem](https://viem.sh), read [this guide](/docs/guides/testing/using-viem) - To learn how to write tests in TypeScript using [Mocha](https://mochajs.org) runner and [ethers.js](https://github.com/ethers-io/ethers.js), read [this guide](/docs/guides/testing/using-ethers) - To learn how to override test settings per-function, read the [inline configuration guide](/docs/guides/testing/inline-configuration) +- To learn how to run your Solidity tests with different settings, read the [Test Profiles guide](/docs/guides/testing/test-profiles) diff --git a/src/content/docs/docs/guides/testing/inline-configuration.mdx b/src/content/docs/docs/guides/testing/inline-configuration.mdx index d00221784..24f9c8f42 100644 --- a/src/content/docs/docs/guides/testing/inline-configuration.mdx +++ b/src/content/docs/docs/guides/testing/inline-configuration.mdx @@ -1,14 +1,14 @@ --- title: Inline configuration for Solidity tests -description: How to use inline comments to override the solidity tests' configuration +description: How to use inline comments to override Solidity test settings sidebar: label: Solidity tests inline config - order: 5 + order: 6 --- -Hardhat lets you override solidity test settings on a per-function basis using NatSpec comments. This is useful when different tests need different parameters. For example, running more fuzz iterations on a critical function, or increasing the depth of an invariant test. +Hardhat lets you override Solidity test settings using NatSpec comments, on a single test function or on a whole test contract. This is useful when different tests need different parameters. For example, a critical function may need more fuzz iterations, while an invariant test may need a greater depth. -Inline overrides take precedence over the global settings defined in your [Solidity tests configuration](/docs/reference/configuration#solidity-tests-configuration). +Inline overrides take precedence over the global settings defined in your [Solidity tests configuration](/docs/reference/configuration#solidity-tests-configuration). They can also be scoped to a [Test Profile](/docs/guides/testing/test-profiles). ## Syntax @@ -38,6 +38,63 @@ function invariantBalanceAlwaysPositive() public { } ``` +## Overriding settings for a whole contract + +A directive above a contract applies to every test it runs, including the ones it inherits: + +```solidity +/// hardhat-config: fuzz.runs = 500 +contract TransferTest is Test { + function testTransferFuzz(uint256 amount) public { + // ... + } +} +``` + +If a test function sets the same setting, its value wins. The contract's other settings still apply to it. + +## Scoping an override to a Test Profile + +An unprefixed directive applies under every [Test Profile](/docs/guides/testing/test-profiles). To apply one under a single profile, prefix its key with the profile's name. This works the same way above a function and above a contract: + +```solidity +/// hardhat-config: ci.fuzz.runs = 10000 +function testTransferFuzz(uint256 amount) public { + // ... +} +``` + +Combine both forms in the same place to use different values. When its profile is selected, the prefixed directive takes precedence over the unprefixed one for the same key: + +```solidity +/// hardhat-config: fuzz.runs = 100 +/// hardhat-config: ci.fuzz.runs = 10000 +function testTransferFuzz(uint256 amount) public { + // ... +} +``` + +This runs 10000 iterations under the `ci` profile, and 100 under any other. + +## Which directive applies + +Directives are applied in this order of priority: + +1. Function-level over contract-level +2. Profile-prefixed over unprefixed, within the same level + +The level comes first, so a function's unprefixed directive beats the contract's prefixed one, even under that profile: + +```solidity +/// hardhat-config: ci.fuzz.runs = 10000 +contract TransferTest is Test { + /// hardhat-config: fuzz.runs = 100 + function testTransferFuzz(uint256 amount) public { + // 100 runs under every profile, including `ci` + } +} +``` + ## Supported configuration keys Keys with the `fuzz.`/`invariant.` prefix apply only to that test type; the rest apply to the test as a whole and work on both fuzz and invariant tests. @@ -67,20 +124,28 @@ Setting `allowInternalExpectRevert` to `true` allows `expectRevert` to work on c /// hardhat-config: allowInternalExpectRevert = true function testInternalRevert() public { vm.expectRevert("some error"); - this.myInternalHelper(); // reverts at the same call depth + myInternalHelper(); // reverts at the same call depth } ``` ## Foundry compatibility -Hardhat also accepts the `forge-config:` prefix and the `default` profile, so existing Foundry inline configuration works without changes: +Hardhat also accepts the `forge-config:` prefix, so you can keep using it: ```solidity -/// forge-config: default.fuzz.runs = 10000 +/// forge-config: fuzz.runs = 10000 /// forge-config: fuzz.max-test-rejects = 500 function testTransferFuzz(uint256 amount) public { // ... } ``` -Omitting the profile is equivalent to using `default`. +The `default.` prefix is the one thing that changes. In Foundry it applies under every profile, because profiles inherit from `default`. In Hardhat it applies only under the `default` profile, so remove `default` to keep the Foundry behavior: + +```solidity del={1} ins={2} +/// forge-config: default.fuzz.runs = 10000 +/// forge-config: fuzz.runs = 10000 +function testTransferFuzz(uint256 amount) public { + // ... +} +``` diff --git a/src/content/docs/docs/guides/testing/test-profiles.mdx b/src/content/docs/docs/guides/testing/test-profiles.mdx new file mode 100644 index 000000000..1efff2ee3 --- /dev/null +++ b/src/content/docs/docs/guides/testing/test-profiles.mdx @@ -0,0 +1,102 @@ +--- +title: Test Profiles +description: How to use Test Profiles in Hardhat 3 +sidebar: + label: Solidity Test Profiles + order: 5 +--- + +import Run from "@hh/Run.astro"; + +Test Profiles let you use different settings to run your Solidity tests. For example, you might use one Test Profile while iterating locally and another in CI, each with the number of fuzz runs that suits that workflow. + +## Configuring a Test Profile + +To configure a Test Profile, use this extended version of the `test.solidity` settings in your config: + +```ts ins={8-15} +// hardhat.config.ts +import { defineConfig } from "hardhat/config"; + +export default defineConfig({ + //... + test: { + solidity: { + profiles: { + default: { + fuzz: { runs: 256 }, + }, + ci: { + fuzz: { runs: 10_000 }, + }, + }, + }, + }, +}); +``` + +Each Test Profile can use the full [Solidity tests configuration](/docs/reference/configuration#solidity-tests-configuration). + +When using `profiles`, a `default` profile is always required. + +Profile names can contain letters, numbers, underscores, and dashes. They can't be `fuzz`, `invariant`, `isolate`, `evmVersion`, or `allowInternalExpectRevert`, as those would be ambiguous with [inline configuration](/docs/guides/testing/inline-configuration) keys. + +If you define your Solidity tests config without explicitly defining Test Profiles, you're actually configuring the default behavior. + +## Choosing a Test Profile + +Use the `--test-profile` argument to choose which Test Profile to use when running your tests. + +For example, to run your Solidity tests with the `ci` profile: + + + +It also works on the `test` task, where it only affects the Solidity tests: + + + +Set the `HARDHAT_TEST_PROFILE` environment variable to select a profile: + + + +If you provide both, the argument takes precedence. If you don't specify a profile, Hardhat uses `default`. + +## Sharing default values between Test Profiles + +To use the same settings in several profiles, define them once in your config file and spread them into each profile: + +```ts ins={4-7,14-15} +// hardhat.config.ts +import { defineConfig } from "hardhat/config"; + +const base = { + isolate: true, + fuzz: { runs: 256 }, +}; + +export default defineConfig({ + //... + test: { + solidity: { + profiles: { + default: base, + ci: { ...base, fuzz: { runs: 10_000 } }, + }, + }, + }, +}); +``` + +Each Test Profile is resolved on its own, starting from Hardhat's defaults, the same way [Build Profiles](/docs/guides/writing-contracts/build-profiles) are. A profile that only sets `fuzz.runs` gets the default value for every other setting, not the value from `default`. + +## Test Profiles and Build Profiles + +Test Profiles and [Build Profiles](/docs/guides/writing-contracts/build-profiles) are independent: Test Profiles only affect how your tests run, and Build Profiles only affect how your contracts are compiled. + +Switching Test Profiles doesn't cause your contracts to be recompiled, so you can switch between profiles freely. If you want to change both profiles, you need to pass both arguments: + + + +## Overriding settings for a single test + +Test Profiles apply to your entire test suite. To override settings for a single test function, use [inline configuration](/docs/guides/testing/inline-configuration), which can also be scoped to a Test Profile. diff --git a/src/content/docs/docs/guides/testing/using-solidity.mdx b/src/content/docs/docs/guides/testing/using-solidity.mdx index 2fbf0d65c..6b5d1bf88 100644 --- a/src/content/docs/docs/guides/testing/using-solidity.mdx +++ b/src/content/docs/docs/guides/testing/using-solidity.mdx @@ -219,3 +219,5 @@ export default defineConfig({ To learn more about how to configure your Solidity tests, read [the Solidity tests Configuration reference](/docs/reference/configuration#solidity-tests-configuration). You can also override config settings for individual test functions using inline configuration. Read the [Inline configuration](/docs/guides/testing/inline-configuration) guide to learn more. + +To define several sets of settings and choose between them when running your tests, read the [Test Profiles](/docs/guides/testing/test-profiles) guide. diff --git a/src/content/docs/docs/guides/writing-contracts/build-profiles.mdx b/src/content/docs/docs/guides/writing-contracts/build-profiles.mdx index aa091eb2f..4b789974d 100644 --- a/src/content/docs/docs/guides/writing-contracts/build-profiles.mdx +++ b/src/content/docs/docs/guides/writing-contracts/build-profiles.mdx @@ -56,3 +56,5 @@ For example, to run your tests with the production profile: If you don't specify a profile, most tasks will use `default`, but each task can choose the profile that works best for its workflow. For example, when you deploy with [Hardhat Ignition](/ignition), it uses `production` by default. + +Build Profiles only affect how your contracts are compiled. To run your Solidity tests with different settings, use [Test Profiles](/docs/guides/testing/test-profiles), which are selected separately with `--test-profile`. diff --git a/src/content/docs/docs/migrate-from-foundry/index.mdx b/src/content/docs/docs/migrate-from-foundry/index.mdx index bd57ff26a..3ce6232a3 100644 --- a/src/content/docs/docs/migrate-from-foundry/index.mdx +++ b/src/content/docs/docs/migrate-from-foundry/index.mdx @@ -166,7 +166,23 @@ Port the compiler settings and project layout from `foundry.toml` so Hardhat fin ::: -3. **Compile your contracts** +3. **Split your profiles into Build and Test Profiles** + + Foundry uses a single profile to hold both compiler and test settings: a `[profile.ci]` section holds them together. Hardhat splits them in two, so each Foundry profile becomes two Hardhat profiles: + - Compiler settings go to a [Build Profile](/docs/guides/writing-contracts/build-profiles) under `solidity.profiles`, selected with `--build-profile`. + - Test settings go to a [Test Profile](/docs/guides/testing/test-profiles) under `test.solidity.profiles`, selected with `--test-profile`. + + If you create both profiles for `ci`, select them together with: + + + + :::caution + + Hardhat profiles don't inherit from each other. In Foundry, every profile is merged over `[profile.default]`, so a `[profile.ci]` that only sets `optimizer_runs` keeps everything else from `default`. In Hardhat, each profile is resolved on its own, you can share settings between profiles through shared data in your config file (i.e. a variable). + + ::: + +4. **Compile your contracts** Run the `build` task to verify that the config is working: @@ -257,6 +273,36 @@ Hardhat 3's Solidity testing is designed for compatibility with Foundry test sui See [the Solidity tests configuration reference](/docs/reference/configuration#solidity-tests-configuration) for the full list of options, including `from`, `txOrigin`, `blockTimestamp`, fork settings, and the `fsPermissions` allowlist that replaces Foundry's `fs_permissions`. + If you have more than one `foundry.toml` profile, port each one's test settings into [Test Profiles](/docs/guides/testing/test-profiles) under `test.solidity.profiles`, and select them with `--test-profile`. Hardhat profiles don't inherit from `default`, so share the settings they have in common through shared data in your config file: + + ```ts + // hardhat.config.ts + import { defineConfig } from "hardhat/config"; + + const sharedTestSettings = { + isolate: true, + }; + + export default defineConfig({ + test: { + solidity: { + profiles: { + default: { + ...sharedTestSettings, + fuzz: { runs: 256 }, + }, + ci: { + ...sharedTestSettings, + fuzz: { runs: 10_000 }, + }, + }, + }, + }, + }); + ``` + + If your tests use `forge-config:` annotations, they will keep working. Hardhat supports defaults for all profiles at the root rather than with a `default.` profile. Read the [inline configuration](/docs/guides/testing/inline-configuration) guide to see how they interact with Test Profiles. + 4. **Enable `ffi` and filesystem access if your tests need them** These are off by default for security reasons. If you use `vm.ffi`, set `test.solidity.ffi: true`. If you use `vm.readFile`/`vm.writeFile`, declare the allowed paths under `test.solidity.fsPermissions`. diff --git a/src/content/docs/docs/reference/configuration.mdx b/src/content/docs/docs/reference/configuration.mdx index cde2f1e86..3b8513e55 100644 --- a/src/content/docs/docs/reference/configuration.mdx +++ b/src/content/docs/docs/reference/configuration.mdx @@ -294,6 +294,36 @@ If write access to a directory is needed, please make sure that it doesn't conta - `includePushBytes`: The flag indicating whether to include push bytes values. Defaults to true. - `shrinkRunLimit`: The maximum number of attempts to shrink a failed sequence. The shrink process is disabled if set to 0. Defaults to 5000. +#### Test Profiles + +Instead of the options above, `test.solidity` can contain a single `profiles` entry: a map of [Test Profile](/docs/guides/testing/test-profiles) names to objects with those same options. + +```ts +// hardhat.config.ts +import { defineConfig } from "hardhat/config"; + +export default defineConfig({ + test: { + solidity: { + profiles: { + default: { + fuzz: { runs: 256 }, + }, + ci: { + fuzz: { runs: 10_000 }, + }, + }, + }, + }, +}); +``` + +The two shapes are mutually exclusive: a config with `profiles` can't also set options at the top level. Setting them at the top level is equivalent to defining only a `default` profile. + +When using `profiles`, a `default` profile is always required. + +Profile names can contain letters, numbers, underscores, and dashes. They can't be `fuzz`, `invariant`, `isolate`, `evmVersion`, or `allowInternalExpectRevert`, as those would be ambiguous with [inline configuration](/docs/guides/testing/inline-configuration) keys. + ### Coverage configuration Hardhat has built-in [code coverage](/docs/guides/testing/code-coverage). You can use the `coverage` entry to configure it: diff --git a/src/content/docs/docs/reference/foundry-compatibility.mdx b/src/content/docs/docs/reference/foundry-compatibility.mdx index f5ce548d7..a2cc9a9d0 100644 --- a/src/content/docs/docs/reference/foundry-compatibility.mdx +++ b/src/content/docs/docs/reference/foundry-compatibility.mdx @@ -47,6 +47,8 @@ Hardhat 3 can run Solidity tests by default, including those previously built fo To learn how to configure the Solidity tests runner, read the [Configuration](/docs/reference/configuration#solidity-tests-configuration) reference documentation. +Foundry's `[profile.]` test settings map to Hardhat's [Test Profiles](/docs/guides/testing/test-profiles), and its `forge-config:` annotations to Hardhat's [inline configuration](/docs/guides/testing/inline-configuration). Both differ from Foundry in ways that matter when migrating, so read the [Foundry migration guide](/docs/migrate-from-foundry) first. + ## Separate build process and test artifacts Hardhat 3 builds smart contracts and tests as separate build processes. On top of that, it doesn't expose the build artifacts of tests to users, and considers them internal and part of the `cache/` directory.