From e3fbbf6c5c5bbd41ad702860e17dc13e0fc84587 Mon Sep 17 00:00:00 2001 From: alangmartini Date: Fri, 1 May 2026 16:09:25 -0300 Subject: [PATCH] docs(v30): clarify required non-indexed fields are now type-validated In v30.0+ the skip-validation gate for `index: false` tightened to also require `optional: true`. Required non-indexed fields are now type validated on write, which can cause imports that previously succeeded under v29.x to return 400 errors after upgrading. Updates the v30.0/30.1/30.2 release notes (Deprecations / behavior changes) and the "Indexing all but some fields" section in collections.md to spell out the new constraint and the recommended fixes (set the field optional, or change the type to match the data shape). --- docs-site/content/30.0/api/README.md | 3 ++- docs-site/content/30.0/api/collections.md | 4 +++- docs-site/content/30.1/api/README.md | 3 ++- docs-site/content/30.1/api/collections.md | 4 +++- docs-site/content/30.2/api/README.md | 3 ++- docs-site/content/30.2/api/collections.md | 4 +++- 6 files changed, 15 insertions(+), 6 deletions(-) diff --git a/docs-site/content/30.0/api/README.md b/docs-site/content/30.0/api/README.md index eb6f22d8..88c96e46 100644 --- a/docs-site/content/30.0/api/README.md +++ b/docs-site/content/30.0/api/README.md @@ -46,7 +46,7 @@ This release contains new features, enhancements, performance improvements, bug - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.0/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.0/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.0/api/search.html#faceting-parameters) @@ -109,6 +109,7 @@ This release contains new features, enhancements, performance improvements, bug - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.0/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.0/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.0/api/collections.md b/docs-site/content/30.0/api/collections.md index e7774c5e..3951eba9 100644 --- a/docs-site/content/30.0/api/collections.md +++ b/docs-site/content/30.0/api/collections.md @@ -391,7 +391,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: diff --git a/docs-site/content/30.1/api/README.md b/docs-site/content/30.1/api/README.md index 3cfdeb24..1d9b69e7 100644 --- a/docs-site/content/30.1/api/README.md +++ b/docs-site/content/30.1/api/README.md @@ -46,7 +46,7 @@ This release contains new features, enhancements, performance improvements, bug - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.1/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.1/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.1/api/search.html#faceting-parameters) @@ -110,6 +110,7 @@ This release contains new features, enhancements, performance improvements, bug - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.1/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.1/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.1/api/collections.md b/docs-site/content/30.1/api/collections.md index a44f0e72..b62b159f 100644 --- a/docs-site/content/30.1/api/collections.md +++ b/docs-site/content/30.1/api/collections.md @@ -401,7 +401,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: diff --git a/docs-site/content/30.2/api/README.md b/docs-site/content/30.2/api/README.md index 1e378cda..9974331e 100644 --- a/docs-site/content/30.2/api/README.md +++ b/docs-site/content/30.2/api/README.md @@ -50,7 +50,7 @@ search, highlighting, filtering, faceting, vector search, and multi-search / con - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.1/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.1/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.1/api/search.html#faceting-parameters) @@ -132,6 +132,7 @@ search, highlighting, filtering, faceting, vector search, and multi-search / con - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.1/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.2/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.2/api/collections.md b/docs-site/content/30.2/api/collections.md index 6420d14b..84f98fcd 100644 --- a/docs-site/content/30.2/api/collections.md +++ b/docs-site/content/30.2/api/collections.md @@ -391,7 +391,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: