diff --git a/docs-site/content/30.0/api/README.md b/docs-site/content/30.0/api/README.md index eb6f22d8..88c96e46 100644 --- a/docs-site/content/30.0/api/README.md +++ b/docs-site/content/30.0/api/README.md @@ -46,7 +46,7 @@ This release contains new features, enhancements, performance improvements, bug - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.0/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.0/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.0/api/search.html#faceting-parameters) @@ -109,6 +109,7 @@ This release contains new features, enhancements, performance improvements, bug - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.0/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.0/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.0/api/collections.md b/docs-site/content/30.0/api/collections.md index e7774c5e..3951eba9 100644 --- a/docs-site/content/30.0/api/collections.md +++ b/docs-site/content/30.0/api/collections.md @@ -391,7 +391,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: diff --git a/docs-site/content/30.1/api/README.md b/docs-site/content/30.1/api/README.md index 3cfdeb24..1d9b69e7 100644 --- a/docs-site/content/30.1/api/README.md +++ b/docs-site/content/30.1/api/README.md @@ -46,7 +46,7 @@ This release contains new features, enhancements, performance improvements, bug - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.1/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.1/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.1/api/search.html#faceting-parameters) @@ -110,6 +110,7 @@ This release contains new features, enhancements, performance improvements, bug - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.1/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.1/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.1/api/collections.md b/docs-site/content/30.1/api/collections.md index a44f0e72..b62b159f 100644 --- a/docs-site/content/30.1/api/collections.md +++ b/docs-site/content/30.1/api/collections.md @@ -401,7 +401,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: diff --git a/docs-site/content/30.2/api/README.md b/docs-site/content/30.2/api/README.md index 1e378cda..9974331e 100644 --- a/docs-site/content/30.2/api/README.md +++ b/docs-site/content/30.2/api/README.md @@ -50,7 +50,7 @@ search, highlighting, filtering, faceting, vector search, and multi-search / con - Configurable token truncation for string fields to improve exact match filtering on long strings, using the new `truncate` parameter in the collection schema. [(Docs)](https://typesense.org/docs/30.1/api/collections.html#field-parameters) - Return an error message when a field is declared that references another field of the same collection. - New `group_max_candidates` search parameter which overrides the behavior of `group_by` queries introduced in [v29.0](https://typesense.org/docs/29.0/api/#deprecations-behavior-changes) where `found` value is an approximation. When `group_max_candidates` is passed, `found` will be accurate up until its value. [(Docs)](https://typesense.org/docs/30.1/api/search.html#grouping-parameters) -- Allow non-indexed nested fields to still be marked as required. +- Allow non-indexed nested fields to still be marked as required (`{"index": false, "optional": false}`). See the behavior changes section below for the validation implication on existing schemas. - Improved synonym matching logic: Previously, synonym matches with a higher number of tokens (query/synonym) would be ranked higher. Now, matches are ranked by how well they match the query/synonyms overall, not just by the number of matched tokens. - Use Transliterator objects pool to enhance tokenization performance of Cyrillic and Chinese languages. - Support for dynamic `facet_return_parent` fields. [(Docs)](https://typesense.org/docs/30.1/api/search.html#faceting-parameters) @@ -132,6 +132,7 @@ search, highlighting, filtering, faceting, vector search, and multi-search / con - ⚠️ The structure of **Analytics Rules** has changed. Old rules will be automatically migrated to the new structure internally. Read more here. [(Docs)](https://typesense.org/docs/30.1/api/analytics-query-suggestions.html) - The export endpoint doesn't stop streaming the response if an error is encountered while loading a document from disk. The error is logged and is also returned in the response stream. - Collections having references to each other are not allowed. If mutual reference is detected, the reference field will not be indexed. +- ⚠️ **Required non-indexed fields are now type-validated on write.** In earlier versions, any field declared with `index: false` silently bypassed type validation regardless of the `optional` setting. From v30.0 onwards, validation is only skipped when the field is **both** `index: false` **and** `optional: true`.

If you previously had a field declared as `{"type": "object", "index": false, "optional": false}` and your documents supplied a value that did not match the declared type (for example, an array of objects against a singular `object` field), imports that previously succeeded will now return `400 Field 'X' has an incorrect type`. To restore the old permissive behavior, either set `"optional": true` on the field, or change the field type so it matches the actual data shape (for example, change `object` to `object[]`). See the [Indexing all but some fields](https://typesense.org/docs/30.2/api/collections.html#indexing-all-but-some-fields) section for examples. ## Upgrading diff --git a/docs-site/content/30.2/api/collections.md b/docs-site/content/30.2/api/collections.md index 6420d14b..84f98fcd 100644 --- a/docs-site/content/30.2/api/collections.md +++ b/docs-site/content/30.2/api/collections.md @@ -391,7 +391,9 @@ float field and we cannot differentiate between a lat/long definition and an act If you have a case where you do want to index all fields in the document, except for a few fields, you can use the `{"index": false, "optional": true}` settings to exclude fields. -Note: it is not currently possible to have a mandatory field excluded from the indexing, hence the setting to optional. +:::warning +Setting only `"index": false` on a required field (`"optional": false`) excludes it from the index but still **type-validates** the document value against the declared field type. So a document that supplies an array against a singular `object` field will be rejected with `400 Field 'X' has an incorrect type`, even though the field is not indexed. To bypass validation entirely, set `"optional": true` as well.

This is a behavior change from pre-v30 releases, where `"index": false` silently skipped validation regardless of the `optional` setting. +::: For eg, if you want to index all fields, except for fields that start with `description_`, you can use a schema like this: