Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ Lint/ConstantDefinitionInBlock:
Lint/EmptyBlock:
Exclude:
- 'spec/chewy/index/import/bulk_request_spec.rb'
- 'spec/chewy/index/witchcraft_spec.rb'
- 'spec/chewy/minitest/helpers_spec.rb'
- 'spec/chewy/rspec/update_index_spec.rb'
- 'spec/chewy/search/scrolling_spec.rb'
Expand Down Expand Up @@ -136,7 +135,6 @@ Naming/VariableNumber:
# Offense count: 3
Style/DocumentDynamicEvalDefinition:
Exclude:
- 'lib/chewy/index/witchcraft.rb'
- 'lib/chewy/repository.rb'
- 'lib/chewy/search/pagination/kaminari.rb'

Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@

### Changes

* **Removal:** `Chewy::Index.witchcraft!` and the `Chewy::Index::Witchcraft` module have been removed. The compiled compose path (default since 8.3.0) delivers equivalent throughput without `method_source` / `parser` / `prism` / `unparser`. Remove any `witchcraft!` call from your indexes — no other changes are needed. ([@AlfonsoUceda][])

## 8.4.0 (2026-06-17)

### New Features
Expand Down
51 changes: 1 addition & 50 deletions docs/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -278,55 +278,6 @@ The compiled path also handles hash inputs, join fields, and nested object field
For a small number of edge cases the compiler falls back to the legacy path automatically: `ignore_blank` fields, `geo_point` fields, indexes with a custom root value proc, and fields whose name is not a valid Ruby identifier.
The fallback is transparent — the compose result is identical either way.

## Witchcraft technology (deprecated)

> **Deprecated**.
> `witchcraft!` is retained only for compatibility with older index definitions and will be removed in a future major release.
> The compiled compose path above is the default for all indexes and delivers equivalent performance without `method_source` / `parser` / `prism` / `unparser` dependencies, with substantially lower boot-time memory (see [#644](https://github.com/toptal/chewy/issues/644)).
> Remove the `witchcraft!` call from your index — no other changes are needed.

Witchcraft was an experimental performance optimization that used `method_source` to read each field value proc's Ruby source, parse it with `parser` or `prism`, rewrite the AST, and `Unparser`-emit a single fused lambda per index.
Concretely it compiled definitions like this:

```ruby
index_scope Product
witchcraft!

field :title
field :tags, value: -> { tags.map(&:name) }
field :categories do
field :name, value: -> (product, category) { category.name }
field :type, value: -> (product, category, crutch) { crutch.types[category.name] }
end
```

into a single lambda roughly equivalent to:

```ruby
-> (object, crutches) do
{
title: object.title,
tags: object.tags.map(&:name),
categories: object.categories.map do |object2|
{
name: object2.name,
type: crutches.types[object2.name]
}
end
}
end
```

It came with several limitations the compiled path does not have:

1. Required `method_source`-parseable proc source — reflowing a value proc could break the build.
2. Did not support value procs with splat arguments.
3. Required dynamically-generated fields to use procs with explicit arguments rather than argumentless `-> { ... }`.
4. Required `method_source`, `parser` (or `prism`), and `unparser` at boot — together adding ~7 MiB of allocations and ~1 MiB of retained memory to every Rails boot, even for apps that did not call `witchcraft!`.

Calling `witchcraft!` now prints a deprecation warning.
Removing the call switches the index to the compiled path automatically; no other code changes are needed.

## Import context

When importing, you often already have data in memory at the call site (precomputed embeddings, batched API responses, aggregations) that crutches would otherwise re-fetch from the database.
Expand Down Expand Up @@ -355,7 +306,7 @@ field :embedding, value: ->(object, crutches, context) {

Both are backward-compatible: existing 1-arg crutch blocks and 1-2 arg field procs continue to work unchanged via arity-based dispatch.

Context flows through the default compiled compose path automatically, and is also honored under the legacy `witchcraft!` path and under parallel imports (`parallel: N`).
Context flows through the default compiled compose path automatically, and is also honored under parallel imports (`parallel: N`).
With `parallel:`, the same context hash is shared across all workers — precompute once on the main process, reuse in every worker.

```ruby
Expand Down
1 change: 0 additions & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,6 @@ See [configuration.md](configuration.md#import-scope-clean-up-behavior) for deta
Some Chewy features require additional gems that are not listed as hard dependencies:

- **`parallel`** — required for `chewy:parallel:*` rake tasks. Install it with `gem 'parallel'` in your Gemfile.
- **`method_source`**, **`parser`** (or **`prism`** on Ruby 3.3+), **`unparser`** — required only for the deprecated `witchcraft!` path. Not needed for the default compiled compose path. Install them only if you still call `witchcraft!` from an index.

If these gems are missing you'll get a `LoadError` when the relevant feature is used.

Expand Down
2 changes: 0 additions & 2 deletions gemfiles/base.gemfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
gem 'database_cleaner'
gem 'elasticsearch-extensions'
gem 'kaminari-core', require: false
gem 'method_source'
gem 'parallel', require: false
gem 'rake'
gem 'redis', require: false
Expand All @@ -14,4 +13,3 @@ gem 'rubocop-rake'
gem 'sidekiq', require: false
gem 'sqlite3'
gem 'timecop'
gem 'unparser', '~> 0.6.15'
2 changes: 0 additions & 2 deletions lib/chewy/index.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
require 'chewy/index/settings'
require 'chewy/index/specification'
require 'chewy/index/syncer'
require 'chewy/index/witchcraft'
require 'chewy/index/compiled'
require 'chewy/index/wrapper'

Expand All @@ -32,7 +31,6 @@ class Index
include Mapping
include Observe
include Crutch
include Witchcraft
include Compiled
include Wrapper

Expand Down
4 changes: 1 addition & 3 deletions lib/chewy/index/compiled.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ class Index
# per-field arity into the call site and inlines hash construction,
# removing the per-field iteration + dispatch overhead of the legacy
# plain compose path. Proc bodies are NOT inlined — procs are called
# via `.call`. In exchange there are no parser/unparser/method_source
# deps and no Ruby/Prism version coupling.
# via `.call`.
#
# `fields:` selection is supported: a dedicated method is generated
# and memoized per unique fields set.
Expand All @@ -28,7 +27,6 @@ module ClassMethods
# Returns true when this index can use the compiled path for the
# given compose call. Builds the per-fields method if needed.
def compiled_compose_available?(fields)
return false if witchcraft?
return false unless root&.children&.present?

ensure_compiled_compose_for!(fields)
Expand Down
4 changes: 1 addition & 3 deletions lib/chewy/index/import.rb
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ def bulk(**options)
payload_errors(error_items)
end

# Composes a single document from the passed object. Uses either witchcraft
# Composes a single document from the passed object. Uses either the compiled
# or normal composing under the hood.
#
# @param object [Object] a data source object
Expand All @@ -140,8 +140,6 @@ def compose(object, crutches = nil, fields: [], context: {})
else
compiled_compose(object, crutches, context, fields)
end
elsif witchcraft? && root.children.present?
cauldron(fields: fields).brew(object, crutches, context)
else
root.compose(object, crutches, fields: fields, context: context)
end
Expand Down
Loading