Skip to content

Commit

Permalink
Merge pull request #6 from Badiapp/total/chewy-v7.1
Browse files Browse the repository at this point in the history
Total/chewy v7.1
  • Loading branch information
lucanioi authored Nov 27, 2023
2 parents 662caeb + 89c7b28 commit 27de527
Show file tree
Hide file tree
Showing 37 changed files with 166 additions and 500 deletions.
26 changes: 0 additions & 26 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,6 @@ workflows:
- rspec-ruby-27-activerecord52
- rspec-ruby-27-mongoid
- rspec-ruby-30-activerecord61
- rspec-ruby-27-activerecord61-es6
- rspec-ruby-30-activerecord61-es6

commands:
rspec-test:
Expand Down Expand Up @@ -214,27 +212,3 @@ jobs:
GEMFILE: gemfiles/rails.6.1.activerecord.gemfile
steps:
- rspec-test

rspec-ruby-27-activerecord61-es6:
docker:
- image: circleci/ruby:2.7
- image: docker.elastic.co/elasticsearch/elasticsearch:6.8.13
environment:
<<: *es-env
working_directory: ~/repo
environment:
GEMFILE: gemfiles/rails.6.1.activerecord.gemfile
steps:
- rspec-test

rspec-ruby-30-activerecord61-es6:
docker:
- image: circleci/ruby:3.0
- image: docker.elastic.co/elasticsearch/elasticsearch:6.8.13
environment:
<<: *es-env
working_directory: ~/repo
environment:
GEMFILE: gemfiles/rails.6.1.activerecord.gemfile
steps:
- rspec-test
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,17 @@

### Bugs Fixed

## 7.1.0 (2021-03-03)

### Changes

* [#766](https://github.com/toptal/chewy/pull/766): **(Breaking)** Drop support for Elasticsearch 6.x ([@rabotyaga][])
* [#765](https://github.com/toptal/chewy/pull/765): Fix ruby 2.7 warnings in rake tasks ([@aglushkov][])

### Bugs Fixed

* [#722](https://github.com/toptal/chewy/issues/722): Remove alias_method_chain, use Module#prepend instead ([@dalthon][])

## 7.0.1 (2021-05-03)

### Changes
Expand Down
65 changes: 27 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ Chewy is an ODM (Object Document Mapper), built on top of the [the official Elas
* [Index definition](#index-definition)
* [Type default import options](#type-default-import-options)
* [Multi (nested) and object field types](#multi-nested-and-object-field-types)
* [Parent and children types](#parent-and-children-types)
* [Geo Point fields](#geo-point-fields)
* [Crutches™ technology](#crutches-technology)
* [Witchcraft™ technology](#witchcraft-technology)
Expand Down Expand Up @@ -55,10 +54,6 @@ Chewy is an ODM (Object Document Mapper), built on top of the [the official Elas

In this section we'll cover why you might want to use Chewy instead of the official `elasticsearch-ruby` client gem.

* Multi-model indices.

Index classes are independent from ORM/ODM models. Now, implementing e.g. cross-model autocomplete is much easier. You can just define the index and work with it in an object-oriented style. You can define several types for index - one per indexed model.

* Every index is observable by all the related models.

Most of the indexed models are related to other and sometimes it is necessary to denormalize this related data and put at the same object. For example, you need to index an array of tags together with an article. Chewy allows you to specify an updateable index for every model separately - so corresponding articles will be reindexed on any tag update.
Expand Down Expand Up @@ -99,6 +94,7 @@ Chewy is compatible with MRI 2.5-3.0¹.

| Chewy version | Elasticsearch version |
| ------------- | ---------------------------------- |
| 7.1.x | 7.x |
| 7.0.0 | 6.8, 7.x |
| 6.0.0 | 5.x, 6.x |
| 5.x | 5.x, limited support for 1.x & 2.x |
Expand Down Expand Up @@ -365,18 +361,6 @@ end

The `value:` option for internal fields will no longer be effective.

### Parent and children types

To define [parent](https://www.elastic.co/guide/en/elasticsearch/guide/current/parent-child-mapping.html) type for a given index_type, you can include root options for the type where you can specify parent_type and parent_id

```ruby
define_type User.includes(:account) do
root parent: 'account', parent_id: ->{ account_id } do
field :created_at, type: 'date'
field :task_id, type: 'integer'
end
end
```
### Geo Point fields

You can use [Elasticsearch's geo mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-point.html) with the `geo_point` field type, allowing you to query, filter and order by latitude and longitude. You can use the following hash format:
Expand Down Expand Up @@ -602,7 +586,7 @@ Imagine that you reset your index in a zero-downtime manner (to separate index),

### Types access

You can access index-defined types with the following API:
You can access index-defined type with the following API:

```ruby
UsersIndex::User # => UsersIndex::User
Expand Down Expand Up @@ -630,11 +614,10 @@ UsersIndex::User.import # import with 0 arguments process all the data specified
UsersIndex::User.import User.where('rating > 100') # or import specified users scope
UsersIndex::User.import User.where('rating > 100').to_a # or import specified users array
UsersIndex::User.import [1, 2, 42] # pass even ids for import, it will be handled in the most effective way
UsersIndex::User.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action.
UsersIndex::User.import User.where('rating > 100'), update_fields: [:email] # if update fields are specified - it will update their values only with the `update` bulk action

UsersIndex.import # import every defined type
UsersIndex.import user: User.where('rating > 100') # import only active users to `user` type.
# Other index types, if exists, will be imported with default scope from the type definition.
UsersIndex.import # same as UsersIndex::User.import
UsersIndex.import user: User.where('rating > 100') # import only specified users
UsersIndex.reset! # purges index and imports default data for all types
```

Expand Down Expand Up @@ -923,27 +906,33 @@ Quick introduction.
The request DSL have the same chainable nature as AR or Mongoid ones. The main class is `Chewy::Search::Request`. It is possible to perform requests on behalf of indices or types:

```ruby
PlaceIndex.query(match: {name: 'London'}) # returns documents of any type
PlaceIndex::City.query(match: {name: 'London'}) # returns cities only.
CitiesIndex.query(match: {name: 'London'}) # or
CitiesIndex::City.query(match: {name: 'London'})
```

Main methods of the request DSL are: `query`, `filter` and `post_filter`, it is possible to pass pure query hashes or use `elasticsearch-dsl`. Also, there is an additional

```ruby
PlaceIndex
CitiesIndex
.filter(term: {name: 'Bangkok'})
.query { match name: 'London' }
.query.not(range: {population: {gt: 1_000_000}})
```

You can query a set of indexes at once:

```ruby
CitiesIndex.indices(CountriesIndex).query(match: {name: 'Some'})
```

See https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html and https://github.com/elastic/elasticsearch-ruby/tree/master/elasticsearch-dsl for more details.

An important part of requests manipulation is merging. There are 4 methods to perform it: `merge`, `and`, `or`, `not`. See [Chewy::Search::QueryProxy](lib/chewy/search/query_proxy.rb) for details. Also, `only` and `except` methods help to remove unneeded parts of the request.

Every other request part is covered by a bunch of additional methods, see [Chewy::Search::Request](lib/chewy/search/request.rb) for details:

```ruby
PlaceIndex.limit(10).offset(30).order(:name, {population: {order: :desc}})
CitiesIndex.limit(10).offset(30).order(:name, {population: {order: :desc}})
```

Request DSL also provides additional scope actions, like `delete_all`, `exists?`, `count`, `pluck`, etc.
Expand All @@ -969,16 +958,16 @@ See [Chewy::Search::Scrolling](lib/chewy/search/scrolling.rb) for details.
It is possible to load ORM/ODM source objects with the `objects` method. To provide additional loading options use `load` method:

```ruby
PlacesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Type` wrappers.
PlacesIndex.load(scope: -> { active }).objects # An array of AR source objects.
CitiesIndex.load(scope: -> { active }).to_a # to_a returns `Chewy::Type` wrappers.
CitiesIndex.load(scope: -> { active }).objects # An array of AR source objects.
```

See [Chewy::Search::Loader](lib/chewy/search/loader.rb) for more details.

In case when it is necessary to iterate through both of the wrappers and objects simultaneously, `object_hash` method helps a lot:

```ruby
scope = PlacesIndex.load(scope: -> { active })
scope = CitiesIndex.load(scope: -> { active })
scope.each do |wrapper|
scope.object_hash[wrapper]
end
Expand All @@ -995,8 +984,8 @@ Performs zero-downtime reindexing as described [here](https://www.elastic.co/blo
```bash
rake chewy:reset # resets all the existing indices
rake chewy:reset[users] # resets UsersIndex only
rake chewy:reset[users,places] # resets UsersIndex and PlacesIndex
rake chewy:reset[-users,places] # resets every index in the application except specified ones
rake chewy:reset[users,cities] # resets UsersIndex and CitiesIndex
rake chewy:reset[-users,cities] # resets every index in the application except specified ones
```

#### `chewy:upgrade`
Expand All @@ -1011,8 +1000,8 @@ See [Chewy::Stash::Specification](lib/chewy/stash.rb) and [Chewy::Index::Specifi
```bash
rake chewy:upgrade # upgrades all the existing indices
rake chewy:upgrade[users] # upgrades UsersIndex only
rake chewy:upgrade[users,places] # upgrades UsersIndex and PlacesIndex
rake chewy:upgrade[-users,places] # upgrades every index in the application except specified ones
rake chewy:upgrade[users,cities] # upgrades UsersIndex and CitiesIndex
rake chewy:upgrade[-users,cities] # upgrades every index in the application except specified ones
```

#### `chewy:update`
Expand All @@ -1024,8 +1013,8 @@ Unlike `reset` or `upgrade` tasks, it is possible to pass type references to upd
```bash
rake chewy:update # updates all the existing indices
rake chewy:update[users] # updates UsersIndex only
rake chewy:update[users,places#city] # updates the whole UsersIndex and PlacesIndex::City type
rake chewy:update[-users,places#city] # updates every index in the application except every type defined in UsersIndex and the rest of the types defined in PlacesIndex
rake chewy:update[users,cities#city] # updates UsersIndex and CitiesIndex (if City type is defined on CitiesIndex)
rake chewy:update[-users,cities#city] # updates every index in the application except UsersIndex and CitiesIndex::City
```
#### `chewy:sync`
Expand All @@ -1039,8 +1028,8 @@ See [Chewy::Type::Syncer](lib/chewy/type/syncer.rb) for more details.
```bash
rake chewy:sync # synchronizes all the existing indices
rake chewy:sync[users] # synchronizes UsersIndex only
rake chewy:sync[users,places#city] # synchronizes the whole UsersIndex and PlacesIndex::City type
rake chewy:sync[-users,places#city] # synchronizes every index in the application except every type defined in UsersIndex and the rest of the types defined in PlacesIndex
rake chewy:sync[users,cities#city] # synchronizes UsersIndex and CitiesIndex (if City type is defined on CitiesIndex)
rake chewy:sync[-users,cities#city] # synchronizes every index in the application except except UsersIndex and CitiesIndex::City
```
#### `chewy:deploy`
Expand All @@ -1064,7 +1053,7 @@ If the number of processes is not specified explicitly - `parallel` gem tries to
```bash
rake chewy:parallel:reset
rake chewy:parallel:upgrade[4]
rake chewy:parallel:update[4,places#city]
rake chewy:parallel:update[4,cities#city]
rake chewy:parallel:sync[4,-users]
rake chewy:parallel:deploy[4] # performs parallel upgrade and parallel sync afterwards
```
Expand Down
3 changes: 1 addition & 2 deletions lib/chewy/fields/root.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ def mappings_hash
mappings[name][:dynamic_templates].concat dynamic_templates
end

mappings[name][:_parent] = parent.is_a?(Hash) ? parent : {type: parent} if parent
mappings
mappings[name]
end

def dynamic_template(*args)
Expand Down
19 changes: 4 additions & 15 deletions lib/chewy/index.rb
Original file line number Diff line number Diff line change
Expand Up @@ -152,23 +152,12 @@ def define_type(target, options = {}, &block)
self.type_hash = type_hash.merge(type_class.type_name => type_class)
end

# Types method has double usage.
# If no arguments are passed - it returns array of defined types:
# Returns defined type:
#
# UsersIndex.types # => [UsersIndex::Admin, UsersIndex::Manager, UsersIndex::User]
# UsersIndex.types # => [UsersIndex::User]
#
# If arguments are passed it treats like a part of chainable query DSL and
# adds types array for index to select.
#
# UsersIndex.filters { name =~ 'ro' }.types(:admin, :manager)
# UsersIndex.types(:admin, :manager).filters { name =~ 'ro' } # the same as the first example
#
def types(*args)
if args.present?
all.types(*args)
else
type_hash.values
end
def types
type_hash.values
end

# Returns defined types names:
Expand Down
4 changes: 1 addition & 3 deletions lib/chewy/index/actions.rb
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,7 @@ def create!(suffix = nil, **options)

body = specification_hash
body[:aliases] = {general_name => {}} if options[:alias] && suffixed_name != general_name
args = {index: suffixed_name, body: body}
args[:include_type_name] = true if Runtime.version >= '6.7.0'
result = client.indices.create(**args)
result = client.indices.create(index: suffixed_name, body: body)

Chewy.wait_for_status if result
result
Expand Down
4 changes: 1 addition & 3 deletions lib/chewy/index/aliases.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ module Aliases

module ClassMethods
def indexes
get_args = {index: index_name}
get_args[:include_type_name] = true if Runtime.version >= '6.7.0'
indexes = empty_if_not_found { client.indices.get(**get_args).keys }
indexes = empty_if_not_found { client.indices.get(index: index_name).keys }
indexes += empty_if_not_found { client.indices.get_alias(name: index_name).keys }
indexes.compact.uniq
end
Expand Down
20 changes: 2 additions & 18 deletions lib/chewy/railtie.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,6 @@ def call(env)
end

module MigrationStrategy
extend ActiveSupport::Concern
included do
alias_method_chain :migrate, :chewy
end

def migrate_with_chewy(*args)
Chewy.strategy(:bypass) { migrate_without_chewy(*args) }
end
end

module Rails5MigrationStrategy
def migrate(*args)
Chewy.strategy(:bypass) { super }
end
Expand All @@ -57,13 +46,8 @@ def migrate(*args)

initializer 'chewy.migration_strategy' do
ActiveSupport.on_load(:active_record) do
if Rails::VERSION::MAJOR >= 5
ActiveRecord::Migration.prepend(Rails5MigrationStrategy)
ActiveRecord::Migrator.prepend(Rails5MigrationStrategy) if defined? ActiveRecord::Migrator
else
ActiveRecord::Migration.send(:include, MigrationStrategy)
ActiveRecord::Migrator.send(:include, MigrationStrategy) if defined? ActiveRecord::Migrator
end
ActiveRecord::Migration.prepend(MigrationStrategy)
ActiveRecord::Migrator.prepend(MigrationStrategy) if defined? ActiveRecord::Migrator
end
end

Expand Down
14 changes: 2 additions & 12 deletions lib/chewy/search/loader.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,10 @@ module Search
# @see Chewy::Search::Scrolling#scroll_objects
class Loader
# @param indexes [Array<Chewy::Index>] list of indexes to lookup types
# @param only [Array<String, Symbol>] list of selected type names to load
# @param except [Array<String, Symbol>] list of type names which will not be loaded
# @param options [Hash] adapter-specific load options
# @see Chewy::Type::Adapter::Base#load
def initialize(indexes: [], only: [], except: [], **options)
def initialize(indexes: [], **options)
@indexes = indexes
@only = Array.wrap(only).map(&:to_s)
@except = Array.wrap(except).map(&:to_s)
@options = options
end

Expand All @@ -31,7 +27,7 @@ def derive_type(index, type)
(@derive_type ||= {})[[index, type]] ||= begin
index_class = derive_index(index)
raise Chewy::UnderivableType, "Can not find index named `#{index}`" unless index_class
index_class.type_hash[type] or raise Chewy::UnderivableType, "Index `#{index}` doesn`t have type named `#{type}`"
index_class.type_hash.values.first
end
end

Expand All @@ -48,8 +44,6 @@ def derive_type(index, type)
def load(hits)
hit_groups = hits.group_by { |hit| [hit['_index'], hit['_type']] }
loaded_objects = hit_groups.each_with_object({}) do |((index_name, type_name), hit_group), result|
next if skip_type?(type_name)

type = derive_type(index_name, type_name)
ids = hit_group.map { |hit| hit['_id'] }
loaded = type.adapter.load(ids, **@options.merge(_type: type))
Expand All @@ -74,10 +68,6 @@ def derive_index(index_name)
def indexes_hash
@indexes_hash ||= @indexes.index_by(&:index_name)
end

def skip_type?(type_name)
@except.include?(type_name) || @only.present? && !@only.include?(type_name)
end
end
end
end
Loading

0 comments on commit 27de527

Please sign in to comment.