Jay Taylor's notes

back to listing index

ruby on rails - Tire Gem Returns 'Can't sort on string types with more than one value per doc, or more than one token per field' - Stack Overflow

[web search]
Original source (stackoverflow.com)
Tags: elasticsearch facets stackoverflow.com
Clipped on: 2012-11-01

I have mapped an associated field as a multi-field. I have set the 'name' property to be analyzed using a snow ball analyzer and 'exact' as not analyzed. I am able to search on this field and filter as well. I am not able to sort on this field. When trying to sort, elastic returns an error "Can't sort on string types with more than one value per doc, or more than one token per field".

I have attempted creating an additional field called 'raw' similar to exact, and this did not work either. Below is my mapping and how I am attempting to perform the sort via the tire gem:

    mapping do
    indexes :sectors, :type => 'object',
      :properties => { :name => { :type => 'multi_field',
                                  :fields => {
                                    :name =>  { :type => 'string', :analyzer => 'snowball' },
                                    :exact => { :type => 'string', :index => 'not_analyzed' , :include_in_all => false }
                                  }
                                  }
                       }
end

def to_indexed_json
    to_json( :include => {
               ...
               :sectors => { :only => ["name"] },
               ...
    })
end

def self.search(params)
  tire.search(:load => true, :page=>params[:page], :per_page => 12) do

    if params[:query].present?
      query { string params[:query], :default_operator => "OR" }
    else
      query { all }
    end

    filter :term, "sectors.name.exact" => params[:sector] if params[:sector].present?
    sort { by 'sectors.name.exact','asc' } if !params[:sort][:sector].blank?
  end
end
asked Oct 14 at 5:36
Image (Asset 1/3) alt= 132
add comment

2 Answers

up vote 3 down vote accepted

There are basically two reasons why you're getting that error:

  • You have more than one value per field: your field is analyzed and the tokenizer produces multiple tokens out of its content, which is not the case with your name.exact field since it's not_analyzed in your mapping
  • You have more than one value per document: you're adding multiple values to the name field, for example providing an array of values while indexing.

I would check what your documents look like. In order to do it quickly you can make a Terms Facet based on a script which returns the number of items included in the name.exact field like the following. If you get back at least one of the facet entries with term greather than 1, it means that you've added multiple values to the field at least once, thus you cannot sort on it.

{
    "query" : {
        "match_all" : {}
    },
    "facets" : {
        "my_facet" : {
            "terms" : {
                "script_field" : "doc['name.exact'].values.length"
            }
        }
    }
}
answered Oct 14 at 10:59
Image (Asset 2/3) alt= 8,03631429
  upvote
 flag
I ran the following: curl -X GET "localhost:9200/myprj/prj/_search?load=true&pretty=true"; -d '{"query":{"match_all":{}},"facets":{"sorters":{"terms":{"script_field":"doc[\u0‌​027sectors.name.exact\u0027].values.length"}}}}' – Macho Geek Oct 14 at 14:20
1 upvote
 flag
Sounds like you have 1 document with 3 terms and 1 with 2 terms. Thanks javanna for the script tip! Very interesting! – dadoonet Oct 15 at 4:53
  upvote
 flag
@MachoGeek I meant a value greather than 1. 399 of your documents have 1 value for that field, one has 2 values, one has 3, 13 documents have no value at all. In order to sort on it you need to have 0 or 1 value. – javanna Oct 15 at 6:52
  upvote
 flag
@javanna, sounds good, I will keep digging. I think I have a little better idea of what to look for now. – Macho Geek Oct 15 at 15:07
  upvote
 flag
Glad to know! You might want to accept one of our answers then. – javanna Oct 15 at 16:22
add / show 1 more comment

IMHO, it should work. Just wondering if your mapping is applied. Can you check it with a curl command?

Something like:

$ curl -XGET 'http://localhost:9200/twitter/tweet/_mapping'

answered Oct 14 at 6:02
Image (Asset 3/3) alt= 3464
  upvote
 flag
Yes it returns: "sectors":{"properties":{"name":{"type":"multi_field","fields":{"name":{"type":"‌​string","analyzer":"snowball"},"exact":{"type":"string","index":"not_analyzed","i‌​nclude_in_all":false},"raw":{"type":"string","index":"not_analyzed","store":"yes"‌​,"include_in_all":false}}}} – Macho Geek Oct 14 at 6:09
add comment

Your Answer

 
community wiki

Not the answer you're looking for? Browse other questions tagged or ask your own question.