- ElasticSearch 基本概念
- ElasticSearch Setting
- ElasticSearch Mapping
- ElasticSearch 基础操作
- ElasticSearch 数据类型
- 参考文献
版本信息:7.13.4
ElasticSearch 基本概念
是文档型数据库,不是一个java应用,需要数据库相当的硬件配置,不具有事务(ACID)特性
- 集群(cluster) 多个节点构成集群
- 节点/实例(node) 相当于关系数据库(Database)
- 路由(routing) 用于控制文档是在哪个分片上,默认路由为其标识ID
- 索引(Index) 相当于关系型数据库中的表(Table)
- 文档(Document) 相当于关系型数据库中表的一行数据(Row)
- 属性(Field) 相当于关系型数量库中表的列(Column)
- DSL(查询语句) 相当于关系型数量库中的查询语句(SQL)
ElasticSearch Setting
每个索引的配置信息
基础设置
index.number_of_shards主分片的数量,一般设置不能超过节点数index.number_of_replicas副本分片的数量,一般设置小于节点数index.refresh_interval刷新的时间,默认为1sindex.default_pipeline默认pipeline,没有pipeline的时候执行index.final_pipeline最终的pipeline,都会进行执行
限制设置
-
index.mapping.total_fields.limit一个索引的属性总数限制,默认值是1000properties以及fields下的字段统计数量 -
index.mapping.depth.limit一个索引属性深度限制,默认是20 -
index.mapping.nested_fields.limit一个索引中nested类型属性次数限制,默认是50 -
index.mapping.nested_objects.limit一个文档中nested类型所有属性对象的长度限制,默认是10000 -
index.mapping.field_name_length.limit索引属性名称长度限制,默认是Long.MAX_VALUE(相当于无限制) -
index.max_result_window查询的最大数,避免深度分页,默认是10000
# 查询setting
GET my-index-0000001/_settings
# 参数,flat_settings,json扁平化
GET my-index-0000001/_settings?flat_settings=true
# 参数 ,默认的参数,include_defaults
GET my-index-0000001/_settings?flat_settings=true&include_defaults=true
# 修改setting
PUT xxx_index_name/_settings
{
"index.mapping.total_fields.limit":3
}
ElasticSearch Mapping
每个索引的属性信息
Mapping配置
dynamic 参数配置,可以在mapping下控制全局,也可以在属性 下配置控制单个属性,单个属性的配置高于全局设置
true自动添加runtime只可在属性下进行配置,配置的属性不可检索,只在查询的时候在_source中显示false新增属性忽略(不添加mapping),不可进行检索,只在查询的时候在_source中显示strict新增属性异常报错
字段动态Mapping
| JSON数据类型 | dynamic:true | dynamic:runtime |
|---|---|---|
null | 不处理 | 不处理 |
true/false | boolean | boolean |
double | float | double |
integer | long | long |
object | object | object |
array | array中第一个非空值 | array中第一个非空值 |
日期类型推断String | date | date |
数字类型推断String | float/long | double/long |
剩下的String | text与.keyword | keyword |
- 日期类型推断参数
date_detection,默认为true- 日期类型格式化参数
dynamic_date_formats,默认为[ yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis],自动转换类型只有yyyy/MM/dd HH:mm:ss||yyyy/MM/dd,epoch_millis可以作为date类型的值,设置了类型的话后面创建数据必须符合该格式
- 日期类型格式化参数
- 数字类型推断参数
numeric_detection,默认为false
# 删除索引,确保是自动映射
DELETE date-demo-001
# 第一次保存插入,动态mapping
POST date-demo-001/_doc
{
"createAt1":"2021/07/28", # date
"createAt2":"21/07/28", # text与.keyword
"createAt3":"2021/07/28 00:00:00", # date
"createAt4":1627462203 # long
}
# 数据可以正常插入
POST date-demo-001/_doc
{
"createAt1":1627462203,
"createAt2":"21/7/8",
"createAt3":1627462203
}
# 删除索引,确保是自动映射
DELETE date-demo-001
# 设置mapping中的动态日期格式
PUT date-demo-001
{
"mappings": {
"dynamic_date_formats":["yyyy/MM/dd HH:mm:ss||yyyy/MM/dd","MM/dd/yyyy"]
}
}
# 第一次保存插入,动态mapping
POST date-demo-001/_doc
{
"createAt1":"2021/07/28", # date
"createAt2":"2021/07/28 00:00:00", # date
"createAt3":"07/28/2021" # date
}
# 设置mapping中的数字类型推断
PUT number-demo-001
{
"mappings": {
"numeric_detection": true
}
}
# 第一次保存插入,动态mapping
POST number-demo-001/_doc
{
"number1":"1.0", # float
"number2":"2" # long
}
模板动态Mapping
分两种类型:某个具体索引级别的模板和索引模板
- 类型匹配
match_mapping_type - 名称匹配/不匹配
match/unmatch - 路径匹配/不匹配
path_match/path_unmatch
具体索引下的模板(优先级高)
# 具体的某个索引下的模板
# 删除索引,确保是自动映射
DELETE dynamic-demo-001
# 设置索引模板
# 1. 不以test开头且以List结尾的object类型转换为nested类型
# 2. name路径下除了middle全部设置为keyword类型
PUT dynamic-demo-001
{
"mappings": {
"dynamic_templates": [
{
"list_as_nested": {
"match_mapping_type": "object",
"match": "*List",
"unmatch":"test*",
"mapping": {
"type": "nested"
}
}
},
{
"full_name": {
"path_match": "name.*",
"path_unmatch":"*.middle",
"mapping": {
"type": "keyword"
}
}
}
]
}
}
# 第一次保存插入,动态mapping
POST dynamic-demo-001/_doc
{
"doctorList": [ # nested
{
"doctorId": 1, # long
"count": 2 # long
}
],
"testList": [ # object
{
"test1": 1 # long
}
],
"arrayList": [ #long
1,
2,
3,
4
],
"name": { # object
"first": "xu", # keyword
"middle": "cheng",# test 与 .keyword
"last": "lin" # keyword
}
}
索引模板
# 创建索引模板
# 删除索引模板
DELETE _index_template/dynamic-demo-01
# 创建索引模板
# 1. 以test开头的object类型转换为nested类型
# 2. name路径下middle设置为keyword类型
PUT _index_template/dynamic-demo-01
{
"index_patterns": [
"dynamic-*"
],
"priority": 1, # 多个模板的话,数值越大,优先级越高
"template": {
"mappings": {
"dynamic_templates": [
{
"list_as_nested": {
"match_mapping_type": "object",
"match": "test*",
"mapping": {
"type": "nested"
}
}
},
{
"full_name": {
"path_match": "*.middle",
"mapping": {
"type": "keyword"
}
}
}
]
}
}
}
# 删除索引,确保是自动映射
DELETE dynamic-demo-002
# 第一次保存插入,动态mapping
POST dynamic-demo-002/_doc
{
"doctorList": [ # object
{
"doctorId": 1, # long
"count": 2 # long
}
],
"testList": [ # nested
{
"test1": 1 # long
}
],
"arrayList": [ # long
1,
2,
3,
4
],
"name": {
"first": "xu", # text 与 .keyword
"middle": "cheng", # keyword
"last": "lin" # text 与 .keyword
}
}
组件模板动态Mapping
# 删除组件模板
DELETE _component_template/component_template_001
# 创建组件模板,test开头的object类型转为为nested类型
PUT _component_template/component_template_001
{
"template": {
"mappings": {
"dynamic_templates": [
{
"list_as_nested": {
"match_mapping_type": "object",
"match": "test*",
"mapping": {
"type": "nested"
}
}
}
]
}
}
}
# 删除组件模板
DELETE _component_template/component_template_002
# 创建组件模板,middle为keyword类型
PUT _component_template/component_template_002
{
"template": {
"mappings": {
"dynamic_templates": [
{
"full_name": {
"path_match": "*.middle",
"mapping": {
"type": "keyword"
}
}
}
]
}
}
}
# 删除索引模板
DELETE _index_template/component_template_compose_001
# 利用组合组件模板创建索引模板
PUT _index_template/component_template_compose_001
{
"index_patterns": [
"template-compose-*"
],
"priority": 1,
"composed_of":["component_template_001","component_template_002"]
}
# 删除索引
DELETE template-compose-001
# 创建索引
POST template-compose-001/_doc
{
"doctorList": [ # object
{
"doctorId": 1, # long
"count": 2 # long
}
],
"testList": [ # nested
{
"test1": 1 # long
}
],
"arrayList": [ # long
1,
2,
3,
4
],
"name": {
"first": "xu", # test 与 .keyword
"middle": "cheng", # keyword
"last": "lin" # test 与 .keyword
}
}
# 获取mapping数据
GET template-compose-001/_mapping
元数据信息
_index索引的名称_type索引类型,7版本只有_doc,8版本可能会进行删除_id文档的ID_score相关度评分_source文档的原始数据_meta元数据的其他元数据信息_routing路由信息_version文档的版本信息,可以外置_seq_no文档的序号,与_primary_term一起构成内部版本控制_primary_term分片的序号,与_seq_no一起构成内部版本控制_ingored索引的时候忽略的字段,参数配置ignore_malformed
Mapping参数
dynamic 动态匹配配置
优先级高于mapping中的配置,默认值为true
# 删除创建
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "text"
},
"area": {
"type": "object",
"dynamic": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": "张三", # text
"area": {
"prod": "AH", # text 与 .keyword
"city": "HF" # text 与 .keyword
},
"age": 30 # 无
}
# 获取mapping信息
GET dynamic-demo-001/_mapping
index 是否索引
默认值为true
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "text",
"index": false
},
"area": {
"type": "object",
"dynamic": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": "张三",
"area": {
"prod": "AH",
"city": "HF"
},
"age": 30
}
# 查询索引数据,查询报错 Cannot search on field [name] since it is not indexed
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name": "张三"
}
}
}
enabled 属性启用
控制索引和mapping,类型为
object或者nested类型与index的区别是查询的时候不报错,mapping也不会变化,默认值为true
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"dynamic": false,
"properties": {
"name": {
"type": "object",
"enabled": false
},
"area": {
"type": "object",
"dynamic": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": "张三",
"area": {
"prod": "AH",
"city": "HF"
},
"age": 30
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": {
"first": "张",
"last": "三"
},
"area": {
"prod": "AH",
"city": "HF"
},
"age": 30
}
GET dynamic-demo-001/_mapping
# 查询索引数据,无法检索到数据
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name.first": "张"
}
}
}
format 格式化匹配
主要是针对日期类型的格式化匹配
# 删除索引
DELETE dynamic-demo-001
# 创建索引mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name":{
"type": "text"
},
"birth":{
"type":"date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 创建索引报错,格式不匹配
POST dynamic-demo-001/_doc
{
"name":"李四",
"birth":"1991 02 27"
}
# 获取mapping
GET dynamic-demo-001/_mapping
# 范围查询
GET dynamic-demo-001/_search
{
"query": {
"range": {
"birth": {
"gte": "1991/02/27",
"lte": "2021/02/27"
}
}
}
}
store 存储
默认是只索引,不存储,只会出现在_source里面,索引文档会存放ES和Lucence,store会存储到Lucence上,默认值为false
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name":{
"type": "text",
"store": true
},
"birth":{
"type":"date",
"format": ["yyyy/MM/dd"],
"store": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 查询数 据
GET dynamic-demo-001/_search
{
"stored_fields": ["name"], # 只返回name字段的值
"query": {
"range": {
"birth": {
"gte": "1991/02/27",
"lte": "2021/02/27"
}
}
}
}
fields 多属性
文本类型会自动mapping出一个keyword属性
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name":{
"type": "text",
"store": true,
"fields": {
"raw":{
"type":"keyword"
}
}
},
"birth":{
"type":"date",
"format": ["yyyy/MM/dd"],
"store": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 查询数据
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name.raw": "张三"
}
}
}
ignore_malformed 忽略格式检查
错误的格式字段在_ingored属性中体现,默认为false
# 创建索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name":{
"type": "text"
},
"birth":{
"type":"date",
"format": ["yyyy/MM/dd"],
"ignore_malformed": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 创建索引,可以正常存入
POST dynamic-demo-001/_doc
{
"name":"李四",
"birth":"1991 02 27"
}
# 查询数据,错误的数据会在_ignored中体现
GET dynamic-demo-001/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 2,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "dynamic-demo-001",
"_type" : "_doc",
"_id" : "-MjIBXsBsXY_v_247cx2",
"_score" : 1.0,
"_source" : {
"name" : "张三",
"birth" : "1991/02/27"
}
},
{
"_index" : "dynamic-demo-001",
"_type" : "_doc",
"_id" : "-cjIBXsBsXY_v_24-MzM",
"_score" : 1.0,
"_ignored" : [
"birth"
],
"_source" : {
"name" : "李四",
"birth" : "1991 02 27"
}
}
]
}
}
ingore_above 超过不检索
字符长度,不是字节长度,只针对keyword类型,默认为Int最大值
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "text",
"fields": {
"first": {
"type": "keyword",
"ignore_above": 2
}
}
},
"birth": {
"type": "date",
"format": [
"yyyy/MM/dd"
],
"ignore_malformed": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"李四王五",
"birth":"1991 02 27"
}
# 数据查询,张三可以进行查询出来,但是李四王五无法进行查询
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name.first": "张三"
}
}
}
doc_values 文档值
关闭后该属性不支持排序、聚合、脚本中使用,默认值为true
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "keyword",
"doc_values": false
},
"birth": {
"type": "date",
"format": [
"yyyy/MM/dd"
],
"ignore_malformed": true
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张三",
"birth":"1991/02/27"
}
# 查询排序,报错Can't load fielddata on [name] because fielddata is unsupported on fields of type [keyword]. Use doc values instead
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name": "张三"
}
},
"sort": [
{
"name": {
"order": "desc"
}
}
]
}
fielddata text类型排序
默认值为false,排序是按分词进行排序的
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "text",
"fielddata": true
},
"birth": {
"type": "date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张物三",
"birth":"1991/02/27",
"height":175
}
# 查询排序
GET dynamic-demo-001/_search
{
"query": {
"match_all": {
}
},
"sort": [
{
"name": {
"order": "desc"
}
}
]
}
eager_global_ordinals 全局序号
默认值为false
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "text",
"fielddata": true,
"eager_global_ordinals": true
},
"birth": {
"type": "date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name":"张物三",
"birth":"1991/02/27",
"height":175
}
# 查询排序
GET dynamic-demo-001/_search
{
"query": {
"match_all": {
}
},
"sort": [
{
"name": {
"order": "desc"
}
}
]
}
properties子属性
针对
object类型和nested类型
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "object",
"properties": {
"first":{
"type":"keyword"
},
"last":{
"type": "keyword"
}
}
},
"birth": {
"type": "date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": {
"first": "张",
"last": "三峰"
},
"birth": "1991/02/27",
"height": 175
}
copy_to 复制
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "object",
"properties": {
"first":{
"type":"keyword",
"copy_to":"full_name"
},
"last":{
"type": "keyword",
"copy_to":"full_name"
}
}
},
"birth": {
"type": "date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": {
"first": "张",
"last": "三峰"
},
"birth": "1991/02/27",
"height": 175
}
# 可以根据full_name进行查询
GET dynamic-demo-001/_search
{
"query": {
"match": {
"full_name": "张三峰"
}
}
}
null_value null值处理
相同类型的值处理
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"sex":{
"type": "integer",
"null_value": -1
},
"birth": {
"type": "date",
"format": ["yyyy/MM/dd"]
}
}
}
}
# 创建索引,没有这个sex节点
POST dynamic-demo-001/_doc
{
"name": "张三峰",
"birth": "1991/02/27",
"height": 175
}
# 创建索引,有sex节点,并且值为null
POST dynamic-demo-001/_doc
{
"name": "张三",
"birth": "1991/02/27",
"height": 175,
"sex": null
}
# 查询为-1的数据,能查到张三
GET dynamic-demo-001/_search
{
"query": {
"match": {
"sex": -1
}
}
}
analyzer 分词器
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"settings": {
"analysis": {
"analyzer": {
"english_stop": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"stop"
]
}
}
}
},
"mappings": {
"properties": {
"remark": {
"type": "text",
"analyzer": "english_stop"
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"text":"I have a appple."
}
# term 查询,根据"I" 查询无法进行查询,只能根据"i"查询
GET dynamic-demo-001/_search
{
"query": {
"term": {
"text": "i"
}
}
}
search_analyzer 查询分词器
默认情况下,查询分词器和分词器一致
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"char_filter": [
"quote"
],
"tokenizer": "pattern",
"filter": [
"lowercase",
"stop"
]
}
},
"char_filter": {
"quote": {
"type": "mapping",
"mappings": [
":) => happy",
":( => sad"
]
}
}
}
},
"mappings": {
"properties": {
"remark": {
"type": "text",
"analyzer": "my_analyzer",
"search_analyzer": "standard"
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"remark":"I hava a apple. I am felling :), Feeling :( today"
}
# 根据":)"无法查询,只能根据"happy"进行查询
GET dynamic-demo-001/_search
{
"query": {
"match": {
"remark": ":)"
}
}
}
normalizer 单一分词处理
针对keyword类型
# 删除索引
DELETE dynamic-demo-001
# 创建mapping
PUT dynamic-demo-001
{
"settings": {
"analysis": {
"normalizer": {
"my_normalizer": {
"filter": [
"lowercase"
]
}
}
}
},
"mappings": {
"properties": {
"name": {
"type": "keyword",
"normalizer": "my_normalizer"
}
}
}
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": "BAR"
}
# 创建索引
POST dynamic-demo-001/_doc
{
"name": "bar"
}
# 查询数据,都可以进行查询到
GET dynamic-demo-001/_search
{
"query": {
"match": {
"name": "BAR"
}
}
}
ElasticSearch 基础操作
-
除了查询操作以外的操作都是使用_bulk批量处理的
-
文档的操作需要检索和存储,检索需要refresh,默认配置是1s(准实时),存储是flush
# refresh
POST dynamic-demo-001/_refresh
# flush
POST dynamic-demo-001/_flush -
删除文档只是标记删除,不是物理删除,后台会定时进行处理,需要物理删除的话需要强制合并
# 强制合并
POST dynamic-demo-001/_forcemerge
新增
POST 与 PUT区别,PUT必须要指定ID
# 新增索引,自动创建ID
POST dynamic-demo-001/_doc
{
"name": "BAR"
}
# 新增索引,ID为1,若是存在的话会进行更新
POST dynamic-demo-001/_doc/1
{
"name": "BAR"
}
# 新增索引,ID为1,若是存在的话会进行更新
PUT dynamic-demo-001/_doc/1
{
"name": "BAR"
}
# 新增索引,ID为1,若是存在的话会报错
POST/PUT dynamic-demo-001/_doc/1?op_type=create
{
"name": "BAR"
}
# 新增索引,ID为2,若是存在的话会报错
POST/PUT dynamic-demo-001/_create/2
{
"name":"bar"
}
修改
修改并不是真正意义上的修改,而是更新了删除标识,再创建了一个文档
# 全量修改
PUT/POST dynamic-demo-001/_doc/1
{
"name": "bar",
"sex":1
}
# 增量修改
POST dynamic-demo-001/_update/1
{
"doc":{
"sex":2
}
}
条件修改
利用脚本script和ingest pipeline进行更新
# 脚本script 更新性别为2的改为1
POST dynamic-demo-001/_update_by_query
{
"script": {
"source": """
ctx._source.sex = 1
""",
"lang": "painless"
},
"query": {
"match": {
"sex": 2
}
}
}
# ingest pipeline 更新性别为1的改为2
# 创建一个修改数据的pipeline
PUT _ingest/pipeline/sex001
{
"description": "性别为2",
"processors": [
{
"set": {
"field": "sex",
"value": "2"
}
}
]
}
# 利用pipeline 批量修改数据
POST dynamic-demo-001/_update_by_query?pipeline=sex001
{
"query": {
"match": {
"sex": 1
}
}
}
删除
删除并不是真正意义上的删除,而是更新删除标识,再创建了一个文档
# 根据文档进行删除
DELETE dynamic-demo-001/_doc/1
条件删除
# 根据条件批量进行删除
POST dynamic-demo-001/_delete_by_query
{
"query": {
"match": {
"name": "bar"
}
}
}
查询
# 根据标识进行查询
GET dynamic-demo-001/_doc/1
# QSL查询
GET dynamic-demo-001/_search
批量操作
_bulk操作
version与version_type版本控制refresh是否立即刷新,默认为falserouting路由,根据路由选择分片pineline管道处理器wait_for_active_shards操作的分片数,默认为1
# index 索引 update 需改 delete 删除 create 新建
PUT _bulk
{"index":{"_index":"dynamic-demo-001","_id":1}}
{"name":"bar","sex":2}
{"index":{"_index":"dynamic-demo-001","_id":2}}
{"name":"bob","sex":1}
{"update":{"_index":"dynamic-demo-001","_id":1}}
{"doc":{"age":20}}
{"delete":{"_index":"dynamic-demo-001","_id":2}}
{"delete":{"_index":"dynamic-demo-001","_id":3}}
{"create":{"_index":"dynamic-demo-001","_id":4}}
{"name":"bar","sex":2}
_mget操作
# 多个查询
GET _mget
{
"docs": [
{
"_index": "dynamic-demo-001",
"_id": 1
},
{
"_index": "dynamic-demo-001",
"_id": 4
}
]
}
# 多个查询
GET dynamic-demo-001/_mget
{
"docs": [
{
"_id": 1
},
{
"_id": 4
}
]
}
# 某个索引下的多个查询
GET dynamic-demo-001/_mget
{
"ids":[1,4]
}
ElasticSearch 数据类型
基础类型
binary
base64的字符串,不存储不索引
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"blob": {
"type": "binary"
}
}
}
}
# 索引数据
PUT my-index-000001/_doc/1
{
"name": "Some binary blob",
"blob": "U29tZSBiaW5hcnkgYmxvYg=="
}
boolean
只有两个值,true 和 false
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"is_published": {
"type": "boolean"
}
}
}
}
# 创建索引,字符串会自动解析为boolean
POST my-index-000001/_doc/1
{
"is_published": "true"
}
# 创建索引
POST my-index-000001/_doc/2
{
"is_published": true
}
# 查询可以查询全部
GET my-index-000001/_search
{
"query": {
"term": {
"is_published": true
}
}
}
keywords
当数值类型满足下面的条件的时候,用keyword类型更好
- 不需要范围查询
- 快速检索,term下的keyword效率高于number类型
keyword一个词项constant_keyword常量词项,固定值,文档的值没有或者必须为该固定值,版本7.7.0wildcard通配词项,版本7.9.0
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"tags":{
"type": "keyword"
},
"level":{
"type":"constant_keyword",
"value":1
},
"name":{
"type":"text"
},
"pinyin":{
"type": "wildcard"
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"name":"测试",
"pinyin":"cs",
"level":1,
"tags":"cs"
}
# 索引
POST my-index-0000001/_doc
{
"name":"环境",
"pinyin":"hj",
"tags":"hj"
}
# 查询索引
GET my-index-0000001/_search
numbers
有精度要求首选
scaled_float,无精度要求首选适合的
byte8字节,-128到127short16字节,-32768到32767integer32字节,-2^31 到 2^31 -1long64字节,-2^63 到 2^63 -1float32字节的小数double64字节的小数half_float16字节的小数scaled_float精度的小数,后台使用long存储unsigned_long无符号的整数, 0 到 2^64-1,版 本7.10.0
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"age": {
"type": "short"
},
"income": {
"type": "scaled_float",
"scaling_factor": 100
},
"name": {
"type": "text"
},
"id": {
"type": "long"
},
"sex":{
"type": "byte"
}
}
}
}
# 创建索引
POST my-index-0000001/_doc
{
"id":4300221305,
"name":"张三",
"age":35,
"income":15617.43,
"sex":1
}
# 查询数据
GET my-index-0000001/_search
dates
日期最好有个格式,格式默认为
strict_date_optional_time_nanos||epoch_millis,参考format
date日期类型date_nacos毫秒的日期类型
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"createDate":{
"type": "date"
},
"traceDate":{
"type": "date_nanos"
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"createDate":"2021-08-03",
"traceDate":"2021-08-03"
}
# 索引
POST my-index-0000001/_doc
{
"createDate":"2021-08-03T12:10:30Z",
"traceDate":"2021-08-03T12:10:30Z"
}
# 索引
POST my-index-0000001/_doc
{
"createDate":"1420070400001",
"traceDate":"1420070400001"
}
# 索引
POST my-index-0000001/_doc
{
"createDate":1420070400001,
"traceDate":1420070400001
}
# 排序查询
GET my-index-0000001/_search
{
"sort": [
{
"createDate": {
"order": "desc"
}
}
]
}
# 获取createDate数据
GET my-index-0000001/_search
{
"fields": [ {"field": "createDate"}],
"_source": false
}
# 获取traceDate数据
GET my-index-0000001/_search
{
"fields": [ {"field": "traceDate"}],
"_source": false
}
# 格式化输出createDate数据
GET my-index-0000001/_search
{
"fields": [ {"field": "createDate","format": "yyyy/MM/dd HH:mm:ss"}],
"_source": false
}
alias
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"name": {
"type": "text"
},
"aliasName": {
"type": "alias",
"path": "name"
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"name":"Mike"
}
# 根据别名进行查询
GET my-index-0000001/_search
{
"query": {
"match": {
"aliasName": "Mike"
}
}
}
对象以及关系类型
object
复杂类型默认都是对象,单个类型的数组为单类型,对象类型的数组为object类型
# 删除索引
DELETE my-index-0000001
# 索引
POST my-index-0000001/_doc
{
"tags": [ # text 与 .keyword
"cs",
"hj"
],
"location": [ # long
11111,
2222
],
"area": { # object
"pro": "AH", # text 与 .keyword
"city": "HF" # text 与 .keyword
},
"cities": [ # object
{
"aliasName": "HF", # text 与 .keyword
"name": "合肥" # text 与 .keyword
}
]
}
# 获取mapping
GET my-index-0000001/_mapping
flatetened
避免属性过多,但使用内部字段进行排序的话需要注意是按照字符串的进行排序的,7.3.0版本的特性
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"area":{
"type": "flattened"
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF"
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF",
"location": [
11111,
2222
]
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF",
"location": [
11111,
2222
],
"cities": [
{
"aliasName": "HF",
"name": "合肥"
}
]
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF",
"source":1,
"location": [
11111,
2222
],
"cities": [
{
"aliasName": "HF",
"name": "合肥"
}
]
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF",
"source":11,
"location": [
11111,
2222
],
"cities": [
{
"aliasName": "HF",
"name": "合肥"
}
]
}
}
# 索引
POST my-index-0000001/_doc
{
"area": {
"pro": "AH",
"city": "HF",
"source":7,
"location": [
11111,
2222
],
"cities": [
{
"aliasName": "HF",
"name": "合肥"
}
]
}
}
# 查询数据
GET my-index-0000001/_search
{
"fields": [{"field": "area"},{"field": "area.city"},{"field": "area.location"},{"field": "area.cities"},{"field": "area.source"}],
"_source": false
}
# 排序,需要注意是按字符串的排序,会导致7在11之前
GET my-index-0000001/_search
{
"sort": [
{
"area.source": {
"order": "desc"
}
}
]
}
nested
针对子文档需要精确检索(需要数组中多个条件)的时候,需要使用该内容
# 删除索引
DELETE my-index-0000001
# 索引
POST my-index-0000001/_doc
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
# 查询,可以查询到数据,不应该查到数据
GET my-index-0000001/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"user.first": "John"
}
},
{
"match": {
"user.last": "White"
}
}
]
}
}
}
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"user":{
"type": "nested"
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
# 查询,查询不到数据,正常
GET my-index-0000001/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"user.first": "John"
}
},
{
"match": {
"user.last": "White"
}
}
]
}
}
}
join
父子文档必须要在同一个分片上
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"id": {
"type": "keyword"
},
"join_field": {
"type": "join",
"relations": {
"question": "answer"
}
}
}
}
}
# 索引父文档
POST my-index-0000001/_doc/1
{
"id": "1",
"text": "This is a question",
"join_field": {
"name": "question"
}
}
# 索引子文档,必须指定父文档标识ID的路由
POST my-index-0000001/_doc?routing=1
{
"id": "2",
"text": "This is a answer2",
"join_field": {
"name": "answer",
"parent":"1"
}
}
# 索引子文档,必须指定父文档标识ID的路由
POST my-index-0000001/_doc?routing=1
{
"id": "3",
"text": "This is a answer1",
"join_field": {
"name": "answer",
"parent":"1"
}
}
# 根据parent_id查询子文档
GET my-index-0000001/_search
{
"query": {
"parent_id":{
"type":"answer",
"id":"1"
}
}
}
# 根据子文档查询父文档
GET my-index-0000001/_search
{
"query": {
"has_child": {
"type": "answer",
"query": {
"match": {
"id": "2"
}
}
}
}
}
# 根据父文档查询子文档
GET my-index-0000001/_search
{
"query": {
"has_parent": {
"parent_type": "question",
"query": {
"match": {
"id": "1"
}
}
}
}
}
结构数据类型
range
integer_range32位整数范围,-2^31 到 2^31 -1float_range32位小数范围long_range64位整数范围,-2^63 到 2^63 -1double_range64位小数范围date_range日期范围ip_rangeip范围
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-0000001
{
"mappings": {
"properties": {
"hight":{
"type": "integer_range"
},
"validateDate":{
"type":"date_range",
"format": ["yyyy/MM/dd HH:mm:ss"]
}
}
}
}
# 索引
POST my-index-0000001/_doc
{
"hight" : {
"gte" : 10,
"lt" : 20
},
"validateDate" : {
"gte" : "2015/10/31 12:00:00",
"lte" : "2015/11/01 00:00:00"
}
}
# 具体值进行查询
GET my-index-0000001/_search
{
"query": {
"match": {
"hight": 15
}
}
}
# 范围查询,默认是intersects
GET my-index-0000001/_search
{
"query": {
"range": {
"hight": {
"gte": 19,
"lt": 35
}
}
}
}
# 范围查询,查询范围之内的数据
GET my-index-0000001/_search
{
"query": {
"range": {
"hight": {
"gte": 5,
"lt": 30,
"relation": "within"
}
}
}
}
# 范围查询,查询范围被完全包含的数据
GET my-index-0000001/_search
{
"query": {
"range": {
"hight": {
"gte": 18,
"lt": 19,
"relation": "contains"
}
}
}
}
ip
# 删除索引
DELETE my-index-0000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"ip_addr": {
"type": "ip"
}
}
}
}
# 索引
PUT my-index-000001/_doc/1
{
"ip_addr": "192.168.1.1"
}
# 查询
GET my-index-000001/_search
{
"query": {
"term": {
"ip_addr": "192.168.0.0/1"
}
}
}
文本搜索类型
text
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"full_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
# 索引
POST my-index-000001/_doc
{
"full_name":"张三"
}
# 索引
POST my-index-000001/_doc
{
"full_name":"张三峰"
}
# 查询
GET my-index-000001/_search
{
"query": {
"match": {
"full_name": "张"
}
}
}
match_only_text
特殊的text类型,不提供评分和高性能,适用日志信息领域,版本7.14.0
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"message": {
"type": "match_only_text"
}
}
}
}
# 索引
POST my-index-000001/_doc
{
"message":"I have a apple."
}
# 索引
POST my-index-000001/_doc
{
"message":"I have a apple two."
}
# 查询,_score 都为1.0
GET my-index-000001/_search
{
"query": {
"match": {
"message": "apple"
}
}
}
token_count
必须要指定分词器analyzer
# 删除索引
DELETE my-index-000001
# 创建mapping
PUT my-index-000001
{
"mappings": {
"properties": {
"full_name": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
},
"length":{
"type":"token_count",
"analyzer":"standard"
}
}
}
}
}
}
# 索引
POST my-index-000001/_doc
{
"full_name":"张三"
}
# 索引
POST my-index-000001/_doc
{
"full_name":"张三峰"
}
# 查询
GET my-index-000001/_search
{
"query": {
"match": {
"full_name.length": 2
}
}
}