springBoot2.1.3集成elasticSearch6.7.1
elasticsearch很火,但是springboot和es之间存在版本问题,如果版本不一致,就会出现如下异常:None of the configured nodes are available....这里我选用的版本是 springboot 2.1.3 + elasticsearch6.7.1 具体版本之间的问题,可以参考:https://github.com/spring-pro...
一.centos7安装elasticSearch6.7.1
//强烈建议:安装进官网下载最新版本!! https://www.elastic.co/guide/en/elasticsearch/reference/current/zip-targz.html //选择文件的方式,有.zip或.gz,这个自己选择,进行解压(在线下载,需要耐心等待) wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.zip wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.zip.sha512 shasum -a 512 -c elasticsearch-6.7.1.zip.sha512 unzip elasticsearch-6.7.1.zip cd elasticsearch-6.7.1/ //下载后,进入安装目录,我的安装目录是/crawler/es/elasticsearch-6.7.1/ cd /crawler/es/elasticsearch-6.7.1/ cd config vi elasticsearch.yml //在文件里面添加如下配置:我们需要将配置设置成所有机器都可以访问。 network.host: 0.0.0.0 http.port: 9500 transport.tcp.port: 9300 transport.tcp.compress: true
此时基本配置已经做好,在这里可以准备启动es,但是需要注意的是,启动es(6+版本)需要非root账号,并且jvm的设置可能会有限制问题,在此,我们做如下两步操作(没有抛错的请忽略)
(这里可以参照https://www.cnblogs.com/phppe...
a.将jvm配置改大:
sudo sysctl -w vm.max_map_count=262144
b.创建额外的用户(非root用户),并授权
第一步:liunx创建新用户 adduser XXX 然后给创建的用户加密码 passwd XXX 输入两次密码。
第二步:切换刚才创建的用户 su XXX 然后执行elasticsearch 会显示Permission denied 权限不足。
第三步:给新建的XXX赋权限,chmod 777 * 这个不行,因为这个用户本身就没有权限,肯定自己不能给自己付权限。所以要用root用户登录付权限。
第四步:root给XXX赋权限,chown -R XXX /你的elasticsearch安装目录。
做了如上操作以后,我们切换到创建用户,进入安装目录并执行./bin/elasticsearch -d (守护线程执行)就好了
二.springBoot2.1.3集成es6.7
上面的es服务器搭建好了以后,集成在springboot上面就会比较简单了,下面做了一个demo:(参照https://www.cnblogs.com/qdhxh...
pom.xml依赖坐标
<dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-elasticsearch</artifactId> </dependency>
配置文件
spring: data: elasticsearch: cluster-name: elasticsearch cluster-nodes: www.iamcrawler.cn:9300 //大哥,这里写你自己的.... repositories: enabled: true
实体类
import com.baomidou.mybatisplus.annotations.TableField; import com.baomidou.mybatisplus.annotations.TableName; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.AllArgsConstructor; import lombok.Builder; import lombok.Data; import lombok.NoArgsConstructor; import org.springframework.data.elasticsearch.annotations.Document; /** * Created by liuliang on 2018/11/1. */ @TableName("goddess_user") @Data @NoArgsConstructor @AllArgsConstructor @Builder @Document(indexName = "wantu_notice_info", type = "doc") public class GoddessUser { @JsonProperty("id") private Long id; @JsonProperty("code") private String code; @TableField("user_name") @JsonProperty("userName") private String userName; @JsonProperty("password") private String password; @JsonProperty("mail") private String mail; @JsonProperty("phone") private String phone; }
Repository
@Component public interface EsUserRepository extends ElasticsearchRepository<GoddessUser, Long> { }
Controller
package cn.iamcrawler.crawlergoddess.elasticsearch.user; import cn.iamcrawler.crawler_common.domain.goddess.GoddessUser; import cn.iamcrawler.crawlergoddess.mapper.GoddessUserMapper; import com.baomidou.mybatisplus.mapper.EntityWrapper; import com.google.common.collect.Lists; import lombok.extern.slf4j.Slf4j; import org.elasticsearch.index.query.MatchQueryBuilder; import org.elasticsearch.index.query.QueryBuilders; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.data.domain.Pageable; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import java.util.ArrayList; import java.util.List; /** * Created by liuliang on 2019/4/8. */ @RestController @RequestMapping("/api/es") @Slf4j public class EsUserController { @Autowired private GoddessUserMapper userMapper; @Autowired EsUserRepository esUserRepository; @PostMapping("/insert/all") public ResponseEntity loadAllUser(){ List<GoddessUser> users = userMapper.selectList(new EntityWrapper<GoddessUser>()); log.info("users:{}",users); users.stream().forEach(goddessUser -> { esUserRepository.save(goddessUser); }); return ResponseEntity.ok(true); } @GetMapping("/search") public ResponseEntity getByName(String name, Pageable pageable){ //按name进行搜索 MatchQueryBuilder builder = QueryBuilders.matchQuery("userName", name); //如果实体和数据的名称对应就会自动封装,pageable分页参数 Page<GoddessUser> users = esUserRepository.search(builder, pageable); //迭代器转list ArrayList<GoddessUser> list = Lists.newArrayList(users); return ResponseEntity.ok(list); }
}
可以看到,上面一个insert进去以后,在此查询的时候,就有数值了,postman示例:
[{ "id": 1, "code": "crawler", "userName": "liuliang", "password": "123456", "mail": "***@sina.com", "phone": "185***6019" } ]
至此,从搭建,到集成简单的demo结束...本文做的时候参考的文档有:
https://my.oschina.net/u/3187...
https://www.cnblogs.com/phppe...
相关推荐
另外一部分,则需要先做聚类、分类处理,将聚合出的分类结果存入ES集群的聚类索引中。数据处理层的聚合结果存入ES中的指定索引,同时将每个聚合主题相关的数据存入每个document下面的某个field下。