利用SolrJ操作solr API完成index操作
使用SolrJ操作Solr会比利用httpClient来操作Solr要简单。SolrJ是封装了httpClient方法,来操作solr的API的。SolrJ底层还是通过使用httpClient中的方法来完成Solr的操作。
1、首先,你需要添加如下jar包
其中apache-solr-solrj-3.4.0.jar、slf4j-api-1.6.1.jar可以在下载的apache-solr-3.4.0的压缩包中的dist中能找到。
2、其次,建立一个简单的测试类,完成Server对象的相关方法的测试工作,代码如下:
package com.hoo.test; import java.io.IOException; import java.net.MalformedURLException; import java.util.ArrayList; import java.util.Collection; import java.util.List; import org.apache.solr.client.solrj.SolrQuery; import org.apache.solr.client.solrj.SolrServer; import org.apache.solr.client.solrj.SolrServerException; import org.apache.solr.client.solrj.impl.CommonsHttpSolrServer; import org.apache.solr.client.solrj.response.QueryResponse; import org.apache.solr.client.solrj.response.UpdateResponse; import org.apache.solr.common.SolrDocumentList; import org.apache.solr.common.SolrInputDocument; import org.apache.solr.common.params.ModifiableSolrParams; import org.apache.solr.common.params.SolrParams; import org.junit.After; import org.junit.Before; import org.junit.Test; import com.hoo.entity.Index; /** * <b>function:</b> Server TestCase * @author hoojo * @createDate 2011-10-19 下午01:49:07 * @file ServerTest.java * @package com.hoo.test * @project SolrExample * @blog http://blog.csdn.net/IBM_hoojo * @email [email protected] * @version 1.0 */ public class ServerTest { private SolrServer server; private CommonsHttpSolrServer httpServer; private static final String DEFAULT_URL = "http://localhost:8983/solr/"; @Before public void init() { try { server = new CommonsHttpSolrServer(DEFAULT_URL); httpServer = new CommonsHttpSolrServer(DEFAULT_URL); } catch (MalformedURLException e) { e.printStackTrace(); } } @After public void destory() { server = null; httpServer = null; System.runFinalization(); System.gc(); } public final void fail(Object o) { System.out.println(o); } /** * <b>function:</b> 测试是否创建server对象成功 * @author hoojo * @createDate 2011-10-21 上午09:48:18 */ @Test public void server() { fail(server); fail(httpServer); } /** * <b>function:</b> 根据query参数查询索引 * @author hoojo * @createDate 2011-10-21 上午10:06:39 * @param query */ public void query(String query) { SolrParams params = new SolrQuery(query); try { QueryResponse response = server.query(params); SolrDocumentList list = response.getResults(); for (int i = 0; i < list.size(); i++) { fail(list.get(i)); } } catch (SolrServerException e) { e.printStackTrace(); } } }
测试运行servercase方法,如果成功创建对象,那你就成功的链接到。
注意:在运行本方法之前,请启动你的solr官方自动的项目。http://localhost:8983/solr/保证能够成功访问这个工程。因为接下来的所有工作都是围绕这个solr工程完成的。如果你现在还不知道,怎么部署、发布官方solr工程,请参考前面的具体章节。
3、Server的有关配置选项参数,server是CommonsHttpSolrServer的实例
server.setSoTimeout(1000); // socket read timeout server.setConnectionTimeout(100); server.setDefaultMaxConnectionsPerHost(100); server.setMaxTotalConnections(100); server.setFollowRedirects(false); // defaults to false // allowCompression defaults to false. // Server side must support gzip or deflate for this to have any effect. server.setAllowCompression(true); server.setMaxRetries(1); // defaults to 0. > 1 not recommended. //sorlr J 目前使用二进制的格式作为默认的格式。对于solr1.2的用户通过显示的设置才能使用XML格式。 server.setParser(new XMLResponseParser()); //二进制流输出格式 //server.setRequestWriter(new BinaryRequestWriter());
4、利用SolrJ完成IndexDocument的添加操作
/** * <b>function:</b> 添加doc文档 * @author hoojo * @createDate 2011-10-21 上午09:49:10 */ @Test public void addDoc() { //创建doc文档 SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", 1); doc.addField("name", "Solr Input Document"); doc.addField("manu", "this is SolrInputDocument content"); try { //添加一个doc文档 UpdateResponse response = server.add(doc); fail(server.commit());//commit后才保存到索引库 fail(response); fail("query time:" + response.getQTime()); fail("Elapsed Time:" + response.getElapsedTime()); fail("status:" + response.getStatus()); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } query("name:solr"); }
在apache-solr-3.4.0\example\solr\conf目录下的schema.xml中可以找到有关于field属性的配置,schema.xml中的field就和上面Document文档中的field(id、name、manu)对应。如果出现ERROR:unknownfield'xxxx'就表示你设置的这个field在schema.xml中不存在。如果一定要使用这个field,请你在schema.xml中进行filed元素的配置。具体请参考前面的章节。
注意:在schema.xml中配置了uniqueKey为id,就表示id是唯一的。如果在添加Document的时候,id重复添加。那么后面添加的相同id的doc会覆盖前面的doc,类似于update更新操作,而不会出现重复的数据。
5、利用SolrJ添加多个Document,即添加文档集合
/** * <b>function:</b> 添加docs文档集合 * @author hoojo * @createDate 2011-10-21 上午09:55:01 */ @Test public void addDocs() { Collection<SolrInputDocument> docs = new ArrayList<SolrInputDocument>(); SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", 2); doc.addField("name", "Solr Input Documents 1"); doc.addField("manu", "this is SolrInputDocuments 1 content"); docs.add(doc); doc = new SolrInputDocument(); doc.addField("id", 3); doc.addField("name", "Solr Input Documents 2"); doc.addField("manu", "this is SolrInputDocuments 3 content"); docs.add(doc); try { //add docs UpdateResponse response = server.add(docs); //commit后才保存到索引库 fail(server.commit()); fail(response); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } query("solr"); }
就是添加一个List集合
6、添加JavaEntityBean,这个需要先创建一个JavaBean,然后来完成添加操作;
JavaBean:Index的代码
package com.hoo.entity; import org.apache.solr.client.solrj.beans.Field; /** * <b>function:</b> JavaEntity Bean;Index需要添加相关的Annotation注解,便于告诉solr哪些属性参与到index中 * @author hoojo * @createDate 2011-10-19 下午05:33:27 * @file Index.java * @package com.hoo.entity * @project SolrExample * @blog http://blog.csdn.net/IBM_hoojo * @email [email protected] * @version 1.0 */ public class Index { //@Field setter方法上添加Annotation也是可以的 private String id; @Field private String name; @Field private String manu; @Field private String[] cat; @Field private String[] features; @Field private float price; @Field private int popularity; @Field private boolean inStock; public String getId() { return id; } @Field public void setId(String id) { this.id = id; } //getter、setter方法 public String toString() { return this.id + "#" + this.name + "#" + this.manu + "#" + this.cat; } }
注意上面的属性是和在apache-solr-3.4.0\example\solr\conf目录下的schema.xml中可以找到有关于field属性的配置对应的。如果你IndexJavaBean中出现的属性在schema.xml的field配置无法找到,那么出出现unknownfiled错误。
添加Bean完成doc添加操作
/** * <b>function:</b> 添加JavaEntity Bean * @author hoojo * @createDate 2011-10-21 上午09:55:37 */ @Test public void addBean() { //Index需要添加相关的Annotation注解,便于告诉solr哪些属性参与到index中 Index index = new Index(); index.setId("4"); index.setName("add bean index"); index.setManu("index bean manu"); index.setCat(new String[] { "a1", "b2" }); try { //添加Index Bean到索引库 UpdateResponse response = server.addBean(index); fail(server.commit());//commit后才保存到索引库 fail(response); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } queryAll(); }
7、添加Bean集合
/** * <b>function:</b> 添加Entity Bean集合到索引库 * @author hoojo * @createDate 2011-10-21 上午10:00:55 */ @Test public void addBeans() { Index index = new Index(); index.setId("6"); index.setName("add beans index 1"); index.setManu("index beans manu 1"); index.setCat(new String[] { "a", "b" }); List<Index> indexs = new ArrayList<Index>(); indexs.add(index); index = new Index(); index.setId("5"); index.setName("add beans index 2"); index.setManu("index beans manu 2"); index.setCat(new String[] { "aaa", "bbbb" }); indexs.add(index); try { //添加索引库 UpdateResponse response = server.addBeans(indexs); fail(server.commit());//commit后才保存到索引库 fail(response); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } queryAll(); }
8、删除索引Document
/** * <b>function:</b> 删除索引操作 * @author hoojo * @createDate 2011-10-21 上午10:04:28 */ @Test public void remove() { try { //删除id为1的索引 server.deleteById("1"); server.commit(); query("id:1"); //根据id集合,删除多个索引 List<String> ids = new ArrayList<String>(); ids.add("2"); ids.add("3"); server.deleteById(ids); server.commit(true, true); query("id:3 id:2"); //删除查询到的索引信息 server.deleteByQuery("id:4 id:6"); server.commit(true, true); queryAll(); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } }
9、查询索引
/** * <b>function:</b> 查询所有索引信息 * @author hoojo * @createDate 2011-10-21 上午10:05:38 */ @Test public void queryAll() { ModifiableSolrParams params = new ModifiableSolrParams(); // 查询关键词,*:*代表所有属性、所有值,即所有index params.set("q", "*:*"); // 分页,start=0就是从0开始,,rows=5当前返回5条记录,第二页就是变化start这个值为5就可以了。 params.set("start", 0); params.set("rows", Integer.MAX_VALUE); // 排序,,如果按照id 排序,,那么将score desc 改成 id desc(or asc) params.set("sort", "score desc"); // 返回信息 * 为全部 这里是全部加上score,如果不加下面就不能使用score params.set("fl", "*,score"); try { QueryResponse response = server.query(params); SolrDocumentList list = response.getResults(); for (int i = 0; i < list.size(); i++) { fail(list.get(i)); } } catch (SolrServerException e) { e.printStackTrace(); } }
10、其他和Server有关方法
/** * <b>function:</b> 其他server相关方法测试 * @author hoojo * @createDate 2011-10-21 上午10:02:03 */ @Test public void otherMethod() { fail(server.getBinder()); try { fail(server.optimize());//合并索引文件,可以优化索引、提供性能,但需要一定的时间 fail(server.ping());//ping服务器是否连接成功 Index index = new Index(); index.setId("299"); index.setName("add bean index199"); index.setManu("index bean manu199"); index.setCat(new String[] { "a199", "b199" }); UpdateResponse response = server.addBean(index); fail("response: " + response); queryAll(); //回滚掉之前的操作,rollback addBean operation fail("rollback: " + server.rollback()); //提交操作,提交后无法回滚之前操作;发现addBean没有成功添加索引 fail("commit: " + server.commit()); queryAll(); } catch (SolrServerException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } }
11、文档查询
/** * <b>function:</b> query 基本用法测试 * @author hoojo * @createDate 2011-10-20 下午04:44:28 */ @Test public void queryCase() { //AND 并且 SolrQuery params = new SolrQuery("name:apple AND manu:inc"); //OR 或者 params.setQuery("name:apple OR manu:apache"); //空格 等同于 OR params.setQuery("name:server manu:dell"); //params.setQuery("name:solr - manu:inc"); //params.setQuery("name:server + manu:dell"); //查询name包含solr apple params.setQuery("name:solr,apple"); //manu不包含inc params.setQuery("name:solr,apple NOT manu:inc"); //50 <= price <= 200 params.setQuery("price:[50 TO 200]"); params.setQuery("popularity:[5 TO 6]"); //params.setQuery("price:[50 TO 200] - popularity:[5 TO 6]"); //params.setQuery("price:[50 TO 200] + popularity:[5 TO 6]"); //50 <= price <= 200 AND 5 <= popularity <= 6 params.setQuery("price:[50 TO 200] AND popularity:[5 TO 6]"); params.setQuery("price:[50 TO 200] OR popularity:[5 TO 6]"); //过滤器查询,可以提高性能 filter 类似多个条件组合,如and //params.addFilterQuery("id:VA902B"); //params.addFilterQuery("price:[50 TO 200]"); //params.addFilterQuery("popularity:[* TO 5]"); //params.addFilterQuery("weight:*"); //0 < popularity < 6 没有等于 //params.addFilterQuery("popularity:{0 TO 6}"); //排序 params.addSortField("id", ORDER.asc); //分页:start开始页,rows每页显示记录条数 //params.add("start", "0"); //params.add("rows", "200"); //params.setStart(0); //params.setRows(200); //设置高亮 params.setHighlight(true); // 开启高亮组件 params.addHighlightField("name");// 高亮字段 params.setHighlightSimplePre("<font color='red'>");//标记,高亮关键字前缀 params.setHighlightSimplePost("</font>");//后缀 params.setHighlightSnippets(1);//结果分片数,默认为1 params.setHighlightFragsize(1000);//每个分片的最大长度,默认为100 //分片信息 params.setFacet(true) .setFacetMinCount(1) .setFacetLimit(5)//段 .addFacetField("name")//分片字段 .addFacetField("inStock"); //params.setQueryType(""); try { QueryResponse response = server.query(params); /*List<Index> indexs = response.getBeans(Index.class); for (int i = 0; i < indexs.size(); i++) { fail(indexs.get(i)); }*/ //输出查询结果集 SolrDocumentList list = response.getResults(); fail("query result nums: " + list.getNumFound()); for (int i = 0; i < list.size(); i++) { fail(list.get(i)); } //输出分片信息 List<FacetField> facets = response.getFacetFields(); for (FacetField facet : facets) { fail(facet); List<Count> facetCounts = facet.getValues(); for (FacetField.Count count : facetCounts) { System.out.println(count.getName() + ": " + count.getCount()); } } } catch (SolrServerException e) { e.printStackTrace(); } }
12、分片查询、统计
/** * <b>function:</b> 分片查询, 可以统计关键字及出现的次数、或是做自动补全提示 * @author hoojo * @createDate 2011-10-20 下午04:54:25 */ @Test public void facetQueryCase() { SolrQuery params = new SolrQuery("*:*"); //排序 params.addSortField("id", ORDER.asc); params.setStart(0); params.setRows(200); //Facet为solr中的层次分类查询 //分片信息 params.setFacet(true) .setQuery("*:*") .setFacetMinCount(1) .setFacetLimit(5)//段 //.setFacetPrefix("electronics", "cat") .setFacetPrefix("cor")//查询manu、name中关键字前缀是cor的 .addFacetField("manu") .addFacetField("name");//分片字段 try { QueryResponse response = server.query(params); //输出查询结果集 SolrDocumentList list = response.getResults(); fail("Query result nums: " + list.getNumFound()); for (int i = 0; i < list.size(); i++) { fail(list.get(i)); } fail("All facet filed result: "); //输出分片信息 List<FacetField> facets = response.getFacetFields(); for (FacetField facet : facets) { fail(facet); List<Count> facetCounts = facet.getValues(); for (FacetField.Count count : facetCounts) { //关键字 - 出现次数 fail(count.getName() + ": " + count.getCount()); } } fail("Search facet [name] filed result: "); //输出分片信息 FacetField facetField = response.getFacetField("name"); List<Count> facetFields = facetField.getValues(); for (Count count : facetFields) { //关键字 - 出现次数 fail(count.getName() + ": " + count.getCount()); } } catch (SolrServerException e) { e.printStackTrace(); } }
分片查询在某些统计关键字的时候还是很有用的,可以统计关键字出现的次数,可以通过统计的关键字来搜索相关文档的信息。
13、Document文档和JavaBean相互转换
这里转换的Bean是一个简单的User对象
package com.hoo.entity; import java.io.Serializable; import org.apache.solr.client.solrj.beans.Field; /** * <b>function:</b> User Entity Bean;所有被添加Annotation @Field 注解的属性将参与index操作 * @author hoojo * @createDate 2011-10-19 下午04:16:00 * @file User.java * @package com.hoo.entity * @project SolrExample * @blog http://blog.csdn.net/IBM_hoojo * @email [email protected] * @version 1.0 */ public class User implements Serializable { /** * @author Hoojo */ private static final long serialVersionUID = 8606788203814942679L; //@Field private int id; @Field private String name; @Field private int age; /** * 可以给某个属性重命名,likes就是solr index的属性;在solrIndex中将显示like为likes */ @Field("likes") private String[] like; @Field private String address; @Field private String sex; @Field private String remark; public int getId() { return id; } //setter 方法上面也可以 @Field public void setId(int id) { this.id = id; } public String getName() { return name; } //getter、setter @Override public String toString() { return this.id + "#" + this.name + "#" + this.age + "#" + this.like + "#" + this.address + "#" + this.sex + "#" + this.remark; } }
测试类代码如下
package com.hoo.test; import org.apache.solr.client.solrj.beans.DocumentObjectBinder; import org.apache.solr.common.SolrDocument; import org.apache.solr.common.SolrDocumentList; import org.apache.solr.common.SolrInputDocument; import org.apache.solr.common.SolrInputField; import org.junit.Test; import com.hoo.entity.User; /** * <b>function:</b>SolrInputDocument implements Map, Iterable * @author hoojo * @createDate 2011-10-19 下午03:54:54 * @file SolrInputDocumentTest.java * @package com.hoo.test * @project SolrExample * @blog http://blog.csdn.net/IBM_hoojo * @email [email protected] * @version 1.0 */ public class SolrInputDocumentTest { public final void fail(Object o) { System.out.println(o); } /** * <b>function:</b> 创建SolrInputDocument * @author hoojo * @createDate 2011-10-21 下午03:38:20 */ @Test public void createDoc() { SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", System.currentTimeMillis()); doc.addField("name", "SolrInputDocument"); doc.addField("age", 22, 2.0f); doc.addField("like", new String[] { "music", "book", "sport" }); doc.put("address", new SolrInputField("guangzhou")); doc.setField("sex", "man"); doc.setField("remark", "china people", 2.0f); fail(doc); } /** * <b>function:</b> 利用DocumentObjectBinder对象将SolrInputDocument 和 User对象相互转换 * @author hoojo * @createDate 2011-10-21 下午03:38:40 */ @Test public void docAndBean4Binder() { SolrDocument doc = new SolrDocument(); doc.addField("id", 456); doc.addField("name", "SolrInputDocument"); doc.addField("likes", new String[] { "music", "book", "sport" }); doc.put("address", "guangzhou"); doc.setField("sex", "man"); doc.setField("remark", "china people"); DocumentObjectBinder binder = new DocumentObjectBinder(); User user = new User(); user.setId(222); user.setName("JavaBean"); user.setLike(new String[] { "music", "book", "sport" }); user.setAddress("guangdong"); fail(doc); // User ->> SolrInputDocument fail(binder.toSolrInputDocument(user)); // SolrDocument ->> User fail(binder.getBean(User.class, doc)); SolrDocumentList list = new SolrDocumentList(); list.add(doc); list.add(doc); //SolrDocumentList ->> List fail(binder.getBeans(User.class, list)); } /** * <b>function:</b> SolrInputDocument的相关方法 * @author hoojo * @createDate 2011-10-21 下午03:44:30 */ @Test public void docMethod() { SolrInputDocument doc = new SolrInputDocument(); doc.addField("id", System.currentTimeMillis()); doc.addField("name", "SolrInputDocument"); doc.addField("age", 23, 1.0f); doc.addField("age", 22, 2.0f); doc.addField("age", 24, 0f); fail(doc.entrySet()); fail(doc.get("age")); //排名有用,类似百度竞价排名 doc.setDocumentBoost(2.0f); fail(doc.getDocumentBoost()); fail(doc.getField("name")); fail(doc.getFieldNames());//keys fail(doc.getFieldValues("age")); fail(doc.getFieldValues("id")); fail(doc.values()); } }