2.基于memcached-session-manager的负载均衡
环境CentOS 6.5,Nginx 1.7.2 ,Memcached 1.4.20
一.导出lib
1.利用maven,构建并导出基于kryo-serializer的lib,可以将下面的依赖版本和配置如下
<properties> <memcached-session-manager.version>1.8.2</memcached-session-manager.version> <spymemcached.version>2.11.4</spymemcached.version> <kryo-serializers.version>0.11</kryo-serializers.version> <kryo.version>1.04</kryo.version> <minlog.version>1.2</minlog.version> <reflectasm.version>1.01</reflectasm.version> <asm.version>3.2</asm.version> </properties> <dependencies> <!-- memcached-session-manager --> <dependency> <groupId>de.javakaffee.msm</groupId> <artifactId>memcached-session-manager</artifactId> <version>${memcached-session-manager.version}</version> <exclusions> <exclusion> <groupId>net.spy</groupId> <artifactId>spymemcached</artifactId> </exclusion> <exclusion> <groupId>com.couchbase.client</groupId> <artifactId>couchbase-client</artifactId> </exclusion> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>jsr305</artifactId> </exclusion> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>annotations</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>de.javakaffee.msm</groupId> <artifactId>memcached-session-manager-tc7</artifactId> <version>${memcached-session-manager.version}</version> <exclusions> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>jsr305</artifactId> </exclusion> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>annotations</artifactId> </exclusion> </exclusions> </dependency> <!-- spymemcached --> <dependency> <groupId>net.spy</groupId> <artifactId>spymemcached</artifactId> <version>${spymemcached.version}</version> </dependency> <!-- kryo-serializer --> <dependency> <groupId>de.javakaffee.msm</groupId> <artifactId>msm-kryo-serializer</artifactId> <version>${memcached-session-manager.version}</version> <exclusions> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>jsr305</artifactId> </exclusion> <exclusion> <groupId>com.google.code.findbugs</groupId> <artifactId>annotations</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>de.javakaffee</groupId> <artifactId>kryo-serializers</artifactId> <version>${kryo-serializers.version}</version> </dependency> <dependency> <groupId>com.googlecode</groupId> <artifactId>kryo</artifactId> <version>${kryo.version}</version> </dependency> <dependency> <groupId>com.googlecode</groupId> <artifactId>minlog</artifactId> <version>${minlog.version}</version> </dependency> <dependency> <groupId>com.googlecode</groupId> <artifactId>reflectasm</artifactId> <version>${reflectasm.version}</version> </dependency> <dependency> <groupId>asm</groupId> <artifactId>asm</artifactId> <version>${asm.version}</version> </dependency> </dependencies>
2.执行下面的命令下载lib
mvn dependency:copy-dependencies -DoutputDirectory=./target/lib
lib就会被自动下载到/target/lib目录下,内容如下图。
二.配置tomcat
配置方式主要见https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration
1.拷贝LIB
将上面导出的lib拷贝到 $CATALINA_HOME/lib/ 目录下
2.配置$CATALINA_HOME/conf/context.xml
以下两种配置选择其中一种,推荐第二种
第一种: sticky sessions + kryo
<Context> ... <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211" failoverNodes="n1" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" /> </Context>
注意:
(1) sticky模式时,配置jvmroute参数,每台tomcat的jvmroute参数都不能一样 (测试这种配置方式不适用于多台Servlet容器,或适用于反向代理服务器可以把相同来源的请求映射到同一台容器上的情况下,不推荐;另网上看到该配置修改transcoderFactoryclass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory" 未验证)
$CATALINA_HOME/conf/server.xml
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
(2) 必须启动两个Memcached实例,比如
/usr/local/memcached/bin/memcached -d -m 256 -u root -p 11211 -c 128 -P /tmp/memcached1.pid /usr/local/memcached/bin/memcached -d -m 256 -u root -p 11212 -c 128 -P /tmp/memcached2.pid
第二种:non-sticky sessions + kryo
<Context> ... <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211" sticky="false" sessionBackupAsync="false" lockingMode="uriPattern:/path1|/path2" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" /> </Context>
3.可选配置$CATALINA_HOME/conf/logging.properties
如果需要调试日志,可以在logging.properties中添加如下内容
de.javakaffee.web.msm.level=FINE
4. 多台Tomcat配置 需要注意下面内容
1) 修改http访问端口(默认为8080端口)
2) 修改Shutdown端口(默认为8005端口)
3) 修改AJP 访问端口(默认为8009端口)
4) 修改http访问(默认 Connector port="8080"),添加 URIEncoding="UTF-8"
三.配置Nginx
修改配置文件 /usr/local/nginx/conf/nginx.conf
下面是Nginx 1.7.2完整事例
#user nobody; user gqshao; worker_processes 1; #error_log logs/error.log; #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; events { // windows系统下不要 use epoll use epoll; worker_connections 1024; } http { include mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log logs/access.log main; sendfile on; #tcp_nopush on; #keepalive_timeout 0; keepalive_timeout 65; #gzip on; gzip on; gzip_types text/javascript text/plain text/css application/xml application/x-javascript; gzip_disable "MSIE [1-6]."; # 配置tomcat服务器 upstream web_server { #ip_hash; server 192.168.158.131:8080 max_fails=3 fail_timeout=30s; server 192.168.158.131:8081 max_fails=3 fail_timeout=30s; server 192.168.158.131:8082 max_fails=3 fail_timeout=30s; } server { # 侦听192.168.158.131的80端口 listen 80; server_name 192.168.158.131; #charset koi8-r; charset utf-8; access_log logs/host.access.log main; location / { proxy_pass http://web_server ; proxy_redirect off ; #后端的Web服务器可以通过X-Forwarded-For获取用户真实IP proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } #error_page 404 /404.html; # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
四.启动和停止
启动
# 启动 memcached sudo /usr/local/memcached/bin/memcached -d -m 256 -u gqshao -p 11211 -c 128 -P /tmp/memcached1.pid sudo /usr/local/memcached/bin/memcached -d -m 256 -u gqshao -p 11212 -c 128 -P /tmp/memcached2.pid # 启动 Tomcat ./startup.sh # 启动 Nginx sudo /usr/local/nginx/sbin/nginx