lb-01：192.168.33.135 nginx-lb centos7 rs-01：192.168.33.131 apache-web centos6.x rs-02：192.168.33.132 nginx-web centos6.x
lb-01 安装nginx,配置nginx源
# cat /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0 enabled=1
yum安装
[root@lb-01 ~]# yum install nginx -y
启动
[root@lb-01 ~]# systemctl start nginx.service
测试
[root@lb-01 ~]# curl -I localhost HTTP/1.1 200 OK Server: nginx/1.10.1 Date: Tue, 28 Jun 2016 21:17:48 GMT Content-Type: text/html Content-Length: 612 Last-Modified: Tue, 31 May 2016 14:09:55 GMT Connection: keep-alive ETag: "574d9b33-264" Accept-Ranges: bytes
其他rs机器rs-01安装apache使用默认yum安装即可，rs-02安装nginx
rs-01安装
# yum install httpd -y
rs-02安装
# yum install nginx -y
rs-01 web访问结果
[root@rs-01 ~]# cat /var/www/html/index.html <h1>this is rs-01 ip 192.168.33.131 </h1> [root@rs-01 ~]# curl localhost <h1>this is rs-01 ip 192.168.33.131 </h1>
rs-02 web访问结果
[root@rs-02 ~]# cat /usr/share/nginx/html/index.html <h1>this is rs-02 ip 192.168.33.132 </h1> [root@rs-02 ~]# curl localhost <h1>this is rs-02 ip 192.168.33.132 </h1>
[root@lb-01 ~]# cat /etc/nginx/conf.d/upstream.conf
upstream blog {
server 192.168.33.131:80 weight=3;
server 192.168.33.132:80 weight=3;
server 192.168.33.133:81 weight=3;
}
server {
listen 80;
server_name blog.zxl.com;
location / {
proxy_pass http://blog;
}
}upsteam池中端口不写也没关系默认是80，如果是别的端口可以填写相关端口即可。upsteam必须在http{}标签内，weight权重。upstream.conf配置文件放在include /etc/nginx/conf.d/*.conf目录下，官方地址http://nginx.org/en/docs/http/ngx_http_upstream_module.html#upstream
upstream模块常用的指令有：
ip_hash：基于客户端IP地址完成请求的分发，它可以保证来自于同一个客户端的请求始终被转发至同一个upstream服务器； keepalive：每个worker进程为发送到upstream服务器的连接所缓存的个数； least_conn：最少连接调度算法； server：定义一个upstream服务器的地址，还可包括一系列可选参数，如： weight：权重；默认1 max_fails：最大失败连接次数，失败连接的超时时长由fail_timeout指定；默认1,0则为禁止失败尝试 fail_timeout：等待请求的目标服务器发送响应的时长；默认10s backup：用于fallback的目的，所有服务均故障时才启动此服务器； down：手动标记其不再处理任何请求；
[root@lb-01 ~]# tail -n 1 /etc/hosts 192.168.33.135 blog.zxl.com
[root@lb-01 ~]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@lb-01 ~]# nginx -s reload
[root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-02 ip 192.168.33.132 </h1> [root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-01 ip 192.168.33.131 </h1> [root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-02 ip 192.168.33.132 </h1> [root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-01 ip 192.168.33.131 </h1>
注：为什么没有访问33.131:81服务，因为我没有这台机器假如有的话，没有81端口。nginx本身自带健康检查，有问题的话会自带剔除
[root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-01 ip 192.168.33.131 </h1> [root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-01 ip 192.168.33.131 </h1> [root@lb-01 ~]# curl http://blog.zxl.com <h1>this is rs-01 ip 192.168.33.131 </h1>
注：可以看到访问一直都是后端rs-01服务
注意：如果upstream中使用ip_hash算法，那么不能使用weight和backup参数
nginx核心模块 http://nginx.org/en/docs/http/ngx_http_core_module.html
location [ = | ~ | ~* | ^~ ] uri { ... }
location @name { ... }
功能：允许根据用户请求的URI来匹配指定的各location以进行访问配置；匹配到时，将被location块中的配置所处理；比如：http://www.zxl.com/images/logo.gif
=：精确匹配；
~：正则表达式模式匹配，匹配时区分字符大小写
~*：正则表达式模式匹配，匹配时忽略字符大小写
^~: URI前半部分匹配，不检查正则表达式$uri: 当前请求的uri，不带参数； $request_uri: 请求的uri，带完整参数； $host: http请求报文中host首部；如果请求中没有host首部，则以处理此请求的虚拟主机的主机名代替； $hostname: nginx服务运行在的主机的主机名； $remote_addr: 客户端IP $remote_port: 客户端Port $remote_user: 使用用户认证时客户端用户输入的用户名； $request_filename: 用户请求中的URI经过本地root或alias转换后映射的本地的文件路径； $request_method: 请求方法 $server_addr: 服务器地址 $server_name: 服务器名称 $server_port: 服务器端口 $server_protocol: 服务器向客户端发送响应时的协议，如http/1.1, http/1.0 $scheme: 在请求中使用scheme, 如https://www.magedu.com/中的https； $http_HEADER: 匹配请求报文中指定的HEADER，$http_host匹配请求报文中的host首部 $sent_http_HEADER: 匹配响应报文中指定的HEADER，例如$http_content_type匹配响应报文中的content-type首部； $document_root：当前请求映射到的root配置；
分别配置三个虚拟主机域名为 www.zxl.com bbs.zxl.com blog.zxl.com
[root@rs-01 conf]# tail -n 18 httpd.conf <VirtualHost *:80> DocumentRoot /var/www/www ServerName www.zxl.com ErrorLog logs/www.zxl.com-error_log CustomLog logs/www.zxl.com-access_log common </VirtualHost> <VirtualHost *:80> DocumentRoot /var/www/bbs ServerName bbs.zxl.com ErrorLog logs/bbs.zxl.com-error_log CustomLog logs/bbs.zxl.com-access_log common </VirtualHost> <VirtualHost *:80> DocumentRoot /var/www/blog ServerName blog.zxl.com ErrorLog logs/blog.zxl.com-error_log CustomLog logs/blog.zxl.com-access_log common </VirtualHost>
注：apache配置虚拟主机时把#NameVirtualHost :80注释需要打开*
创建虚拟主机站点目录
[root@rs-01 httpd]# mkdir /var/www/{www,bbs,blog} -pv
mkdir: created directory `/var/www/www‘
mkdir: created directory `/var/www/bbs‘
mkdir: created directory `/var/www/blog‘rs-01节点配置hosts文件
[root@rs-01 ~]# tail -n 1 /etc/hosts 192.168.33.131 www.zxl.com 192.168.33.131 blog.zxl.com 192.168.33.131 bbs.zxl.com
站点文件内容
[root@rs-01 ~]# cat /var/www/www/index.html apache www 131 [root@rs-01 ~]# cat /var/www/bbs/index.html apache bbs 131 [root@rs-01 ~]# cat /var/www/blog/index.html apache blog 131
使用curl访问测试
[root@rs-01 ~]# curl www.zxl.com apache www 131 [root@rs-01 ~]# curl blog.zxl.com apache blog 131 [root@rs-01 ~]# curl bbs.zxl.com apache bbs 131
rs-02节点wwww虚拟主机配置文件如下
[root@rs-02 conf.d]# cat www.conf
server {
listen 80;
server_name www.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/www;
}
}rs-02节点bbs虚拟主机配置文件如下
[root@rs-02 conf.d]# cat bbs.conf
server {
listen 80;
server_name bbs.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/bbs;
}
}rs-02节点blog虚拟主机配置文件如下
[root@rs-02 conf.d]# cat blog.conf
server {
listen 80;
server_name blog.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/blog;
}
}检查rs-02节点配置文件语法以及重新加载
[root@rs-02 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@rs-02 conf.d]# nginx -s reload
创建站点配置文件
[root@rs-02 conf.d]# mkdir /usr/share/nginx/{www,bbs,blog} -pv
mkdir: created directory `/usr/share/nginx/www‘
mkdir: created directory `/usr/share/nginx/bbs‘
mkdir: created directory `/usr/share/nginx/blog‘创建站点内容
[root@rs-02 ~]# cat /usr/share/nginx/www/index.html nginx www 132 [root@rs-02 ~]# cat /usr/share/nginx/bbs/index.html nginx bbs 132 [root@rs-02 ~]# cat /usr/share/nginx/blog/index.html nginx blog 132
rs-02节点配置hosts文件
[root@rs-02 conf.d]# tail -n 1 /etc/hosts 192.168.33.132 www.zxl.com 192.168.33.132 blog.zxl.com 192.168.33.132 bbs.zxl.com
rs-02节点进行测试访问
[root@rs-02 ~]# curl www.zxl.com nginx www 132 [root@rs-02 ~]# curl bbs.zxl.com nginx bbs 132 [root@rs-02 ~]# curl blog.zxl.com nginx blog 132
配置hosts文件
[root@lb-01 ~]# tail -n 3 /etc/hosts 192.168.33.135 www.zxl.com 192.168.33.135 blog.zxl.com 192.168.33.135 bbs.zxl.com
访问blog.zxl.com
[root@lb-01 conf.d]# curl blog.zxl.com apache www 131 [root@lb-01 conf.d]# curl blog.zxl.com nginx blog 132
注：为什么会出现和预期的不一样呢？因为lb根本不知道该去找个节点，因为后端都是80端口。lb需要进行参数设置
proxy_set_header Host $host ；获取后端head的信息，代理那个后端主机，完整示例如下
[root@lb-01 conf.d]# cat upstream.conf
upstream blog {
server 192.168.33.131:80 weight=3;
server 192.168.33.132:80 weight=3;
}
server {
listen 80;
server_name blog.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
}
}其实就是在location中添加了proxy_set_header Host $host;
检查lb上nginx语法以及重新加载并测试
[root@lb-01 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@lb-01 conf.d]# nginx -s reload
结果就是可以rr权重了
[root@lb-01 conf.d]# curl blog.zxl.com apache blog 131 [root@lb-01 conf.d]# curl blog.zxl.com nginx blog 132
bbs和www和blog类似，完整示例如下
[root@lb-01 conf.d]# cat upstream.conf
upstream blog {
server 192.168.33.131:80 weight=3;
server 192.168.33.132:80 weight=3;
}
server {
listen 80;
server_name blog.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
}
}
server {
listen 80;
server_name bbs.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
}
}
server {
listen 80;
server_name www.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
}
}注：bbs和www以及blog使用相同的upstream池，因为后端节点都是一样的
检查语法以及测试
[root@lb-01 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@lb-01 conf.d]# nginx -s reload
可以看到每个站点都是rr轮询访问的
[root@lb-01 conf.d]# curl www.zxl.com apache www 131 [root@lb-01 conf.d]# curl www.zxl.com nginx www 132 [root@lb-01 conf.d]# curl bbs.zxl.com apache bbs 131 [root@lb-01 conf.d]# curl bbs.zxl.com nginx bbs 132
后端节点如何查看来自客户端访问的真实ip地址？看的话只能从日志看了，所以设置好后端节点的日志，apache上面定义虚拟主机的时候已经配置好了，下面配置nginx节点的日志，示例如下
rs-02后端节点bbs虚拟主机log日志
[root@rs-02 conf.d]# cat bbs.conf
server {
listen 80;
server_name bbs.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/bbs;
access_log logs/bbs.access.log main;
}
}rs-02后端节点blog虚拟主机log日志
[root@rs-02 conf.d]# cat blog.conf
server {
listen 80;
server_name blog.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/blog;
access_log logs/blog.access.log main;
}
}rs-02后端节点www虚拟主机log日志
[root@rs-02 conf.d]# cat www.conf
server {
listen 80;
server_name www.zxl.com;
location / {
index index.html index.htm;
root /usr/share/nginx/www;
access_log logs/www.access.log main;
}
}注：access__log添加字段是记录log日志的
创建rs-02节点虚拟主机记录log日志的目录
[root@rs-02 conf.d]# mkdir /usr/share/nginx/logs
检查语法以及重新加载
[root@rs-02 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@rs-02 conf.d]# nginx -s reload
在自己电脑上设置hosts文件绑定域名进行访问，然后查看后端节点日志看看是来自哪个ip地址？C:\Windows\System32\drivers\etc\hosts
192.168.33.135 www.zxl.com 192.168.33.135 blog.zxl.com 192.168.33.135 bbs.zxl.com
rs-02节点日志
[root@rs-02 logs]# tail bbs.access.log 192.168.33.135 - - [27/Jun/2016:09:53:07 +0800] "GET / HTTP/1.0" 200 14 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" 192.168.33.135 - - [27/Jun/2016:09:53:10 +0800] "GET / HTTP/1.0" 200 14 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" [root@rs-02 logs]# tail blog.access.log 192.168.33.135 - - [27/Jun/2016:10:02:39 +0800] "GET / HTTP/1.0" 200 15 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" 192.168.33.135 - - [27/Jun/2016:10:02:56 +0800] "GET / HTTP/1.0" 304 0 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" [root@rs-02 logs]# tail www.access.log 192.168.33.135 - - [27/Jun/2016:10:03:33 +0800] "GET / HTTP/1.0" 200 14 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-"
注：可以看到上面ip地址来自lb负载均衡器
rs-01节点日志
[root@rs-01 httpd]# tail www.zxl.com-access_log 192.168.33.135 - - [17/Jun/2016:07:22:34 +0800] "GET / HTTP/1.0" 200 15 [root@rs-01 httpd]# tail bbs.zxl.com-access_log 192.168.33.135 - - [17/Jun/2016:07:23:00 +0800] "GET / HTTP/1.0" 304 - [root@rs-01 httpd]# tail blog.zxl.com-access_log 192.168.33.135 - - [17/Jun/2016:07:22:53 +0800] "GET / HTTP/1.0" 200 16
注：上面ip地址来自lb负载均衡器
后端节点如何才能记录来自真实的ip地址访问呢？
其实后端节点nginx已经准备接收参数默认已经开启了，http{}标签中，log_format main ....$http_x_forwarded_for,$http_x_forwarded_for记录客户端的真实ip地址
那么apache呢？修改示例如下
[root@rs-01 conf]# sed -n ‘498p‘ httpd.conf
LogFormat "\"%{X-Forwarded-For}i\" %l %u %t \"%r\" %>s %b" common修改配置文件后重新启动服务
[root@rs-01 conf]# /etc/init.d/httpd restart Stopping httpd: [ OK ] Starting httpd: httpd: apr_sockaddr_info_get() failed for rs-01 httpd: Could not reliably determine the server‘s fully qualified domain name, using 127.0.0.1 for ServerName
完成示例如下
[root@lb-01 conf.d]# cat upstream.conf
upstream blog {
server 192.168.33.131:80 weight=3;
server 192.168.33.132:80 weight=3;
}
server {
listen 80;
server_name blog.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 80;
server_name bbs.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
server {
listen 80;
server_name www.zxl.com;
location / {
proxy_pass http://blog;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}注：上面server段标签中的proxy_set_header X-Forwarded-For $remote_addr;与proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;功能都是一样获取来自真实的客户端访问的ip地址
检查lb的nginx语法以及重新加载服务
[root@lb-01 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@lb-01 conf.d]# nginx -s reload
打开浏览器访问bbs.zxl.com 
查看rs-01的bbs站点日志访问
[root@rs-01 ~]# tail -n 2 /var/log/httpd/bbs.zxl.com-access_log "192.168.33.1" - - [17/Jun/2016:20:07:35 +0800] "GET / HTTP/1.0" 200 15 "192.168.33.1" - - [17/Jun/2016:20:07:35 +0800] "GET / HTTP/1.0" 200 15
注：可以看到已经记录了来自客户端访问的真实ip地址了
查看rs-02的bbs站点日志访问
[root@rs-02 ~]# tail -n 2 /usr/share/nginx/logs/bbs.access.log 192.168.33.135 - - [27/Jun/2016:22:57:04 +0800] "GET / HTTP/1.0" 200 14 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "192.168.33.1" 192.168.33.135 - - [27/Jun/2016:22:57:40 +0800] "GET / HTTP/1.0" 200 14 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "192.168.33.1
注：可以看到已经记录了来自客户端访问的真实ip地址了
其实lb负载均衡器配置文件中的location中可以把一些n多参数使用配置文件引用即可，就拿blog站点示例如下
[root@lb-01 conf.d]# cat upstream.conf
upstream blog {
server 192.168.33.131:80 weight=3;
server 192.168.33.132:80 weight=3;
}
server {
listen 80;
server_name blog.zxl.com;
location / {
proxy_pass http://blog;
#proxy_set_header Host $host;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
include /etc/nginx/conf.d/a.conf;
}
}a.conf文件内容如下
[root@lb-01 conf.d]# cat a.conf proxy_redirect off; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; client_body_buffer_size 128k; proxy_connect_timeout 30; proxy_send_timeout 15; proxy_read_timeout 15;
使用www站点来做说明
lb-nginx配置如下
[root@lb-01 conf.d]# cat upstream.conf
upstream static_pools {
server 192.168.33.131:80 weight=3;
}
upstream dynamic_pools {
server 192.168.33.132:80 weight=3;
}
server {
listen 80;
server_name www.zxl.com;
location /static/ {
proxy_pass http://static_pools;
include /etc/nginx/conf.d/a.conf;
}
location /dynamic/ {
proxy_pass http://dynamic_pools;
include /etc/nginx/conf.d/a.conf;
}
}include引用的a.conf配置文件
检查lb-nginx配置文件语法以及重新加载
[root@lb-01 conf.d]# nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful [root@lb-01 conf.d]# nginx -s reload
rs-02节点配置如下
[root@rs-02 nginx]# pwd /usr/share/nginx [root@rs-02 nginx]# ll total 20 drwxr-xr-x. 2 root root 4096 Jun 27 09:26 bbs drwxr-xr-x. 2 root root 4096 Jun 27 09:26 blog drwxr-xr-x. 2 root root 4096 Jun 27 09:18 html drwxr-xr-x. 2 root root 4096 Jun 27 09:56 logs drwxr-xr-x. 2 root root 4096 Jun 27 09:26 www [root@rs-02 nginx]# mkdir www/dynamic -p [root@rs-02 nginx]# echo dynamic > www/dynamic/index.html
rs-02节点测试访问www.zxl.com
[root@rs-02 nginx]# curl www.zxl.com nginx www 132 [root@rs-02 nginx]# curl www.zxl.com/dynamic/index.html dynamic [root@rs-02 nginx]# curl www.zxl.com/static/index.html <html> <head><title>404 Not Found</title></head> <body bgcolor="white"> <center><h1>404 Not Found</h1></center> <hr><center>nginx/1.0.15</center> </body> </html>
同理得rs-01节点也是如此
rs-01节点配置如下
[root@rs-01 ~]# mkdir /var/www/www/static/ [root@rs-01 ~]# echo static >/var/www/www/static/index.html
rs-01节点测试访问结果如下
[root@rs-01 ~]# curl www.zxl.com/static/index.html static [root@rs-01 ~]# curl www.zxl.com/dynamic/index.html <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Found</h1> <p>The requested URL /dynamic/index.html was not found on this server.</p> <hr> <address>Apache/2.2.15 (CentOS) Server at www.zxl.com Port 80</address> </body></html>
以上访问就已经实现动态分离了根据lb-nginx的字符串匹配，使用浏览器访问结果也是一样的
lb-nginx配置文件内容如下
server {
listen 80;
server_name www.zxl.com;
location ~ .*. (gif|jpg|jpeg|png|bmp|swf|css|js)$ {
proxy_pass http://static_pools;
include a.conf;
}
location ~ .*. (php|php5)$ {
proxy_pass http://dynamic_pools;
include a.conf;
}
}本文出自 “村里的男孩” 博客，请务必保留此出处http://noodle.blog.51cto.com/2925423/1794729
原文地址：http://noodle.blog.51cto.com/2925423/1794729
