kvm管理平台webvirtmgr的部署

git地址：https://github.com/retspen/webvirtmgr
环境：
system:centos6.2
libvirt: 0.10.2
webvirtmgr的安装，参考：https://github.com/retspen/webvirtmgr/wiki/Install-WebVirtMgr
1.安装虚拟化环境

yum -y groupinstall ‘Virtualization Tools‘
yum -y groupinstall ‘Virtualization‘
yum -y groupinstall ‘Virtulization Client‘
yum -y groupinstall ‘Virtulization Platform‘

2.安装依赖包

$ sudo yum -y install http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
$ sudo yum -y install git python-pip libvirt-python libxml2-python python-websockify supervisor nginx

3.克隆代码

$ git clone git://github.com/retspen/webvirtmgr.git
$ cd webvirtmgr
$ sudo pip install -r requirements.txt # or python-pip (RedHat, Fedora, CentOS, OpenSuse)

用pip安装django不太顺利，报错：
[root@vm03 ~]# pip install Django==1.5.5
Collecting Django==1.5.5
Could not find a version that satisfies the requirement Django==1.5.5 (from versions: 1.7plus-email-as-username-1.7.0, 4store-0.3, 503-0.1)
No matching distribution found for Django==1.5.5
折腾许久，只能去django官网下载了个Django-1.5.12.tar.gz包来安装

4.Django环境初始化
执行syncdb时，报错，未安装mysqldb
于是下载MySQL-python-1.2.5.zip，解压安装，又是一个报错：command ‘gcc’ failed with exit status 1
原因是未安装：mysql-devel，python-devel，mysql-python

$ ./manage.py syncdb
$ ./manage.py collectstatic

5.配置nginx
vim /etc/nginx/conf.d/webvirtmgr.conf 内容如下：

server {
    listen 80 default_server;

    server_name $hostname;
    #access_log /var/log/nginx/webvirtmgr_access_log; 

    location /static/ {
        root /var/www/webvirtmgr/webvirtmgr; # or /srv instead of /var
        expires max;
    }

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 600;
        proxy_read_timeout 600;
        proxy_send_timeout 600;
        client_max_body_size 1024M; # Set higher depending on your needs 
    }
}

注释掉nginx的default内容，vim /etc/nginx/conf.d/default.conf 。然后启动nginx

 sudo service nginx restart

6.启动django

./manage.py runserver 0.0.0.0:8000

访问：http://x.x.x.x:8000 (x.x.x.x - your server IP address )，输入创建的django用户和密码，如果没有创建，请用python manager.py createsuperuser,命令创建。登录后如下图所示

webvirtmgr客户端就这样搭建完了，接下来需要配置虚拟机所在宿主机的，参考git地址：https://github.com/retspen/webvirtmgr/wiki/Setup-Host-Server

配置宿主机环境：
system: centos 6.6
libvirt: 0.10.2
1.下载并执行脚本：libvirt-bootstrap.sh
如果虚拟机比较多，该脚本执行时间会比较长，因为会执行service libvirt-guests restart，会将所有运行的虚拟机挂起然后再恢复，感觉这一步不是必须的，因为我有一台只设置ssh认证，也可以正常连接。

curl http://retspen.github.io/libvirt-bootstrap.sh | sudo sh

2.设置防火墙：

 sudo iptables -I INPUT -m state --state NEW -m tcp -p tcp --dport 16509 -j ACCEPT

3.设置TCP认证Setup TCP authorization

参考：https://github.com/retspen/webvirtmgr/wiki/Setup-TCP-authorization，webvirtmgr新建服务器连接时需要此账号

$ sudo saslpasswd2 -a libvirt fred
Password: xxxxxx
Again (for verification): xxxxxx

查看新加的账号

$ sudo sasldblistusers2 -f /etc/libvirt/passwd.db
fred@webvirtmgr.net: userPassword

删除新加的账号：

sudo saslpasswd2 -a libvirt -d fred

验证新加的账号：

virsh -c qemu+tcp://IP_address/system nodeinfo
Please enter your authentication name: fred@webvirtmgr.net
Please enter your password: xxxxxx
CPU model:           x86_64
CPU(s):              2
CPU frequency:       2611 MHz
CPU socket(s):       1
Core(s) per socket:  2
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:         2019260 kB

注意：账号全名带hostname，如fred@webvirtmgr.net

4.设置ssh认证
ssh和tcp设置一种即可，其实就是设置无密码登录，要注意的是从webvirtmgr的什么用户到宿主机的什么用户的无密码登录，比如我用root跑的django webvirtmgr，而宿主机也是root跑的virsh，所以需要设置root到root的无密码登录。而git官网推荐的是用nginx用户跑django webvirtmgr，webvirtmgr用户跑的virsh，所以设置的是nginx用户到宿主机webvirtmgr用户的无密码登录。
参考：https://github.com/retspen/webvirtmgr/wiki/Setup-SSH-Authorization

5.使用tcp认证连接服务器
访问：http://xxxx:8000，xxxx是webvirtmgr的ip地址，点击new connection

使用TCP连接，label填写ip地址，FQDN/IP 填写ip地址，用户名填写第三步设置tcp认证所增加的账号，这里的用户名不用加@hostname，点击添加完成。如下图：

6.使用ssh认证连接服务器
访问访问：http://xxxx:8000，xxxx是webvirtmgr的ip地址，点击new connection，使用ssh认证涉及到一个问题，比如我用账号webvirtmgr登录了宿主机，但是使用命令virsh list的时候，并不能列出虚拟机信息，只有root用户使用virsh list时才能查看信息。于是我设置了nginx到宿主机root的ssh认证，但是在使用webvirtmgr网页登录时，发现并不能登录，需要手动输入，而且手动输入后发现，是可用的。

后来找到原因了，因为我用root跑的webvirtmgr，所以需要设置webvirtmgr服务器的root用户宿主机的root用户的无密码登录，设置后，可以使用。

7.虚拟机控制台使用
点击虚机实例，每台虚机最后的按钮”控制台”，点击后发现报错，如下图所示

查看django后台的错误为：
ERROR:django.request:Internal Server Error: /console/
Traceback (most recent call last):
File “/usr/lib/python2.6/site-packages/django/core/handlers/base.py”, line 113, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File “/data/apps/webvirtmgr/console/views.py”, line 58, in console
response.set_cookie(‘token’, token)
AttributeError: ‘str’ object has no attribute ‘set_cookie’
查找其源代码，发现可能的错误是由于虚机没有安装vnc等图形管理工具，于是在虚机的xml配置文件添加如下配置：
virsh edit oss.webvirtmgr.001

<graphics type=‘vnc‘ port=‘5910‘ autoport=‘no‘ listen=‘0.0.0.0‘>
      <listen type=‘address‘ address=‘0.0.0.0‘/>
</graphics>

再重启虚机，如果还是不行，就先undefine，然后重新根据xml配置create试试。下载个VNC-Viewer，输入IP：5910，看能否访问，如下图所示：说明设置VNC是成功

但是从网页点击控制台按钮，发现还是不行，报错是server disconnected

访问时django后台只有如下信息：
[26/Apr/2016 05:59:56] “GET /console/?token=4-bcc6ca3b-ca6d-6e51-505a-9c86cf58a6b2 HTTP/1.1” 200 7270

参考这篇：http://ju.outofmemory.cn/entry/145497
安装了novnc，仍然报同样的错，百度谷歌也找不到相应的信息。yum -y install novnc后，无意间在服务器上发现了这个命令novnc-server

于是敲了如下命令：novnc_server –vnc IPADDR:5920，其中IPADDR为宿主机ip地址，5920是定义的虚拟机的vnc端口号。

[root@oss-webvirtmgr-001 ~]# novnc_server –vnc 192.168.200.230:5920
Warning: could not find self.pem
Starting webserver and WebSockets proxy on port 6080
WebSocket server settings:
- Listen on :6080
- Flash security policy server
- Web server. Web root: /usr/share/novnc
- No SSL/TLS support (no cert file)
- proxying from :6080 to 192.168.200.230:5920

Navigate to this URL:

http://oss-webvirtmgr-001:6080/vnc.html?host=oss-webvirtmgr-001&port=6080

Press Ctrl-C to exit

1: 10.1.196.211: Plain non-SSL (ws://) WebSocket connection
1: 10.1.196.211: Version hybi-13, base64: ‘False’
1: connecting to: 192.168.200.230:5920

结果居然ok了，console控制台起来了

问题是：假如我有30台虚拟机，每个虚拟机都有一个vnc端口，我是不是得写30条这样的命令，监控每一个端口？？

后来发现是由于有一个脚本没起，才会导致这种情况，起了这个脚本后，所有控制台应该都能正常访问。pyhton脚本为
webvirtmgr/console/webvirtmgr-console
有兴趣的可以研究下脚本内容：

#!/usr/bin/env python

import os
import sys
import logging

DIR_PATH = os.path.dirname(os.path.abspath(__file__))
ROOT_PATH = os.path.abspath(os.path.join(DIR_PATH, ‘..‘, ‘‘))
# VENV_PATH = ROOT_PATH + ‘/venv/lib/python2.7/site-packages‘
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webvirtmgr.settings")
CERT = DIR_PATH + ‘/cert.pem‘

if ROOT_PATH not in sys.path:
    sys.path.append(ROOT_PATH)
# if VENV_PATH not in sys.path:
# sys.path.append(VENV_PATH)

import Cookie
import socket
from webvirtmgr.settings import WS_PORT, WS_HOST, WS_CERT

from vrtManager.connection import CONN_SSH, CONN_SOCKET

import re

from tunnel import Tunnel

from optparse import OptionParser
parser = OptionParser()

parser.add_option("-v",
                  "--verbose",
                  dest="verbose",
                  action="store_true",
                  help="Verbose mode",
                  default=False)

parser.add_option("-d",
                  "--debug",
                  dest="debug",
                  action="store_true",
                  help="Debug mode",
                  default=False)

parser.add_option("-H",
                  "--host",
                  dest="host",
                  action="store",
                  help="Listen host",
                  default=WS_HOST)

parser.add_option("-p",
                  "--port",
                  dest="port",
                  action="store",
                  help="Listen port",
                  default=WS_PORT or 6080)

parser.add_option("-c",
                  "--cert",
                  dest="cert",
                  action="store",
                  help="Certificate file path",
                  default=WS_CERT or CERT)

(options, args) = parser.parse_args()

FORMAT="%(asctime)s - %(name)s - %(levelname)s : %(message)s"
if options.debug:
    logging.basicConfig(level=logging.DEBUG,format=FORMAT)
    options.verbose=True
elif options.verbose:
    logging.basicConfig(level=logging.INFO,format=FORMAT)
else:
    logging.basicConfig(level=logging.WARNING,format=FORMAT)

try:
    from websockify import WebSocketProxy

    try:
        from websockify import ProxyRequestHandler
    except ImportError:
        USE_HANDLER = False
    else:
        USE_HANDLER = True
except ImportError:
    try:
        from novnc.wsproxy import WebSocketProxy
    except ImportError:
        print(‘Unable to import a websockify implementation, ‘ +
              ‘please install one‘)
        sys.exit(1)
    else:
        USE_HANDLER = False


def get_connection_infos(token):
    from instance.models import Instance
    from vrtManager.instance import wvmInstance

    try:
        temptoken = token.split(‘-‘, 1)
        host = int(temptoken[0])
        uuid = temptoken[1]
        instance = Instance.objects.get(compute_id=host, uuid=uuid)
        conn = wvmInstance(instance.compute.hostname,
                           instance.compute.login,
                           instance.compute.password,
                           instance.compute.type,
                           instance.name)
        if instance.compute.hostname.count(‘:‘):
            connhost = instance.compute.hostname.split(‘:‘)[0]
            connport = instance.compute.hostname.split(‘:‘)[1]
        else:
            connhost = instance.compute.hostname
            connport = 22
        connuser = instance.compute.login
        conntype = instance.compute.type
        console_host = conn.get_console_listen_addr()
        console_port = conn.get_console_port()
        console_socket = conn.get_console_socket()
    except Exception,e:
        logging.error(‘Fail to retrieve console connexion infos for token %s : %s‘ % (token,e))
        raise
    return (connhost, connport, connuser, conntype, console_host,
            console_port, console_socket)


class CompatibilityMixIn(object):
    def _new_client(self, daemon, socket_factory):
        cookie = Cookie.SimpleCookie()
        cookie.load(self.headers.getheader(‘cookie‘))
        if ‘token‘ not in cookie:
            self.msg(‘No token cookie found !‘)
            return False
        token = cookie[‘token‘].value

        (connhost, connport, connuser, conntype, console_host, console_port,
         console_socket) = get_connection_infos(token)

        cnx_debug_msg = "Connexion infos :\n"
        cnx_debug_msg += "- connhost : ‘%s‘\n" % connhost 
        cnx_debug_msg += "- connport : ‘%s‘\n" % connport 
        cnx_debug_msg += "- connuser : ‘%s‘\n" % connuser 
        cnx_debug_msg += "- conntype : ‘%s‘\n" % conntype 
        cnx_debug_msg += "- console_host : ‘%s‘\n" % console_host 
        cnx_debug_msg += "- console_port : ‘%s‘\n" % console_port 
        cnx_debug_msg += "- console_socket : ‘%s‘\n" % console_socket 
        logging.debug(cnx_debug_msg)

        if console_socket and conntype == CONN_SOCKET:
            # Local socket on local host
            self.msg(‘Try to open local socket %s‘ % console_socket)
            tsock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
            tsock.connect(console_socket)
        elif console_socket or re.match(‘^127\.‘, console_host):
            # Need tunnel to physical host
            if conntype != CONN_SSH:
                self.msg("Need a tunnel to access console but can‘t mount " +
                         "one because it‘s not a SSH host")
                raise
            try:
                # generate a string with all placeholders to avoid TypeErrors
                # in sprintf
                # https://github.com/retspen/webvirtmgr/pull/497
                error_msg = "Try to open tunnel on %s@%s:%s on console %s:%s "
                error_msg += "(or socket %s)"
                self.msg(error_msg % (connuser, connhost, connport,
                         console_host, console_port, console_socket))
                tunnel = Tunnel()
                fd = tunnel.open(connhost, connuser, connport,
                                 console_host, console_port, console_socket)
                tsock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
            except Exception as e:
                self.msg("Fail to open tunnel : %s" % e)
                raise
            self.msg("Tunnel openned")
        else:
            # Direct access
            self.msg("connecting to: %s:%s" % (connhost, console_port))
            tsock = socket_factory(connhost, console_port, connect=True)
            tunnel = None

        if self.verbose and not daemon:
            print(self.traffic_legend)

        # Start proxying
        try:
            self.msg("Start proxying")
            self.do_proxy(tsock)
        except:
            if tunnel:
                self.vmsg(
                    "%s:%s (via %s@%s:%s) : Target closed" %
                    (console_host, console_port, connuser, connhost, connport))
                if tsock:
                    tsock.shutdown(socket.SHUT_RDWR)
                    tsock.close()
                if tunnel:
                    tunnel.close()
            raise


if USE_HANDLER:
    class NovaProxyRequestHandler(ProxyRequestHandler, CompatibilityMixIn):
        def msg(self, *args, **kwargs):
            self.log_message(*args, **kwargs)

        def vmsg(self, *args, **kwargs):
            if self.verbose:
                self.msg(*args, **kwargs)

        def new_websocket_client(self):
            """
            Called after a new WebSocket connection has been established.
            """
            # Setup variable for compatibility
            daemon = self.server.daemon
            socket_factory = self.server.socket

            self._new_client(daemon, socket_factory)

else:
    class NovaWebSocketProxy(WebSocketProxy, CompatibilityMixIn):

        def new_client(self):
            """
            Called after a new WebSocket connection has been established.
            """
            # Setup variable for compatibility
            daemon = self.daemon
            socket_factory = self.socket

            self._new_client(daemon, socket_factory)

if __name__ == ‘__main__‘:
    if USE_HANDLER:
        # Create the WebSocketProxy with NovaProxyRequestHandler handler
        server = WebSocketProxy(RequestHandlerClass=NovaProxyRequestHandler,
                                listen_host=options.host,
                                listen_port=options.port,
                                source_is_ipv6=False,
                                verbose=options.verbose,
                                cert=options.cert,
                                key=None,
                                ssl_only=False,
                                daemon=False,
                                record=False,
                                web=False,
                                traffic=False,
                                target_host=‘ignore‘,
                                target_port=‘ignore‘,
                                wrap_mode=‘exit‘,
                                wrap_cmd=None)
    else:
        # Create the NovaWebSockets proxy
        server = NovaWebSocketProxy(listen_host=options.host,
                                    listen_port=options.port,
                                    source_is_ipv6=False,
                                    verbose=options.verbose,
                                    cert=options.cert,
                                    key=None,
                                    ssl_only=False,
                                    daemon=False,
                                    record=False,
                                    web=False,
                                    target_host=‘ignore‘,
                                    target_port=‘ignore‘,
                                    wrap_mode=‘exit‘,
                                    wrap_cmd=None)
    server.start_server()