标签：
There are several ways to deploy openstack, Devstack is easily for developer to deploy Open Stack.
Please follow this URL to set up your Open Stack.
http://docs.openstack.org/developer/devstack/
root@minicloud-allinone-controller0-i7bnc6baumzl:~# dpkg -S /etc/modprobe.d/qemu-system-x86.conf qemu-system-x86: /etc/modprobe.d/qemu-system-x86.conf root@minicloud-allinone-controller0-i7bnc6baumzl:~# cat /etc/modprobe.d/qemu-system-x86.conf options kvm_intel nested=1
echo “options kvm-intel nested=1″ | sudo tee /etc/modprobe.d/kvm-intel.conf
rmmod kvm_intel modprobe kvm_intel
libvirt_cpu_mode = host-passthrough
There are two ways to support PCI PassThrough.
For PCI SR-IOV device we have a specially way to setup.
(1) Enable the Intel VT-d extensions http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
Setup in your bios. Check you have enable it.
$sudo dmesg | grep -e DMAR -e IOMMU
(2) Activate Intel VT-d in the kernel http://www.zerg.sc/technote/?p=241
$ sudo echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts
update the the kernel cmdline.
https://www.gnu.org/software/grub/manual/html_node/Simple-configuration.html
$ sudo su $ sudo sed -i -e ‘s/\(^GRUB_CMDLINE_LINUX_DEFAULT.*\)"$/\1 intel_iommu=on"/‘ /etc/default/grub $ sudo update-grub $ sudo grep intel_iommu=on /boot/grub/grub.cfg
Why we still need the legacy UIO framework, for there are some thing wrong with vfio and igb in some PC.
http://www.linux-kvm.org/page/How_to_assign_devices_with_VT-d_in_KVM
Set parameter KVM allow_unsafe_assigned_interrupts
Transient configure (take effect Immediately)
$ sudo echo 1 > /sys/module/kvm/parameters/allow_unsafe_assigned_interrupts
Or persistent
$ sudo echo "options kvm allow_unsafe_assigned_interrupts=1" >> /etc/modprobe.d/kvm_iommu.conf
Search the PCI device
$ lspci -n 01:00.0 0200: 8086:10fb (rev 01) 01:00.1 0200: 8086:10fb (rev 01) $ lspci 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
Bind pci_stub driver to PCI device
$ sudo modprobe pci_stub $ sudo su(take effect Immediately) $ echo "8086 10fb" > /sys/bus/pci/drivers/pci-stub/new_id $ echo 0000:01:00.0 > /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind $ echo 0000:01:00.0 > /sys/bus/pci/drivers/pci-stub/bind $ echo 0000:01:00.1 > /sys/bus/pci/devices/0000\:01\:00.1/driver/unbind $ echo 0000:01:00.1 > /sys/bus/pci/drivers/pci-stub/bind
Start attach PCI device to guest and start
$ echo "
<hostdev mode=‘subsystem‘ type=‘pci‘ managed=‘yes‘>
<driver name=‘kvm‘/>
<source>
<address domain=‘0x0000‘ bus=‘0x01‘ slot=‘0x00‘ function=‘0x1‘/>
</source>
</hostdev>" > new-pci.xml
$ virsh attach-device guest-name --config new-pci.xml
$ virsh start guest-name
There are tons of tutorials on how to get VT-d working with KVM/libvirt/Qemu/etc. It’s still a fairly new and unused feature, which means that it is getting updated pretty frequently. Unfortunately, the Internet doesn’t update as quickly, so most of the documentation out there is not accurate or the best approach. readlink You can refer this when any trouble.
We need this command to load the kernel modules at boot time
$ sudo echo "vfio vfio_iommu_type1 vfio_pci" >> /etc/modules
Set vfio_iommu_type1 allow_unsafe_interrupts parameters If you‘re overriding the ivrs table to enable intremap you dont need this line Persistent (need reboot.)
$ echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/vfio_iommu_type1.conf
Or transient (take effect Immediately)
$ sudo echo 1 > /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts
Disable AppArmor for libvirtd
Actually, we do not need to disable Apparmor, this is a bug on ubuntu, we need to work around it. (It may can not works)
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1276719
http://libvirt.org/drvqemu.html#securitysvirtaa
$ apt-get install apparmor-utils $ sudo aa-complain /usr/sbin/libvirtd $ sudo su $ for x in /etc/apparmor.d/libvirt/libvirt-*[0-9a-b];do aa-complain $x; done
Or (It may can not works)
$ sudo ln -s /etc/apparmor.d/usr.sbin.libvirtd /etc/apparmor.d/disable/ $ sudo ln -s /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper /etc/apparmor.d/disable/ $ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd $ sudo apparmor_parser -R /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper $ sudo service libvirt-bin restart
Or remove it (the best way)
http://www.techytalk.info/disable-and-remove-apparmor-on-ubuntu-based-linux-distributions/
$ sudo /etc/init.d/apparmor stop $ sudo update-rc.d -f apparmor remove $ sudo apt-get --purge remove apparmor apparmor-utils libapparmor-perl libapparmor1
It may cause the ubuntu-desktop crash, please reinstall it.
$ sudo apt-get install ubuntu-desktop
Find the all the pci devices in sam iommu group
We must binding all devices in same group to the vfio-pci driver creates.
$ readlink /sys/bus/pci/devices/0000\:01\:00.0/iommu_group ../../../../kernel/iommu_groups/1 $ ls -l /sys/bus/pci/devices/0000\:01\:00.0/iommu_group/devices
Bind vfio driver to PCI device
$ sudo su $ echo "8086 10fb" > /sys/bus/pci/drivers/vfio-pci/new_id $ echo "0000:01:00.1" > /sys/bus/pci/devices/0000\:01\:00.1/driver/unbind $ echo "0000:01:00.1" > /sys/bus/pci/drivers/vfio-pci/bind $ echo "0000:01:00.0" > /sys/bus/pci/devices/0000\:01\:00.0/driver/unbind $ echo "0000:01:00.0" > /sys/bus/pci/drivers/vfio-pci/bind
Start attach PCI device to guest and start
$ echo "
<hostdev mode=‘subsystem‘ type=‘pci‘ managed=‘yes‘>
<driver name=‘vfio‘/>
<source>
<address domain=‘0x0000‘ bus=‘0x01‘ slot=‘0x00‘ function=‘0x1‘/>
</source>
</hostdev>" > new-pci.xml
$ virsh attach-device guest-name --config new-pci.xml
$ virsh start guest-name
Persistent configure for igb module.
$ sudo echo ‘options igb max_vfs=7‘ > /etc/modprobe.d/igb.conf $ sudo echo "igb" >> /etc/modules $ sudo update-initramfs -k all -t -u
Transient configure for igb module.
$ sudo su $ sudo rmmod igb && sudo modprobe igb max_vfs=7
It maill failed as follow on PC:
Jan 10 04:42:18 localhost kernel: [ 8441.914658] ixgbe 0000:01:00.1: SR-IOV: bus number out of range Jan 10 04:42:18 localhost kernel: [ 8441.914662] ixgbe 0000:01:00.1 0000:01:00.1 (uninitialized): Failed to enable PCI sriov: -12
This is intel official document, it is some different from the above.
Download the latest stable kernel. https://www.kernel.org/
$ wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.18.1.tar.xz $ tar -xvf linux-3.18.1.tar.xz $ cd linux-3.18.1 $ cp /boot/config-3.13.0-43-generic ./.config $ echo "" | make oldconfig
Please enable CONFIG_IP_NF_NAT the config, kernel 3.17 change the macro define for nat.
$ sed -i -e "s/# CONFIG_IP_NF_NAT.*/CONFIG_IP_NF_NAT=m/" ./.config $ make -j8 $ make modules_install $ make install $ sudo reboot
$ git clone git://git.qemu.org/qemu.git $ apt-get install libglib2.0-dev zlib1g-dev libspice-server-dev libspice-protocol-dev $ ./configure ‘--target-list=x86_64-softmmu‘ ‘--enable-debug‘ ‘--enable-kvm‘ ‘--enable-spice‘ ‘--prefix=/home/shhfeng/qemu/‘ $ make
$ gunzip # uncompress your initrd-*.img file $ mkdir tmp $ cd tmp/ $ cpio -i --make-directories < "your uncompress initrd direction." $ tree -C .
# tree: option -C color show，-n shutdown color show.
or
$ cd tmp/ $ gzip -dc ../sth.gz |cpio -id
# edit your initrd，
$ cd tmp/
# add or remove *.ko
$ find ./ | cpio -H newc -o > /boot/initrd-2.6.22.6.img-ahci $ gzip /boot/initrd-2.6.22.6.img-ahci $ mv initrd-2.6.22.6.img-ahci.gz /boot/initrd-2.6.22.6.img-ahci
Please read the docs in /opt/stack/nova/doc/source/devref first. Then you will get some ideas about Open Stack. Read up these files, you can ignore them if you already know them.
http://docs.openstack.org/api/quick-start/content/index.html#Compute_API_Quick_Start
It introduce the gerrit work flow.
It introduce how to set up development environment.
Introduce the profile of the Open Stack.
Open Stack rpc is different with the traditional SUN IPC, it base on a AMQP broker. http://www.rabbitmq.com/getstarted.html
Learn more about implementation of rpc. Please seen the Open Stack Common Library.
$ git clone git://git.openstack.org/openstack/oslo.git
Also learn more about kombu:
http://kombu.readthedocs.org/en/latest/
All OpenStack services use *green thread* model of threading, If your know coroutine, you will learn greenthread quickly.
All the above files also can be access by this url link.
http://docs.openstack.org/developer/nova/devref/rpc.html
You had better do some exercise about some technologies that Open Stack uses. This is helpful for you to know Open Stack.
Tutorial:
https://github.com/rabbitmq/rabbitmq-tutorials/ http://wenku.baidu.com/view/800285ea0975f46527d3e19b.html (a Chinese Document for Rabbitmq) http://www.openstack.cn/p136.html (a Chinese Simple Document for Rabbitmq)
Tutorial Source Code:
https://github.com/rabbitmq/rabbitmq-tutorials/
Tutorial Source Code:
http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html
Read the Red hat REST API Guide. It is useful to understand REST API. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.0/html-single/REST_API_Guide/index.html#sect-REST_API_Guide-Common_Features-Resources-Creating_Resources
IRC: https://wiki.openstack.org/wiki/IRC
Dev: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
General: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Workflow: https://wiki.openstack.org/wiki/Gerrit_Workflow
http://docs.openstack.org/user-guide/content/log_in_dashboard.html
Try to access http://localhost/
http://docs.openstack.org/api/quick-start/content/index.html#Compute_API_Quick_Start
I have write a small script to get token ID, as follow, you can name it as “get-token”:
#!/bin/bash return=`curl -i ‘http://127.0.0.1:5000/v2.0/tokens‘ -X POST -H "Content-Type: application/json" -H "Accept: application/json" -d ‘{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "123"}}}‘` echo $return echo "-----------------------------------------------------------------------------" ID="" for x in $return; do if [ "x$ID" = "xtokenID" ] ;then id=`echo $x | tr -d ‘[",]‘` export TOKEN=$id echo "TOKEN=$id" ID="" break fi if [ "x$x" = ‘x"id":‘ ] ;then ID="tokenID" fi done echo ""
This script just is a sample, you can improve it.
Run it by
$ . get-token
Then you can run other commands use the TOKEN environment value, such as get all the tenents, as follow:
$ curl -i -X GET http://127.0.0.1:35357/v2.0/tenants -H "User-Agent: python-keystoneclient" -H "X-Auth-Token: $TOKEN"
http://docs.openstack.org/api/quick-start/content/index.html#Compute_API_Quick_Start
You should export the OS_USERNAME and OS_TENANT_NAME environment by the follow command firstly.
$ cd devstack/ $ . openrc OS_USERNAME OS_TENANT_NAME
http://docs.openstack.org/juno/config-reference/content/section_compute-scheduler.html
update your /etc/nova/nova.conf
Add the follow option to “[DEFAULT]” section.
$ grep -e DEFAULT -e pci_passthrough -e pci_alias -e scheduler_available_filters /etc/nova/nova.conf
[DEFAULT]
pci_passthrough_whitelist={"product_id":"10fb", "vendor_id":"8086", "deviceids":"0000:01:00.0, 0000:01:00.1"}
pci_passthrough_whitelist={"product_id":"8c20", "vendor_id":"8086", "deviceids":"0000:00:1b.0"}
pci_alias = {"vendor_id":"8086", "product_id":"10fb", "name":"intelnic"}
scheduler_available_filters = nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter
Create a instance with pci-passthrough device.
https://wiki.openstack.org/wiki/Pci_passthrough
$ nova flavor-list $ nova flavor-key m1.tiny set pci_passthrough:alias=intelnic:1 $ nova boot pci --flavor m1.tiny --image=cirros-0.3.2-x86_64-uec
I usually use “print” for debugging. The “print” is on the stack screen terminal.
$ cd devstack/ $ ./rejoin-stack.sh
Or
$ screen –ls There is a screen on: 13846.stack $ screen -r stack
Some useful operation:
run a command by “Ctrl + A + :”, you can input help command to show more info. Swith to any window by “Ctrl + A + ‘ ”, input the windows number you want to switch. Swith to any window by “Ctrl + A + " ”, select the windows you want to switch.
https://wiki.openstack.org/wiki/SmallTestingGuide
标签：
原文地址：http://www.cnblogs.com/shaohef/p/4478925.html
