|
|
| Line 1: |
Line 1: |
| − | ..
| + | 代码关键点： |
| − | This work is licensed under a Creative Commons Attribution 3.0 Unported
| + | 100MBraw文件格式 |
| − | License.
| + | data（100MB）|footer(512byte) |
| | | | |
| − | http://creativecommons.org/licenses/by/3.0/legalcode
| + | 100MB动态文件格式 |
| | + | copyoffooter（512）|head（1024）|bat（512）|batmap（1024）|footer（512） 总共3584byte |
| | + | bat的512计算方法：100MB/2MB(block size)=50entries->*4byte(每个地址4byte)=200byte->按照section补齐=512byte |
| | + | batmap的计算方法：bathead（512byte）+map（512byte） |
| | | | |
| − | =================================
| + | vhd-util snapshot |
| − | Support passthrough of USB device
| + | 只有对一个正在使用的VHD或VHD的SNAP（VM正在运行）进行SNAPSHOT，快照文件的depth才会加1，并设置parent为进行快照的对象 |
| − | =================================
| + | 代码看vhd_util_find_snapshot_target，如果给一个差分卷做快照，且差分卷是纯净的，则找查分卷的parent_locator，将其parent |
| | | | |
| − | Include the URL of your launchpad blueprint:
| + | 在100MB动态文件基础上创建增量快照 |
| | + | 和100MB动态文件比较，快照的差别是增加了parentlocator数据，磁盘格式为： |
| | + | copyoffooter（512）|head（1024）|bat（512）|batmap（1024）|parentlocator（1536，3个locator，存放的数据只是父镜像文件的相对地址）|footer（512） 总共5120byte |
| | + | parentlocator就是平台码，用来表示文件格式，代码里头会产生PLAT_CODE_MACX（File URL）、PLAT_CODE_W2KU（windows系统，绝对路径）、PLAT_CODE_W2RU（windows系统、相对路径） |
| | | | |
| − | https://blueprints.launchpad.net/nova/+spec/usb-passthrough
| + | 100MB动态vhd文件扩容到200MB |
| | + | vhd-util resize -n parent.img -s 200 -j journal |
| | + | -j填写什么？journal是个什么东东？ |
| | + | 新增100M空间，故（1）bat条目增加100MB/2MB->50entries*4byte=200byte，第一次分配的bat空间为512byte已满足要求,batmap增加7byte，原来batmap分配的空间是512byte也满足要求。 |
| | + | 故文件物理大小不变仍是3584byte，虚拟大小是200MB |
| | | | |
| − | We provide VDI(Virtual Desktop) and server virtualization solutions for
| + | vhd-util modify -n parent.img -s 314572800 |
| − | customers, our customers have strong requirements for using USB devices.
| + | 修改文件的大小，其效果是将footer按照目的大小做了偏移，但是footer里头记录的文件大小没变，功能有问题 |
| | + | vhd-util create -n parent.img -s 100 -r #创建100M固定卷 |
| | + | vhd-util modify -n parent.img -s 314572800 #修改大小为300M |
| | + | vhd-util query -n parent.img -sv #查看物理大小和虚拟大小 |
| | + | 100 #虚拟大小没变 |
| | + | 314572800 #物理大小变成300M |
| | + | 另外，对于动态卷都没有变化，why？？？？？？ |
| | | | |
| − | The typical use cases and our solutions are described as below:
| |
| | | | |
| − | 1.In VDI solution, customers want to use local USB printers or USB scanners
| + | 快照现在只测试了2层的，搞明白读写后要测试多层的，或者拿个虚拟机来测试？？ |
| − | with TC(Thin-Client), because remote desktop protocol like ICA have already
| |
| − | supported USB-redirection, so customers only need to attach USB device to
| |
| − | TC, the protocol can map USB device to VM.
| |
| | | | |
| − | 2. In virtualization solution, when starting or restarting some
| + | uvp的virVhdOpen接口文件打开模式的VHD_OPEN_IO_WRITE_SPARSE这个模式是什么意思？ |
| − | business-critical applications, a connected USB-KEY is needed for
| |
| − | authentication, some applications even need a daily authentication by USB-KEY
| |
| − | . we suggest the following solutions:
| |
| | | | |
| − | (1) Using physical 'USB-HUB' box and technology of USB-redirection over
| + | 100MB动态vhd母卷，在此基础创建一个链接克隆子卷，然后合并链接克隆子卷到母卷 |
| − | TCP/IP. Customers need to buy USB-HUB and install software in guest os,
| + | ./kvmvhd-test create parent.img 100 3 |
| − | the software helps redirecting USB device to VM.
| + | ./kvmvhd-test linkclone parent.img son.img 0 #这两个步骤本质上和创建增量快照是一样的 |
| − | | |
| − | (2) Using USB-Passthrough functions provided by our virtualization software.
| |
| − | The end users(normally application or system administrators) insert USB
| |
| − | devices to host that containing the VM, then can see USB device list in
| |
| − | portal and choose USB device to attach.
| |
| − | | |
| − | This solution has advantages that
| |
| − | | |
| − | 1. It doesn't need additional physical devices.
| |
| − | | |
| − | 2. It doesn't need a special server to run spice client for USB-Redirection.
| |
| − | | |
| − | 3. Business-critical applications commonly need stable and long-standing
| |
| − | USB-KEY to attach, USB-Passthrough maybe more stable than USB-Redirection
| |
| − | over TCP/IP or remote desktop protocol.
| |
| − | | |
| − | As described above, I think USB-Passthrough is valuable in Openstack.
| |
| − | | |
| − | This BP is focus on how to provide USB-Passthrough function in Openstack
| |
| − | | |
| − | | |
| − | Problem description
| |
| − | ===================
| |
| − | | |
| − | Use cases:
| |
| − | | |
| − | In private cloud, a end user wants to create a VM to run ERP software which
| |
| − | needs a USB-KEY for authentication, he proposes request to system
| |
| − | administrator, they perform the following steps:
| |
| − | | |
| − | 1. The administrator goes to the machine room and chooses a host to insert
| |
| − | USB-KEY to.
| |
| − | | |
| − | 2. The system automatically discovery the device and store device information
| |
| − | to DB.
| |
| − | | |
| − | 3. The administrator queries USB-KEY information from API or portal.
| |
| − | | |
| − | 4. The administrator creates a flavor with USB-KEY information.
| |
| − | | |
| − | 5. The end user creates a VM with flavor created above, the system chooses
| |
| − | the host which contains the USB-KEY to run VM and attach USB-KEY to it.
| |
| − | | |
| − | | |
| − | Proposed change
| |
| − | ===============
| |
| − | | |
| − | The idea about how to implement this function can refer to pci-passthrough.
| |
| − | | |
| − | We should add the following functions to nova:
| |
| − | | |
| − | 1. Nova should support to auto-discover usb devices and store them in DB.
| |
| − | | |
| − | 2. Nova should support to create a flavor which contains usb device.
| |
| − | information.
| |
| − | | |
| − | 3. Nova should support to create VM with a flavor which contains usb device
| |
| − | information.
| |
| − | | |
| − | Alternatives
| |
| − | ------------
| |
| − | | |
| − | None
| |
| − | | |
| − | Data model impact
| |
| − | -----------------
| |
| − | | |
| − | The idea of how to implement this function can refer to pci-passthrough.
| |
| − | | |
| − | 1. In libvirt driver layer, add function to discover USB devices.
| |
| − | | |
| − | There maybe some USB devices in a host, like USB controller, USB keyboard,
| |
| − | USB disk, but user only want to use USB disk. we can do the following
| |
| − | steps to find proper device:
| |
| − | | |
| − | (1) Libvirt driver use self._conn.listDevices('usb_device', 0) to get
| |
| − | USB device information.
| |
| − | | |
| − | (2) Use black-list function to filter out USB controller and
| |
| − | USB keyboard, only retain USB disk.
| |
| − | | |
| − | For example: usb_passthrough_blacklist =
| |
| − | '[{"vendor_id": "1d6b", "product_id": "0002"}]', which is the
| |
| − | 'Linux ehci_hcd'
| |
| − | | |
| − | 2. In DB layer, (1) add a new table 'usb_devices' to store USB device
| |
| − | information. (2) add a key-value pair to instance_system_metadata table to
| |
| − | store USB device information which will be assigned to the VM.
| |
| − | | |
| − | usb_devices:
| |
| − | | |
| − | +-----------------+--------------+------+-----+---------+----------------+
| |
| − | | Field | Type | Null | Key | Default | Extra |
| |
| − | +-----------------+--------------+------+-----+---------+----------------+
| |
| − | | created_at | datetime | YES | | NULL | |
| |
| − | | updated_at | datetime | YES | | NULL | |
| |
| − | | deleted_at | datetime | YES | | NULL | |
| |
| − | | deleted | int(11) | NO | | NULL | |
| |
| − | | id | int(11) | NO | PRI | NULL | auto_increment |
| |
| − | | compute_node_id | int(11) | NO | MUL | NULL | |
| |
| − | | address | varchar(12) | NO | | NULL | |
| |
| − | | product_id | varchar(4) | YES | | NULL | |
| |
| − | | vendor_id | varchar(4) | YES | | NULL | |
| |
| − | | bus_id | int(10) | YES | | NULL | |
| |
| − | | device_id | int(10) | YES | | NULL | |
| |
| − | | port_id | int(10) | YES | | NULL | |
| |
| − | | dev_id | varchar(255) | YES | | NULL | |
| |
| − | | status | varchar(36) | NO | | NULL | |
| |
| − | | extra_info | text | YES | | NULL | |
| |
| − | | instance_uuid | varchar(36) | YES | MUL | NULL | |
| |
| − | +-----------------+--------------+------+-----+---------+----------------+
| |
| − | | |
| − | 3. In scheduler layer, add a filter to find proper host which contains the
| |
| − | usb devices and create VM in this host.
| |
| − | | |
| − | 4. The schema of flavor which contains USB device information may like this:
| |
| − | {"usb_devices":[{"id": 5}]}
| |
| − | | |
| − | 5. How to distinguish USB device is important because user wants to specify
| |
| − | which USB device to which VM.
| |
| − | | |
| − | I consider it as follows:
| |
| − | | |
| − | 1. The identifier of a USB device which are supported by libvirt are
| |
| − | vendor id, product id, bus and device. Bellow is a example:
| |
| − | | |
| − | <hostdev mode='subsystem' type='usb'>
| |
| − | <source>
| |
| − | <vendor id='0x136b'/>
| |
| − | <product id='0x0003'/>
| |
| − | <address bus='2' device='3'/>
| |
| − | </source>
| |
| − | </hostdev>
| |
| − | | |
| − | Different USB devices may have the same vendor id and product id, so they are
| |
| − | not appropriate to distinguish devices.
| |
| − | | |
| − | The bus+device maybe a good choice to distinguish devices, but I have made some
| |
| − | tests and found that the device id will change every time reboot host or
| |
| − | detach/reattach USB device from/to host.
| |
| − | | |
| − | I have tested vsphere and saw some documents, they in fact pass-through
| |
| − | physical port of a host to VM, if user insert a USB device to this port,
| |
| − | guest os will automatic discovery the new devices.
| |
| − | | |
| − | I think libvirt doesn't support this function by now.
| |
| − | | |
| − | So I think bus+device maybe the best choice.
| |
| − | | |
| − | 5. Use which usb controller is also an important thing to consider. I have
| |
| − | made some tests with libvirt-1.2.0-1.el6.x86_64 and qemu-kvm-1.6.2-00001.x86_64
| |
| − | in centos 6.5. The test results are:
| |
| − | | |
| − | xp(sp3) win7 centos6.5
| |
| − | uhci ehci uhci ehci uhci ehci
| |
| − | create vm OK NOK NOK OK OK OK
| |
| − | detach usb OK OK OK OK
| |
| − | atach usb OK OK OK OK
| |
| − | start vm OK OK OK OK
| |
| − | restart OK OK OK OK
| |
| − | suspend/resume OK OK OK OK
| |
| − | | |
| − | xp(sp3)+ehci: Device is in abnormal status. After shutdown VM the device will
| |
| − | not be normally released to hypervisor. Is the guest os problem?
| |
| − | | |
| − | win7+uhci: Device is in abnormal status. After shutdown VM the device will be
| |
| − | normally released to hypervisor. A related bug about it in
| |
| − | https://bugs.launchpad.net/qemu/+bug/685096
| |
| − | | |
| − | So I think the default uhci controller can not meet the requirements. I
| |
| − | suggest two options:
| |
| − | | |
| − | Option 1: Create a default ehci USB controller in libvirt driver, all usb
| |
| − | devices use this default controller. Guest os should support ehci controller
| |
| − | by installing driver or they will not been supported by this function.
| |
| − | | |
| − | Option 2: User can specify USB controller type when specify USB device, then
| |
| − | system create USB controller(ehci, xhci) for USB device.
| |
| − | | |
| − | I suggest option 2 because although it is more complex to implement but it
| |
| − | is more flexible for future extension.
| |
| − | | |
| − | 6. Restriction on max supported USB devices: I think a VM with 2 USB devices
| |
| − | will cover 99% use scenarios. So it is not necessary to consider USB hub
| |
| − | or some other mechanisms to extend more ports for more USB device. Every
| |
| − | kind of USB controller has only one.
| |
| − | | |
| − | REST API impact
| |
| − | ---------------
| |
| − | | |
| − | * API for query USB device information of a hypervisor
| |
| − | | |
| − | V2 API specification:
| |
| − | | |
| − | GET v2/{tenant_id}/os-hypervisors/{hypervisor_hostname}
| |
| − | | |
| − | V3 API specification:
| |
| − | | |
| − | GET v3/os-hypervisors/{hypervisor_hostname}
| |
| − | | |
| − | The response JSON contains the variable "usb_device_stats"
| |
| − | {"hypervisor":{"usb_device_stats":[{"id": 1,"compute_node_id": 1,
| |
| − | "address": "usb_2_3", "product_id": "1520","vendor_id": "8086",
| |
| − | "bus_id": "2", "device_id": "3","port_id": "", "status":"available",
| |
| − | "extra_info": """instance_uuid":""}]}}
| |
| − | | |
| − | | |
| − | Security impact
| |
| − | ---------------
| |
| − | | |
| − | None
| |
| − | | |
| − | Notifications impact
| |
| − | --------------------
| |
| − | | |
| − | None
| |
| − | | |
| − | Other end user impact
| |
| − | ---------------------
| |
| − | | |
| − | I have implemented a demo and made some tests:
| |
| − | | |
| − | 1. If migrate(live-migration, with shared storage or not) a VM with USB device
| |
| − | from host A to host B, because the destination host doesn't have USB device
| |
| − | the migration may be failed.
| |
| − | | |
| − | 2. Currently I want to use <address bus='2' device='3'/> to exclusively
| |
| − | identify a physical USB device of a host. But I found that every time reboot
| |
| − | host or detach/re-attach USB device from/to host the device id will change.
| |
| − | | |
| − | For this problem I suggest to refer to pci-passthrough:
| |
| − | | |
| − | (1) System change the 'invalid' device's status to 'deleted', and create a new
| |
| − | device record in DB.
| |
| − | | |
| − | (2) If user starts VM with 'invalid' device, the libvirt driver will
| |
| − | automatically ignore the 'invalid' device when construct xml. So VM will start
| |
| − | normally but user can not see USB device in guest os.
| |
| − | | |
| − | (3) If user wants to use USB device again, him can create a new flavor with
| |
| − | this 'new' USB device and resize VM with the new flavor.
| |
| − | | |
| − | 3. System has restriction on max supported USB device number because a single
| |
| − | USB controller has restricted port numbers.For example a uhci controller has
| |
| − | 2 ports so only supports two USB devices.
| |
| − | | |
| − | 4. Have restriction on mas supported USB devices.
| |
| − | | |
| − | 5. Currently I only plan to provide this function in KVM so not consider too
| |
| − | much for xen.
| |
| − | | |
| − | | |
| − | Performance Impact
| |
| − | ------------------
| |
| − | | |
| − | None
| |
| − | | |
| − | Other deployer impact
| |
| − | ---------------------
| |
| − | | |
| − | None
| |
| − | | |
| − | Developer impact
| |
| − | ----------------
| |
| − | | |
| − | I have implemented a demo and made some tests:
| |
| − | | |
| − | 1. This new function doesn't affect other server actions including
| |
| − | start, stop, reboot, pause/unpause, suspend/resume, rebuild.
| |
| − | | |
| − | 2. We need to add some processes of USB devices in resize function,
| |
| − | like resize VM from flavor A with USB device A to flavor B with USB device B.
| |
| − | | |
| − | Implementation
| |
| − | ==============
| |
| − | | |
| − | Assignee(s)
| |
| − | -----------
| |
| − | | |
| − | Primary assignee:
| |
| − | <Jing Yuan>
| |
| − | | |
| − | | |
| − | Work Items
| |
| − | ----------
| |
| − | | |
| − | Step 1: Implement function of discover usb device in libvirt driver.
| |
| − | | |
| − | Step 2: Implement function of periodically update USB device information
| |
| − | from nova-compute to DB.
| |
| − | | |
| − | Step 3: Implement function of query USB devices of hypervisor.
| |
| − | | |
| − | Step 4: Implement function of create VM with USB device.
| |
| − | | |
| − | Step 5: Make changes to other functions which are affected by this new
| |
| − | function.
| |
| − | | |
| − | Dependencies
| |
| − | ============
| |
| − | | |
| − | None
| |
| − | | |
| − | | |
| − | Testing
| |
| − | =======
| |
| − | | |
| − | It is necessary to add tempest for this new function.
| |
| − | | |
| − | Documentation Impact
| |
| − | ====================
| |
| − | | |
| − | It is necessary to add doc for how to use this new function.
| |
| − | | |
| − | | |
| − | References
| |
| − | ==========
| |
| − | | |
| − | None
| |
100MB动态文件格式
copyoffooter（512）|head（1024）|bat（512）|batmap（1024）|footer（512） 总共3584byte
bat的512计算方法：100MB/2MB(block size)=50entries->*4byte(每个地址4byte)=200byte->按照section补齐=512byte
batmap的计算方法：bathead（512byte）+map（512byte）
vhd-util snapshot
只有对一个正在使用的VHD或VHD的SNAP（VM正在运行）进行SNAPSHOT，快照文件的depth才会加1，并设置parent为进行快照的对象
代码看vhd_util_find_snapshot_target，如果给一个差分卷做快照，且差分卷是纯净的，则找查分卷的parent_locator，将其parent
在100MB动态文件基础上创建增量快照
和100MB动态文件比较，快照的差别是增加了parentlocator数据，磁盘格式为：
copyoffooter（512）|head（1024）|bat（512）|batmap（1024）|parentlocator（1536，3个locator，存放的数据只是父镜像文件的相对地址）|footer（512） 总共5120byte
parentlocator就是平台码，用来表示文件格式，代码里头会产生PLAT_CODE_MACX（File URL）、PLAT_CODE_W2KU（windows系统，绝对路径）、PLAT_CODE_W2RU（windows系统、相对路径）
100MB动态vhd文件扩容到200MB
vhd-util resize -n parent.img -s 200 -j journal
-j填写什么？journal是个什么东东？
新增100M空间，故（1）bat条目增加100MB/2MB->50entries*4byte=200byte，第一次分配的bat空间为512byte已满足要求,batmap增加7byte，原来batmap分配的空间是512byte也满足要求。
故文件物理大小不变仍是3584byte，虚拟大小是200MB
vhd-util modify -n parent.img -s 314572800
修改文件的大小，其效果是将footer按照目的大小做了偏移，但是footer里头记录的文件大小没变，功能有问题
vhd-util create -n parent.img -s 100 -r #创建100M固定卷
vhd-util modify -n parent.img -s 314572800 #修改大小为300M
vhd-util query -n parent.img -sv #查看物理大小和虚拟大小
100 #虚拟大小没变
314572800 #物理大小变成300M
另外，对于动态卷都没有变化，why？？？？？？
100MB动态vhd母卷，在此基础创建一个链接克隆子卷，然后合并链接克隆子卷到母卷
./kvmvhd-test create parent.img 100 3
./kvmvhd-test linkclone parent.img son.img 0 #这两个步骤本质上和创建增量快照是一样的
