Jump to: navigation, search

Nova VM Diagnostics


The 'nova diagnostics' command provides diagnostics information for the specific VM. The purpose of the command is to provide the administrator with the information that will enable them to troubleshoot issues with a VM. Each driver has its own diagnostics capabilities and information that can be exposed. Currently get_diagnostics returns a dictionary of data specific to the driver.

At the moment the data returned has the following formats:


Property Value
cpu0_time 283520000000
hdd_errors -1
hdd_read 106782
hdd_read_req 37
hdd_write 0
hdd_write_req 0
memory 2097152
memory-actual 2097152
memory-rss 1050096
memory-swap_in 0
vda_errors -1
vda_read 65447936
vda_read_req 8146
vda_write 7746560
vda_write_req 896
vnet0_rx 41195
vnet0_rx_drop 0
vnet0_rx_errors 0
vnet0_rx_packets 282
vnet0_tx 39239
vnet0_tx_drop 0
vnet0_tx_errors 0
vnet0_tx_packets 280


Property Value
vmware:annotation None
vmware:balloonedMemory 0
vmware:bootTime 2013-12-16T03:20:28.040652
vmware:compressedMemory 0
vmware:connectionState connected
vmware:consolidationNeeded False
vmware:consumedOverheadMemory 20
vmware:cpuReservation 0
vmware:device [{u'key': 4000, u'runtimeState': {u'vmDirectPathGen2InactiveReasonOther': [u'vmNptIncompatibleNetwork'], u'vmDirectPathGen2Active': False}}]
vmware:distributedCpuEntitlement 0
vmware:distributedMemoryEntitlement 828
vmware:faultToleranceState notConfigured
vmware:featureRequirement [{u'featureName': u'cpuid.AES', u'value': u'Bool:Min:1', u'key': u'cpuid.AES'}]
vmware:ftLatencyStatus gray
vmware:ftLogBandwidth -1
vmware:ftSecondaryLatency -1
vmware:guestFullName Other (32-bit)
vmware:guestHeartbeatStatus gray
vmware:guestId otherGuest
vmware:guestMemoryUsage 384
vmware:host {u'_type': u'HostSystem', u'value': u'host-207'}
vmware:hostMemoryUsage 110
vmware:installBootRequired False
vmware:instanceUuid cde2fd89-5d39-4e3b-a6fc-589042a74e78
vmware:maxCpuUsage 2400
vmware:maxMemoryUsage 512
vmware:memoryOverhead 21417984
vmware:memoryReservation 0
vmware:memorySizeMB 512
vmware:name cde2fd89-5d39-4e3b-a6fc-589042a74e78
vmware:numCpu 1
vmware:numEthernetCards 1
vmware:numMksConnections 0
vmware:numVirtualDisks 1
vmware:onlineStandby False
vmware:overallCpuDemand 0
vmware:overallCpuUsage 0
vmware:powerState poweredOn
vmware:privateMemory 90
vmware:recordReplayState inactive
vmware:sharedMemory 0
vmware:ssdSwappedMemory 0
vmware:staticCpuEntitlement 2400
vmware:staticMemoryEntitlement 539
vmware:suspendInterval 0
vmware:swappedMemory 0
vmware:template False
vmware:toolsInstallerMounted False
vmware:uptimeSeconds 124
vmware:uuid 4238fd65-cd75-51e7-0c53-e71cd01a753f
vmware:vmPathName [local-] cde2fd89-5d39-4e3b-a6fc-589042a74e78/cde2fd89-5d39-4e3b-a6fc-589042a74e78.vmx


Property Value
cpu0 0.0002
last_update 1387273492
memory 536870912.0000
memory_target 536870912.0000
vbd_xvda_read 0.0
vbd_xvda_write 0.0
vbd_xvdd_read 0.0
vbd_xvdd_write 0.0
vif_0_rx 0
vif_0_tx 0


The proposal is to have the drivers return the following information in a dictionary. If the driver is unable to return the value or does not have access to it at the moment then it does not need to return the key, that is all of the values are optional. The table below has the key and a description of the value returned:

Key Description
state The current state of the VM. The possible values are: 'pending', 'running', 'paused', 'shutdown', 'crashed', 'suspended' and 'building' (String)
driver A string denoting the driver on which the VM is running. Examples may be: 'libvirt', xenserver', 'docker', 'hyper-v' and 'vmware' (String)
uptime The amount of time in seconds that the VM has been running (Integer)
num_cpus The number of vCPUs (Integer)
num_vnics The number of vNICS (Integer)
num_disks The number of disks (Integer)
cpu_details An array of details (a dictionary) per vCPU (see below)
network_details An array of details (a dictionary) per vNIC (see below)
disk_details An array of details (a dictionary) per disk (see below)
memory_details A dictionary of memory details (see below)
driver_private_data A dictionary of private data from the driver. This is driver specific and each driver can return information valuable for diagnosing VM issues. The raw data should be versioned.

CPU details

The cpu details will be an array of dictionaries per each virtual CPU.

Key Description
time CPU Time in nano seconds (Integer)

Network details

The network details will be an array of dictionaries per each virtual NIC.

Key Description
rx_octets Received octets (Integer)
rx_errors Received errors (Integer)
rx_drop Received packets dropped (Integer)
rx_packets Received packets (Integer)
tx_octets Transmitted Octets (Integer)
tx_errors Transmit errors (Integer)
tx_drop Transmit dropped packets (Integer)
tx_packets Transmit packets (Integer)

Disk details

The disk details will be an array of dictionaries per each virtual disk.

Key Description
reads Disk reads in bytes(Integer)
read_requests Read request (Integer)
writes Disk writes in bytes (Integer)
write_requests Write requests (Integer)
errors Disk errors (Integer)

Memory details

Key Description
maximum Amount of memory provisioned for the VM in MB (Integer)
used Amount of memory used by the VM in MB (Integer)


Below is an example of the dictionary data returned by the fake driver.

          {'state': 'running',
           'driver': 'fake',
           'uptime': 7,
           'num_cpus': 1,
           'num_vnics': 1,
           'num_disks': 1,
           'cpu_details': [{'time': 1024}]
           'network_details': [{'rx_octets': 0,        
                                'rx_errors': 0,        
                                'rx_drop': 0,  
                                'rx_packets': 0,       
                                'tx_octets': 0,        
                                'tx_errors': 0,        
                                'tx_drop': 0,
                                'tx_packets': 0}],      
           'disk_details': [{'reads':0, 
                             'read_requests': 0,
                             'writes': 0,
                             'write_requests': 0,
                             'errors': 0}],
           'memory_details': {'maximum': 512, 'used': 256},
           'driver_private_data': output}


  • V2 REST API - no changes. This will ensure that the code is backward compatible and users that have applications written against the V2 API will continue to work. In this case only the raw data will be returned.
  • V3 REST API- the data defined above will be returned

Implementation Details

An additional parameter will be added to the method get_diagnostics. This will be "common_api". By default this will be True. When the V2 API invokes the call it will be set as False.

The blueprint can be seen at: https://blueprints.launchpad.net/nova/+spec/v3-diagnostics