Difference between revisions of "Ceilometer/blueprints/support-resources-pipeline-item"

Latest revision as of 07:47, 15 November 2013

Generic way to specify the endpoint to poll

Related blueprints:

• https://blueprints.launchpad.net/ceilometer/+spec/support-resources-pipeline-item

During the openstack HongKong summit session of how to improve the central agent, it is agreed that we should have a way to allow the administrator to specify which endpoints to poll to get the data from, for the central agent. This will gives the administrators much more flexibility and fine grained control of their deployment. Here we propose the following design to support this.

Changes to the YAML pipeline definition file and pipeline manager

• Adding new optional list item 'resources' in the pipeline definition

The interpretation of the content of the resources list should depend on different pollsters. The content could directly specifying which endpoints the pollster should poll from, or specify a file contains the list of endpoints the pollsters want to poll from. The pipeline manager will only load the resources from the pipeline file and do a very simple type check.

The new pipeline file would be something like the following:

-
name: meter_pipeline
interval: 600
meters:
        - "meterFoo"
resources:
        - snmp://10.4.4.4
        - ipmi://10.5.5.5
        - file:///foo/bar/resources_file
transformers:
publishers:
       - rpc://

Question: Should we explicitly require that the content of the resources should be URI compatible?

Changes to the Pollster class

• new parameter added to the method get_samples to specify the endpoints to get data from

def get_samples(self, manager, cache, resources=[]):

• new method added to the pollster to verify the resources is legal to this pollster

def verify_resources(self, manager, resources):
    '''Verify if the resources are legal to this pollster'''
    return True

Changes to AgentManger and PollingTask

• The PollingTasks class should be modified to support passing resources defined in the pipeline configuration to the corresponding pollsters, and also to verify the resources.

Question: should we make the changes only to the central manager's PollingTask or to the base PollingTask class?

Detect the changes in the resources

We want to avoid restarting the central agent when the resources are incrementally added into the pipeline configuration file. There are 2 ways to achieve this:

• Adding reload in the pipeline manager to detect the change of the pipeline file. The PollingTask should try 'reload' the new resources and then pass them to the pollsters in each interval iteration.

Question: What if other content of the pipeline changes, e.g. the interval, the publisher, the meter, etc.? That definitely requires a restart to setup a whole different set of PollingTasks.

• Put all the endpoints definition in a separate file other than the pipeline.yaml, and pass the file name of that file as the resources to the pollsters. Let the pollsters to detect the changes of the content and reload the new endpoints definitions.

Question: Should we only allow setting the endpoints in this way, or should we also allow putting the endpoints directly into the resources list?