Difference between revisions of "Monasca/Logging"
(→Monasca Log API) |
(→Monasca Log API) |
||
| Line 53: | Line 53: | ||
=== Monasca Log API === | === Monasca Log API === | ||
| + | <br /> | ||
| + | '''!!!! under construction !!!!''' | ||
| + | <br /> | ||
| + | |||
We currently consider two possible approaches: | We currently consider two possible approaches: | ||
# We could mimic the Loggly API in order to let end users benefit from the simplicity of the API usage. | # We could mimic the Loggly API in order to let end users benefit from the simplicity of the API usage. | ||
| + | |||
# Or we could aim achieve a high consistency with existing metric API. In the later approach we would simplified speaking copy the existing Monasca | # Or we could aim achieve a high consistency with existing metric API. In the later approach we would simplified speaking copy the existing Monasca | ||
API and tweak it a bit. We would do this intentionally for consistency reasons. As a result, this would be kind of similar to Victors approach, | API and tweak it a bit. We would do this intentionally for consistency reasons. As a result, this would be kind of similar to Victors approach, | ||
| Line 63: | Line 68: | ||
To summarize the two API approaches again: | To summarize the two API approaches again: | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
# Mimicking Loggly API: | # Mimicking Loggly API: | ||
* simpler | * simpler | ||
* authentication via a customer token. Maybe the X-Auth-Token can be used for that purpose. | * authentication via a customer token. Maybe the X-Auth-Token can be used for that purpose. | ||
* no required structure or fields like host, name, dimensions, timestamp (user is responsible and can organize his data for his needs) | * no required structure or fields like host, name, dimensions, timestamp (user is responsible and can organize his data for his needs) | ||
| + | * the request body contains only the payload (log entry) and no meta-data (simpler for parsing the payload (special characters)). | ||
* tags could be used to organize/filter logs easily (probably also possible with the Monasca “dimensions”) | * tags could be used to organize/filter logs easily (probably also possible with the Monasca “dimensions”) | ||
| + | |||
| + | # Consistency with Monasca metric API: | ||
| + | * Simple speaking: copy Monasca Metric API | ||
| + | * replace “metric” with “log” | ||
| + | * instead of a “double value” (in the JSON of the request body) we will expect a “string value” in JSON format (must be valid JSON! escape special characters...) | ||
Revision as of 15:14, 4 February 2015
This page documents the Monasca Logging solution that is in progress.
Note:
Currently we only have this page to discuss our proposals. For that reason some "metric related stuff" can be found here too. We will move or remove it whenever the discussion is finished.
Monasca Agent
Needed functionality.
OpenStack Hypervisor/VM Monitoring
The Agent should monitor each created VM (in detail)
- Install agent automatically on each created VM.
Metrics as well as logs can be forwarded to Monasca.
The Agent should monitor each created VM (basic metrics like CPU, RAM)
Metrics only!
Sources:
- Nova
Supported by agent?
- KVM
Supported by agent?
- VMware vSphere
vSphere is not supported by agent. A new plugin must be developed (blueprint...).
- VMware ESXi
Probably not supported by agent?
User:
- OpenStack administrator
- OpenStack user (multi tenancy! Can agent add the tenant ID per VM?)
Hypervisor --> Nova --> Ceilometer --> Monasca-ceilometer-plugin --> Monasca
Metrics only!
- KVM
KVM sends data to Nova, but Ceilometer (currently?) doesn't poll that data. Monasca-ceilometer-plugin is not in "product status", that means no data forwarding to Monasca.
- VMware vSphere
Although Nova provides support for VMware vCenter Server, Ceilometer does not. It does not collect metrics of virtual machines deployed on VMware virtualization platform. It will be necessary to extend this support in Ceilometer. The aim is to implement a polling agent that queries samples from VMware vCenter Server.
Ceilometer/blueprints/vmware-vcenter-server
Collect and forward Log Data
The agent should be extended to collect log data and forward them to the Monasca Log API.
Log Management Backend/Server
Integrate ELK stack into the existing Monasca architecture.
Log Data Flow
Monasca Log API
!!!! under construction !!!!
We currently consider two possible approaches:
- We could mimic the Loggly API in order to let end users benefit from the simplicity of the API usage.
- Or we could aim achieve a high consistency with existing metric API. In the later approach we would simplified speaking copy the existing Monasca
API and tweak it a bit. We would do this intentionally for consistency reasons. As a result, this would be kind of similar to Victors approach, however we would prefer to have a dedicated API in order to let them evolve separately.
Currently, we are thinking about which approach would be better for customers and acceptable for Monasca.
To summarize the two API approaches again:
- Mimicking Loggly API:
- simpler
- authentication via a customer token. Maybe the X-Auth-Token can be used for that purpose.
- no required structure or fields like host, name, dimensions, timestamp (user is responsible and can organize his data for his needs)
- the request body contains only the payload (log entry) and no meta-data (simpler for parsing the payload (special characters)).
- tags could be used to organize/filter logs easily (probably also possible with the Monasca “dimensions”)
- Consistency with Monasca metric API:
- Simple speaking: copy Monasca Metric API
- replace “metric” with “log”
- instead of a “double value” (in the JSON of the request body) we will expect a “string value” in JSON format (must be valid JSON! escape special characters...)
