Mission

Provide specific API interface to end-user which would allow to manipulate with database log files. This feature provides the ability to access log files via Swift so the can download them for Auditing/Troubleshooting purposes.

Why does Trove needs it ?

Short BIO. Audit.

Auditing is a core component to compliance and security programs, and a generally accepted practice by IT operations. Relational databases were the first enterprise application to embed auditing as a native platform feature. Yet part of this heritage is the stigma associated with auditing. Vendors provided the basic function, they failed to provide the performance and ease of management demanded by database administrators (DBAs), who remain loathe to enable the feature, and remain one of the last holdouts for accepting the use of database audit trails.

The reasons for their distaste of native audit are fully justified: from performance and data management nightmares, to the fact that these audit trails were not originally designed for the tasks they are being applied to. Regardless, regulatory compliance demands accurate and complete records of transactions, and relational database audit trails produce just that.

Security and compliance drive the use of database auditing, and these log files provide a unique perspective that needs to be captured. Database vendors are building better auditing features into their products, minimizing historic impediments, but careful deployment is required to avoid these known issues.

Database Audit

Database auditing is the examination of audit or transaction logs for the purpose of tracking changes with data or database structure. Databases can be set to capture alterations to data and metadata.

Database Management will be audited approximately once every three years using a risk-based approach. Databases to be reviewed include those databases supporting mission critical university functions. The following topics should be addressed during the review:

Database Administration (policies and procedures, capacity planning)
Database Maintenance and Tuning (operational effectiveness)
Database Integrity (referential integrity, triggers)
Database Security (database security settings, auditing/logging)
Database Backup and Recovery (availability)

The minimum requirements set forth in the “general overview and risk assessment” section below must be completed for the audit to qualify for core audit coverage. Following completion of the general overview and risk assessment, the auditor will use professional judgment to select specific areas for additional focus and audit testing.

Design

Log manipulations are designed to let user perform log investigations. Since Trove is PaaS - level project, it's user cannot interact with compute instance directly, only with database through given API (database operations). Deployer would decide which log files would be available for trove user.

API Schema

API Details

Three new resources, dblog-create, dblog-list, dblog-show will be exposed as part of the Trove API.

The dblog-list is used to provide an ability to list all available(availability defined by Trove) database logging filenames.

The dblog-show is used to provide an ability to list all available(availability defined by Trove) database logging filenames per datastore version.

The dblog-create is used to provide an ability to save database logging file into the Swift container.

To implement this capability, the create/modify/list instance operations will be extended in a manner that does not break the existing 1.0 contract. These operations will permit a user to create a new database logging file entry for already existed instance, list all available database logging filenames for all registered datstore versions (basically, for all datastore managers across all versions), show all available database logging filenames per certain datastore version (manager).

DownloadDBLogFile request parameters and DBLog response object

Description: Downloads current database log file into Swift container.

Request Parameters:

Parameter name	Description	Type	Required
Instance ID or Name	The customer-assigned name of the instance that contains the log files.	String	Yes
Database logging filename	Available database logging filename	String	Yes

Response Elements

Name	Description	Type	Errors
LogFileData	The following elements are returned in a structure named DBLog	DBLog	DBInstanceNotFound. HTTP 404

DBLog response object

DBLog is responsible for database log file stored in Swift container

Name	Description	Type
Instance ID	instance UUID ID	String
Filename in container	Log file name	String

API Calls

Get the list of all available database logging files for all datastore versions. HTTP method GET

Request

(No message body)

Response

{
   "dblogs": {
                      {
                         "datastore_version_manager": "mysql",
                         "datastore_log_files": "general_log, log_slow_queries, log-error"
                      },
                      {
                         "datastore_version_manager": "cassandra"
                         "datastore_log_files": "system_log"
                      },
                      {
                         "datastore_version_manager": "redis"
                         "datastore_log_files": "system_log"
                      }
   }}

Get the list of all available database logging files per given datastore version. HTTP method GET