Difference between revisions of "Security/Guidelines/logging guidelines"

Revision as of 19:39, 29 May 2014

paulmo (#openstack-security IRC channel) is currently editing this page; please check back later for a finished wiki. --- 2014/05/29

Overview

OpenStack needs logging and notification security guidelines and best practices to prevent accidental leakage of confidential information to unauthorized users. This wiki is an in progress attempt to gain the OpenStack community consensus on what those standards should be and how they should be implemented.

Disclaimer: This is not OpenStack Security Group (OSSG) approved at this point.

Problem Definition

There is no standard/structured logging and notification data format across OpenStack projects which would enable OpenStack operators to unambiguously identify and filter out confidential data which should not be exposed to certain users. Let's take a look at a simple architectural diagram:

There are many examples of accidental credential disclosure to unauthorized users within OpenStack. Some non-exhaustive examples:

• (Ceilometer) Log contains DB password in plain text (CVE-2013-6384) [OSSA 2013-031]
• (Keystone) Plaintext passwords are logged
• (Nova) Clear text password has been print in log by some API call

In order to prevent accidental leakage of confidential information to unauthorized users, there are some guidelines to assist in isolating this confidential data for easy/accurate filtering on the back end log management tools. Why is this important? There have been several OpenStack security issues around logging in the past:

• (Ceilometer) Log contains DB password in plain text (CVE-2013-6384) [OSSA 2013-031]
• (Keystone) Plaintext passwords are logged
• (Nova) Clear text password has been print in log by some API call

etc...

Logs should have a format that enables grouping of confidential data especially when logging data such as:

• Exceptions: Unless the developer is sure that an exception will never contain confidential information, exceptions should be identified as confidential. This has historically been especially problematic with database exceptions which may contain real field data.
  • Recommend parsing the specific exception or error and providing an abstracted/safe version back to the user
• Passwords: Never log plain text passwords
• Private Keys: Never log plain text private keys
• PII: Minimize Personally Identifiable Information (PII) logging where possible
• Local Server State: Avoid logging local server state which may provide hints to attackers (examples: file paths, code file names, user account names, PRNG state)
• Tenant/Project ID Checking: If a user identifier (tenant/project ID) is not present in the log record or does not match the current authenticated user, do not show this log data to the user
• Log Insecure Configurations: If a configuration option causes the system to enter a potentially less secure state, log a message to this effect for operators to see

OpenStack's Oslo Log is capable of creating formatted logs with a section for confidential data. The following example contains two pieces of variable data: key_name which is not confidential data (and will equal 'ssh') and key_value which is a confidential key that should not be visible to anyone but admins/operators.

Note: This is a contrived example for simplicity. If the key_value is a public ssh key, it probably isn't critical to hide it in the logs from the authorized user that it belongs to. If the key_value is a private ssh key, it shouldn't be logged to begin with.

Bad Example:

LOG.debug("User set %s key to value %s" % [key_name, key_value])

Revised/Good Example:

LOG.debug("User set %s key" % [key_name], extra={private={value=key_value}})

Note that the extra->private structure is used to hold all confidential data within logs so that it may be filtered out later before a user views logs. In this example, the key value is moved to a 'private' dictionary which makes filtering out confidential data from logs easier as there will be a single keyword to locate in log entries if these guidelines are followed. An authenticated user may see that an ssh key has been changed but an operator may see the actual ssh key value in the logs.

Log level definitions: http://stackoverflow.com/questions/2031163/when-to-use-log-level-warn-vs-error This is a nice writeup about when to use each log level. Here is a brief description:

• Debug: Shows everything and is likely not suitable for normal production operation due to the sheer size of logs generated
• Info: Usually indicates successful service start/stop, versions and such non-error related data
• Warning: Indicates that there might be a systemic issue; potential predictive failure notice
• Error: An error has occurred and an administrator should research the event
• Critical: An error has occurred and the system might be unstable; immediately get administrator assistance