Heat/Blueprints/InstanceUsers

Blueprint Instance Users

https://blueprints.launchpad.net/heat/+spec/instance-users

Problem Statement

We have a need to enable API authentication from within instances launched as part of a Heat stack. The instances, once launched, are not directly controlled by heat, so we have to treat them as being implicitly untrusted - we need to limit privileges of whatever credentials are deployed inside the instance to the absolute minimum possible that allow authentication with our API, ideally also reinforced via policy RBAC to minimise the API surface these credentials can access.

The key requirement is that we should be able to validate the identity of the agent connecting to the API, and map that to a specific Heat resource

The lifecycle of the credentials should be manageable on a per-resource basis
Some use-cases may require impersonation of the stack owning user (e.g posting metrics to ceilometer?_
Many other use cases do not require impersonation (e.g signals for wait condition notification, where we can map the identity inside heat)

What we do now

Currently, if we create any of the following resources, we create a new keystone user, in the same tenant as the user creating the stack, and assign that user to the "heat_stack_user" role, which is restricted to specific API actions via policy.json.

AWS::AutoScaling::ScalingPolicy
AWS::CloudFormation::WaitConditionHandle
OS::Heat::HARestarter
AWS::IAM::User

The problem with this is that every user creating a heat stack containing one of the following resources needs to be a keystone admin, otherwise they can't create the users. This is a big problem for "real world" deployments of Heat, where most users are not expected to have keystone administrative privileges.

Additionally, the reliance on the ec2tokens keystone functionality is problematic for some users, as their environments do not enable this keystone extension.

Proposed Solutions

1. Use credentials associated with a trust

Create one dedicated "heat_proxy_user" in a dedicated project, which has no roles assigned in any project
When we create a heat stack, we create a trust between the user creating the stack, and the heat_proxy_user, enabling impersonation and delegating only the "heat_stack_user" role, which can be used to restrict access via policy.json. The stack owning user is not restricted because we could use a rule like "if heat_stack_user and not Member"
Each resource which requires credentials creates an ec2 keypair using a trust-scoped token, meaning the resulting credentials are associated with the trust_id, so they can be used to perform role-limited impersonation of the stack-owning user

Pros: Solves the admin requirement, could work with the v2 keystone API (so backportable, possibly, to Havana)

Cons: Doesn't solve the dependence on ec2tokens, arguably complex, security implications of having shared proxy user (despite impersonation, because...bugs happen)

2. Use Domain separation and username/password

This suggestion was made by ayoung (initially in the context of using trusts, but I'm not sure we actually need to do impersonation in all cases)

Create a keystone domain specific to heat
For each heat stack, we create a new project in the heat domain
For each resource requiring credentials, we create a keystone user, in the stack project of the heat domain
The user has a randomly created password, which we can deploy in the instance
The user can be assigned the heat_stack_user role to limit the API surface accessible
Optionally, we could create a trust between the stack owning user and the heat "resource user", where impersonation is required (still not sure we actually need it in this case)

Pros: Solves the admin requirement (keystone user creation is performed by the heat service admin user), relatively simple, password auth solves dependency on ec2tokens auth.

Cons: Requires v3 keystone API (probably can't backport to Heat stable/havana)

Additional info/ideas:

Could add a "deploy_credentials" property to OS::Nova::Server, so the username/password are provided to the instance via nova metadata key/value pairs
We could use python-heaclient as the agent which talks to heat, e.g heat stack-signal mystack --data "complete!"
Could still derive an ec2-keypair from the heat-domain-resource-user to support compatibility with current heat-cfntools/boto
heat-cfntools could potentially be ported to use the ReST API and python-heaclient's python bindings instead of boto

3. Use OAuth access/secret keys and signed requests

OAuth has been mentioned as a possible solution to this problem, but currently it looks like there is too much missing from the keystone implementation to consider it a viable option, in particular the client support, request signing/validation and auth middleware do not yet exist.

4. x509 certificates

TODO: ayoung mentioned this as a possible solution, need more info

5. Resource-scoped keypairs

TODO: stevebaker