The OpenStack vulnerability management team (VMT) is responsible for coordinating the progressive disclosure of a vulnerability.
Members of the team are independent and security-minded folks who ensure that vulnerabilities are dealt with in a timely manner and that downstream stakeholders are notified in a coordinated and fair manner. Where a member of the team is employed by a downstream stakeholder, the member does not give their employer prior notice of any vulnerabilities. In order to reduce the disclosure of vulnerability in the early stages, membership of this team is intentionally limited to a maximum of 3 people.
The Vulnerability Management team provides patches fixing vulnerabilities for the two previous releases of OpenStack.
Each security bug is assigned a VMT coordinator (member from the vulnerability management team) that will drive the fixing and disclosure process. Here are the steps we follow.
A report can be received either as a private encrypted email to one of the VMT members, or as a Launchpad security bug (check the box marked "this is a security issue").
The first steps are to confirm the sanity of the report, create a Launchpad bug, and subscribe the project PTL for confirmation of impact.
Fix development and approval
The reporter, or the PTL, or any person that the PTL deems necessary to develop the fix is added to the security bug subscription list. A fix is proposed as a patch to the current master branch and attached to the bug.
Once that's done, core developers of the project are added to the bug subscription list so that the proposed patch can be pre-approved for merging. Patches need to be pre-approved so that they can be fast-tracked through review at disclosure time.
In the mean time, the VMT coordinator prepares a vulnerability description that will be communicated to downstream stakeholders, and will serve as the basis for the Security Advisory that will be finally published.
The description should properly credit the reporter and accurately describe impact and mitigation mechanisms. The description is validated by the reporter and the PTL.
To ensure full traceability, it is good to get a CVE assigned before the issue is communicated to a larger public. Once the description is approved, it is sent to a CNA in an encrypted+signed email in order to get a CVE assigned.
Downstream stakeholders warning
Once the fix and the description are approved, and the CVE is assigned, a signed email with the vulnerability description is sent to the downstream stakeholders. The disclosure date is set to 3-5 business days, excluding Monday/Friday and holiday periods, at 1500 UTC. No stakeholder is supposed to deploy public patches before disclosure date.
Make sure you have a core developer and a stable maintainer available to push the fix at disclosure time.
On the disclosure hour, push patch to Gerrit for review on master and supported stable branches, open bug, fast-track approvals (referencing the bug). Shortly thereafter (potentially waiting for the first test runs to complete), publish the advisory to the OpenStack ML.
Extent of disclosure
The science of vulnerability management is somewhere around being able to assess impact and severity of a report, being able to design security patches, being an obsessive process-following perfectionist and respecting the rule of lesser disclosure.
Lesser disclosure is about disclosing the vulnerability details to an increasing number of people over time, but only to the people that are necessary to reach the next step. here is a map of how the vulnerability will be disclosed:
- Receive encrypted email from original reporter: Reporter, Coordinator
- Create security-restricted Launchpad bug entry: Reporter, vuln-mgmt team
- Warn PTL of affected project to confirm impact: Reporter, vuln-mgmt team, PTL
- Develop fix: Reporter, vuln-mgmt team, PTL + key devs
- Get fix pre-approved by Core team: Reporter, vuln-mgmt team, core devs
- Send issue/fix to downstream stakeholders, get CVE: Reporter, vuln-mgmt team, core devs, downstream stakeholders
- Push fix to Gerrit, open bug: World (active)
- Issue advisory: World (passive)
Vulnerability reporters retain final control over the disclosure of their findings. If for some reason they are uncomfortable with our process, their choice of disclosure terms prevails.
OpenStack as an upstream project is used in a number of distributions, products, private and public service offerings that are negatively affected by vulnerabilities. In the spirit of responsible disclosure, this ecosystem, collectively known as the downstream stakeholders, needs to be warned in advance to be able to prepare patches and roll them out in a coordinated fashion on disclosure day. The embargo period is kept voluntarily small (3-5 business days), as a middle ground between keeping the vulnerability under cover for too long and not giving a chance to downstream stakeholders to react.
If you're currently not a referenced stakeholder and think you should definitely be included on that email distribution list, please submit an email with a rationale to member(s) of the VMT.
Each incoming vulnerability will be classified into one of three categories, each triggering a different workflow.
|Critical||Directly-exploitable vulnerability, allowing the attacker to escalate rights, take down VMs...|
|Normal||Indirect vulnerability that could result in security issue under certain circumstances.|
|Low||Non-exploitable vulnerabilities due to architectural choices that could be improved|
Template for impact description ($DESCRIPTION)
Title: $TITLE Reporter: $CREDIT Products: $PROJECT Affects: All versions Description: $CREDIT reported a vulnerability in... By doing... a... may... resulting in... Only setups.... are affected.
Template for vulnerability description email
- To: Downstream stakeholders
- Subject: Vulnerability in OpenStack $PROJECT ($CVE)
This is an advance warning of a vulnerability discovered in OpenStack, to give you, as downstream stakeholders, a chance to coordinate the release of fixes and reduce the vulnerability window. Please treat the following information as confidential until the proposed public disclosure date. $DESCRIPTION Proposed patch: See attached diff. This proposed patch will be merged to Nova master and stable/diablo branches on public disclosure date. CVE: $CVE Proposed public disclosure date/time: $DISCLOSURE, 1500UTC Please do not make the issue public (or release public patches) before this coordinated embargo date. Regards, -- $VMT_COORDINATOR_NAME OpenStack Vulnerability Management Team
Proposed patch is attached, email must be GPG-signed.
Template for OpenStack Security Advisories
- To: email@example.com, firstname.lastname@example.org
- Subject: [OSSA $NUM] $TITLE ($CVE)
OpenStack Security Advisory: $NUM CVE: $CVE Date: December 13, 2011 $DESCRIPTION Fixes: Folsom: $GITHUB_MASTER_COMMIT 2012.1: $GITHUB_ESSEX_COMMIT 2011.3: $GITHUB_DIABLO_COMMIT References: http://cve.mitre.org/cgi-bin/cvename.cgi?name=$CVE https://bugs.launchpad.net/nova/+bug/$BUG Notes: This fix will be included in the $MILESTONE development milestone and in a future $NEXTSTABLE release. -- $VMT_COORDINATOR_NAME OpenStack Vulnerability Management Team
- Email must be GPG-signed.
- $CVE must always be of the form CVE-YYYY-XXXX
- $NUM is of the form YYYY-XX