Jump to: navigation, search

Difference between revisions of "SwiftQuotas"

 
m (Text replace - "SwiftSpec" to "SwiftSpec")
 
(8 intermediate revisions by 3 users not shown)
Line 1: Line 1:
__NOTOC__
+
 
* '''Launchpad Entry''': [[SwiftSpec]]:storage-quota
+
* '''Launchpad Entry''': SwiftSpec:storage-quotas
 
* '''Created''': 1 Mar 2012
 
* '''Created''': 1 Mar 2012
 
* '''Contributors''': Everett Toews
 
* '''Contributors''': Everett Toews
 +
 +
== Quick Note ==
 +
 +
I see some similar work at https://review.openstack.org/#/c/19758/ http://www.zmanda.com/blogs/?p=1043 and https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
 +
 +
It'd be good to get some collaboration going here.
  
 
== Summary ==
 
== Summary ==
  
In some deployment scenarios, such as a private cloud, the provider may want to limit the tenants (accounts)/users to a maximum allowable amount of storage via a quota.
+
In some deployment scenarios, such as a private cloud, the provider may want to limit the tenants (accounts) to a maximum allowable amount of storage via a quota.
  
 
== Release Note ==
 
== Release Note ==
  
A storage quota prevents accounts/users from using more than their allowed storage.
+
A storage quota prevents accounts from using more than their allowed storage.
  
A storage quota of 0 indicates unlimited storage for an account/user.
+
A storage quota of -1 indicates unlimited storage for an account (-1 also indicates unlimited in Nova quotas).
  
 
== Rationale ==
 
== Rationale ==
Line 18: Line 24:
 
This is necessary for providers that cannot simply add disk at will as storage usage grows. There are any number of factors that may prevent a provider from being able to easily add disk.
 
This is necessary for providers that cannot simply add disk at will as storage usage grows. There are any number of factors that may prevent a provider from being able to easily add disk.
  
== User stories ==
+
== Assumptions ==
  
Account: <quota>
+
The quota values are stored per account.
  User1: <quota> (<derived quota>)
+
All values are in bytes.
  User2: <quota> (<derived quota>)
 
  
'''1. User Quota Storage'''
+
== User stories ==
Account: 0
 
  User1: 0
 
  User2: 6
 
 
 
When the account has a zero value, the quota is determined strictly by the user value.
 
  
'''2. Evenly Distributed Storage'''
+
'''1. Account Unlimited Storage'''
Account: 6
+
* Account quota: -1
  User1: 0 (3)
+
* When the quota has a -1 value, storage is unlimited.
  User2: 0 (3)
 
  
When an account has a non-zero value and all users in that account have a zero value, the account storage is evenly distributed amongst the users.
+
'''2. Account No Storage'''
 +
* Account quota: 0
 +
* When the quota has a 0 value, storage is not allowed.
  
'''3. Evenly Distributed Remainder Storage'''
+
'''3. Account Limited Storage'''
Account: 6
+
* Account quota: >1
  User1: 0 (2)
+
* When the quota has a >1 value, storage is limited to that value.
  User2: 0 (2)
 
  User3: 2
 
  
When an account has a non-zero value and some (not all) users in that account have a zero value, the remainder of the account storage is evenly distributed amongst the users with a zero value.
+
'''4. Account goes over-quota'''
  
'''4. Sum Total Storage'''
+
Error code returned from middleware and the client is responsible for the actual notification (email/sms/whatever)
Account: 6
 
  User1: 4
 
  User2: 2
 
  
When the sum of all of the user values totals to the account value, the quota is determined strictly by the user value.
+
If an account goes over-quota, it will go into an Over-Quota Mode. Below are just examples, the middleware should be generic enough to be easily tweakable (pluggable?) to add different modes. The only mode to be implemented as part of this spec is the Read/Delete-only mode.
  
'''5. Sum Total Storage'''
+
Potential Over-Quota Modes:
Account: 6
+
* Read/Delete-only mode
  User1: 2
+
* Auto-increase quota
  User2: 2
+
* All stop
 +
* Specific rate-limiting (degraded service for an over quota account)
  
When the sum of all of the user values is less than the account value, the quota is determined strictly by the user value.
+
'''5. Account goes under-quota'''
 
+
* Business as usual
'''6. Over-Sum Total Storage'''
 
Account: 6
 
  User1: 4
 
  User2: 4
 
 
 
When the sum of all of the user values is greater than the account value, this is a counter example and should not be allowed.
 
 
 
'''Note:''' After writing out all of these use cases I could easily be convinced that really only cases 1 and 2 are necessary (at least to begin with).
 
 
 
== Assumptions ==
 
 
 
The quota values is stored per account and per user. All values are in GBs.
 
  
 
== Design ==
 
== Design ==
Line 79: Line 64:
 
== Implementation ==
 
== Implementation ==
  
TBD
+
# get auth token
 +
# get usage
 +
# get quota
 +
# filter in Swift middleware
 +
 
 +
Quotas could live outside of Swift and be part of Auth as they are account (tenant) level info.
 +
 
 +
Where does the usage data come from?
 +
* HEAD on the account (RFE: number of bytes transferred since XXX) HEADs are cheap and caching will keep this performant, returns billable bytes, not raw bytes
 +
* slogging (for rolled-up storage and bandwidth )
  
=== UI Changes ===
+
== UI Changes ==
  
The Swift CLI will need to be able to CRUD quotas for accounts and users.
+
The Swift CLI will need to be able to CRUD quotas for accounts.
 +
You should be able to view quota usage in Horizon.
  
=== Code Changes ===
+
== Code Changes ==
  
 
Code changes should include an overview of what needs to change, and in some cases even the specific details.
 
Code changes should include an overview of what needs to change, and in some cases even the specific details.
  
=== Migration ===
+
== Migration ==
  
 
Include:
 
Include:

Latest revision as of 23:31, 17 February 2013

  • Launchpad Entry: SwiftSpec:storage-quotas
  • Created: 1 Mar 2012
  • Contributors: Everett Toews

Quick Note

I see some similar work at https://review.openstack.org/#/c/19758/ http://www.zmanda.com/blogs/?p=1043 and https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota

It'd be good to get some collaboration going here.

Summary

In some deployment scenarios, such as a private cloud, the provider may want to limit the tenants (accounts) to a maximum allowable amount of storage via a quota.

Release Note

A storage quota prevents accounts from using more than their allowed storage.

A storage quota of -1 indicates unlimited storage for an account (-1 also indicates unlimited in Nova quotas).

Rationale

This is necessary for providers that cannot simply add disk at will as storage usage grows. There are any number of factors that may prevent a provider from being able to easily add disk.

Assumptions

The quota values are stored per account. All values are in bytes.

User stories

1. Account Unlimited Storage

  • Account quota: -1
  • When the quota has a -1 value, storage is unlimited.

2. Account No Storage

  • Account quota: 0
  • When the quota has a 0 value, storage is not allowed.

3. Account Limited Storage

  • Account quota: >1
  • When the quota has a >1 value, storage is limited to that value.

4. Account goes over-quota

Error code returned from middleware and the client is responsible for the actual notification (email/sms/whatever)

If an account goes over-quota, it will go into an Over-Quota Mode. Below are just examples, the middleware should be generic enough to be easily tweakable (pluggable?) to add different modes. The only mode to be implemented as part of this spec is the Read/Delete-only mode.

Potential Over-Quota Modes:

  • Read/Delete-only mode
  • Auto-increase quota
  • All stop
  • Specific rate-limiting (degraded service for an over quota account)

5. Account goes under-quota

  • Business as usual

Design

TBD

Implementation

  1. get auth token
  2. get usage
  3. get quota
  4. filter in Swift middleware

Quotas could live outside of Swift and be part of Auth as they are account (tenant) level info.

Where does the usage data come from?

  • HEAD on the account (RFE: number of bytes transferred since XXX) HEADs are cheap and caching will keep this performant, returns billable bytes, not raw bytes
  • slogging (for rolled-up storage and bandwidth )

UI Changes

The Swift CLI will need to be able to CRUD quotas for accounts. You should be able to view quota usage in Horizon.

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

  • data migration, if any
  • redirects from old URLs to new ones, if any
  • how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.