Scheduler/NormalizedWeights - Revision history

Aloga at 16:26, 9 December 2013

2013-12-09T16:26:32Z

Aloga at 16:15, 9 December 2013

2013-12-09T16:15:22Z

Aloga at 13:06, 10 June 2013

2013-06-10T13:06:22Z

Aloga at 11:15, 10 June 2013

2013-06-10T11:15:41Z

Aloga: Aloga moved page Scheduler/NormalizedWeigts to Scheduler/NormalizedWeights

2013-06-10T11:13:10Z

Aloga moved page Scheduler/NormalizedWeigts to Scheduler/NormalizedWeights

Aloga at 08:58, 5 June 2013

2013-06-05T08:58:54Z

Aloga at 08:58, 5 June 2013

2013-06-05T08:58:41Z

Aloga at 08:57, 5 June 2013

2013-06-05T08:57:57Z

Aloga: Created page with " TOC == Overview == Currently the weighter is using the raw values instead of normalizing them. This makes difficult to properly use multipliers for establishing the rel..."

2013-05-23T09:21:12Z

Created page with " __TOC__ == Overview == Currently the weighter is using the raw values instead of normalizing them. This makes difficult to properly use multipliers for establishing the rel..."

New page

__TOC__

== Overview ==

Currently the weighter is using the raw values instead of normalizing
them. This makes difficult to properly use multipliers for establishing
the relative importance between two weighter (one big magnitude could
shade a smaller one).

This blueprint aims to introduce weight normalization so that we can apply
multipliers easily. The commit does not change the behavior per-se
since currently we only have 1 weighter. All the weights will be
normalized between 0.0 and 1.0, so that the final weight for a host
will be.

weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...

Furthermore, two kinds of normalization will be provided.

* In the first case the list will be normalized using its lowest and highest value.
* In the second case, the weighter should be able to specify the upper and lower values.

Comment from Lindgren:

<blockquote>
If I have a weigher for a resource that ranges between 1 and 100 and normalize the weights of two objects, the normalization will produce equal results for weights that differ on the scale (as long as the same object has the highest weight).

For weights [0, 1], this will result in normalized values [0.0, 1.0]. For weights [99, 100] this will still result in the same [0.0, 1.0].

Lets assume I have two objects and two weighers. Both weighers produce values in the same range (this could possibly be already normalized values like utilization of resources which range from 0.0 to 1.0). The weighers give weights [0.1, 0.2] and [1.0, 0.1] which without further normalization gives final weights [1.1, 0.3]. With the current way of calculating normalization, the result will instead be [1.0, 1.0] which has an effect on the ordering of the objects.
</blockquote>

← Older revision		Revision as of 16:26, 9 December 2013
Line 59:		Line 59:
	that it can really influence the final weight. However, if we introduce some		that it can really influence the final weight. However, if we introduce some
	nodes with a lager RAM, we should need to reconfigure it.		nodes with a lager RAM, we should need to reconfigure it.
		+
		+	== Upgrade Notes ==
		+
		+	* https://review.openstack.org/#/c/27160/
		+	* Weights are normalized, so there is no need to inflate multipliers
		+	artificially. The maximum weight that a weigher will put for a node is 1.0
		+	and the minimum is 0.0.
		+	* nova.cells.weights.weight_offset.WeightOffsetWeigher introduces a new
		+	configuration option 'offset_weight_multiplier'
		+	* https://review.openstack.org/#/c/36417/
		+	* Introduce stacking flags for weighers. Negative multipliers should not be
		+	using for stacking, but the weighers are still compatible (the issue a
		+	deprecation warning message).
		+	* https://review.openstack.org/#/c/55873/
		+	* The mute_weight_value has been removed so the operators need
		+	to adjust the mute_weight_multiplier to a proper value if they weren't
		+	using the defaults. If they're using the defaults, no action is needed,
		+	since the multiplier has been adjusted.
		+	* https://review.openstack.org/#/c/44255/
		+	* New flag that allows an operator to weigh the nodes based on the % of RAM.

@@ Line 10: / Line 10: @@
 instead of normalizing them. This makes difficult to properly use multipliers
 for establishing the relative importance between two weighters (one big magnitude
-could shade a smaller one).
+will shade a smaller one).
 For example, a weigher that is returning 0 and 1 as values will need a multiplier
-that is big enough to be taking into account with regard to the RAM weigher, that
+that is big enough, so that it is taken into account with regard to the RAM weigher,
-would return higher values. Some workarounds are the usage of big enough
+that would return higher values (i.e. free RAM). The current workaround is the
-multipliers for each weigher, or using a large enough weight value. However,
+usage of big enough multipliers for each weigher, or using a large enough weight
-even if a system is properly configured, a RAM upgrade would imply a
+value. However, even if a system is properly configured, a RAM upgrade would
-review and reconfiguracion of the weighers.
+imply a review and reconfiguracion of the weighers (if the infrastructure is
 This blueprint aims to introduce weight normalization so that we can apply
@@ Line 26: / Line 27: @@
 This way it is easier for a resource provider to configure and establish the
-importance between the weighers.
+importance between the weighers, since the operator knows well in advance that
 Two kinds of normalization will be provided:
@@ Line 32: / Line 34: @@
 * If the weigher specifies the upper and lower values, the weighed objects will be normalized with regard to these values.
 * In the case that the weigher does not supply the lower and upper limits for the weighed objects, the maximum and minimum values from the weighed objects will be used.
 == Examples ==
@@ Line 39: / Line 46: @@
 Currently the RamWeigher returns the available RAM in MB as the weight for a
 node. Assume that we are implementing a weigher that will return 0 and 1
-values. If we have nodes with some GB of RAM, we should put a multiplier big
+values (for example true or false for a cached instance image). If we have
-enough to make the new weigher influence the final weight. If we upgrade the
+nodes with some GB of RAM, we should put a multiplier big enough to make the
-nodes to some hundreds of GB of RAM, we readapt also the weigher. If we have
+new weigher influence the final weight. If we upgrade the nodes to some hundreds
-an heterogeneous cluster, we should study carefully how we can make this
+of GB of RAM, we readapt also the weigher. If we have an heterogeneous cluster,
-weigher really count.
+we should study carefully how we can make this weigher really count.
 === Cells RamByInstanceTypeWeigher and MuteChildWeigher ===
@@ Line 52: / Line 59: @@
 that it can really influence the final weight. However, if we introduce some
 nodes with a lager RAM, we should need to reconfigure it.

@@ Line 4: / Line 4: @@
 == Overview ==
-Currently the scheduler is using the raw values directly from the available weighers,
+Weighs were originally used in the scheduler, but they are also used in the
-instead of normalizing them. This makes difficult to properly use multipliers for establishing
+cells code. This is still applicable to them also.
 the relative importance between two weighters (one big magnitude could
-shade a smaller one). For example, a weigher that is returning 0 and 1 as
+Currently we are using the raw values directly from the available weighers,
-values will need a multiplier that is big enough to be taking into account with
+instead of normalizing them. This makes difficult to properly use multipliers
-regard to the RAM weigher, that would return higher values.
+for establishing the relative importance between two weighters (one big magnitude
 This blueprint aims to introduce weight normalization so that we can apply
-multiple weighers easily. The commit does not change the behavior per-se
+multiple weighers easily. All the weights will be normalized between 0.0 and
-since currently we only have 1 weigher. All the weights will be
+.0, so that the final weight for a host will be as follows:
      weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...
@@ Line 28: / Line 34: @@
 == Examples ==
 == Comments ==
 See commends in https://review.openstack.org/#/c/27160/ :

@@ Line 4: / Line 4: @@
 == Overview ==
-Currently the weighter is using the raw values instead of normalizing
+Currently the scheduler is using the raw values directly from the available weighers,
-them. This makes difficult to properly use multipliers for establishing
+instead of normalizing them. This makes difficult to properly use multipliers for establishing
 the relative importance between two weighters (one big magnitude could
-shade a smaller one). For example, a weighter that is returning 0 and 1 as
+shade a smaller one). For example, a weigher that is returning 0 and 1 as
 values will need a multiplier that is big enough to be taking into account with
-regard to the RAM weighter, that returns higher values.
+regard to the RAM weigher, that would return higher values.
 This blueprint aims to introduce weight normalization so that we can apply
-multipliers easily. The commit does not change the behavior per-se
+multiple weighers easily. The commit does not change the behavior per-se
-since currently we only have 1 weighter. All the weights will be
+since currently we only have 1 weigher. All the weights will be
 normalized between 0.0 and 1.0, so that the final weight for a host
 will be as follows:
@@ Line 20: / Line 20: @@
 This way it is easier for a resource provider to configure and establish the
-importance between the weighters.
+importance between the weighers.
 Two kinds of normalization will be provided:
-* If the weighter specifies the upper and lower values, the weighted objects will be normalized with regard to these values.
+* If the weigher specifies the upper and lower values, the weighed objects will be normalized with regard to these values.
-* In the case that the weighter does not supply the lower and upper limits for the weighted objects, the maximum and minimum values from the weighted objects will be used.
+* In the case that the weigher does not supply the lower and upper limits for the weighed objects, the maximum and minimum values from the weighed objects will be used.
 == Examples ==
 == Comments ==
-Comment from Lindgren in https://review.openstack.org/#/c/27160/ :
+See commends in https://review.openstack.org/#/c/27160/ :

@@ Line 29: / Line 29: @@
 == Examples ==
-=== Comments ===
+== Comments ==
 Comment from Lindgren in https://review.openstack.org/#/c/27160/ :

← Older revision	Revision as of 11:13, 10 June 2013
(No difference)

Scheduler/NormalizedWeights - Revision history

Aloga at 16:26, 9 December 2013

Aloga at 16:15, 9 December 2013

Aloga at 13:06, 10 June 2013

Aloga at 11:15, 10 June 2013

Aloga: Aloga moved page Scheduler/NormalizedWeigts to Scheduler/NormalizedWeights

Aloga at 08:58, 5 June 2013

Aloga at 08:58, 5 June 2013

Aloga at 08:57, 5 June 2013

Aloga: Created page with " __TOC__ == Overview == Currently the weighter is using the raw values instead of normalizing them. This makes difficult to properly use multipliers for establishing the rel..."

Aloga: Created page with " TOC == Overview == Currently the weighter is using the raw values instead of normalizing them. This makes difficult to properly use multipliers for establishing the rel..."