gradient_descent_adaptive_stepsize()

template<typename Value , typename FunctionEval , typename GradientEval , typename IsImprovement >

auto syten::GradientOpt::gradient_descent_adaptive_stepsize	(	Value	x,
		FunctionEval	func,
		GradientEval	grad,
		IsImprovement	improv,
		Index	num_steps
	)

Gradient descent with adaptive stepsize.

On each gradient update, it checks if the current stepsize leads to an improvement, if not, it reduces the step-size by a factor of 2. It exits if no improvement is possible.

Template Parameters

Value	Domain value type
FunctionEval	function evaluation type, takes a Value object and returns an object of (unnamed) type X
GradientEval	gradient evaluation type, takes a type X and the associated Value object and returns a type X
IsImprovement	given two type X object, should return true if the second object is better than the first.

The domain value type must support SDef * Value → Value, Value + Value → Value and Value += Value.

Parameters

x	initial value
func	function object used to evaluate the function at a specific point
grad	gradient object used to evaluate the gradient at a specific function value and point
improv	improvement test object used to check whether the second passed value is an improvement over the first one
num_steps	maximal number of steps to take

Returns

a tuple of the final domain value object and associated image value object.

Remarks

This function requires an improvement to be possible at every step. Sometimes this is not achieved in the first few steps of a gradient descent, but only once an appropriate region has been reached. In such cases, it may be better to first use gradient_descent and only after a few steps call gradient_descent_adaptive_stepsize.

References std::make_tuple(), std::move(), and SYTEN_SMALL_THRESHOLD.

Here is the call graph for this function: