Using Hyperparameter Tuning for the Model

Enabling HP Tuning

Hyperparameter Tuning is activated within the forecasting pipeline when at least one of the following configuration sections is defined:

hyperparameter_tuning_parameters

This section configures the specifics of the hyperparameter tuning process. By setting parameters here, you can customize how the tuning operates, including the number of iterations, the search method, and evaluation criteria.

tuning_search_params

This function defines the hyperparameter space that the model will explore during tuning. Specifying a range of possible values for each hyperparameter allows the model to test various combinations and determine which settings optimize performance.

Important Considerations for Time Series Data

If you plan to use cross-validation with the time series splitter, you must specify the date_col parameter. This ensures that data rows are ordered chronologically before they are split, which is crucial for maintaining the integrity of time-dependent data.

Default Parameters and Values

If tuning_search_params are not specified, HP Tuning will revert to exploring a default set of parameters and values that are tailored to each specific model. You can review these defaults on the models’ respective pages under Models. Additionally, if you set a starting value for any parameter under algorithm_parameters, it is essential to ensure that this value falls within the exploration range, even if it is the default range.

Sections

hyperparameter_tuning_parameters

Configures hyper parameters tuning.

tuning_search_params

Defines the Hyper parameters Space to explore

---

Example

hyperparameter_tuning_parameters:
scorer: filtered_neg_mean_absolute_percentage_error
scorer_args:
min_val: 5
cv: 5
tuning_search_params:
learning_rate:
  - 0.01
  - 0.03
  - 0.05
  - 0.1
gamma:
  - 0.1
  - 1.0
  - 10.0
min_child_weight:
  - 0.1
  - 1.0
  - 10.0
max_depth:
  - 1
  - 3
  - 5
n_estimators:
  - 1
  - 3
  - 5

---

Utilizing the SMAC Engine for Hyperparameter Tuning

The SMAC (Sequential Model-based Algorithm Configuration) engine offers an advanced alternative to traditional Grid Search methods, which systematically explore every specified combination of parameters. While thorough, Grid Search becomes impractically time-consuming as the number of potential combinations increases due to the growing parameter space.

Advantages of SMAC

SMAC improves efficiency by intelligently exploring the configuration space. Instead of testing all possible values, it uses a model-based approach that requires only the definition of a lower and an upper bound for each parameter. This method allows SMAC to focus on more promising areas of the parameter space, based on the performance outcomes of previous iterations.

Configuration with SMAC

• max_iter: This parameter specifies the maximum number of iterations that SMAC will execute. Setting this limit helps control the duration of the tuning process, making it adaptable to various computational time constraints.

By employing SMAC, you can achieve a more effective and time-efficient search for the optimal set of hyperparameters, especially beneficial when dealing with large datasets or complex models.

hyperparameter_tuning_parameters:
  scorer: filtered_neg_mean_absolute_percentage_error
  scorer_args:
    min_val: 5
  cv: 5
  engine: smac
  max_iter: 50
tuning_search_params:
  learning_rate:
  - 0.01
  - 10.0
  gamma:
  - 0.1
  - 10.0
  min_child_weight:
  - 0.1
  - 10.0
  max_depth:
  - 1
  - 5
  n_estimators:
  - 15
  - 500

To efficiently guide the exploration of the Configuration Space, you can specify initial values for certain parameters using the algorithm_parameters setting. This approach sets a baseline from which the exploration process begins, potentially accelerating convergence to optimal solutions by starting closer to an effective parameter configuration.

Example Configuration

For example, if you wish to initiate the parameter exploration with specific settings, you might specify:

• max_depth=5: This sets the maximum depth of the trees (for tree-based models).

• learning_rate=0.03: This establishes the step size at each iteration to minimize the loss function.

• max_delta_step=0: This parameter is used to limit the maximum delta step we allow each tree's weight estimation to be.

algorithm_parameters:
  max_depth: 5
  learning_rate: 0.03
  max_delta_step: 0

Please make sure that the starting parameters are consistent with the corresponding exploration range.