Breaking News: Grepper is joining You.com. Read the official announcement!

hyperparameters

Akshay R answered on February 22, 2022 Popularity 10/10 Helpfulness 2/10

answer hyperparameters

related hyperparameter tuning

hyperparameters

Comment

def build_model(hp):
  model = keras.Sequential()
  model.add(keras.layers.Dense(
      hp.Choice('units', [8, 16, 32]),
      activation='relu'))
  model.add(keras.layers.Dense(1, activation='relu'))
  model.compile(loss='mse')
  return model

Popularity 10/10 Helpfulness 2/10 Language python

Source: keras.io

Tags: hyperparameters python

Link to this answer
Share Copy Link

Contributed on Feb 22 2022

Akshay R

0 Answers Avg Quality 2/10

Closely Related Answers

hyperparameter tuning

Comment

- Finding optimal combination of parameters for a model
- parameters: 
    - the ones that is set by the model after learning from dataset 
    - eg: co-efficients of linear regression, node decision by the decision trees
    - accessible by attribute (in the attribute section in the documentation)
- hyperparameters : 
    - the ones that we have the option to set before creating the model
    - print the estimator to see what it contains
    - accessible by parameter (in the parameter section in the documentation)
- Silly things to do (some examples):
    - Creating a random forest with just 2 or 3 trees
    - 1,2 neighbors in knn algorithm
    - increasing a hyperparameter by a small amount
    - Be aware of conflicting hyperparameter choices (The 'newton-cg', 'sag' and 'lbfgs' solvers support only l2 penalties.)
- Visualize if the hyperparameter has any effect:
    - Graph of learning curve : hyperparameter on X-axis and accuracy on Y-axis
- Problem: So many models can be build. But among these, find an optimal model that yields optimal result.
- Solution: Train with a set of adjustable parameters and compare the results to find the optimal model
- Rule of thumb : Cross validation is used to estimate the generalization performance.
- Curse of dimensionality : exhaustively searching results in exponential increase of dimensions with the increase of grid.
- Best practice : Do this when you really need optimal solution since it does not make a bad model into a good model.
- optimal hyperparameters = set of hyperparameters corresponding to the best CV score.
- Some algorithms:
    - Uninformed Search:
        - Grid Search : 
            - Find result for all possible combination of parameters 
            - Guaranteed best result
            - time consuming process, resource intensive
        - Random Search : 
            - Randomly choose a number of combinations of given parameters 
            - A good result but may not be the absolute best
            - fast 
            - Idea : You are unlikely to keep completely missing the 'good area' for a long time when randomly picking new spots
    - Informed Search :
        - Coarse to Fine: (Hybrid of grid search and randomized search)
            1. Random search
            2. Find promising areas (Narrow Down)
            3. Grid search in the smaller area or skip for step 4
            4. Continue from step 1 until optimal score is obtained
            - Idea : Narrow down the optimal area. The best result will be in that area.
        - Bayesian Optimization :
            - Inferring the probability of best result by deducing the outcome of past results
            - eg: Given a certain event occurs due to another event, we get more clearer idea of the probability of certain outcome
            - Idea : Getting better as we get more evidence.
        - Genetic Algorithms :
            1. We can create some models (that have hyperparameter settings)
            2. We can pick the best (by our scoring function). These are the ones that 'survive'
            3. We can create new models that are similar to the best ones
            4. We add in some randomness so we don't reach a local optimum
            5. Repeat until we are happy!
            - Idea : With evolution in gene sequence combination of best genes in new generations, we can obtain the optimal being

Popularity 9/10 Helpfulness 3/10 Language whatever

Source: Grepper

Tags: whatever whateve

Link to this answer
Share Copy Link

Contributed on Jan 03 2024

Innocent Iguana

0 Answers Avg Quality 2/10

hyperparameters

Contents

More Related Answers

hyperparameters

Closely Related Answers

hyperparameter tuning

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.