63213_en-US

liliconcom

Fit Model Deep Dive Materials

Fit Model Deep Dive

More than you probably wanted to know about fitting fixed effects linear models with the Fit Model platform

 

• Platform for fitting univariate and multivariate general linear models.

• Univariate: One response. Several of the fitting personalities support simultaneous fitting of multiple univariate responses.
• Linear model: When the response is a linear function of the parameters
• Parameter: an unknown coefficient that quantifies the influence of an input by itself or in combination with other inputs.
• Fixed effect: parameters assumed to take a finite (usually small) number of values.
• Random effect: parameters taking values from a (typically normal) distribution.
• General: the model can be made linear using transformation or distributional assumptions.
• Error: From simple (univariate Gaussian) to complex (correlated multivariate).

Intro

Linear Model Examples

 

 

 

 

 

Platform Comparison

Standard Least Squares

• Fixed effects, simple random effects (blocking, plots, VCs)
• Univariate Gaussian error
• Typically of the form:

Standard Least Squares

Flow

• Fit a model (with or without variable selection)
• Check assumptions, fix if needed/possible
• Reduce
• Check assumptions, fix if needed/possible
• Deploy or reject

Standard Least Squares

Example: Big Class

• Fit a model:  
• Main effects, 2 factor interactions, quadratic (height²)
• Check assumptions: Residuals (Row Diagnostics) and Lack of Fit. You can also save residuals and plot/test them against categorical factor levels for homogeneous variance (homoscedasticity). You’re looking for non-random patterns and unusual data points. Don’t read too much into these plots since they are rough indicators.
• Reduce: Remove non significant and/or unimportant variables
• Recheck
• Deploy/Reject

How Input Modeling Type Affects Interpretation of Main Effect Estimates

• Continuous (no coding): change in response as input is changed one unit. The slope of the regression line associated with the input.
• Continuous (coding): the same as above when the input has been recoded using the formula
• This corresponds to a change in response going from the input midpoint to an extreme setting.
• Nominal: difference between the average for the level and the average of all level averages.
• Ordinal: change in response going from the previous to the current level.

How Input Modeling Type Affects Interpretation of Intercept

• Intercept: the value when all the inputs are at their zero setting. What zero is depends on modeling type:
• Continuous (no coding): zero is the value zero
• CC: the value of the center point, i.e., ½(max + min)
• N: because of how nominal inputs are parameterized, it is unlikely there is a level corresponding to 0.
• O: The lowest level

Accounting for Variability

Intro

• Two types:
• Assignable (systematic): can be attributed to changes made to the inputs. Also known as the signal.
• Random (noise): variability with no know cause (yet).
• 
•  When we fit a model to the data, we’ll put “hats” on the response and parameters to indicate the responses are expected values (fitted) and the parameters are estimates.
• 

Accounting for Variability

Statistical Tests

• A large number of statistical tests look at the ratio of the accountable variability to the noise. The assumption is that this variability is in actuality mislabeled noise. If this is true, the ratio should be about 1. The larger the ratio, the more likely the assumption is false.
• Given certain assumptions (e.g., the data is independent and identically distributed, or IID) the ratio follows a know statistical distribution. This distribution will depend on how variability is quantified.
• Given a know statistical distribution, a probabilistic statement can be made about how likely a specific value is.

 

Accounting for Variability

Main ANOVA

 

• Degrees of freedom: number of independent observations used to calculate the value of interest.
• Model:  MEs, one for each continuous effect, N Levels: 1 for each categorical effect. For 2FIs and higher, multiply MEs together.
• Total:  N observations: 1
• Error:  df_Total: df_Model

 

Accounting for Variability

Main ANOVA

• Mean square: SS/df
• F: ratio of mean squares (MS_Model/MS_Error )
• R²: ratio of SS_Model/SS_Total
• RMSE: square root of the MS_Error
• R²adj: 

Accounting for Variability

Lack of Fit

• Pure error: variability under constant conditions. Three things are required to estimate pure error:
• Replication
• You’re not using the same method for estimating model error and pure error
• There are df left over to estimate error (i.e., you’re not fitting a saturated model)
• Assumption: the model error (MS_error) is a good estimate of random error. If it is appreciably larger, the additional variability may be due to a poorly fitting model.
• Caution:  A large ratio could also be because the pure error is underestimated.

Accounting for Variability

Effects

• The statistical test for a given effect compares the size of the systematic variability that can be explained by the effect to the noise variability.
• Correlated effects share variability with each other. This shared variability can only be associated with the entire group and can’t be separated among the individual effects.
• One solution is to only look at the variability contributed by an effect after the influence of all other effects in the model has been taken into consideration. This is known as Type III SS.
• Coding and (not) centering polynomials changes the correlation structure.