Need: Most of the statistical/Machine Learning techniques requires the normality assumption for significance testing as well as for modeling, but in the real time scenario most of the times the data distribution doesn’t satisfy the normality conditions.
Solution
Transformation is one method of converting, non- normal distribution into normal distribution, but what kind of transformation we need to apply on the original (non-normal) distribution is always remails question.
A power transform will make the probability distribution of a variable more Gaussian. This is often described as removing a skew in the distribution, although more generally is described as stabilizing the variance of the distribution.in these transforms, we will find a parameter (lambda or λ) that best transforms a variable to a Gaussian probability distribution.
The examples are
- Box-Cox Transform
- Yeo-Johnson Transform
Below are some common values for lambda
- λ = −1.0 is a reciprocal transform.
- λ = −0.5 is a reciprocal square root transform.
- λ = 0.0 is a log transform.
- λ = 0.5 is a square root transform.
- λ = 1.0 is no transform.
The difference between Box-cox and Yeo-Johnson is Yeo-Johnson does not require the values for each input variable to be strictly positive like Box-Cox. It supports zero values and negative values.
This means we can apply it to our dataset without scaling it first.
For code please check the below GitHub link
https://github.com/drstatsvenu/Tranform-distributions
https://github.com/rajanalavrao/transformation_normal
