Cheat sheetCore Ml Cheat SheetA comprehensive guide for Core Ml Cheat Sheet🧠 Core ML Cheat Sheet — Complete Cheat Sheet1. Problem FramingTopicDescriptionSupervised learningLearn a mapping from features `X` to labeled targets `y`.\n\n- **Regression:** target is continuous.\n- **Classification:** target is categorical.\n- **Ranking:** learn an ordering or relevance score.Unsupervised learningFind structure without labeled targets.\n\n- **Clustering:** group similar observations.\n- **Dimensionality reduction:** compress features while preserving useful structure.\n- **Density estimation:** model how likely observations are under a distribution.Semi-supervised learningCombine a smaller labeled set with a larger unlabeled set; useful when labels are expensive.Self-supervised learningCreate training targets from the input itself, such as masked-token prediction; central to modern representation learning.Train/validation/test split- **Train:** fit parameters.\n- **Validation:** choose hyperparameters, thresholds, or model variants.\n- **Test:** final unbiased estimate; touch only after model decisions are frozen.2. Bias, Variance, and GeneralizationTopicDescriptionBiasSystematic error from an overly restrictive model; high bias usually means underfitting.VarianceSensitivity to the particular training sample; high variance usually means overfitting.Irreducible noiseError that cannot be removed from the chosen input/target formulation.Expected prediction errorA useful conceptual decomposition is: `Expected error ≈ Bias² + Variance + Noise`.Overfitting controlsUse regularization, simpler models, more data, early stopping, feature selection, augmentation, or cross-validation.3. Linear RegressionTopicDescriptionModel`ŷ = Xw + b`.ObjectiveOrdinary least squares minimizes: `MSE = (1/n) Σ(yᵢ - ŷᵢ)²`.Closed-form solutionFor appropriately conditioned full-rank data: `w = (XᵀX)⁻¹Xᵀy`.Gradient descentUpdate `w ← w - η ∇L(w)` until convergence.AssumptionsLinearity, independent errors, appropriate error variance, and no problematic multicollinearity; normality is mainly relevant for exact small-sample inference.When to useStrong baseline for continuous targets when relationships are approximately linear and interpretability matters.4. Regularized Linear ModelsTopicDescriptionRidge regressionObjective: `MSE + λ||w||₂²`.\n\n- Shrinks coefficients toward zero.\n- Handles correlated features better than plain OLS.\n- Usually keeps all features nonzero.Lasso regressionObjective: `MSE + λ||w||₁`.\n\n- Encourages sparse coefficients.\n- Useful for feature selection.\n- Can behave unstably with highly correlated predictors.Elastic NetCombines L1 and L2 penalties: `MSE + λ[α||w||₁ + (1-α)||w||₂²]`.\n\n- Often preferred when many features are correlated.Regularization intuitionLarger `λ` means stronger shrinkage and typically lower variance but higher bias.5. Logistic RegressionTopicDescriptionModel`p(y=1|x) = σ(wᵀx+b)` where `σ(z)=1/(1+e⁻ᶻ)`.Log-odds`log(p/(1-p)) = wᵀx+b`.LossBinary cross-entropy: `- [y log p + (1-y) log(1-p)]`.Multiclass formUse softmax: `p_k = exp(z_k) / Σ_j exp(z_j)`.When to useExcellent interpretable baseline for classification, especially with sparse/high-dimensional features such as text.6. k-Nearest NeighborsTopicDescriptionIdeaPredict from the labels/values of the closest training examples.Key hyperparameter`k` controls smoothness.\n\n- Small `k`: low bias, high variance.\n- Large `k`: higher bias, lower variance.DistanceCommon choices include Euclidean, Manhattan, and cosine distance.Critical preprocessingScale numeric features because distance is sensitive to units.When to useSmall-to-medium datasets with meaningful local geometry; weak choice for very large datasets or high-dimensional raw features.7. Decision TreesTopicDescriptionSplit principleChoose a feature and threshold that best reduces node impurity.Gini impurity`Gini = 1 - Σ_k p_k²`.Entropy`H = -Σ_k p_k log₂ p_k`.Regression treesCommonly minimize squared error within leaves.StrengthsNonlinear, handles interactions, little preprocessing, and naturally models mixed feature effects.RisksDeep trees overfit and are unstable to small data changes.Controls`max_depth`, `min_samples_leaf`, pruning, `max_features`, and minimum impurity decrease.8. Random ForestTopicDescriptionIdeaTrain many randomized decision trees and average/vote their predictions.Why it worksBagging reduces variance; random feature selection decorrelates trees.StrengthsStrong tabular baseline, robust, little feature engineering, supports nonlinear interactions.WeaknessesLarger memory/model size; less interpretable than a single tree; poor extrapolation in regression.Feature importance cautionImpurity-based importance can be biased; permutation importance is often more reliable.9. Gradient BoostingTopicDescriptionCore ideaBuild weak learners sequentially so each new learner reduces the current model's loss.Generic update`F_m(x) = F_{m-1}(x) + η h_m(x)`.Important controlsNumber of trees, learning rate, tree depth, subsampling, and regularization.XGBoost / LightGBM / CatBoostAll are highly competitive boosting families with different engineering choices.\n\n- **XGBoost:** explicit regularization and mature ecosystem.\n- **LightGBM:** histogram/leaf-wise growth; often fast on large tabular data.\n- **CatBoost:** strong handling of categorical features and ordered boosting.When to useOften a first-choice class of models for structured/tabular prediction.10. Support Vector MachinesTopicDescriptionHard-margin ideaFind a separating hyperplane maximizing geometric margin.Soft-margin SVMTrade margin violations against model complexity using parameter `C`.\n\n- High `C`: penalize violations strongly.\n- Low `C`: tolerate more violations and regularize more.Kernel trickMap examples implicitly into richer feature spaces. Common kernels: linear, polynomial, RBF.RBF kernel`K(x,x') = exp(-γ||x-x'||²)`. High `γ` makes influence more local and can overfit.When to useMedium-sized datasets, high-dimensional features, and problems where margins are meaningful.11. Naive BayesTopicDescriptionPrincipleApply Bayes' rule with a conditional-independence assumption: `P(y|x) ∝ P(y) Π_i P(x_i|y)`.Variants- **Gaussian:** continuous features.\n- **Multinomial:** counts/frequencies.\n- **Bernoulli:** binary features.StrengthsVery fast, low data requirement, strong text baseline.WeaknessConditional independence is often unrealistic, so probability estimates may be imperfect even when classification is good.12. ClusteringTopicDescriptionk-MeansMinimize within-cluster squared distance: `Σ_i ||x_i - μ_{c_i}||²`.k-Means assumptionsClusters are roughly compact and spherical under the chosen feature scaling.Choosing kUse domain knowledge, elbow diagnostics, silhouette score, or downstream utility; do not treat a single heuristic as definitive.DBSCANDensity-based clustering using neighborhood radius `ε` and minimum samples. Good for arbitrary shapes and outlier detection; sensitive to scale and varying densities.Hierarchical clusteringBuild a tree of nested clusters; useful when a hierarchy itself is informative.13. Dimensionality ReductionTopicDescriptionPCAFind orthogonal directions maximizing variance.PCA covariance viewEigenvectors of the covariance matrix give principal directions; eigenvalues quantify captured variance.Explained variance ratio`λ_i / Σ_j λ_j`.Scaling before PCAUsually standardize when features use different units.t-SNE / UMAPPrimarily visualization/embedding tools; neighborhood structure can be informative, but global distances should not be overinterpreted.14. Ensemble MethodsTopicDescriptionBaggingParallel models trained on bootstrap samples; mainly reduces variance.BoostingSequential models correcting prior errors; often reduces bias and can also control variance with regularization.StackingUse predictions from base models as features for a meta-model; use out-of-fold predictions to prevent leakage.Voting / averagingCombine class probabilities or numeric predictions from diverse models.15. Feature EngineeringTopicDescriptionNumerical transformsLog, square root, ratios, clipping, winsorization, and interaction terms can expose useful structure.Categorical encodingOne-hot encoding is simple; target/mean encoding needs leakage-safe folds; hashing helps very high cardinality.Missing valuesImpute using training-only statistics; add missingness indicators when absence is informative.Scaling- Standardization: `(x-μ)/σ`.\n- Min-max: `(x-min)/(max-min)`.\n- Robust scaling: center/scale using medians and quantiles.Feature selectionFilter, wrapper, and embedded methods reduce noise, cost, and leakage risk.16. Class ImbalanceTopicDescriptionResamplingOversample minority, undersample majority, or use synthetic methods such as SMOTE.Class weightsIncrease loss contribution from rare classes.MetricsAccuracy can be misleading; prefer precision, recall, F1, PR-AUC, balanced accuracy, or cost-weighted utility.Threshold tuningClassification thresholds should be selected on validation data based on business cost, not assumed to be `0.5`.17. Cross-ValidationTopicDescriptionk-Fold CVSplit into `k` folds; train on `k-1`, validate on the remaining fold, and average scores.Stratified k-FoldPreserves class proportions for classification.Group k-FoldKeeps related entities together, preventing the same entity from appearing in both train and validation.Time-series splitRespect chronology; never train on future data to predict the past.Nested CVUse inner CV for tuning and outer CV for unbiased model evaluation.18. Leakage ChecklistTopicDescriptionTarget leakageA feature contains information created after the prediction point or derived from the target.Preprocessing leakageFit scalers, imputers, encoders, and selectors on training folds only.Temporal leakageUse only information available at prediction time.Duplicate leakageNear-duplicates across train/test can inflate performance.19. Model Selection at a GlanceSituationStrong starting pointSmall, interpretable regressionLinear/RidgeSparse high-dimensional classificationLogistic/Linear SVMTabular nonlinear dataGradient boostingFast robust tabular baselineRandom ForestStrong local structure, small datak-NNText count featuresNaive Bayes or linear modelsNeed nonlinear interactions + interpretabilityShallow tree / monotonic boostingClustering compact groupsk-MeansArbitrary-shape clusters + noiseDBSCAN20. Hyperparameter IntuitionTopicDescriptionLearning rateSmaller usually needs more iterations but can improve optimization stability.Model capacityDepth, number of parameters, tree count, and feature complexity increase capacity.RegularizationPenalties, subsampling, dropout-like controls, and early stopping reduce effective complexity.Search strategyRandom search often explores broad spaces more efficiently than naive grids; Bayesian optimization can focus trials using previous results.21. Final Selection ChecklistTopicDescriptionDefine the prediction unitSpecify exactly what one row/example represents.Define the prediction timestampDocument which information is legal at inference time.Pick a primary metricAlign with the real cost of false positives, false negatives, ranking errors, or numeric error.Establish a baselineCompare against simple heuristics and simple linear models.Validate realisticallyUse stratification, groups, or time-aware splits as required.Inspect errorsSlice performance by segment, class, geography, device, time, or other meaningful cohorts.Calibrate when probabilities matterA classifier can rank well yet output poorly calibrated probabilities.