Skip to content
AI360Xpert
Playgrounds
Visuals
Concepts
Learning Paths
Interview Questions
Papers
Explore
Hire Us
Learning Paths
Ai Research Engineer
A comprehensive guide for Ai Research Engineer
""
1
PREREQUISITES: MATHEMATICS & COMPUTER SCIENCE FOUNDATIONS
2
CORE MACHINE LEARNING & THEORY
3
DEEP LEARNING FOUNDATIONS
4
ADVANCED AI TOPICS & CUTTING-EDGE RESEARCH
5
AI ENGINEERING & INFRASTRUCTURE
6
THE RESEARCH LIFECYCLE
Ai Research Engineer
Linear Algebra
Vectors, Matrices,
Tensors, and Vector
Spaces
Eigenvalues,
Eigenvectors, and
Matrix Decompositions
SVD
PCA
Cholesky
Matrix Calculus and
Differentiating
through Linear Algebra
Operations
Calculus & Optimization
Multivariate Calculus
Partial Derivatives
Gradients
Jacobians
Hessians
Gradient Descent and
its Variants
SGD
Momentum
Adam
AdamW
Sophia
Constrained
Optimization, Lagrange
Multipliers, and
Convex Optimization
Probability & Statistics
Probability
Distributions, Bayes'
Theorem, Expectations,
and Moments
Statistical
Significance,
Hypothesis Testing,
and Maximum Likelihood
Estimation
MLE
Information Theory
Entropy
Cross-Entropy
KL Divergence
Mutual Information
Computer Science Fundamentals
Data Structures and
Algorithms
Trees
Graphs
Dynamic Programming
Computational
Complexity and Big O
Notation
Parallel Computing,
Concurrency, and
Distributed Systems
Basics
Supervised & Unsupervised Learning
Linear & Logistic
Regression, Support
Vector Machines
SVMs
Ensemble Methods
Random Forests
Gradient Boosting -
XGBoost
LightGBM
Clustering (K-Means,
DBSCAN) and
Dimensionality
Reduction
PCA
t-SNE
UMAP
Theoretical Machine Learning
PAC Learning, VC
Dimension, and
Statistical Learning
Theory
Bias-Variance
Tradeoff,
Generalization Bounds,
and Overfitting
Empirical Risk
Minimization and
Regularization
Strategies
Neural Networks Basics
Perceptrons, MLPs, and
Modern Activation
Functions
GELU
Swish
SiLU
Forward Propagation,
Backpropagation, and
Computational Graphs
Initialization
Strategies and
Normalization Layers
Batch
Layer
Group
RMSNorm
Deep Learning Theory
The Neural Tangent
Kernel (NTK) and
Infinite-Width
Networks
Grokking, Double
Descent, and the
Lottery Ticket
Hypothesis
Optimization
Landscapes and Loss
Surface Geometry
Computer Vision (CV)
Convolutional Neural
Networks (CNNs) and
Modern Architectures
ResNet
ConvNeXt
Object Detection,
Segmentation, and
Dense Prediction Tasks
3D Vision, Neural
Radiance Fields
(NeRFs), and 3D
Gaussian Splatting
Natural Language Processing (NLP)
Tokenization, Subword
Algorithms (BPE,
WordPiece), and
Vocabulary Design
Word Embeddings, RNNs,
and the Transition to
Attention
The Transformer
Architecture
Self-Attention
Positional Encodings
KV Cache
Graph and Geometric Deep Learning
Graph Convolutional
Networks (GCNs) and
Message Passing
Graph Attention
Networks (GATs) and
Transformers on Graphs
Equivariant Neural
Networks and
Physics-Informed ML
Data Curation and Synthetic Data
Pre-training Data
Pipelines
Web Crawling
Deduplication
Quality Filtering
PII Removal
Synthetic Data
Generation and
Self-Play
Self-Instruct
Rejection Sampling
Data Contamination,
Leakage Detection, and
De-duplication against
Benchmarks
Large Language Models (LLMs)
Pre-training
Objectives, Scaling
Laws, and
Compute-Optimal
Training
Chinchilla
Fine-tuning (SFT),
Instruction Tuning,
and Prompt Engineering
In-Context Learning,
Chain of Thought
(CoT), and Reasoning
Capabilities
Agents, Retrieval, and Tool Use
Vector Databases,
Embeddings, and
Advanced RAG
Chunking
Re-ranking
Tool Calling, Function
Calling, and API
Integration
Agentic Workflows
(ReAct, Reflection,
Planning) and
Multi-Agent Systems
Reinforcement Learning (RL)
Markov Decision
Processes (MDPs),
Q-Learning, and Policy
Gradients
Deep Reinforcement
Learning
DQN
PPO
SAC
Offline RL,
Multi-Agent RL, and
Hierarchical RL
Alignment and Post-Training
Reinforcement Learning
from Human Feedback
(RLHF) and RLAIF
Direct Preference
Optimization (DPO,
KTO) and Reward
Modeling
AI Safety, Mechanistic
Interpretability, and
Red Teaming
Generative Models
Variational
Autoencoders (VAEs)
and Generative
Adversarial Networks
GANs
Diffusion Models
DDPMs
SDE-based formulations
Latent Diffusion
Flow Matching,
Continuous Normalizing
Flows, and
Autoregressive
Generation
Self-Supervised and Multimodal Learning
Contrastive Learning
(SimCLR, CLIP) and
Masked Modeling
MAE
Vision-Language Models
(VLMs) and Cross-Modal
Attention
Any-to-Any
Multimodality
Audio
Video
Text
Image Fusion
Emerging Architectures and Methods
State Space Models
(Mamba, Jamba) and
Linear RNNs
Mixture of Experts
(MoE) and Routing
Algorithms
Test-Time Compute
(System 2 Thinking,
Search) and Process
Reward Models
PRMs
Rigorous Evaluation and Benchmarking
Static Benchmarks
(MMLU, GSM8K,
SWE-bench) and their
Limitations
Automated Evaluation
and LLM-as-a-Judge
LMSYS Chatbot Arena
Evaluating Open-Ended
Generation, Tool Use,
and Long-Context
Capabilities
Deep Learning Frameworks
PyTorch Mastery
Autograd
Custom `nn.Module`s
`torch.compile`
Hooks
JAX Ecosystem
Functional ML
vmap
pmap
jit
Flax/Haiku
Distributed Training and High-Performance Computing
Data Parallelism
(FSDP) and Model
Parallelism
Tensor/Pipeline/Sequen
ce
Parallelism
GPU/TPU Architecture,
CUDA/C++ Basics, and
Custom Triton Kernels
Large-Scale
Infrastructure
Megatron-LM
DeepSpeed
Slurm
Memory Optimization and Efficient Inference
Parameter-Efficient
Fine-Tuning
LoRA
QLoRA
Adapters
Memory Optimizations
Mixed Precision
ZeRO
Activation
Checkpointing
Efficient Inference
FlashAttention
PagedAttention
Continuous Batching
vLLM
Model Compression
Quantization - AWQ
GPTQ
Pruning
Knowledge Distillation
MLOps and Experiment Tracking
Experiment Tracking
Weights & Biases
MLflow
TensorBoard
Model Versioning,
Artifact Management,
and Checkpointing
CI/CD for Machine
Learning and
Continuous Training
Pipelines
Reading, Analyzing, and Synthesizing Papers
Literature Review
Methodologies and
Keeping up with arXiv
Reproducing Results
and Implementing
Papers from Scratch
Critical Analysis,
Identifying Gaps, and
Formulating Novel
Hypotheses
Designing Experiments and Prototyping
Managing Compute
Budgets and Designing
Scaling Experiments
Ablation Studies,
Empirical Rigor, and
Statistical
Significance of
Results
Rapid Prototyping,
Iterative Debugging,
and Managing Technical
Debt
Scientific Communication and Open Source
Writing Conference
Papers
NeurIPS
ICLR
ICML Formatting
Guidelines
Creating Effective
Visualizations and
Communicating Complex
Ideas
Open Sourcing Code,
Releasing Models
(Hugging Face), and
Dissemination