Skip to content
AI360Xpert
Learning Paths

Deep Learning Engineer

A comprehensive guide for Deep Learning Engineer

""1PREREQUISITES & FUNDAMENTALS2NEURAL NETWORK FOUNDATIONS3DEEP LEARNING FRAMEWORKS4CONVOLUTIONAL NEURAL NETWORKS (CNNS) & COMPUTER VISION5SEQUENCE MODELS & NATURAL LANGUAGE PROCESSING (NLP)6ADVANCED DEEP LEARNING ARCHITECTURES7OPTIMIZATION & DISTRIBUTED TRAINING8GENERATIVE AI & LARGE LANGUAGE MODELS (LLMS)9DEEP LEARNING IN PRODUCTION (MLOPS)10HARDWARE & DEPLOYMENT OPTIMIZATIONDeep Learning EngineerProgramming & Software EngineeringPython ProgrammingData StructuresOOPAsync/AwaitData ManipulationNumPyPandasPolarsVersion ControlGitGitHubGitLabSoftware DesignPatterns & Clean CodeApplied MathematicsLinear AlgebraVectorsMatricesEigenvaluesSVDCalculusDerivativesPartial DerivativesChain RuleProbability &StatisticsDistributionsBayes TheoremHypothesis TestingInformation TheoryEntropyCross-EntropyKL DivergenceMachine Learning BasicsSupervised vsUnsupervised LearningLinear & LogisticRegressionSupport VectorMachines & DecisionTreesEnsemble MethodsRandom ForestGradient BoostingXGBoostEvaluation MetricsAccuracyPrecisionRecallF1ROC-AUCArtificial Neural Networks (ANNs)Perceptrons &Multi-LayerPerceptronsMLPsActivation FunctionsSigmoidTanhReLULeakyReLUGELUSwishForward Propagation &BackpropagationTraining Neural NetworksLoss FunctionsMSECross-EntropyHuber LossFocal LossGradient Descent &VariantsSGDMini-batchNesterovRegularizationTechniquesL1L2DropoutEarly StoppingWeight DecayNormalizationBatch NormLayer NormInstance NormGroup NormWeight InitializationStrategiesXavier/GlorotHeKaimingPyTorchTensors and AutogradBuilding nn.ModuleclassesDataLoaders and CustomDatasetsPyTorch Lightning &Ecosystem BoilerplateReductionTensorFlow & KerasEager Execution vsGraph ModeKeras Sequential andFunctional APIstf.data API for DataPipelinesTensorBoard forVisualizationJAX & EcosystemJAX PrimitivesjitgradvmappmapNeural NetworkLibrariesFlaxHaikuOptax for OptimizationCNN ArchitecturesConvolutions, Pooling,and StridesClassic ArchitecturesLeNetAlexNetVGGAdvanced ArchitecturesResNetInceptionEfficientNetConvNeXtObject Detection & SegmentationTwo-stage DetectorsR-CNNFaster R-CNNOne-stage DetectorsYOLO seriesSSDSemantic SegmentationFCNU-NetDeepLabInstance & PanopticSegmentationMask R-CNNDETRAdvanced Vision TasksImage Generation &Style TransferVision TransformersViTSwin Transformer3D Computer VisionNeRF3D Gaussian SplattingPoint CloudsVideo Understanding &Action RecognitionTraditional Sequence ModelsRecurrent NeuralNetworksRNNsLong Short-Term Memory(LSTMs) & GatedRecurrent UnitsGRUsBidirectional RNNsWord Embeddings & RepresentationWord2Vec & GloVeFastText &Character-levelEmbeddingsContextual EmbeddingsELMoTokenizationAlgorithmsBPEWordPieceSentencePieceTiktokenAttention Mechanisms & TransformersSelf-Attention &Multi-Head AttentionThe TransformerArchitectureVaswani et al.Encoder-only ModelsBERTRoBERTaDeBERTaDecoder-only ModelsGPT seriesLLaMAMistralEncoder-Decoder ModelsT5BARTState Space Models (SSMs) & AlternativesLinear State SpaceModelsMambaS4Hybrid ArchitecturesJambaRWKV and LinearAttentionGenerative Adversarial Networks (GANs)Generator andDiscriminator NetworksDCGAN, WGAN, andStyleGANConditional GANscGANPix2PixCycleGANAutoencoders & DiffusionVanilla and SparseAutoencodersVariationalAutoencodersVAEsDenoising DiffusionProbabilistic ModelsDDPMsLatent DiffusionStable DiffusionFluxFlow Matching &Continuous-Time ModelsSelf-Supervised Learning (SSL)Contrastive LearningSimCLRMoCoBYOLMasked Image ModelingMAEBEiTSelf-DistillationDINODINOv2Reinforcement Learning (RL) & Deep RLMarkov DecisionProcesses (MDPs) &Q-LearningDeep Q-NetworksDQNPolicy Gradients &Actor-Critic MethodsPPOSACInverse RL & OfflineRLGraph Neural Networks (GNNs)Graph ConvolutionsGCNsMessage Passing NeuralNetworksMPNNsGraph AttentionNetworksGATsAdvanced OptimizersMomentum, RMSprop,AdaGradAdam, AdamW, and LAMBLearning RateSchedulersCosine AnnealingOne-Cycle PolicyWarmupTuning StrategiesGrid Search and RandomSearchBayesian OptimizationOptunaHyperoptPopulation BasedTrainingPBTDistributed TrainingData ParallelismDDPModel ParallelismTensorPipeline ParallelismFully Sharded DataParallelFSDPDeepSpeedGradient Checkpointing& Mixed PrecisionTrainingFP16BF16Ring Attention &Context ScalingLarge Language Model ArchitecturesScaling Laws &Emergent AbilitiesMixture of ExpertsMoEPrompt Engineering &In-Context LearningRetrieval-AugmentedGeneration (RAG) &Vector DatabasesAgentic LLM WorkflowsLangChainLlamaIndexReActFine-Tuning & AlignmentInstruction TuningSupervised Fine-TuningParameter-EfficientFine-TuningPEFTLoRAQLoRADoRAReinforcement Learningfrom Human FeedbackRLHFDirect PreferenceOptimizationDPOORPOKTOMultimodal ModelsContrastiveLanguage-ImagePretrainingCLIPSigLIPVisual QuestionAnsweringVQAVision-Language ModelsLLaVABLIPQwen-VLAudio & VideoGenerationSoraWaveNetVITSModel Serving & DeploymentREST APIsFastAPIFlaskgRPC and ProtocolBuffersDedicated ServersTorchServevLLMTriton InferenceServerContainerization & OrchestrationDocker &Containerization BestPracticesKubernetes (K8s) forML WorkloadsKubeflow & MLflow forLifecycle ManagementContinuous Integration & Data ManagementCI/CD for MachineLearningFeature StoresFeastHopsworksData Version ControlDVCMonitoring & MaintenanceModel Drift and DataDrift DetectionLogging and TelemetryPrometheusGrafanaELK StackA/B Testing, ShadowDeployments, CanaryDeploymentsHardware AcceleratorsGPUsCUDAcuDNNROCmTPUs & SpecializedASICsAWS InferentiaGoogle TPUNPUs and LPUsGroqNeural ProcessingUnitsEdge DevicesJetson NanoRaspberry PiModel Optimization TechniquesQuantizationPTQQAT8-bit/4-bitGPTQAWQGGUFPruningMagnitudeStructuredUnstructuredKnowledge DistillationAttentionOptimizationsFlashAttentionPagedAttentionSpeculative DecodingDeployment FrameworksONNX and ONNX RuntimeTensorRT for NVIDIAGPUsOpenVINO for IntelHardwareEdge & MobileFrameworksTFLiteCoreMLExecuTorch