Geeky Codes | AI Engineering, Data Engineering & Machine Learning Tutorials
Learn how Spark stores intermediate DataFrames and RDDs in memory, disk, or both—and when persistence can dramatically improve performance. If you work with Apache Spark long enough, you’ll eventually encounter a situation like this: You have a DataFrame that takes several minutes to compute, and you use it multiple times in your pipeline. Without persistence, … Read more
Preparing for AI Engineer interviews requires understanding that roles differ significantly across companies. Interviews may focus on AI theory, system engineering, or LLM interactions. Candidates should analyze job descriptions, prepare specific questions, and structure responses around the role’s requirements. Emphasizing reasoning over memorization enhances interview readiness and relevance.
Learn how Spark DataFrames transform data lazily, how transformations build execution plans, and which operations can trigger expensive shuffles. Apache Spark Learning Path APACHE SPARK LEARNING PATH Where Are We Now? We’ve already learned how Spark reads data and how its execution engine works. Now we need to understand the most common thing we actually … Read more
Machine Learning is everywhere. When Netflix recommends a movie, your bank detects a suspicious transaction, Google ranks search results, or an e-commerce website recommends a product, Machine Learning may be working behind the scenes. But what exactly is Machine Learning? The simplest definition is: Machine Learning is a way of building systems that learn patterns … Read more
A practical guide to lexical + semantic retrieval, score fusion, reranking, Python implementation, and production RAG architecture Introduction A common mistake when building a RAG system is assuming that vector search is enough. You convert documents into embeddings, store them in a vector database, retrieve the top-k chunks, and send them to the LLM. It … Read more
After interviewing 50+ AI Engineer candidates, I noticed a pattern: impressive GenAI projects can get you through the first 10 minutes—but strong fundamentals are what separate candidates who build AI systems from those who truly understand them. I’ve interviewed 50+ AI Engineer candidates over the past few months. Almost every resume looked impressive. RAG. AI … Read more
A practical framework for turning ambiguous business problems into production-ready ML solutions Part 1: AI Engineer Interview Questions Part 2: AI Engineer Interview QuestionsPart 3: AI Engineer Interview QuestionsPart 4: AI Engineer Interview Questions Part 5: AI Engineer Interview Questions I recently came across an AI Engineer interview question that sounded deceptively simple: “Your movie is releasing … Read more
RAG, LLMs, Agentic AI, LangGraph, and Production GenAI Part 1: AI Engineer Interview Questions Part 2: AI Engineer Interview QuestionsPart 3: AI Engineer Interview QuestionsPart 4: AI Engineer Interview Questions Part 5: AI Engineer Interview Questions AI Engineer interviews are no longer limited to questions like: “What is an embedding?” or: “What is RAG?” Interviewers increasingly want … Read more
Before self-attention changed AI forever, recurrent neural networks tried to solve sequence modeling. Here’s why they eventually hit a wall. This is Part 1 of a 5-part series on Transformers. 📚 Blog Series Every Revolution Starts With a Problem In 2017, Google published a paper that fundamentally changed artificial intelligence. Attention Is All You Need. … Read more
Understanding Nested Parallelism,FastAPI, OpenMP Thread Contention, and the Right Way to Deploy CPU-Bound Machine Learning Models Imagine This… You have trained a Random Forest classifier using Scikit-Learn. Everything works perfectly during development. You deploy it behind FastAPI, add a bit of asynchronous programming using run_in_executor(), and your API happily serves requests. Then someone runs a … Read more
PyTorch CNN Shape Mismatch Error is one of the most common and frustrating issues faced by machine learning engineers and deep learning beginners. If you’ve encountered the dreaded: error while training or evaluating a Convolutional Neural Network (CNN), you’re not alone. The good news is that this error is usually easy to diagnose once you … Read more
This concluding section wraps up critical concepts regarding Auto Loader scalability, DLT pipeline modes, Databricks SQL caching, Workflow concurrency, and the key differences between ingestion methods. Let’s bring it home! 🚀 Part 36: Ingestion Scalability Question 51: Auto Loader at Massive Scale A data engineering team is using Auto Loader in Directory Listing mode to … Read more
In this section, we are focusing on advanced Delta Lake optimization, streaming-static joins, Unity Catalog security features, and Databricks SQL alerting. 🛡️ Part 26: Unity Catalog Row & Column Security Question 41: Dynamic Data Masking A company policy dictates that only members of the hr_team group are allowed to see the actual values in the … Read more
In this section, we dive deep into Change Data Capture (CDC) with Delta Live Tables, Spark memory management, Databricks Workflows routing, and secure data sharing. Let’s keep crushing it! 🔄 Part 16: Delta Live Tables (DLT) & CDC Question 31: Handling Change Data Capture (CDC) A data engineer is building a Delta Live Tables (DLT) … Read more
In this section, we tackle advanced ingestion with Auto Loader, Spark internals, Delta Live Tables (DLT) architectures, and Databricks Workflows. Let’s keep the momentum going! 📥 Part 10: Advanced Auto Loader & Streaming Question 21: Handling Evolving Schemas A data engineer is using Auto Loader to ingest JSON files. The upstream team frequently adds new … Read more
🌊 Part 5: Delta Lake Mechanics Question 11: Accidental Data Deletion A junior data engineer accidentally ran a DELETE statement that removed millions of valid rows from a Delta table named sales_prod exactly two hours ago. The table has not been VACUUMed. Which Databricks SQL command should you use to recover the data? ✅ Correct … Read more
Preparing for a Databricks Data Engineering certification? Mastering the concepts is only half the battle; you also need to know how to navigate tricky multiple-choice scenarios. In this post, we’ll break down 10 real-world exam-style questions covering PySpark, Unity Catalog, Databricks Workflows, and Performance Optimization. More importantly, we won’t just give you the answers—we’ll explain … Read more
Large Language Models (LLMs) are pretrained on vast datasets to understand language and various concepts. Enterprises often fine-tune these models for specific domains. Techniques like full fine-tuning and Parameter-Efficient Fine-Tuning (PEFT) optimize model adaptation while minimizing costs and risks like catastrophic forgetting. Understanding these strategies is essential for AI applications.
“Our RAG chatbot worked perfectly in the POC.But once we scaled to 50,000 documents… accuracy dropped to 60%.” If you’ve worked with enterprise RAG systems, you’ve probably heard this story. And if you ask most engineers what went wrong, you’ll hear answers like: ❌ These sound smart❌ They sometimes help❌ But they miss the real … Read more
This blog explores how to measure ROI for Generative AI (GenAI) in healthcare. It outlines key performance indicators (KPIs) related to clinical outcomes, operational efficiency, finance, and patient experience. It emphasizes the importance of clear objectives, baseline measurements, and best practices to successfully assess GenAI’s impact on healthcare organizations.