-
GenAI – Techniques To Train The LLM On Domain Specific Data .
GenAI – Techniques To Train The LLM On Domain Specific Data . Table Of Contents: Prompt Engineering. Instruction Tuning. Retrieval Augmented Generation(RAG) Parameter Efficient Finetuning(PEFT) Full Fine Tuning. Continued Pretraining(DAPT/TAPT). Reinforcement Learning. Knowledge Injection. (1) Prompt Engineering. (2) Instruction Tuning (3) RAG (Retrieval-Augmented Generation) (4) Parameter Efficient Fine-Tuning. (5) Full Fine Tuning (6) Continued Pretraining(DAPT/TAPT). (7) Reinforcement Learning (8) Knowledge Injection
-
GenAI – What Is Hallucination In GenAI ?
GenAI – What Is Hallucination In GenAI ? Table Of Contents: What Is Hallucination In GenAI ? Examples Of GenAI Model Hallucination . Why Does Hallucination Happen ? How To Fix Hallucination ? (1) What Is Hallucination In GenAI ? (2) Examples Of GenAI Model Hallucination. (3) Why Does Hallucination Happen ? (4) How To Fix Hallucination In GenAI ?
-
GenAI – How To Train LLM On Domain Specific Data ?
GenAI – Why We Need To Train Our LLM Model On Domain Specific Data ? Table Of Contents: Why We Need To Train Our LLM Model On Domain Specific Data ? What Is Hallucination In LLM ? What Techniques Are There To Train The LLM Model On Domain Specific Data ? What Is Fine Tuning LLM Model ? What Is RAG Based Training LLM Model ? (1) Why We Need To Train Our LLM Model On Domain Specific Data ?
-
GenAI – Steps Involved Building LLM Model.
GenAI – Steps Involved Building LLM Models Table Of Contents: Define The Problem Statement . Select The LLM . Prepare the Data (Optional, if fine-tuning or custom RAG) . Design the Interaction (Prompt Engineering) . (Optional) Fine-tune / Adapt the Model . Set Up the Backend (Model Serving) . Build the Frontend (User Interface) . Integrate Memory, Tools, or Search (Advanced Features) . Test and Evaluate the Application . Deploy to Production . (1) Define The Problem Statement (2) Select The LLM (3) Prepare the Data (Optional, if fine-tuning or custom RAG) (4) Design the Interaction (Prompt Engineering) (5) (Optional)
-
GenAI – Introduction To Generative AI.
GenAI – GenAI Roadmap For Beginners. Table Of Contents: What Is Generative AI? Examples Of Generative AI. Generative AI Tools. Best Open-Source Text Generation Models Best Closed-Source (Proprietary) Text Generation Models (1) What Is Generative AI ? (2) Examples Of Generative AI. (3) Generative AI Tools. (4) Best Open-Source Text Generation Models (5) Best Closed-Source (Proprietary) Text Generation Models
-
PySpark – PySpark Graphs
PySpark – PySpark GraphX / GraphFrames Table Of Contents: What is a Graph in PySpark? Example Of PySpark Graph. Why Use Graphs in PySpark? Where Does The Pyspark Graph Is Used In Real Life? (1) What is a Graph in PySpark? (2) Example Of PySpark Graph. from graphframes import GraphFrame g = GraphFrame(vertices, edges) (3) Why Use Graphs in PySpark? (4) PySpark Real Life Examples
-
PySpark – PySpark Streaming
PySpark – PySpark Streaming Table Of Contents: What is Spark Streaming? What is Structured Streaming? Key Concepts Example Code Spark Streaming vs. Structured Streaming Use Cases (1) What Is Spark Streaming ? (2) What Is Structured Streaming ? (3) Key Concepts: (4) Example Code from pyspark.sql import SparkSession spark = SparkSession.builder.appName("StructuredStreamingExample").getOrCreate() # Read stream from a socket source df = spark.readStream.format("socket").option("host", "localhost").option("port", 9999).load() # Word count logic words = df.selectExpr("explode(split(value, ' ')) as word") word_counts = words.groupBy("word").count() # Write the results to the console query = word_counts.writeStream.outputMode("complete").format("console").start() query.awaitTermination() (5) Spark Streaming vs. Structured Streaming (6) Use Cases
-
PySpark – PySpark MLlib
PySpark – PySpark MLLib Table Of Contents: What is PySpark MLlib? Two APIs in MLlib Why Use MLlib? Key Features Example ML Pipeline (End-to-End) Commonly Used Classes When to Use PySpark MLlib? (1) What Is PySpark MLLib? (2) Two APIs in MLlib (3) Why Use MLlib? (4) Key Features (5) What is a PySpark Pipeline? model.transform(data) model = pipeline.fit(data) from pyspark.ml import Pipeline from pyspark.ml.feature import StringIndexer, VectorAssembler from pyspark.ml.classification import LogisticRegression # Step 1: Convert label to numeric indexer = StringIndexer(inputCol="purchased", outputCol="label") # Step 2: Assemble features assembler = VectorAssembler(inputCols=["age", "salary"], outputCol="features") # Step 3: Model lr = LogisticRegression(featuresCol="features",
-
PySpark – PySpark SQL
PySpark – PySpark SQL Table Of Contents: What is PySpark SQL? Why Use PySpark SQL? Setting It Up (Step-by-Step) SQL vs DataFrame APIs (Both Supported!) Advanced Features in PySpark SQL Input Data Formats Performance Optimizations Real-World Use Cases Summary (1) What is PySpark SQL? (2) Why Use PySpark SQL? (3) Setting It Up (Step-by-Step) Step 1: Create a SparkSession from pyspark.sql import SparkSession spark = SparkSession.builder .appName("PySparkSQLDemo") .getOrCreate() Step 2: Load Data into a DataFrame df = spark.read.csv("employees.csv", header=True, inferSchema=True) df.show() Step 3: Register DataFrame as SQL Table (Temp View) df.createOrReplaceTempView("employees") Step 4: Run SQL Queries! result = spark.sql(""" SELECT
-
PySpark – DataFrames
PySpark – Dataframes Table Of Contents: What Is PySpark DataFrames. Why Use DataFrames In PySpark? How To Create DataFrames In PySSark? Common DataFrame Operations. Lazy Evaluation. Under the Hood: Catalyst & Tungsten (1) What Is PySpark DataFrames. (2) Why Use DataFrames in PySpark? (3) How to Create a DataFrame? From A List: from pyspark.sql import SparkSession spark = SparkSession.builder.appName("Example").getOrCreate() data = [("Alice", 30), ("Bob", 25), ("Charlie", 35)] columns = ["Name", "Age"] df = spark.createDataFrame(data, columns) df.show() From A CSV File: df = spark.read.csv("employees.csv", header=True, inferSchema=True) df.show() (4) Common DataFrame Operations Filtering Rows: df.filter(df.Age > 30).show() Selecting Columns: df.select("Name").show() Group and
