ISBMS · PGDM 2025–27 · SEMESTER III · SESSION 2

slide1. ISBMS · PGDM 2025–27 · SEMESTER III · SESSION 2 OF 10 Module 2
Prompt Engineering & RAG for Financial Analysis Tejas Jadhav, CFA, FRM
Faculty · Agentic AI & Advanced Analytics in Finance (PGDM-SEM3-SPEC-AIFINANCE)
tejasgjadhav.github.io/AIFINANCE TODAY 3 hours
Hour 1 prompts · Hour 2
RAG pipeline · Hour 3 lab Plain English
A prompt is written in English.
Hours 1 and 2 need no code. 1
Q&A BOT ON AN IPO PROSPECTUS
BUILT BEFORE YOU LEAVE ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 01<br>
slide2. THE PLAN What we do in these three hours HOUR 1 · 60 MIN
The prompts ▪ Zero-shot, few-shot, chain-of-thought ▪ Role prompting — “You are a CFO…” ▪ Output formats: JSON, markdown tables ▪ Hallucination — the risk and the fixes ▪ Pair activity: one query, four prompts HOUR 2 · 60 MIN
The pipeline ▪ Why a model cannot read 200 pages ▪ Chunking a financial document ▪ Embeddings and vector databases ▪ Retrieval, then re-ranking ▪ Self-consistency and tree-of-thought HOUR 3 · 60 MIN
The lab ▪ One NSE IPO prospectus as the source ▪ LlamaIndex + ChromaDB + Claude API ▪ Build the Q&A bot ▪ Ten investor questions, scored ▪ Case: SEBI DRHP analysis using RAG Our goal for today: a bot that answers investor questions from a 200-page IPO prospectus and shows the page each answer came from. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 02<br>
slide3. HOUR 1 · PROMPTING Three ways to ask: zero-shot, few-shot, chain-of-thought ZERO-SHOT Just ask. You state the task and nothing else. The model answers from its training alone. EXAMPLE
“Classify this cost as capex or opex: licence fee for the new core banking system.” Use it for routine tasks the model has seen a thousand times. FEW-SHOT Show it first. You give two or three solved examples, then the real task. The model copies the pattern. EXAMPLE
Show two costs already labelled capex or opex, then the third for the model to label the same way. Use it when the format must match yours exactly. CHAIN-OF-THOUGHT Ask for the steps. You ask it to reason step by step before answering. The steps become visible, so errors can be caught. EXAMPLE
“Compute the current ratio step by step, then end with one line stating the ratio.” Use it for arithmetic and multi-step logic. The order of use: start zero-shot. Add examples when the format matters. Ask for the steps when there is arithmetic. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 03<br>
slide4. HOUR 1 · PROMPTING Role prompting and output formatting ROLE PROMPTING “You are a CFO…” A role sets vocabulary, depth and audience in one line. The same question returns a different answer for a CFO, an auditor and a retail investor. EXAMPLE
“You are the CFO of a mid-size NBFC. Explain the Q1 margin drop to the board in five bullets, no jargon.” A role is context, not magic. It adds no facts the model does not have. OUTPUT FORMATTING Ask for the shape you will use. Prose is for reading. JSON and markdown tables are for code and spreadsheets. Fix the format and the same prompt runs every quarter. EXAMPLE
Return only: {"company": "", "revenue_cr": 0, "period": ""} — no other text. A fixed format is what lets Python validate the answer. The skeleton: state the role at the top, the task in the middle, the output format at the bottom. Every reusable prompt this semester follows it. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 04<br>
slide5. HOUR 1 · HALLUCINATION Hallucination: the risk, then the fixes THE RISK It fills the gap with plausible words. An LLM has no database of truth — Module 1 said it. It predicts the next word. When it lacks a fact, it can produce an EPS, a fine, a circular number that reads exactly like a real one. EXAMPLE
Asked for an EPS with no document attached, a model can return ₹42.18 in full confidence. The number does not exist. THE FOUR FIXES Ground it. Attach the filing, or retrieve it with RAG. The model answers from the page, not from memory. Allow “not found”. Tell it to say so when the source is silent. An allowed no beats a confident guess. Demand citations. Every number carries the page or clause it came from. Verify the numbers. Recompute in Python or Excel anything that leaves your desk. The desk rule: an unverified number never leaves your desk. Check it against the filing before anyone else sees it. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 05<br>
slide6. HOUR 1 · HALLUCINATION Hallucination: what it is, and its three forms WHAT IT IS A hallucination is a confident answer that is false. The model predicts the next plausible word. When it lacks a fact, it fills the gap and does not flag the guess. THE QUESTION IT GOT WRONG One wrong answer cost $100 billion. Google demoed its chatbot Bard in February 2023. The demo asked: “What new discoveries from the James Webb Space Telescope can I tell my nine-year-old about?”
Bard answered that the telescope took the very first picture of a planet outside our solar system. The first such picture was actually taken in 2004.
Alphabet’s shares fell about nine percent that day, roughly $100 billion of market value. THE THREE FORMS 1 · Fact hallucination.
It states a wrong fact with full confidence. Example: an EPS quoted to two decimals for a company that never published one. 2 · Source hallucination.
It invents the evidence. Example: a New York brief cited six court cases ChatGPT made up, and the court fined the lawyers $5,000 (Mata v. Avianca, 2023). 3 · Context hallucination.
It contradicts the document it was given. Example: Air Canada’s bot promised a refund its own policy pages never offered, and a tribunal made the airline pay (2024). How RAG grounds it: the bot answers only from the pages it retrieved, and every answer carries its page, so a wrong one is caught in one look. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 06<br>
slide7. HOUR 2 · RAG BASICS What is RAG? Retrieval-Augmented Generation THE FULL FORM RAG stands for Retrieval-Augmented Generation. It connects the model to documents it never saw in training, so it answers from your pages instead of its memory. R Retrieval The bot searches your document first and pulls out the passages that match the question. A Augmented Those passages are pasted into the prompt, next to the question, as context the model must use. G Generation The model writes the answer from those passages and cites the page each fact came from. THE CLASSROOM ANALOGY A plain chatbot sits a closed-book exam and answers from memory. A RAG bot sits an open-book exam: it finds the page first, answers from it, and writes the page number in the margin. Say it as one sentence: retrieve the pages, augment the prompt, generate the answer. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 07<br>
slide8. HOUR 2 · THE RAG PIPELINE RAG: the bot reads your document before it answers THE PROBLEM The model never read your prospectus — its training ended before the filing existed, and 200 pages do not fit a prompt comfortably. So we fetch the right pages first. 1 · DOCUMENT We start from one 200-page IPO prospectus (DRHP). 2 · CHUNK Split it into passages a few hundred tokens long. 3 · EMBED Each chunk becomes numbers that capture meaning — Sentence Transformers. 4 · STORE Chunks and embeddings go into a vector database — ChromaDB, Pinecone. 5 · RETRIEVE Your question is embedded too; the store returns the closest chunks. 6 · ANSWER The model writes from those chunks and cites the pages. Retrieval-Augmented Generation: retrieve the right pages first, then generate the answer from them. The R happens before the G. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 08<br>
slide9. HOUR 2 · QUALITY Four choices that decide whether the bot is right CHUNKING Cut by meaning, not by size. Split at section boundaries and keep every table whole. Overlap the edges so no sentence is cut in half. EMBEDDINGS Meaning becomes numbers. Close meanings land close together, so “revenue” finds “turnover” and “top line”. Sentence Transformers is the standard tool. RETRIEVE & RE-RANK Shortlist, then interview. The first search is quick and pulls about 20 chunks that look close. A slower second look keeps the best 5. The model reads only those 5. ASK MORE THAN ONCE Ask again, then vote. Self-consistency asks the same question three times and takes the most common answer. Tree-of-thought tries a few solution paths and keeps the best. Both cost tokens, so save them for hard questions. When the bot is wrong: look first at which chunks it was given. Most bad answers are retrieval failures, not model failures. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 09 EXAMPLE
Screen 100 CVs fast, shortlist 20, interview 5. Re-ranking is the interview. EXAMPLE
Three analysts compute one ratio. Two get 1.8, one gets 2.4. You go with 1.8.<br>
slide10. HOUR 3 · LAB The lab: a Q&A bot on a real IPO prospectus We build one bot on one prospectus and put ten questions to it. Everything runs in VS Code. THE BUILD 1 Download one NSE IPO prospectus (DRHP) from sebi.gov.in, about 200 pages. 2 LlamaIndex splits and embeds it; ChromaDB stores the chunks. 3 Claude answers each question from the retrieved chunks and cites the pages. TRY IT FREE AT HOME
No paid account is needed. Upload the same PDF to any free AI chat, ask the same ten questions, and compare its answers with the class bot’s cited pages. THE TEST ▪ Where does the fresh-issue money go?
▪ What is the promoter holding after the offer?
▪ Which three risk factors come first?
You write the other seven. Each answer is scored right, wrong or not found, and the cited page is checked in the PDF. INDUSTRY CASE · SEBI DRHP ANALYSIS USING RAG
Every DRHP is public on sebi.gov.in the day it is filed. The same pipeline gives an analyst a first pass in minutes, and the cited pages make it checkable. What you have at the end: a bot that answered ten investor questions from a document it had never seen, and a score sheet saying how often it was right. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 10<br>
slide11. PRACTICE QUESTIONS · PART A Six questions on today’s basics Q1 When do you move from zero-shot to few-shot, and what do the examples buy you?
TIP Move when the output format or the judgement must match yours. The examples teach the pattern. Q2 Why does “You are a CFO…” change the answer? Name the three things a role sets.
TIP Vocabulary, depth and audience. It adds no new facts. Q3 Why ask for JSON instead of prose when the output feeds a spreadsheet?
TIP A fixed format can be parsed and validated by code. Prose cannot. Q4 What is hallucination, and what are its three types?
TIP A confident answer that is false. Fact: a wrong number stated with confidence. Source: invented evidence. Context: it contradicts the document it was given. Q5 Why build RAG instead of pasting all 200 pages into the chat?
TIP Cost, context limits and focus. Retrieval sends only the few passages that matter. Q6 A search for “revenue” must also find “turnover”. Which stage makes that work?
TIP Embeddings. They map meaning to numbers, so synonyms land close together. The tip is the shape of the answer, not the whole answer. Two or three sentences each in the exam. MODULE 2 · 11<br>
slide12. PRACTICE QUESTIONS · PART A Q7 What is the CRAFT prompt technique? Name the three things a role sets.
TIP Context, Role, Action, Format, Tone. A role sets vocabulary, depth and audience, and it adds no new facts. Q8 How will you reduce hallucination? (This was asked in an Accenture strategy interview.)
TIP Give it the document (RAG), demand a page citation for every number, allow “I don't know”, fix the output format, and check every number with code. Q9 You are a research analyst valuing Zomato for a long-term horizon. Write the best prompt in CRAFT format.
TIP Context: Zomato, latest annual results, long-term view. Role: equity research analyst. Action: value it by DCF and peer multiples, list the risks. Format: one-page memo with a table. Tone: plain, board-ready. Q10 How will you build a RAG pipeline for the 550-page Shiprocket DRHP, which would cost over a million tokens to paste? List the steps.
TIP Load the PDF, chunk it by section, embed each chunk, store the vectors in a database, retrieve the top chunks for each question, and generate the answer with page citations. The tip is the shape of the answer, not the whole answer. Two or three sentences each in the exam. MODULE 2 · 12<br>
slide13. PRACTICE QUESTIONS · PART B Three cases from the desk Q11 The CFO wants last quarter’s segment results as a board-ready table, from the same prompt every quarter. Write the prompt’s three parts.
TIP State the role at the top, the task in the middle, and the exact table format at the bottom, with a “not found” rule for missing lines. Q12 Your prospectus bot answers a risk-factor question with text from the marketing section. Which stage failed, and what do you change?
TIP Retrieval, or the chunking before it. Chunk by section, keep headings with their text, and re-rank so the risk-factor chunks win. Q13 The bot returns a promoter stake the PDF never states. Which two controls stop that number reaching the investor?
TIP Require a page citation for every number, and instruct “not found” when the source is silent. A human checks the cited page before it goes out. Before Session 3: pick any live DRHP on sebi.gov.in, read its risk-factors section, and write the ten questions you would put to a bot about it. ISBMS · PGDM 2025–27 · AGENTIC AI & ADVANCED ANALYTICS IN FINANCE MODULE 2 · 13<br>