First Year Project Presentation Submitted by TARZ Try Before You Buy : AI-Powered Fashion Decision-Support Platform Faculty Mentor: Dr. Ravinder Beniwal Project Overview AI fashion decision-support platform that answers one checkout
"First Year Project Presentation Submitted by TARZ" is the property of its rightful owner. Permission is granted to
download and print the materials on this website for personal, non-commercial use only, and to display it
on your personal computer provided you do not modify the materials and that you retain all copyright
notices contained in the materials. By downloading content from our website, you accept the terms of this
agreement.
Presentation Transcript
01
First Year Project Presentation
Submitted by TARZ
Try Before You Buy : AI-Powered Fashion Decision-Support Platform Faculty Mentor: Dr. Ravinder Beniwal<br>
02
Project Overview AI fashion decision-support platform that answers one checkout question – “Is this garment suitable for me?” – using the shopper’s body shape, skin undertone, size, style and occasion.
Body Analysis Agent reads a single photo (MediaPipe Pose + OpenCV) or manual measurements to infer body shape, skin undertone, bust/hip estimate and a recommended size with a fit-confidence value.
Smart Catalog ranks 28 garments with a transparent weighted score over size, shape, tone, style and occasion; every term is shown to the user.
Virtual Try-On chains Kolors, IDM-VTON and Replicate diffusion services with bounded time-outs to preview the garment on the user’s own photo.
Honest Stylist Agent – the server computes the verdict; the LLM (Groq gpt-oss-120b, Gemini fallback) only explains it from evidence and may suggest only server-provided alternatives.<br>
03
Specific Objectives To build an end-to-end decision-support system that tells a shopper whether a garment suits them, instead of showing a generic “Recommended for You” list.
To extract a body profile from a single photo – body shape, skin undertone, approximate bust/hip and recommended size – using MediaPipe and OpenCV, with a manual-measurement fallback.
To design an explainable weighted score S(p) = Σ w·s over size, shape, tone, style and occasion, whose terms are visible to the user.
To integrate diffusion-based virtual try-on with bounded time-outs and fallback services, so that a slow or failed backend never blocks the user.
To ground the LLM stylist – the server decides the verdict, the LLM only explains it and cannot invent products (ID guard + template fallback).<br>
04
Key Features AI Body Analysis – 33 MediaPipe landmarks from one photo give body shape (5 classes), skin undertone, bust/hip estimate and size S–XXL with a fit confidence in [0, 1]
Explainable Smart Catalog – 5-term weighted score: size 30%, tone 25%, shape 20%, style 15%, occasion 10%; verdict is suitable at 0.70 or above, partly suitable at 0.45 or above
Virtual Try-On – garment letterboxed to 768×1024 and sent to Kolors, then IDM-VTON, then Replicate (30 s connect + 90 s predict limit per stage)
Honest Stylist Agent – JSON evidence record in, explanation out; invented product IDs are dropped, and a template fallback never contradicts the verdict
Privacy & Honesty – “Delete my data” removes the profile and photo; photo-based values are flagged as approximate; API keys stay on the server<br>
05
Project Usecases & Scope Five-Screen MVP – Landing, Analyze (photo or manual measurements), Smart Catalog with filters, Try-On, and Stylist; no cart, payment or admin panel
Primary Use Case: Fit Confidence at Checkout – upload a photo, get body shape, size and fit confidence, then browse a catalog scored for you
Virtual Try-On for Visualization – preview the selected garment on your own photo before deciding to buy
Explainable Styling Support – the Stylist gives reasons tied to named score terms and alternatives only from the server’s candidate list
Scope – 28 garments (20 women’s, 8 men’s, 9 categories) including kurtis, sarees and suits; evaluated on synthetic profiles, with no user study yet<br>
06
Prototype/Project Flow diagram/Architecture<br>
07
Data & Resources Catalog Data – 28 curated garments with price, MRP, rating, colour family, silhouette, style and occasion tags (catalog.json)
User Data – photo and profile (shape, undertone, measurements, size) kept in SQLite and local uploads; deletable through the API; optional Cloudinary mirror for try-on results
Pre-Trained Models – MediaPipe Pose, OpenCV, Kolors and IDM-VTON via Hugging Face Spaces (Replicate fallback), Groq gpt-oss and Gemini 2.5 Flash for the stylist; no custom training
Software Stack – React + Vite (frontend), FastAPI + Python 3.12 (backend), SQLite (database), pytest (9 automated tests)
Deployment – Vercel (frontend) and Render (backend); scoring runs on a single CPU core with no special hardware<br>
08
Methodology Pipeline Approach – Input → Body Analysis → Smart Catalog → Try-On → Stylist; each stage is independent and has its own fallback
Three-Tier Architecture – React/Vite client → FastAPI REST endpoints (/analyze, /recommend, /tryon, /stylist, /catalog, /profile) → SQLite + external AI services
Body Analysis Agent – landmark widths scaled by the user’s height give chest and hip circumference; shape from ratios; size by minimum mean distance to the size-chart bands, with fit confidence c
Scoring & Verdict – S(p) = Σ w·s(p) computed on the server; the LLM never changes the verdict
Grounded Stylist & Fallbacks – evidence-only prompt and candidate-ID whitelist; Groq → Gemini → template for advice; Kolors → IDM-VTON → Replicate for try-on<br>
09
Results & Impact Tested Core – all 9 automated tests pass; analyse → recommend → explain takes about 9.5 ms (median) without external calls
Hallucination Guard – in a simulation of 1,000 LLM answers, none of 1,200 invented product IDs got through
Size Robustness – the recommended size stayed the same in 91%, 76%, 56% and 41% of trials under 2%, 3%, 5% and 8% photo scale error, so photo sizes are flagged as approximate
Explainability Insight – removing tone drops the top-5 overlap to 48%, while removing size flips 42.7% of verdicts: nominal weight is not effective influence
Impact – an auditable, honest blueprint for explainable fashion AI, reusable for e-commerce integration<br>
10
Limitations & Future Work No User Study – evaluation uses 90 synthetic profiles and 28 garments, so no claims about recommendation quality or user satisfaction
Uncalibrated Geometry – single frontal 2-D photo, hand-set constants, generic size charts, and no waist estimate from photos
Skin-Tone Heuristics – hand-set thresholds not yet validated across skin tones; a manual override is provided
Privacy & Security – photos are unencrypted until deleted, try-on images are not removed by “Delete my data”, and the API has no rate limiting
Future Work – calibrate on tape-measured samples, run a user study, validate skin-tone logic, grow the catalog, add authentication and encryption<br>
