Evaluating Large Language Models on Computer

trish-goza
Published  · 15 slides · 0 views
↓ Download
Evaluating Large Language Models on Computer
1 / 1
Evaluating Large Language Models on Computer - slide 1 of 15 Evaluating Large Language Models on Computer - slide 2 of 15 Evaluating Large Language Models on Computer - slide 3 of 15 Evaluating Large Language Models on Computer - slide 4 of 15 Evaluating Large Language Models on Computer - slide 5 of 15 Evaluating Large Language Models on Computer - slide 6 of 15 Evaluating Large Language Models on Computer - slide 7 of 15 Evaluating Large Language Models on Computer - slide 8 of 15 Evaluating Large Language Models on Computer - slide 9 of 15 Evaluating Large Language Models on Computer - slide 10 of 15 Evaluating Large Language Models on Computer - slide 11 of 15 Evaluating Large Language Models on Computer - slide 12 of 15 Evaluating Large Language Models on Computer - slide 13 of 15 Evaluating Large Language Models on Computer - slide 14 of 15 Evaluating Large Language Models on Computer - slide 15 of 15
Description
Evaluating Large Language Models on Computer Science Education Using University Exam Questions Edan Gabay, Yael Maoz, Jonathan Stahl, Naama Maoz, Abdo Amer, Orr Eilat, Adi Haviv, Hanoch Levy, Michal Kleinbort, Amir Rubinstein Tel Aviv

Related Topics

Download this presentation From Below

"Evaluating Large Language Models on Computer" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

01
Evaluating Large Language Models on Computer Science Education Using University Exam Questions Edan Gabay, Yael Maoz, Jonathan Stahl, Naama Maoz, Abdo Amer, Orr Eilat, Adi Haviv, Hanoch Levy, Michal Kleinbort, Amir Rubinstein
Tel Aviv University<br>
02
TL;DR We built a dataset of theoretical CS questions from TAU exams to test LLMs, focusing on problems that require algorithmic and logical reasoning, not just coding.
We evaluated the performance of Claude 3.5 and GPT-4o on the dataset
LLMs scored ~55% - not enough to pass as a student <br>
03
About the team This work was carried out by a team of undergrad students as part of a workshop in the Computer Science B.Sc. Program at TAU.
Supervised and lead by a teaching assistant pursuing her Ph.D. and three faculty members from the Computer Science Department Edan Gabay, Yael Maoz, Jonathan Stahl, Naama Maoz, Abdo Amer, Orr Eilat, Adi Haviv, Hanoch Levy, Michal Kleinbort, Amir Rubinstein
Tel Aviv University<br>