SpeakScribe

Overview

SpeakScribe was developed at PennApps, the University of Pennsylvania's hackathon, with the goal of optimizing the learning and teaching process. For educators, the tool generates personalized assignments based on provided lesson plans, using GPT-3.5 Turbo to tailor activities to students' preferences and exporting them as PDFs. For students, with the teacher's permission, SpeakScribe records lectures, generates transcriptions, summaries, and key points, deepening their understanding of the content. The inspiration came from the need to process information more efficiently in the modern educational environment, especially with the growth of online learning. We wanted a solution that could convert speech to text and summarize visual data (like images into summaries, with future plans for charts) so students could focus on learning instead of manual note-taking. We used Python for the backend and React TypeScript for the frontend.

Key highlights

• Developed as a team during the PennApps hackathon at the University of Pennsylvania
• Creation of personalized assignments for students using GPT-3.5 Turbo
• Lecture recording, transcription, and summarization functionality for students
• Overcame challenges in installing Python packages and integrating pre-trained AI models
• Valuable learning about using API keys, handling real-time speech functions, and teamwork collaboration under pressure
• Future plans include expansion to more languages, summarization of long lectures, and reading charts/tables