Skip to content
All projects

Local Speech AI · Full-Stack Application / v0.5.0

AI Meeting Assistant

A local-first full-stack AI application that turns meeting and lecture recordings into transcripts, summaries, key points, action items, and searchable knowledge without sending meeting content to a cloud AI service.

Next.jsTypeScriptTailwind CSSPythonFastAPIfaster-whisperOllamaSQLitePydanticGitHub Actions

The problem

Meeting recordings can contain sensitive information, while cloud transcription and AI services introduce privacy and dependency concerns. I wanted to explore an end-to-end workflow where speech recognition, language-model processing, and meeting history stay on the machine running the backend.

My approach

I combined faster-whisper for multilingual speech-to-text with FastAPI orchestration, Ollama for structured analysis and transcript-grounded Q&A, SQLite persistence, and a Next.js interface. The Whisper model is cached per backend process and processing timings are exposed in the UI to make local-model latency visible.

Project overview

Audio or video recordings are uploaded to the FastAPI backend and stored locally under a generated meeting ID.

faster-whisper performs multilingual speech-to-text locally; the model is reused within the backend process to avoid repeated initialization.

The transcript is analyzed by a local Ollama model into a title, summary, key points, and action items, then persisted in SQLite.

Users can ask questions grounded in the stored transcript, revisit meeting history, export notes or transcripts, and delete local meeting data.

System workflow

01

Upload

Meeting or lecture recording → local media storage + meeting ID

02

Transcribe

faster-whisper → multilingual local transcript

03

Analyze

Transcript → Ollama → structured summary, key points, and action items

04

Ask

Stored transcript + question → grounded local answer

Engineering decisions

Local speech-to-text

The multilingual faster-whisper small model runs on CPU with int8 compute, VAD, and model reuse for a practical local transcription workflow.

Structured local analysis

Ollama converts transcript content into validated meeting fields while the backend normalizes model output before persistence.

Grounded Q&A

Questions are answered from the selected meeting transcript, with instructions to state when requested information is absent instead of inventing it.

Challenges and fixes

Local inference latency

Separated and measured transcription and analysis time, cached the Whisper model, and tuned the local workflow so performance bottlenecks are visible and can be optimized independently.

Unreliable structured LLM output

Improved backend parsing and normalization after malformed JSON responses during development so analysis failures do not silently corrupt meeting data.

Privacy boundary

Designed the application around local speech recognition, local LLM inference, SQLite, and local media/transcript storage rather than a hosted AI API.

Result and learning

Version 0.5.0 implements the complete upload-to-knowledge workflow: local transcription, structured AI analysis, persistent meeting history, grounded transcript Q&A, deletion controls, exports, and visible Whisper/AI timing. GitHub Actions verifies the Python backend and production Next.js build.

The key learning: a useful local meeting assistant is more than a transcription model. Speech recognition, robust structured LLM output, persistent history, grounded answers, and visible processing time have to work as one coherent system.

Scope and limitations

Explore the implementation

This case study is based on the project’s own documentation.

More projects

Let’s talk about AI and software development