Add files via upload
This commit is contained in:
m15-ai
GitHub
parent
b8ce4dac61
commit
adef09c069
333
README.md
Normal file
333
README.md
Normal file
@ -0,0 +1,333 @@
|
|||||||
|
# Local Voice Assistant (Offline, Real-Time AI)
|
||||||
|
|
||||||
|
**Lightweight, low-latency voice assistant running fully offline on a Raspberry Pi or Linux machine.**
|
||||||
|
Powered by PyAudio, Vosk STT, Piper TTS, and local LLMs via Ollama.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Features
|
||||||
|
|
||||||
|
- 🎙️ **Microphone Input** using PyAudio
|
||||||
|
- 🔊 **Real-Time Transcription** with [Vosk](https://alphacephei.com/vosk/)
|
||||||
|
- 🧠 **LLM-Powered Responses** using [Ollama](https://ollama.com) with models like `gemma2:2b`, `qwen2.5:0.5b`
|
||||||
|
- 🗣️ **Natural Voice Output** via [Piper TTS](https://github.com/rhasspy/piper)
|
||||||
|
- 🎛️ Optional **Noise & Filter FX** using SoX for realism
|
||||||
|
- 🔧 ALSA **Volume Control**
|
||||||
|
- 🧩 Modular Python code ready for customization
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🛠 Requirements
|
||||||
|
|
||||||
|
- Raspberry Pi 5 or Linux desktop
|
||||||
|
- Python 3.9+
|
||||||
|
- PyAudio, NumPy, requests, soxr, pydub, vosk
|
||||||
|
- SoX + ALSA utilities
|
||||||
|
- Ollama with one or more small LLMs (e.g., Gemma or Qwen)
|
||||||
|
- Piper TTS with ONNX voice models
|
||||||
|
|
||||||
|
Install dependencies:
|
||||||
|
|
||||||
|
```
|
||||||
|
pip install pyaudio requests soxr numpy pydub vosk
|
||||||
|
sudo apt install sox alsa-utils
|
||||||
|
```
|
||||||
|
|
||||||
|
## ⚙️ JSON Configuration
|
||||||
|
|
||||||
|
Place a config file at va_config.json:
|
||||||
|
|
||||||
|
```
|
||||||
|
{
|
||||||
|
"volume": 8,
|
||||||
|
"mic_name": "Plantronics",
|
||||||
|
"audio_output_device": "Plantronics",
|
||||||
|
"model_name": "gemma2:2b",
|
||||||
|
"voice": "en_US-kathleen-low.onnx",
|
||||||
|
"enable_audio_processing": false,
|
||||||
|
"history_length": 6,
|
||||||
|
"system_prompt": "You are a helpful assistant."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: if the configuration file is not found, defaults withing the main python app will be used:
|
||||||
|
|
||||||
|
```
|
||||||
|
# ------------------- CONFIG FILE LOADING -------------------
|
||||||
|
DEFAULT_CONFIG = {
|
||||||
|
"volume": 9,
|
||||||
|
"mic_name": "Plantronics",
|
||||||
|
"audio_output_device": "Plantronics",
|
||||||
|
"model_name": "qwen2.5:0.5b",
|
||||||
|
"voice": "en_US-kathleen-low.onnx",
|
||||||
|
"enable_audio_processing": False,
|
||||||
|
"history_length": 4,
|
||||||
|
"system_prompt": "You are a helpful assistant."
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 🔁 What `history_length` Means
|
||||||
|
|
||||||
|
The `history_length` setting controls how many previous exchanges (user + assistant messages) are included when generating each new reply.
|
||||||
|
|
||||||
|
- A value of `6` means the model receives the last 6 exchanges, plus the system prompt.
|
||||||
|
- This allows the assistant to maintain **short-term memory** for more coherent conversations.
|
||||||
|
- Setting it lower (e.g., `2`) increases speed and memory efficiency.
|
||||||
|
|
||||||
|
### ✅ `requirements.txt`
|
||||||
|
|
||||||
|
```
|
||||||
|
pyaudio
|
||||||
|
vosk
|
||||||
|
soxr
|
||||||
|
numpy
|
||||||
|
requests
|
||||||
|
pydub
|
||||||
|
```
|
||||||
|
|
||||||
|
If you plan to run this on a Raspberry Pi, you may also need:
|
||||||
|
|
||||||
|
```
|
||||||
|
soundfile # for pydub compatibility on some distros
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🐍 Install with Virtual Environment
|
||||||
|
|
||||||
|
```
|
||||||
|
# 1. Clone the repo
|
||||||
|
|
||||||
|
git clone https://github.com/your-username/voice-assistant-local.git
|
||||||
|
cd voice-assistant-local
|
||||||
|
|
||||||
|
# 2. Create and activate a virtual environment
|
||||||
|
|
||||||
|
python3 -m venv env
|
||||||
|
source env/bin/activate
|
||||||
|
|
||||||
|
# 3. Install dependencies
|
||||||
|
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# 4. Install SoX and ALSA utilities (if not already installed)
|
||||||
|
|
||||||
|
sudo apt install sox alsa-utils
|
||||||
|
|
||||||
|
# 5. (Optional) Test PyAudio installation
|
||||||
|
|
||||||
|
python -m pip install --upgrade pip setuptools wheel
|
||||||
|
```
|
||||||
|
|
||||||
|
> 💡 If you get errors installing PyAudio on Raspberry Pi, try:
|
||||||
|
>
|
||||||
|
> ```
|
||||||
|
> sudo apt install portaudio19-dev
|
||||||
|
> pip install pyaudio
|
||||||
|
> ```
|
||||||
|
|
||||||
|
## 🆕 🔧 Piper Installation (Binary)
|
||||||
|
|
||||||
|
Piper is a standalone text-to-speech engine used by this assistant. It's **not a Python package**, so it must be installed manually.
|
||||||
|
|
||||||
|
#### ✅ Install Piper
|
||||||
|
|
||||||
|
1. Download the appropriate Piper binary from:
|
||||||
|
👉 https://github.com/rhasspy/piper/releases
|
||||||
|
|
||||||
|
For Ubuntu Linux, download:
|
||||||
|
`piper_linux_x86_64.tar.gz`
|
||||||
|
|
||||||
|
2. Extract it:
|
||||||
|
|
||||||
|
```
|
||||||
|
tar -xvzf piper_linux_x86_64.tar.gz
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Move the binary into your project directory:
|
||||||
|
|
||||||
|
```
|
||||||
|
mkdir -p bin/piper
|
||||||
|
mv piper bin/piper/
|
||||||
|
chmod +x bin/piper/piper
|
||||||
|
```
|
||||||
|
|
||||||
|
4. ✅ Done! The script will automatically call it from `bin/piper/piper`.
|
||||||
|
|
||||||
|
## 📂 Directory Example
|
||||||
|
|
||||||
|
```
|
||||||
|
voice_assistant.py
|
||||||
|
va_config.json
|
||||||
|
requirements.txt
|
||||||
|
bin/
|
||||||
|
└── piper/
|
||||||
|
└── piper ← (binary)
|
||||||
|
voices/
|
||||||
|
└── en_US-kathleen-low.onnx
|
||||||
|
└── en_US-kathleen-low.onnx.json
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 🔌 Finding Your USB Microphone & Speaker
|
||||||
|
|
||||||
|
To configure the correct audio devices, use these commands on your Raspberry Pi or Linux terminal:
|
||||||
|
|
||||||
|
1. List Microphones (Input Devices)
|
||||||
|
|
||||||
|
```
|
||||||
|
python3 -m pip install pyaudio
|
||||||
|
python3 -c "import pyaudio; p = pyaudio.PyAudio(); \
|
||||||
|
[print(f'{i}: {p.get_device_info_by_index(i)}') for i in range(p.get_device_count())]"
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for your microphone name (e.g., Plantronics) and use that as mic_name.
|
||||||
|
2. List Speakers (Output Devices)
|
||||||
|
|
||||||
|
```
|
||||||
|
aplay -l
|
||||||
|
```
|
||||||
|
|
||||||
|
Example output:
|
||||||
|
|
||||||
|
```
|
||||||
|
card 3: Device [USB PnP Sound Device], device 0: USB Audio [USB Audio]
|
||||||
|
```
|
||||||
|
|
||||||
|
Use this info to set your audio_output_device to something like:
|
||||||
|
|
||||||
|
```
|
||||||
|
"audio_output_device": "USB PnP"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🔧 Ollama Installation (Required)
|
||||||
|
|
||||||
|
Ollama is a local model runner for LLMs. You need to install it separately (outside of Python).
|
||||||
|
|
||||||
|
#### 💻 Install Ollama
|
||||||
|
|
||||||
|
On **Linux (x86 or ARM)**:
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -fsSL https://ollama.com/install.sh | sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Or follow detailed instructions:
|
||||||
|
👉 https://ollama.com/download
|
||||||
|
|
||||||
|
Then start the daemon:
|
||||||
|
|
||||||
|
```
|
||||||
|
ollama serve
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 📥 Download the Models
|
||||||
|
|
||||||
|
After Ollama is installed and running, open a terminal and run:
|
||||||
|
|
||||||
|
##### ✅ For Gemma 2B:
|
||||||
|
|
||||||
|
```
|
||||||
|
ollama run gemma2:2b
|
||||||
|
```
|
||||||
|
|
||||||
|
##### For Qwen 0.5B:
|
||||||
|
|
||||||
|
```
|
||||||
|
ollama run qwen2.5:0.5b
|
||||||
|
```
|
||||||
|
|
||||||
|
This will automatically download and start the models. You only need to run this once per model.
|
||||||
|
|
||||||
|
##### ⚠️ Reminder
|
||||||
|
|
||||||
|
> Ollama is **not a Python package** — it is a background service.
|
||||||
|
> Do **not** add it to `requirements.txt`. Just make sure it’s installed and running before launching the assistant.
|
||||||
|
|
||||||
|
## 🎤 Installing Piper Voice Models
|
||||||
|
|
||||||
|
To enable speech synthesis, you'll need to download a **voice model (.onnx)** and its matching **config (.json)** file.
|
||||||
|
|
||||||
|
#### ✅ Steps:
|
||||||
|
|
||||||
|
1. Visit the official Piper voices list:
|
||||||
|
📄 https://github.com/rhasspy/piper/blob/master/VOICES.md
|
||||||
|
|
||||||
|
2. Choose a voice you like (e.g., `en_US-lessac-medium` or `en_US-amy-low`).
|
||||||
|
|
||||||
|
3. Download **both** files for your chosen voice:
|
||||||
|
|
||||||
|
- `voice.onnx`
|
||||||
|
- `config.json`
|
||||||
|
|
||||||
|
4. If you wish, you can rename the ONNX file and config file using the same base name. For example:
|
||||||
|
|
||||||
|
```
|
||||||
|
amy-low.onnx
|
||||||
|
amy-low.json
|
||||||
|
```
|
||||||
|
|
||||||
|
5. Place both files in a directory called `voices/` next to your script.
|
||||||
|
Example Directory Structure:
|
||||||
|
|
||||||
|
```
|
||||||
|
voice_assistant.py
|
||||||
|
voices/
|
||||||
|
├── amy-low.onnx
|
||||||
|
└── amy-low.json
|
||||||
|
```
|
||||||
|
|
||||||
|
6. Update your `config.json`:
|
||||||
|
|
||||||
|
```
|
||||||
|
"voice": "amy-low.onnx"
|
||||||
|
```
|
||||||
|
|
||||||
|
> ⚠️ Make sure both `.onnx` and `.json` are present in the `voices/` folder with matching names (excluding the extension).
|
||||||
|
|
||||||
|
## 🧪 **Performance Report**
|
||||||
|
|
||||||
|
The script prints out debug timing for the STT, LLM, and TTS parts of the pipeline. I asked ChatGPT4 to analyze some of the results i obtained.
|
||||||
|
|
||||||
|
**System:** Ubuntu laptop, Intel Core i5
|
||||||
|
**Model:** `qwen2.5:0.5b` (local via Ollama)
|
||||||
|
**TTS:** `piper` with `en_US-kathleen-low.onnx`
|
||||||
|
**Audio:** Plantronics USB headset
|
||||||
|
|
||||||
|
------
|
||||||
|
|
||||||
|
### 📊 **Timing Metrics (avg)**
|
||||||
|
|
||||||
|
| Stage | Metric (ms) | Notes |
|
||||||
|
| -------------- | ------------- | --------------------------------------- |
|
||||||
|
| STT Parse | 4.5 ms avg | Vosk transcribes near-instantly |
|
||||||
|
| LLM Inference | ~2,200 ms avg | Ranges from ~1s (short queries) to 5s |
|
||||||
|
| TTS Generation | ~1,040 ms avg | Piper ONNX performs well on CPU |
|
||||||
|
| Audio Playback | ~7,250 ms avg | Reflects actual audio length, not delay |
|
||||||
|
|
||||||
|
### ✅ Observations
|
||||||
|
|
||||||
|
- **STT speed is excellent** — under 10 ms consistently.
|
||||||
|
- **LLM inference is snappy** for a 0.5b model running locally. Your best response came in under 1.1 sec.
|
||||||
|
- **TTS is consistent and fast** — Kathleen-low voice is fully synthesized in ~800–1600 ms.
|
||||||
|
- **Playback timing matches response length** — no lag, just actual audio time.
|
||||||
|
- End-to-end round trip time from speaking to hearing a reply is about **8–10 seconds**, including speech and playback time.
|
||||||
|
|
||||||
|
## 💡 Use Cases
|
||||||
|
|
||||||
|
- Offline smart assistants
|
||||||
|
|
||||||
|
- Wearable or embedded AI demos
|
||||||
|
|
||||||
|
- Voice-controlled kiosks
|
||||||
|
|
||||||
|
- Character-based roleplay agents
|
||||||
|
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
MIT © 2024 M15.ai
|
||||||
Loading…
Reference in New Issue
Block a user