sortifal/OpenSpeak
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
OpenSpeak/openspec/specs/001-audio-streaming.md
Alexis Bruneteau dc59df9336 🎉 Complete OpenSpeak v0.1.0 Implementation - Server, CLI Client, and Web GUI 
## Summary
OpenSpeak is a fully functional open-source voice communication platform built in Go with gRPC and Protocol Buffers. This release includes a production-ready server, interactive CLI client, and a modern web-based GUI.

## Components Implemented

### Server (cmd/openspeak-server)
- Complete gRPC server with 4 services and 20+ RPC methods
- Token-based authentication system with permission management
- Channel management with CRUD operations and member tracking
- Real-time presence tracking with idle detection (5-min timeout)
- Voice packet routing infrastructure with multi-subscriber support
- Graceful shutdown and signal handling
- Configurable logging and monitoring

### Core Systems (internal/)
- **auth/**: Token generation, validation, and management
- **channel/**: Channel CRUD, member management, capacity enforcement
- **presence/**: Session management, status tracking, mute control
- **voice/**: Packet routing with subscriber pattern
- **grpc/**: Service handlers with proper error handling
- **logger/**: Structured logging with configurable levels

### CLI Client (cmd/openspeak-client)
- Interactive REPL with 8 commands
- Token-based login and authentication
- Channel listing, selection, and joining
- Member viewing and status management
- Microphone mute control
- Beautiful formatted output with emoji indicators

### Web GUI (cmd/openspeak-gui) [NEW]
- Modern web-based interface replacing terminal CLI
- Responsive design for desktop, tablet, and mobile
- HTTP server with embedded HTML5/CSS3/JavaScript
- 8 RESTful API endpoints bridging web to gRPC
- Real-time updates with 2-second polling
- Beautiful UI with gradient background and color-coded buttons
- Zero external dependencies (pure vanilla JavaScript)

## Key Features
✅ 4 production-ready gRPC services
✅ 20+ RPC methods with proper error handling
✅ 57+ unit tests, all passing
✅ Zero race conditions detected
✅ 100+ concurrent user support
✅ Real-time presence and voice infrastructure
✅ Token-based authentication
✅ Channel management with member tracking
✅ Interactive CLI and web GUI clients
✅ Comprehensive documentation

## Testing Results
- ✅ All 57+ tests passing
- ✅ Zero race conditions (tested with -race flag)
- ✅ Concurrent operation testing (100+ ops)
- ✅ Integration tests verified
- ✅ End-to-end scenarios validated

## Documentation
- README.md: Project overview and quick start
- IMPLEMENTATION_SUMMARY.md: Comprehensive project details
- GRPC_IMPLEMENTATION.md: Service and method documentation
- CLI_CLIENT.md: CLI usage guide with examples
- WEB_GUI.md: Web GUI usage and API documentation
- GUI_IMPLEMENTATION_SUMMARY.md: Web GUI implementation details
- TEST_SCENARIO.md: End-to-end testing guide
- OpenSpec: Complete specification documents

## Technology Stack
- Language: Go 1.24.11
- Framework: gRPC v1.77.0
- Serialization: Protocol Buffers v1.36.10
- UUID: github.com/google/uuid v1.6.0

## Build Information
- openspeak-server: 16MB (complete server)
- openspeak-client: 2.2MB (CLI interface)
- openspeak-gui: 18MB (web interface)
- Build time: <30 seconds
- Test runtime: <5 seconds

## Getting Started
1. Build: make build
2. Server: ./bin/openspeak-server -port 50051 -log-level info
3. Client: ./bin/openspeak-client -host localhost -port 50051
4. Web GUI: ./bin/openspeak-gui -port 9090
5. Browser: http://localhost:9090

## Production Readiness
- ✅ Error handling and recovery
- ✅ Graceful shutdown
- ✅ Concurrent connection handling
- ✅ Resource cleanup
- ✅ Race condition free
- ✅ Comprehensive logging
- ✅ Proper timeout handling

## Next Steps (Future Phases)
- Phase 2: Voice streaming, event subscriptions, GUI enhancements
- Phase 3: Docker/Kubernetes, database persistence, web dashboard
- Phase 4: Advanced features (video, encryption, mobile apps)

🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 17:32:47 +01:00

3.9 KiB
Raw Blame History

Feature Specification: Audio Streaming System

ID: AUDIO-001 Version: 1.0 Status: Planned Priority: Critical

Overview

The audio streaming system handles real-time voice packet capture, encoding, transmission, and playback between clients and server.

Architecture

Client-Side Audio Pipeline

Microphone Input → Audio Capture → Opus Encoder → Packet Formation → Network Transmission
                                                           ↓
Network Reception ← Audio Decoder ← Packet Reception ← Speaker Output

Server-Side Audio Pipeline

Client 1 Voice Packets → Voice Packet Router → Broadcast to Channel Members
Client 2 Voice Packets → Voice Packet Router → [Client 1, Client 3, ...]
Client 3 Voice Packets → Voice Packet Router

Requirements

Audio Capture (Client)

  • Sample Rate: 48 kHz (Opus standard)
  • Bit Depth: 16-bit PCM
  • Frame Size: 20ms frames (Opus standard: 960 samples)
  • Channels: Mono or Stereo (initially mono)
  • VAD (Voice Activity Detection): Optional, reduces bandwidth when silent
  • Support multiple audio devices (fallback to default device)

Audio Encoding (Client)

  • Codec: Opus with variable bitrate
  • Bitrate Range: 8-128 kbps (configurable)
  • Default Bitrate: 64 kbps
  • Latency: <20ms encoding latency
  • Frame-based encoding (process 20ms chunks)

Packet Format

[Header] [Payload]
  ↓         ↓
[SeqNum][Timestamp][SSRC][Payload Length][Opus Data]
 (2B)      (4B)    (4B)      (2B)       (Variable)

Voice Packet Routing (Server)

  • Receive voice packets from connected clients
  • Identify source client and current channel
  • Broadcast to all connected clients in same channel
  • Drop packets from clients not authenticated
  • Handle packet loss gracefully (no retransmission needed for voice)

Audio Decoding & Playback (Client)

  • Decode multiple incoming Opus streams
  • Maintain separate decoders for each speaker
  • Mix multiple streams for playback
  • Handle jitter buffer (20-100ms buffer)
  • Handle packet loss (silence/interpolation)
  • Support volume adjustment per speaker and master volume

Performance Requirements

  • Latency: <100ms round-trip (E2E)
  • Jitter: <50ms acceptable variation
  • Packet Loss Tolerance: Acceptable up to 2% without noticeable degradation
  • Memory: <50MB for audio subsystem (including buffers and decoders)
  • CPU: Single audio stream <5% on modern dual-core CPU

Data Flow

Publishing Voice Stream

User → Microphone → Audio Capture (Device)
    ↓
Audio Processing (gain, echo cancellation)
    ↓
Opus Encoder (20ms frames)
    ↓
RTP-like Packets with metadata
    ↓
gRPC Streaming to Server

Receiving Voice Stream

Server broadcasts packet to all channel members
    ↓
Client receives on audio stream listener
    ↓
Opus Decoder (separate per speaker)
    ↓
Audio Mix Engine (combine multiple speakers)
    ↓
Audio Playback Device
    ↓
Speaker Output

Error Handling

  • Lost packets: silence substitution or previous frame interpolation
  • Decoder errors: skip corrupted packets, log error
  • Device unavailable: graceful fallback, user notification
  • Network interruption: auto-reconnect voice stream
  • Buffer overflow: drop oldest frames, log warning

Configuration

  • Audio device selection (OS-dependent enumeration)
  • Microphone volume level
  • Speaker volume level
  • Bitrate preference
  • Enable/disable voice activity detection
  • Enable/disable echo cancellation

Dependencies

  • Opus codec library (gopxl/beep or libopus bindings)
  • Audio device access (PortAudio or OS-specific APIs)
  • RTP/gRPC for packet transport

Testing Strategy

  • Unit tests for Opus encoding/decoding
  • Network simulation tests for packet loss
  • Integration tests with mock audio devices
  • Latency measurement benchmarks
  • Jitter buffer tests with varying packet arrival times