Multi-Modal Autonomous Agent & Desktop OS

Commercial AI assistants operate as rigid cloud SaaS silos lacking system-level desktop access, leaking sensitive context across networks, incurring high recurring infrastructure costs, and lacking verifiable guardrails when executing consequential actions.

• Architected a local-first, privacy-preserving agent runtime leveraging LiveKit Agents framework and Google Gemini Live for bidirectional, low-latency audio streaming.
• Engineered a native Swift background daemon monitoring global hardware keycodes (Right Command double-tap) to trigger instant push-to-talk voice capture without window focus.
• Built persistent browser automation with authenticated profiles, enabling headless WhatsApp Web monitoring and messaging via background launchd daemons.
• Integrated native macOS automation via AppleScript and Python to orchestrate Calendar, Mail, Reminders, and filesystem workflows with automatic classification.
• Engineered multi-tier token governance and cost optimization: decoupled high-frequency scheduled tasks (inbox triage, digest dispatch) to zero-token direct script execution, reserving frontier reasoning LLMs strictly for dynamic multi-turn interactions and cutting projected monthly API spend from >$300/mo to <$25/mo.
• Implemented self-maintaining memory and reflection architecture: automated 2:00 AM routines that review daily cross-channel communications, extract durable facts and commitments into persistent stores, and maintain continuous context across sessions.
• Enforced strict Human-in-the-Loop (HITL) safety architecture: zero permanent file deletions, mandatory user confirmation before sending communications, and tool-enforced verification.
• Implemented multi-model failover routing: streaming Gemini Live for conversational voice, lightweight Gemini Flash Lite for batch triage, and local Ollama for zero-connectivity offline fallback.

• Delivered an always-on, hands-free personal operating system managing cross-stream non-profit operations, calendar commitments, and inbox sorting.
• Eliminates context-switching friction by executing tasks via voice and WhatsApp on the go.
• Demonstrates advanced 0-to-1 AI product delivery, hardware/software integration, and cost-optimized system architecture.

• Runs locally as native macOS launchd daemons (com.jeevan.rightcmdlistener, com.jeevan.whatsappwatcher) ensuring 24/7 background availability with minimal memory footprint.
• Multi-client support across terminal console mode, Next.js web frontend, and Flutter mobile client.
• Configured automated cron triggers for morning executive briefings, daytime inbox triage, and overnight reflections.

• Consequential action risks mitigated through strict approval boundaries where agent proposes actions but requires explicit user confirmation.
• System sleep limitations handled via hybrid architecture separating heavy local tasks from lightweight messaging alerts.