LegacyOracle: Personal AI with Memory and Context
LegacyOracle is my experimental personal AI system that combines a conversational interface with long-term memory and contextual understanding. Unlike typical chat systems, LegacyOracle maintains a comprehensive knowledge graph about topics and preferences, allowing for more personalized and consistent interactions over time.
Project Overview
The LegacyOracle project emerged from a simple observation: current AI assistants have no real memory. They can only “remember” what’s within their context window, and once a conversation ends, that information is lost unless explicitly saved.
I wanted to create an AI companion that could:
- Remember past conversations and facts about me
- Build connections between different pieces of information over time
- Provide consistent responses based on accumulated knowledge
- Run completely locally for privacy
- Feel more like interacting with a person than a stateless system
Key Features
Persistent Memory System
LegacyOracle’s core innovation is its memory architecture:
- Knowledge graph storage of facts, preferences, and relationships
- Episodic memory of past conversations with timestamps and context
- Semantic memory for general knowledge understanding
- Memory consolidation processes that run in the background
Natural Language Interface
Interaction is designed to feel natural and conversational:
- Text-based REPL interface for precise control
- Context awareness across multiple sessions
- Custom prompt engineering for consistent persona
- Support for complex, multi-turn conversations
Knowledge Graph Implementation
The system builds and maintains relationships between concepts:
- Neo4j graph database for storing knowledge
- Automatic extraction of entities and relationships
- Confidence scoring for fact reliability
- Visual browser to explore knowledge connections
Facial Interface (Experimental)
Optional visual components for a more personal experience:
- Camera-based facial recognition for user identification
- Expression analysis for emotional context
- Animated avatar representation of the Oracle
- Completely optional and privacy-respecting
Current Status
The basic conversation and memory systems are functional, with the knowledge graph implementation in active development. Facial recognition is in early testing phase.
Development Log
| Version | Date | Major Changes | |||
|---|---|---|---|---|---|
| Version 0.4 (Current) | May 2025 | Implemented basic knowledge graph storage, entity relationship extraction | Version 0.4 (Current) | May 2025 | Implemented basic knowledge graph storage, entity relationship extraction, graph visualization tools |
| Version 0.3 | April 2025 | Added persistent memory across sessions, simple fact storage and retrieval | |||
| Version 0.2 | March 2025 | Created conversation engine with context windows, improved prompt design | |||
| Version 0.1 | February 2025 | Established basic architecture and interface, command-line REPL |
Technical Architecture
Conversation Engine
LegacyOracle’s conversation system is built around local language models:
- Local LLM (currently using Mistral or LLaMA 3) for core interaction
- Custom prompt engineering to ensure consistent persona
- Context management system that includes:
- Current conversation
- Recent exchanges
- Relevant past interactions
- Known facts about the user
Memory System
The multi-layered memory architecture consists of:
- Short-term memory: Active conversation and recent context
- Long-term memory: Persistent storage of facts and past conversations
- Knowledge graph: Neo4j database for relationships between concepts
- Memory consolidation: Background processes that organize and connect information
Interface
The user interface options include:
- Command-line REPL for text interaction
- Early-stage web UI for more visual interaction
- Optional camera module for facial recognition (experimental)
- Knowledge graph visualization tools
Technology Stack
- Backend: Python with FastAPI
- Database: Neo4j for knowledge graph, SQLite for conversation history
- AI: Local language models via llama.cpp
- Frontend: Command-line interface and Vue.js for web UI
- Vision: OpenCV for facial recognition (optional)
Future Roadmap
The LegacyOracle project has an ambitious development plan:
| Feature | Description | Target Release |
|---|---|---|
| Enhanced knowledge graph | More sophisticated relationship inference and connection building | Version 0.5 |
| Voice interface | Speech recognition and synthesis for natural conversation | Version 0.6 |
| Emotion detection | Recognition of user emotional states for contextual responses | Version 0.7 |
| Smart home integration | Connecting to home automation systems for expanded capabilities | Version 0.8 |
| Memory visualization and editing | User tools to view and modify the AI’s knowledge about them | Version 0.9 |
Project Demonstration
Here’s an example of LegacyOracle’s capabilities, showing how it builds and uses knowledge over time:
Conversation 1 (Day 1)
User: Hello there, I'm Lewis.
LegacyOracle: Hello Lewis, nice to meet you. I'm LegacyOracle. How are you today?
User: I'm doing well, just got back from a run. I try to run three times a week.
LegacyOracle: That's great! Regular exercise like running is excellent for physical and mental health. Do you have a favorite route or time of day to run?
User: I usually run in the morning, around the park near my house. I've been training for a half marathon.
LegacyOracle: Morning runs are refreshing! And training for a half marathon is impressive. When is the event, and is this your first one?
User: It's in September, and yes, it's my first half marathon. I'm a bit nervous about it.
LegacyOracle: You've got plenty of time to prepare if it's in September. First-time nerves are completely normal, but with your consistent three-times-weekly training, you're taking the right approach. Would you like any training tips?Conversation 2 (Two Weeks Later)
User: Hi there, I had a great run today!
LegacyOracle: That's wonderful to hear, Lewis! How was your morning run in the park? Is your half marathon training going well?
User: Yes, I actually increased my distance today. I managed 15km for the first time.
LegacyOracle: 15km is a significant milestone! You're making excellent progress toward your September half marathon. How are you feeling after the longer distance?
User: Pretty tired but satisfied. I think I need new running shoes though.
LegacyOracle: That sense of accomplishment despite the fatigue is what makes running so rewarding. Regarding shoes, that's a wise consideration—proper footwear is crucial for distance training, especially as you increase your mileage. Do you have a preferred brand or store for running shoes?Notice how the system remembered Lewis’s running habits, the upcoming half marathon in September, and that he typically runs in the morning at a nearby park—all without being explicitly prompted to recall this information.
Privacy and Ethics
LegacyOracle is built with strong privacy principles:
- All data remains on the user’s device
- No cloud connectivity required
- User has full control over stored information
- Optional tools to view and edit the knowledge graph
- “Forget” functionality to remove specific memories or topics
Ethically, the project acknowledges the responsibility of creating AI systems with persistent memory about users. All development is approached with careful consideration of potential impacts and user agency.
Early Testing Program
I’m looking for a small group of technical testers who are interested in providing feedback on LegacyOracle’s memory and conversation capabilities.
If you’re curious about AI systems with persistent memory and would like to help shape this project, please contact me through the form below. Include “LegacyOracle Testing” in the subject line and share a bit about your interest in the project.