Skip to content
Project-MOSO-AIPublic

About

Privacy-first, local-first adaptive AI assistant platform with on-device LLM inference, memory engine, and cross-platform support.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

01001101 01001111 01010011 01001111 01001101 01000101 01001101 01001111 01010010 01011001 00100000 01000001 01001001 00100000 01010000 01010010 01001001 01010110 01000001 01000011 01011001 00100000 01000110 01001001 01010010 01010011 01010100 00001010 MOSO AI PRIVACY-FIRST ADAPTIVE INTELLIGENCE

Status License Version


Viewing access does not grant usage rights. MOSO Source Available License. Written permission required for use, modification, or distribution.


What is MOSO?

MOSO is a privacy-first, local-first AI assistant that runs entirely on your device. It sees your screen, understands your system, and acts on your behalf — all without sending data to the cloud.

User Goal → Brain (plan) → Muscles (act) → Eyes (verify) → Memory (learn)

Architecture

                    ┌──────────┐
                    │  BRAIN   │  Goal understanding, planning, reasoning
                    └────┬─────┘
                         │
          ┌──────────────┼──────────────┐
          │              │              │
          ▼              ▼              ▼
       ┌──────┐    ┌──────────┐    ┌─────────┐
       │ EYES │    │  MEMORY  │    │ MUSCLES │
       │      │    │          │    │         │
       │ OCR  │    │ Episodic │    │ Mouse   │
       │ Screenshot│ │ Semantic│    │ Keyboard│
       │ Window│    │ Procedural   │ Browser │
       │ Detect│    │ Vector   │    │ Terminal│
       └──────┘    └──────────┘    │ Files   │
                                   └─────────┘
          │              │              │
          └──────────────┼──────────────┘
                         │
                    ┌────▼─────┐
                    │ LEARNER  │  Skills from experience
                    └────┬─────┘
                         │
                    ┌────▼─────┐
                    │  SAFETY  │  Risk scoring, permissions
                    └────┬─────┘
                         │
                    ┌────▼─────┐
                    │    UI    │  Aura orb + chat
                    └──────────┘

Seven organs. No app-specific code. No cloud dependency.

Full architecture: See docs/ARCHITECTURE.md for complete organ interfaces, database schemas, event flow, and migration plan.

Module What It Does Key Files
Brain Plans, reasons, reflects agents/ (planner, executor, verifier)
Eyes Sees your screen vision/ (OCR, screenshots, window detection)
Muscles Acts on your system tools/ + computer_use/ (mouse, keyboard, files, terminal)
Memory Remembers everything memory/ (episodic, semantic, procedural, vector search)
Learning Self-improvement engine learning/ (experience, reflection, skill builder, generalization, optimizer, curriculum, evaluation)
Safety Blocks dangerous actions risk/ (permissions, privacy, risk scoring)
UI Desktop orb + conversation moso_ui/ (PySide6 floating orb)

Quick Start

Step 1: Install Python 3.12+

# Check your version
python --version

Step 2: Clone and install

git clone https://github.com/Project-MOSO-AI/MOSO.git
cd MOSO

# Create virtual environment
python -m venv .venv

# Activate it
# Windows:
.venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate

# Install dependencies
pip install -r moso_core/requirements.txt

Step 3: Download a model

# Lightweight (~2.3 GB)
python scripts/download_model.py phi3-mini

# Or larger (~5.2 GB, better quality)
python scripts/download_model.py qwen3-8b

Step 4: Run MOSO

# Launch desktop UI (Aura orb)
python run.py

# Or run module demos
python run.py demo

# Or run tests
python run.py test

Step-by-Step Usage

1. Basic Chat

from moso_core.inference.base import InferenceConfig
from moso_core.orchestration.orchestrator import Orchestrator

config = InferenceConfig(model_path="models/phi3-mini-3.8b-q4_k_m.gguf")
orchestrator = Orchestrator(config)

result = orchestrator.process("Hello, what can you do?")
print(result.text)

2. Enable Memory

orchestrator.enable_memory()

# MOSO remembers across sessions
orchestrator.process("My name is Alice")
orchestrator.process("What's my name?")  # → "Alice"

3. Use Tools

orchestrator.enable_tools()
orchestrator.enable_identity()

from moso_core.tools.models import ToolRequest

# Launch an app
req = ToolRequest(
    tool_name="app_tool",
    parameters={"action": "launch_application", "app_name": "code"},
)
result = orchestrator.tools.execute_tool(req, identity=orchestrator.identity_verifier)

# Read a file
req = ToolRequest(
    tool_name="file_tool",
    parameters={"action": "read_file", "path": "notes.txt"},
)
result = orchestrator.tools.execute_tool(req)

4. Automate Tasks

orchestrator.enable_agents()

summary = orchestrator.agents.plan_and_execute(
    "create a python project named my_app",
    requester="owner",
)
print(summary.overall_status)  # GoalStatus.COMPLETED

5. Control Your Desktop

orchestrator.enable_computer_use()

# Take a screenshot
result = orchestrator.computer_use.execute_action({"action": "capture_screen"})

# Type text
result = orchestrator.computer_use.execute_action(
    {"action": "type_text", "text": "Hello from MOSO"}
)

# Execute a sequence
results = orchestrator.computer_use.execute_sequence([
    {"action": "move_to", "x": 500, "y": 200},
    {"action": "click"},
    {"action": "type_text", "text": "MOSO AI"},
])

6. See Your Screen

orchestrator.enable_vision()

ctx = orchestrator.vision.build_context()
print(f"Active window: {ctx.active_window}")
print(f"Text on screen: {ctx.text_content[:200]}")

7. Ask About Your System

orchestrator.enable_system_intelligence()

print(orchestrator.system_intelligence.get_hardware_summary())
print(orchestrator.system_intelligence.explain("what is RAM"))

8. Research the Web

orchestrator.enable_realtime()

response = orchestrator.research("latest Python 3.13 features")
print(response.formatted_report)

9. Launch Desktop UI

# Install PySide6 first
pip install PySide6

# Launch the Aura orb
python -m moso_ui.main

The floating orb gives you:

  • Click to open chat
  • Drag to reposition
  • System tray for settings

Enable Everything at Once

config = InferenceConfig(model_path="models/qwen3-8b-q4_k_m.gguf")
orchestrator = Orchestrator(config)
orchestrator.enable_all(model_path="models/qwen3-8b-q4_k_m.gguf")

# Now MOSO can: chat, remember, see, act, plan, research, and understand your system

Project Structure

MOSO/
├── moso_core/                # Core AI engine
│   ├── agents/               # Brain — planning, execution, verification
│   ├── computer_use/         # Muscles — mouse, keyboard, screen, windows
│   ├── desktop/              # Desktop agent — perceive→reason→act→verify loop
│   ├── identity/             # Owner verification (voice, behavior, device)
│   ├── inference/            # Model backends (llama.cpp, ONNX Runtime)
│   ├── llm/                  # LLM integration (local, OpenAI, Anthropic, Ollama)
│   ├── learner/              # Learning engine — skills from experience
│   ├── memory/               # Memory — episodic, semantic, procedural, vector
│   ├── orchestration/        # Central hub — connects all modules
│   ├── realtime/             # Web research — fetch, verify, analyze, summarize
│   ├── resources/            # System monitoring — CPU, RAM, storage, network
│   ├── risk/                 # Safety — risk scoring, privacy, permissions
│   ├── tools/                # Tools — file, app, browser, terminal
│   ├── vision/               # Eyes — OCR, screenshots, window detection
│   └── voice/                # Voice pipeline — STT, TTS, speaker verify
├── moso_ui/                  # Desktop UI — Aura floating orb
├── backend/                  # FastAPI backend
├── tests/                    # Test suite (249 tests)
├── scripts/                  # Utility scripts
├── models/                   # Local GGUF model storage
├── run.py                    # Entry point
└── llms.txt                  # LLM context summary

Key Concepts

Experience → Skill Loop

Every action MOSO takes is recorded:

Screen Before → Action → Screen After → Success/Fail

When the same sequence succeeds multiple times, MOSO extracts a skill:

"Play {song_name} on Spotify"
Variables: song_name
Success rate: 95%

Skills are stored in procedural memory and reused for future tasks.

Risk Scoring

Before any action, MOSO calculates a risk score:

Score Level What Happens
< 0.2 LOW Allowed
0.2–0.5 MEDIUM Allowed with review
0.5–0.8 HIGH Blocked
≥ 0.8 CRITICAL Blocked

Identity Levels

Level Score Permissions
Owner 95+ Full access
Likely 80–94 Standard access
Guest 60–79 Limited
Unknown < 60 Read-only

Tech Stack

Component Technology Why
Language Python 3.12 Huge AI ecosystem
Inference llama.cpp Best local inference
Memory SQLite + WAL Plenty for local AI
Vision pytesseract + mss Stable OCR
Automation pyautogui + pygetwindow Desktop control
UI PySide6 Native desktop
API FastAPI Fast, simple
Config Pydantic Clean settings
Testing pytest Standard

Deploy with Docker

Option 1: Pull from GHCR (recommended)

# Pull the latest image
docker pull ghcr.io/project-moso-ai/moso:latest

# Run with your models directory mounted
docker run -d \
  -p 8000:8000 \
  -v ./models:/models:ro \
  -v moso-data:/data/moso \
  --name moso \
  ghcr.io/project-moso-ai/moso:latest

# Verify it's running
curl http://localhost:8000/health

Option 2: Docker Compose (development)

# Build and run locally with hot-reload
docker compose -f docker-compose.dev.yml up --build

# Or use the default compose (builds from source)
docker compose up --build

Option 3: Docker Compose (production)

# Pull from GHCR, persistent volumes
docker compose -f docker-compose.prod.yml up -d

# Check status
docker compose -f docker-compose.prod.yml ps
docker compose -f docker-compose.prod.yml logs moso

API Endpoints

Once running, MOSO exposes these endpoints:

Endpoint Method Description
/health GET Health check + module status
/chat POST Send a message, get a response
/execute POST Execute a tool (file, app, browser, terminal)
/plan POST Create and execute a multi-step plan
/memory/store POST Store a memory (episodic, semantic, procedural)
/memory/search POST Search memories
/memory/recent GET Get recent memories
/skills GET List learned skills
/system GET System resource status
/system/diagnostics GET Run system diagnostics
/identity GET Identity verification status

Example API calls

# Health check
curl http://localhost:8000/health

# Chat
curl -X POST http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello, what can you do?"}'

# Execute a tool
curl -X POST http://localhost:8000/execute \
  -H "Content-Type: application/json" \
  -d '{"tool_name": "file_tool", "parameters": {"action": "read_file", "path": "README.md"}}'

# Store a memory
curl -X POST http://localhost:8000/memory/store \
  -H "Content-Type: application/json" \
  -d '{"content": "User prefers dark mode", "memory_type": "semantic", "category": "preference"}'

# System status
curl http://localhost:8000/system

Environment Variables

Variable Default Description
MOSO_MODEL_PATH "" Path to GGUF model file
MOSO_N_CTX 2048 LLM context size
MOSO_HOME ~/.moso Data directory (memory, settings, logs)
LOG_LEVEL info Logging level (debug, info, warning, error)

Roadmap

Phase Focus Status
1 Voice Pipeline ✅
2 Identity Engine ✅
3 Memory Engine ✅
4 Resource Manager ✅
5 Tool Engine ✅
6 Agent System ✅
7 Computer Use ✅
8 Screen Vision ✅
9 LLM Integration ✅
10 Aura UI ✅
11 Risk & Privacy ✅
12 Real-Time Intelligence ✅
13 Knowledge Graph ✅
14 Learning Engine (7 sub-engines) 🔄 In Progress
15 Architecture v2 (7-organ redesign) 📋 Planned

Tests

# Run all tests
python -m pytest tests/ -v

# Run specific test file
python -m pytest tests/test_risk.py -v

License

MOSO Source Available License v1.0

Copyright (c) 2024-2026 MOSO AI

Viewing access does not grant usage rights. Written permission required for use, modification, or distribution.


MOSO AI — Privacy-First Adaptive Intelligence
License · GitHub

About

Privacy-first, local-first adaptive AI assistant platform with on-device LLM inference, memory engine, and cross-platform support.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages