# Claude Code MCP

Claude Code MCP is an implementation of [Claude Code](https://gist.github.com/transitive-bullshit/487c9cb52c75a9701d312334ed53b20c) as a [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server. This project allows you to use Claude Code's powerful software engineering capabilities through the standardized MCP interface.

# ⚠ DISCLAIMER ⚠
Claude Code MCP is a auto-generated project by DevinAI, who was prompted to analyse the Claude Code codebase, and generate an MCP server.

This is a proof of concept that I don't advise anyone to use.

## What is Claude Code?

Claude Code is Anthropic's CLI tool for software engineering tasks, powered by Claude. It provides a set of tools and capabilities that help developers with:

- Code generation and editing
- Code review and analysis
- Debugging and troubleshooting
- File system operations
- Shell command execution
- Project exploration and understanding

The original implementation is available as a JavaScript module that defines prompts and tools for interacting with Claude's API.

## What is MCP?

The Model Context Protocol (MCP) is a standardized interface for AI models that enables consistent interaction patterns across different models and providers. MCP defines:

- **Tools**: Functions that models can call to perform actions
- **Resources**: External data that models can access
- **Prompts**: Predefined conversation templates

By implementing Claude Code as an MCP server, we make its capabilities available to any MCP-compatible client, allowing for greater interoperability and flexibility.

## Features

- Full implementation of Claude Code functionality as an MCP server
- Provides tools for file operations, shell commands, and code analysis
- Exposes resources for accessing file system and environment information
- Includes prompts for general CLI interaction and code review
- Compatible with any MCP client
- TypeScript implementation with full type safety

## Installation

```bash
# Clone the repository
git clone https://github.com/auchenberg/claude-code-mcp.git
cd claude-code-mcp

# Install dependencies
npm install

# Build the project
npm run build
```

## Usage

### Running as a standalone server

```bash
# Start the server
npm start
```

### Using with MCP clients

Claude Code MCP can be used with any MCP client. Here's an example of how to connect to it using the MCP TypeScript SDK:

```typescript
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";

const transport = new StdioClientTransport({
  command: "node",
  args: ["dist/index.js"]
});

const client = new Client(
  {
    name: "example-client",
    version: "1.0.0"
  },
  {
    capabilities: {
      prompts: {},
      resources: {},
      tools: {}
    }
  }
);

await client.connect(transport);

// Use Claude Code through MCP
const result = await client.callTool({
  name: "bash",
  arguments: {
    command: "ls -la"
  }
});

console.log(result);
```

## Available Tools

Claude Code MCP provides the following tools:

- **bash**: Execute shell commands with security restrictions and timeout options
- **readFile**: Read files from the filesystem with options for line offsets and limits
- **listFiles**: List files and directories with detailed metadata
- **searchGlob**: Search for files matching a glob pattern
- **grep**: Search for text in files with regex pattern support
- **think**: A no-op tool for thinking through complex problems
- **codeReview**: Analyze and review code for bugs, security issues, and best practices
- **editFile**: Create or edit files with specified content

### Tool Details

#### bash

```typescript
{
  command: string;  // The shell command to execute
  timeout?: number; // Optional timeout in milliseconds (max 600000)
}
```

The bash tool includes security restrictions that prevent execution of potentially dangerous commands like `curl`, `wget`, and others.

#### readFile

```typescript
{
  file_path: string; // The absolute path to the file to read
  offset?: number;   // The line number to start reading from
  limit?: number;    // The number of lines to read
}
```

#### searchGlob

```typescript
{
  pattern: string;  // The glob pattern to match files against
  path?: string;    // The directory to search in (defaults to current working directory)
}
```

#### grep

```typescript
{
  pattern: string;   // The regular expression pattern to search for
  path?: string;     // The directory to search in (defaults to current working directory)
  include?: string;  // File pattern to include in the search (e.g. "*.js", "*.{ts,tsx}")
}
```

## Available Resources

- **file**: Access file contents (`file://{path}`)
  - Provides direct access to file contents with proper error handling
  - Returns the full text content of the specified file

- **directory**: List directory contents (`dir://{path}`)
  - Returns a JSON array of file information objects
  - Each object includes name, path, isDirectory, size, and modified date

- **environment**: Get system environment information (`env://info`)
  - Returns information about the system environment
  - Includes Node.js version, npm version, OS info, and environment variables

## Available Prompts

- **generalCLI**: General CLI prompt for Claude Code
  - Provides a comprehensive system prompt for Claude to act as a CLI tool
  - Includes guidelines for tone, style, proactiveness, and following conventions
  - Automatically includes environment details

- **codeReview**: Prompt for reviewing code
  - Specialized prompt for code review tasks
  - Analyzes code for bugs, security vulnerabilities, performance issues, and best practices

- **prReview**: Prompt for reviewing pull requests
  - Specialized prompt for PR review tasks
  - Analyzes PR changes and provides comprehensive feedback

- **initCodebase**: Initialize a new CLAUDE.md file with codebase documentation
  - Creates documentation for build/lint/test commands and code style guidelines
  - Useful for setting up a new project with Claude Code

## Development

```bash
# Run in development mode with auto-reload
npm run dev
```

## Architecture

Claude Code MCP is built with a modular architecture:

```
claude-code-mcp/
├── src/
│   ├── server/
│   │   ├── claude-code-server.ts  # Main server setup
│   │   ├── tools.ts               # Tool implementations
│   │   ├── prompts.ts             # Prompt definitions
│   │   └── resources.ts           # Resource implementations
│   ├── utils/
│   │   ├── bash.ts                # Shell command utilities
│   │   └── file.ts                # File system utilities
│   └── index.ts                   # Entry point
├── package.json
├── tsconfig.json
└── README.md
```

The implementation follows these key principles:

1. **Modularity**: Each component (tools, prompts, resources) is implemented in a separate module
2. **Type Safety**: Full TypeScript type definitions for all components
3. **Error Handling**: Comprehensive error handling for all operations
4. **Security**: Security restrictions for potentially dangerous operations

## Implementation Details

### MCP Server Setup

The main server is set up in `claude-code-server.ts`:

```typescript
export async function setupClaudeCodeServer(server: McpServer): Promise<void> {
  // Set up Claude Code tools
  setupTools(server);

  // Set up Claude Code prompts
  setupPrompts(server);

  // Set up Claude Code resources
  setupResources(server);
}
```

### Tool Implementation

Tools are implemented using the MCP SDK's tool registration method:

```typescript
server.tool(
  "toolName",
  "Tool description",
  {
    // Zod schema for tool arguments
    param1: z.string().describe("Parameter description"),
    param2: z.number().optional().describe("Optional parameter description")
  },
  async ({ param1, param2 }) => {
    // Tool implementation
    return {
      content: [{ type: "text", text: "Result" }]
    };
  }
);
```

### Resource Implementation

Resources are implemented using the MCP SDK's resource registration method:

```typescript
server.resource(
  "resourceName",
  new ResourceTemplate("resource://{variable}", { list: undefined }),
  async (uri, variables) => {
    // Resource implementation
    return {
      contents: [{
        uri: uri.href,
        text: "Resource content"
      }]
    };
  }
);
```

## License

MIT

## Acknowledgements

- [Claude Code](https://gist.github.com/transitive-bullshit/487c9cb52c75a9701d312334ed53b20c) by Anthropic
- [Model Context Protocol](https://modelcontextprotocol.io)
- [MCP TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk)

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## Disclaimer

This project is not officially affiliated with Anthropic. Claude Code is a product of Anthropic, and this project is an independent implementation of Claude Code as an MCP server.


# ESP32-S3 BLE Gateway with MCP Server

A hybrid ESP32-S3 gateway that manages BLE devices, runs an MCP server, and integrates with Home Assistant via MQTT.

## Features

- BLE device management (scan, connect, read/write/control)
- MCP server for AI chat-based control
- MQTT client for Home Assistant integration
- Support for Ecoflow, EG4, and other BLE devices
- Auto-discovery and unified device API

## Hardware

- ESP32-S3 (recommended for MCP server capabilities)
- BLE devices (Ecoflow, EG4, etc.)

## Setup

1. Install ESP-IDF v4.4+
2. Configure WiFi and MQTT settings in `config.h`
3. Build and flash the firmware

## MCP Server

The ESP32 runs a lightweight MCP server accessible via HTTP/WebSocket for device control.

## MQTT Topics

- `homeassistant/ble/{device_id}/state` - Device state
- `homeassistant/ble/{device_id}/control` - Device control
- `homeassistant/ble/{device_id}/availability` - Device availability

## Usage

1. Devices are automatically discovered via BLE
2. Control via MCP server chat interface
3. Monitor and control via Home Assistant MQTT integration


# pindex

To install dependencies:

```bash
bun install
```

To run:

```bash
bun run index.ts
```

This project was created using `bun init` in bun v1.2.16. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime.


[![Docker Size](https://img.shields.io/docker/image-size/jokobsk/netalertx?label=Size&logo=Docker&color=0aa8d2&logoColor=fff&style=for-the-badge)](https://hub.docker.com/r/jokobsk/netalertx)
[![Docker Pulls](https://img.shields.io/docker/pulls/jokobsk/netalertx?label=Pulls&logo=docker&color=0aa8d2&logoColor=fff&style=for-the-badge)](https://hub.docker.com/r/jokobsk/netalertx)
[![GitHub Release](https://img.shields.io/github/v/release/jokob-sk/NetAlertX?color=0aa8d2&logoColor=fff&logo=GitHub&style=for-the-badge)](https://github.com/jokob-sk/NetAlertX/releases)
[![Discord](https://img.shields.io/discord/1274490466481602755?color=0aa8d2&logoColor=fff&logo=Discord&style=for-the-badge)](https://discord.gg/NczTUTWyRr)
[![Home Assistant](https://img.shields.io/badge/Repo-blue?logo=home-assistant&style=for-the-badge&color=0aa8d2&logoColor=fff&label=Add)](https://my.home-assistant.io/redirect/supervisor_add_addon_repository/?repository_url=https%3A%2F%2Fgithub.com%2Falexbelgium%2Fhassio-addons)

# NetAlertX - Network Visibility & Asset Intelligence Framework

![main][main]

<details>
  <summary>📷 Click for more screenshots</summary>

  | ![Main screen][main] | ![device_details 1][device_details]  | ![Screen network][network] |
  |----------------------|----------------------|----------------------|
  | ![presence][presence] | ![maintenance][maintenance] | ![settings][settings]  |
  | ![sync_hub][sync_hub] | ![report1][report1] | ![device_nmap][device_nmap]  |

  Head to [https://netalertx.com/](https://netalertx.com/) for even more gifs and screenshots 📷.

</details>

Centralized network visibility and continuous asset discovery.

Monitor devices, detect change, and stay aware across distributed networks.

NetAlertX provides a centralized "Source of Truth" (NSoT) for network infrastructure. Maintain a real-time inventory of every connected device, identify Shadow IT and unauthorized hardware to maintain regulatory compliance, and automate compliance workflows across distributed sites.

NetAlertX is designed to bridge the gap between simple network scanning and complex SIEM tools, providing actionable insights without the overhead.

## Table of Contents

- [Quick Start](#quick-start)
- [Features](#features)
- [Documentation](#documentation)
- [Security \& Privacy](#security--privacy)
- [FAQ](#faq)
- [Troubleshooting Tips](#troubleshooting-tips)
- [Everything else](#everything-else)

## Quick Start

> [!WARNING]
> ⚠ **Important:** The docker-compose has recently changed. Carefully read the [Migration guide](https://docs.netalertx.com/MIGRATION/?h=migrat#12-migration-from-netalertx-v25524) for detailed instructions.

Start NetAlertX in seconds with Docker:

```bash
docker run -d \
  --network=host \
  --restart unless-stopped \
  -v /local_data_dir:/data \
  -v /etc/localtime:/etc/localtime:ro \
  --tmpfs /tmp:uid=20211,gid=20211,mode=1700 \
  -e PORT=20211 \
  -e APP_CONF_OVERRIDE='{"GRAPHQL_PORT":"20214"}' \
  ghcr.io/jokob-sk/netalertx:latest
```

Note: Your `/local_data_dir` should contain a `config` and `db` folder.

To deploy a containerized instance directly from the source repository, execute the following BASH sequence:
```bash
git clone https://github.com/jokob-sk/NetAlertX.git
cd NetAlertX
docker compose up --force-recreate --build
# To customize: edit docker-compose.yaml and run that last command again
```

Need help configuring it? Check the [usage guide](https://docs.netalertx.com/README) or [full documentation](https://docs.netalertx.com/).

For Home Assistant users: [Click here to add NetAlertX](https://my.home-assistant.io/redirect/supervisor_add_addon_repository/?repository_url=https%3A%2F%2Fgithub.com%2Falexbelgium%2Fhassio-addons)

For other install methods, check the [installation docs](#documentation)

---
### || [Docker guide](https://docs.netalertx.com/DOCKER_INSTALLATION) || [Releases](https://github.com/netalertx/NetAlertX/releases) || [Docs](https://docs.netalertx.com/) || [Plugins](https://docs.netalertx.com/PLUGINS) || [Website](https://netalertx.com)
---

## Features

### Discovery & Asset Intelligence

Continuous monitoring for unauthorized asset discovery, connection state changes, and IP address management (IPAM) drift. Discovery & scan methods include: **arp-scan**,  **Pi-hole - DB import**,  **Pi-hole - DHCP leases import**, **Generic DHCP leases import**, **UNIFI controller import**, **SNMP-enabled router import**. Check the [Plugins](https://docs.netalertx.com/PLUGINS#readme) docs for a full list of avaliable plugins.

### Notification gateways

Send notifications to more than 80+ services, including Telegram via [Apprise](https://hub.docker.com/r/caronc/apprise), or use native [Pushsafer](https://www.pushsafer.com/), [Pushover](https://www.pushover.net/), or [NTFY](https://ntfy.sh/) publishers.

### Integrations and Plugins

Feed your data and device changes into [Home Assistant](https://docs.netalertx.com/HOME_ASSISTANT), read [API endpoints](https://docs.netalertx.com/API), or use [Webhooks](https://docs.netalertx.com/WEBHOOK_N8N) to setup custom automation flows. You can also
build your own scanners with the [Plugin system](https://docs.netalertx.com/PLUGINS#readme) in as little as [15 minutes](https://www.youtube.com/watch?v=cdbxlwiWhv8).

### Workflows

The [workflows module](https://docs.netalertx.com/WORKFLOWS) automates IT governance by enforcing device categorization and cleanup policies. Whether you need to assign newly discovered devices to a specific Network Node, auto-group devices from a given vendor, unarchive a device if detected online, or automatically delete devices, this module provides the flexibility to tailor the automations to your needs.

[127 more lines in file. Use offset=101 to continue.]


# 🐋 Docker MCP server

An MCP server for managing Docker with natural language!

## 🪩 What can it do?

- 🚀 Compose containers with natural language
- 🔍 Introspect & debug running containers
- 📀 Manage persistent data with Docker volumes

## ❓ Who is this for?

- Server administrators: connect to remote Docker engines for e.g. managing a
  public-facing website.
- Tinkerers: run containers locally and experiment with open-source apps
  supporting Docker.
- AI enthusiasts: push the limits of that an LLM is capable of!

## Demo

A quick demo showing a WordPress deployment using natural language:

https://github.com/user-attachments/assets/65e35e67-bce0-4449-af7e-9f4dd773b4b3

## 🏎 Quickstart

### Install

#### Claude Desktop

On MacOS: `~/Library/Application\ Support/Claude/claude_desktop_config.json`

On Windows: `%APPDATA%/Claude/claude_desktop_config.json`

  Install from PyPi with uv

If you don't have `uv` installed, follow the installation instructions for your
system:
[link](https://docs.astral.sh/uv/getting-started/installation/#installation-methods)

Then add the following to your MCP servers file:

```
"mcpServers": {
  "mcp-server-docker": {
    "command": "uvx",
    "args": [
      "mcp-server-docker"
    ]
  }
}
```

  Install with Docker

Purely for convenience, the server can run in a Docker container.

After cloning this repository, build the Docker image:

```bash
docker build -t mcp-server-docker .
```

And then add the following to your MCP servers file:

```
"mcpServers": {
  "mcp-server-docker": {
    "command": "docker",
    "args": [
      "run",
      "-i",
      "--rm",
      "-v",
      "/var/run/docker.sock:/var/run/docker.sock",
      "mcp-server-docker:latest"
    ]
  }
}
```

Note that we mount the Docker socket as a volume; this ensures the MCP server
can connect to and control the local Docker daemon.

## 📝 Prompts

### 🎻 `docker_compose`

Use natural language to compose containers. [See above](#demo) for a demo.

Provide a Project Name, and a description of desired containers, and let the LLM
do the rest.

This prompt instructs the LLM to enter a `plan+apply` loop. Your interaction
with the LLM will involve the following steps:

1. You give the LLM instructions for which containers to bring up
2. The LLM calculates a concise natural language plan and presents it to you
3. You either:
   - Apply the plan
   - Provide the LLM feedback, and the LLM recalculates the plan

#### Examples

- name: `nginx`, containers: "deploy an nginx container exposing it on port
  9000"
- name: `wordpress`, containers: "deploy a WordPress container and a supporting
  MySQL container, exposing Wordpress on port 9000"

#### Resuming a Project

When starting a new chat with this prompt, the LLM will receive the status of
any containers, volumes, and networks created with the given project `name`.

This is mainly useful for cleaning up, in-case you lose a chat that was
responsible for many containers.

## 📔 Resources

The server implements a couple resources for every container:

- Stats: CPU, memory, etc. for a container
- Logs: tail some logs from a container

## 🔨 Tools

### Containers

- `list_containers`
- `create_container`
- `run_container`
- `recreate_container`
- `start_container`
- `fetch_container_logs`
- `stop_container`
- `remove_container`

### Images

- `list_images`
- `pull_image`
- `push_image`
- `build_image`
- `remove_image`

### Networks

- `list_networks`
- `create_network`
- `remove_network`

### Volumes

- `list_volumes`
- `create_volume`
- `remove_volume`

## 🚧 Disclaimers

### Sensitive Data

**DO NOT CONFIGURE CONTAINERS WITH SENSITIVE DATA.** This includes API keys,
database passwords, etc.

Any sensitive data exchanged with the LLM is inherently compromised, unless the
LLM is running on your local machine.

If you are interested in securely passing secrets to containers, file an issue
on this repository with your use-case.

### Reviewing Created Containers

Be careful to review the containers that the LLM creates. Docker is not a secure
sandbox, and therefore the MCP server can potentially impact the host machine
through Docker.

For safety reasons, this MCP server doesn't support sensitive Docker options
like `--privileged` or `--cap-add/--cap-drop`. If these features are of interest
to you, file an issue on this repository with your use-case.

## 🛠 Configuration

This server uses the Python Docker SDK's `from_env` method. For configuration
details, see
[the documentation](https://docker-py.readthedocs.io/en/stable/client.html#docker.client.from_env).

### Connect to Docker over SSH

This MCP server can connect to a remote Docker daemon over SSH.

Simply set a `ssh://` host URL in the MCP server definition:

```
"mcpServers": {
  "mcp-server-docker": {
    "command": "uvx",
    "args": [
      "mcp-server-docker"
    ],
    "env": {
      "DOCKER_HOST": "ssh://myusername@myhost.example.com"
    }
  }
}
```

## 💻 Development

Prefer using Devbox to configure your development environment.

See the `devbox.json` for helpful development commands.

After setting up devbox you can configure your Claude MCP config to use it:

```
  "docker": {
    "command": "/path/to/repo/.devbox/nix/profile/default/bin/uv",
    "args": [
      "--directory",
      "/path/to/repo/",
      "run",
      "mcp-server-docker"
    ]
  },
```


I'm making good progress but there are still many more to read. Let me continue reading more READMEs in batches.

Let me continue reading more README files:


# Android MCP Server

An MCP (Model Context Protocol) server that provides programmatic control over
Android devices through ADB (Android Debug Bridge). This server exposes
various Android device management capabilities that can be accessed by MCP
clients like [Claude desktop](https://modelcontextprotocol.io/quickstart/user)
and Code editors
(e.g. [Cursor](https://docs.cursor.com/context/model-context-protocol))

## Features

- 🔧 ADB Command Execution
- 📸 Device Screenshot Capture
- 🎯 UI Layout Analysis
- 📱 Device Package Management

## Prerequisites

- Python 3.x
- ADB (Android Debug Bridge) installed and configured
- Android device or emulator (not tested)

## Installation

1. Clone the repository:

```bash
git clone https://github.com/minhalvp/android-mcp-server.git
cd android-mcp-server
```

2. Install dependencies:
This project uses [uv](https://github.com/astral-sh/uv) for project
management via various methods of
[installation](https://docs.astral.sh/uv/getting-started/installation/).

```bash
uv python install 3.11
uv sync
```

## Configuration

The server supports flexible device configuration with multiple usage scenarios.

### Device Selection Modes

**1. Automatic Selection (Recommended for single device)**

- No configuration file needed
- Automatically connects to the only connected device
- Perfect for development with a single test device

**2. Manual Device Selection**

- Use when you have multiple devices connected
- Specify exact device in configuration file

### Configuration File (Optional)

The configuration file (`config.yaml`) is **optional**. If not present, the server will automatically select the device if only one is connected.

#### For Automatic Selection

Simply ensure only one device is connected and run the server - no configuration needed!

#### For Manual Selection

1. Create a configuration file:

```bash
cp config.yaml.example config.yaml
```

2. Edit `config.yaml` and specify your device:

```yaml
device:
  name: "your-device-serial-here" # Device identifier from 'adb devices'
```

**For auto-selection**, you can use any of these methods:

```yaml
device:
  name: null              # Explicit null (recommended)
  # name: ""              # Empty string
  # name:                 # Or leave empty/comment out
```

### Finding Your Device Serial

To find your device identifier, run:

```bash
adb devices
```

Example output:

```
List of devices attached
13b22d7f        device
emulator-5554   device
```

Use the first column value (e.g., `13b22d7f` or `emulator-5554`) as the device name.

### Usage Scenarios

| Scenario | Configuration Required | Behavior |
|----------|----------------------|----------|
| Single device connected | None | ✅ Auto-connects to the device |
| Multiple devices, want specific one | `config.yaml` with `device.name` | ✅ Connects to specified device |
| Multiple devices, no config | None | ❌ Shows error with available devices |
| No devices connected | N/A | ❌ Shows "no devices" error |

**Note**: If you have multiple devices connected and don't specify which one to use, the server will show an error message listing all available devices.

## Usage

An MCP client is needed to use this server. The Claude Desktop app is an example
of an MCP client. To use this server with Claude Desktop:

1. Locate your Claude Desktop configuration file:

   - Windows: `%APPDATA%\Claude\claude_desktop_config.json`
   - macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`

2. Add the Android MCP server configuration to the `mcpServers` section:

```json
{
  "mcpServers": {
    "android": {
      "command": "path/to/uv",
      "args": ["--directory", "path/to/android-mcp-server", "run", "server.py"]
    }
  }
}
```

Replace:

- `path/to/uv` with the actual path to your `uv` executable
- `path/to/android-mcp-server` with the absolute path to where you cloned this
repository

<https://github.com/user-attachments/assets/c45bbc17-f698-43e7-85b4-f1b39b8326a8>

### Available Tools

The server exposes the following tools:

```python
def get_packages() -> str:
    """
    Get all installed packages on the device.
    Returns:
        str: A list of all installed packages on the device as a string
    """
```

```python
def execute_adb_command(command: str) -> str:
    """
    Executes an ADB command and returns the output.
    Args:
        command (str): The ADB command to execute
    Returns:
        str: The output of the ADB command
    """
```

```python
def get_uilayout() -> str:
    """
    Retrieves information about clickable elements in the current UI.
    Returns a formatted string containing details about each clickable element,
    including their text, content description, bounds, and center coordinates.

    Returns:
        str: A formatted list of clickable elements with their properties
    """
```

```python
def get_screenshot() -> Image:
    """
    Takes a screenshot of the device and returns it.
    Returns:
        Image: the screenshot
    """
```

```python
def get_package_action_intents(package_name: str) -> list[str]:
    """
    Get all non-data actions from Activity Resolver Table for a package
    Args:
        package_name (str): The name of the package to get actions for
    Returns:
        list[str]: A list of all non-data actions from the Activity Resolver
        Table for the package
    """
```

## Contributing

Contributions are welcome!

## Acknowledgments

- Built with
[Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction)


# TTS-Story

A web-based Text-to-Speech application supporting multiple TTS engines including **Kokoro-82M** and **Chatterbox**, with both local GPU inference and Replicate cloud API options for generating multi-voice audiobooks and stories.

## Features

### TTS Engines
- **Multi-Engine Support**: Choose from four TTS engine options:
  - **Kokoro · Local GPU** - Run Kokoro-82M locally on your NVIDIA GPU
  - **Kokoro · Replicate** - Use Kokoro via Replicate cloud API
  - **Chatterbox · Local GPU** - Run Chatterbox locally with voice cloning (~8GB VRAM required)
  - **Chatterbox · Replicate** - Use Chatterbox via Replicate cloud API (`resemble-ai/chatterbox-turbo`)
- **Unified Replicate API**: Single API token works for both Kokoro and Chatterbox Replicate engines
- **Chatterbox Voice Cloning**: Upload your own voice recordings (10-15 seconds recommended) to clone any voice
- **Chatterbox Voice Management**: Add, rename, delete, and preview custom voice prompts with drag-and-drop bulk upload

### Voice & Audio
- **Multi-Voice Support**: Use Kokoro-82M voices for any number of characters in your story
- **Custom Voice Blending**: Mix any combination of Kokoro voices with weighted ratios to create reusable "custom_*" voice codes
- **Speaker Tags & Auto Detection**: Automatically parse `[speaker1]...[/speaker1]` or `[alice]...[/alice]` tags
- **Smart Text Chunking**: Automatically splits long texts into manageable chunks
- **Seamless Audio Merging**: Merges chunks into a single file with configurable crossfade
- **Intro & Inter-Segment Silence Controls**: Dial in precise empty space before the first line and between chunks

### AI & Processing
- **Gemini Pre-Processing**: Automatically decides between whole-text or chapter-based Gemini runs with speaker-memory context
- **Speaker Memory Between Chunks**: Gemini requests carry forward discovered speaker tags for consistency
- **Local GPU Processing**: Run entirely on your machine for privacy and speed
- **Cloud API Option**: Use Replicate API when you don't have local GPU resources

### Job Management & Library
- **Job Queue**: Submit multiple jobs, track real-time progress with ETA, cancel, and download results
- **Job Queue Tab**: Dedicated UI to monitor all jobs with progress bars and chunk counts
- **Audio Library**: Browsable list of all completed outputs with inline players, **engine indicator** showing which TTS engine was used, and delete/clear controls
- **Chapter Collections + Full Audiobook**: Toggle per-chapter outputs and optionally create a single combined audiobook

### UI & Configuration
- **Available Voices & Previews**: Browse all Kokoro voices grouped by language, generate preview samples
- **Configurable Settings**: Control TTS engine, speed, chunk size, output format, bitrate, crossfade
- **Dynamic Gemini Controls**: Save your Gemini API key, fetch the latest available Gemini models on demand
- **Web Interface**: Modern single-page UI built with Flask and vanilla JS

## Available Voices

### Kokoro Voices

TTS-Story exposes the full Kokoro-82M voice set, grouped by language.

### American English 🇺🇸 (lang_code `a`)
- Female: `af_alloy`, `af_aoede`, `af_bella`, `af_heart`, `af_jessica`, `af_kore`, `af_nicole`, `af_nova`, `af_river`, `af_sarah`, `af_sky`
- Male: `am_adam`, `am_echo`, `am_eric`, `am_fenrir`, `am_liam`, `am_michael`, `am_onyx`, `am_puck`, `am_santa`

### British English 🇬🇧 (lang_code `b`)
- Female: `bf_alice`, `bf_emma`, `bf_isabella`, `bf_lily`
- Male: `bm_daniel`, `bm_fable`, `bm_george`, `bm_lewis`

### Spanish 🇪🇸 (lang_code `e`)
- `ef_dora`, `em_alex`, `em_santa`

### French 🇫🇷 (lang_code `f`)
- `ff_siwis`

### Hindi 🇮🇳 (lang_code `h`)
- `hf_alpha`, `hf_beta`, `hm_omega`

### Japanese 🇯🇵 (lang_code `j`)
- `jf_alpha`, `jf_gongitsune`, `jf_nezumi`, `jf_tebukuro`, `jm_kumo`

### Mandarin Chinese 🇨🇳 (lang_code `z`)
- `zf_xiaobei`, `zf_xiaoni`, `zf_xiaoxiao`, `zf_xiaoyi`

### Brazilian Portuguese 🇧🇷 (lang_code `p`)
- `pf_dora`, `pm_alex`, `pm_santa`

All of these voices are browsable in the **Available Voices** tab, where you can generate and play preview samples.

### Chatterbox Voices

Chatterbox supports voice cloning from audio recordings. To add custom voices:

1. Go to the **Available Voices** tab and scroll to the **Chatterbox Available Voices** section
2. Upload a voice recording (WAV, MP3, M4A, FLAC, or OGG format)
   - **Recommended duration**: 10-15 seconds of clear speech
   - Avoid background noise for best results
3. Give the voice a descriptive name and click **Save Voice**
4. Your custom voices appear in all Chatterbox voice dropdowns, sorted alphabetically

You can also drag-and-drop multiple audio files for bulk upload. Each voice can be previewed, renamed, or deleted from the management interface.

## Installation

### Prerequisites
- Python 3.9 or higher
- NVIDIA GPU with CUDA support (optional, for local GPU inference)
- Internet connection (for downloading dependencies)

### Automatic Installation (Recommended)

1. **Clone or download the repository**
```bash
git clone <your-repo-url>
cd TTS-Story
```

2. **Run the setup script**
```bash
setup.bat
```

The setup script will automatically:
- ✅ Detect your Python version
- ✅ Create a Python virtual environment
- ✅ Detect your NVIDIA GPU and CUDA version
- ✅ Install PyTorch with appropriate CUDA support (or CPU-only if no GPU)
- ✅ Download and install espeak-ng automatically
- ✅ Install all other required dependencies
- ✅ Download the Rubber Band CLI and wire it up for high-quality pitch/tempo FX
- ✅ Verify the installation

**Supported CUDA Versions:**
- CUDA 12.9, 12.8, 12.6, 12.4, 12.1
- CUDA 11.8
- CPU-only (automatic fallback if no GPU detected)

3. **Start the application**
```bash
run.bat
```

4. **Open your browser**
```
http://localhost:5000
```

### Manual Installation

If you prefer to install manually or the automatic setup fails:

1. **Install espeak-ng**
   - Download from [espeak-ng releases](https://github.com/espeak-ng/espeak-ng/releases)
   - Install the `espeak-ng-X64.msi` file for Windows

2. **Install Rubber Band CLI (for pitch/tempo FX quality)**
   - Download the Windows zip from [breakfastquay.com/rubberband](https://breakfastquay.com/rubberband/)
   - Extract it and add the folder containing `rubberband.exe` to your `PATH`

2. **Create virtual environment**
```bash
python -m venv venv
venv\Scripts\activate
```

3. **Install PyTorch with CUDA support**
```bash
# For CUDA 12.1 (most common)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# For CPU only
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```

4. **Install other dependencies**
```bash
pip install -r requirements.txt
```

5. **Run the application**
```bash
python app.py
```

## Usage

### Selecting a TTS Engine

TTS-Story supports four TTS engine options. In the **Settings** tab, choose your preferred default engine:

| Engine | Description | Requirements |
|--------|-------------|--------------|
| **Kokoro · Local GPU** | Run Kokoro-82M locally | NVIDIA GPU with CUDA |
| **Kokoro · Replicate** | Kokoro via cloud API | Replicate API token |
| **Chatterbox · Local GPU** | Chatterbox with voice cloning | NVIDIA GPU (~8GB VRAM) |
| **Chatterbox · Replicate** | Chatterbox via cloud API | Replicate API token |

You can also override the engine per-job in the **Generate** tab.

### Basic Workflow

1. Open your browser to `http://localhost:5000`
2. In **Settings**, select your preferred TTS engine
3. If using Replicate engines, enter your API token in the **Replicate API** section
4. Paste your text with or without speaker tags in the **Generate** tab
5. Select a **Default Voice** (used for plain text / unassigned speakers)
6. If you use speaker tags, TTS-Story automatically analyzes the text and lets you assign voices per speaker
7. Click **Generate Audio**
8. The job is added to the **Job Queue**, processed in the background, and the result appears in:
   - **Job Queue** tab (with real-time progress, ETA, and player)
   - **Library** tab (all past generations with engine indicator)

### Using Chatterbox Voice Cloning

When using a Chatterbox engine:

1. First, add voice recordings in **Available Voices → Chatterbox Available Voices**
2. In the **Generate** tab, select your cloned voice from the **Reference Voice** dropdown
3. Each speaker can use a different cloned voice for multi-character stories

### Quick Test Previews

- A shared "Quick Test Text" field lives above the Assigned Voices section so you can type once and preview any speaker with matching FX.
- Each speaker row includes an inline Quick Test button beside the tone controls.

**Note:** Local GPU modes run entirely on your machine and never use cloud APIs, ensuring complete privacy and no API costs.

### Silence & Timing Controls

- **Intro Silence (ms):** Adds empty space before the very first spoken line
- **Silence Between Segments (ms):** Inserts a gap after each chunk/line before the next one begins
- Both settings are configurable in the **Generation Settings** panel (0–2000 ms, 100 ms steps)

### Replicate API Setup

Both Kokoro · Replicate and Chatterbox · Replicate use the same API token:

1. Get your API key from [Replicate](https://replicate.com) (starts with `r8_...`)
2. In **Settings**, enter your token in the **Replicate API** section
3. Click **Save Settings**
4. Select either Replicate engine from the dropdown

### Speaker Tag Format

You can use either numbered speakers or named speakers:

**Numbered Format:**
```
[speaker1]Hello, my name is Alice.[/speaker1]
[speaker2]Nice to meet you, Alice! I'm Bob.[/speaker2]
[speaker1]It's great to meet you too![/speaker1]
```

**Named Format:**
```
[narrator]Once upon a time, in a land far away...[/narrator]
[alice]Hello, my name is Alice.[/alice]
[bob]Nice to meet you, Alice! I'm Bob.[/bob]
[narrator]And so their adventure began.[/narrator]
```

You can use any alphanumeric name (letters, numbers, underscores). The system will automatically detect all unique speakers and let you assign voices to each one.

### Gemini Pre-Processing Workflow

Need to tidy a manuscript or add consistent speaker tags before running TTS? Use the **Prep Text with Gemini** button:

1. Enter your Gemini API key and model in **Settings**, then click **Fetch Available Models** if you want to load the latest list directly from Google.
2. Paste your story in the **Generate** tab and decide whether "Generate separate audio files for each chapter" should be enabled.
3. Select a **Prompt Preset** (see below) or write your own custom prompt.
4. Click **Prep Text with Gemini**:
   - If chapter splitting is enabled, TTS-Story reuses the detected chapter list and sends each one to Gemini separately with your pre-prompt and the running speaker list.
   - If chapter splitting is disabled, the whole manuscript (plus pre-prompt) is sent in a single Gemini request to respect the context window.
   - A real-time progress bar shows which chapter or full-text step is running.
5. When Gemini finishes, the cleaned/expanded narrative replaces the input field. Chapter headings stay inside the narrator tags so audio splitting still works.
6. Re-run **Analyze Text** if needed. Your voice assignments and FX settings remain untouched unless you explicitly reset them.

Because the speaker list is tracked across sections, characters that appear later continue to use the same tag, which keeps the voice assignment UI tidy and prevents duplicate dropdowns.

#### Pre-loaded Prompt Presets

TTS-Story includes three pre-configured Gemini prompt presets optimized for different use cases:

| Preset | Best For | Description |
|--------|----------|-------------|
| **Chatterbox Natural Dialogue Conversation** | Chatterbox engines | Transforms text into natural-sounding dialogue with paralinguistic tags (laughter, sighs, pauses) and human speech quirks. Ideal for conversational content where you want expressive, lifelike output. |
| **Chatterbox Audio Book Conversion** | Chatterbox engines | Maintains strict adherence to the original text while converting symbols and abbreviations that TTS engines struggle with into speakable words (e.g., "/" → "slash", "-" → "dash", "Dr." → "Doctor"). |
| **Kokoro Audio Book Conversion** | Kokoro engines | Preserves the exact text of the book while adding speaker tags and preparing the content for TTS conversion. Focuses on accurate speaker identification and proper text segmentation without modifying the original prose. |

Select a preset from the dropdown in the Gemini section, or create your own custom prompts and save them for reuse.

### Plain Text Mode

If no speaker tags are found, the entire text will be processed with a single voice.

### Job Queue & Library

- **Job Queue** tab shows all jobs with:
  - Real-time progress bars and chunk counts
  - ETA estimates during processing
  - Status indicators (`queued`, `processing`, `completed`, `failed`, `cancelled`)
  - Per-job controls (cancel, download)
- **Library** tab lists all completed outputs (sorted newest first) with:
  - **Engine indicator** showing which TTS engine was used (Kokoro, Kokoro Replicate, Chatterbox, Chatterbox Replicate)
  - Inline audio players
  - Download links
  - Delete and "Clear All" controls

### Available Voices & Previews

- **Available Voices** tab lists all Kokoro-82M voices grouped by language.
- You can:
  - Generate preview samples for all voices
  - Regenerate (overwrite) samples if you change text or update voices
  - Click any voice to play its preview sample

### Custom Voice Blends

- Open the **Custom Voice Blends** panel inside the **Available Voices** tab to create bespoke voices.
- Click **New Custom Voice** (or Edit on any card) to open the modal where you can:
  - Name the blend and choose its language group (lang_code)
  - Add one or more component voices and set their mix weights (e.g., 0.5 narrator + 0.5 af_heart)
  - Optionally add notes for future reference
- Saved blends appear in the grid with metadata (code, language, updated time) and can be edited or deleted at any time.
- All custom voices automatically show up in:
  - Default voice dropdowns
  - Per-speaker assignment selects (grouped by language under “Custom Blends” optgroups)
  - `/api/voices` responses (`custom_voices` arrays per language) so automation scripts can use them.
- When the generator encounters a `custom_*` voice, the backend blends the component embeddings on the fly and caches the tensor for fast reuse.

> Tip: The API exposes the full CRUD workflow under `/api/custom-voices`, so you can script voice creation or keep predefined blends in source control.

## Configuration

Settings are stored in `config.json`:

```json
{
  "replicate_api_key": "",
  "chunk_size": 500,
  "sample_rate": 24000,
  "speed": 1.0,
  "output_format": "mp3",
  "output_bitrate_kbps": 128,
  "crossfade_duration": 0.1,
  "intro_silence_ms": 0,
  "inter_chunk_silence_ms": 0,
  "tts_engine": "kokoro",
  "chatterbox_turbo_local_device": "auto",
  "chatterbox_turbo_local_temperature": 0.8,
  "chatterbox_turbo_replicate_model": "resemble-ai/chatterbox-turbo",
  "gemini_api_key": "",
  "gemini_model": "gemini-2.0-flash"
}
```

### TTS Engine Options

| Value | Description |
|-------|-------------|
| `kokoro` | Kokoro-82M local GPU inference |
| `kokoro_replicate` | Kokoro via Replicate cloud API |
| `chatterbox_turbo_local` | Chatterbox local GPU with voice cloning |
| `chatterbox_turbo_replicate` | Chatterbox via Replicate cloud API |

Any settings you override in the Generate tab (format, bitrate, engine) are sent along with the job payload while keeping the saved defaults intact.

## Project Structure

```
TTS-Story/
├── app.py                 # Flask web server
├── requirements.txt       # Python dependencies
├── setup.bat             # Windows setup script
├── run.bat               # Windows run script
├── config.json           # Configuration file
├── src/
│   ├── tts_engine.py     # TTS engine registry and factory
│   ├── replicate_api.py  # Replicate API integration (Kokoro)
│   ├── text_processor.py # Text chunking and parsing
│   ├── audio_merger.py   # Audio file merging
│   ├── voice_manager.py  # Voice configuration and preview sample metadata
│   ├── voice_sample_generator.py # Batch generation of voice preview samples
│   └── engines/
│       ├── kokoro_engine.py              # Kokoro-82M local engine
│       ├── chatterbox_turbo_local_engine.py    # Chatterbox local GPU engine
│       └── chatterbox_turbo_replicate_engine.py # Chatterbox Replicate engine
├── static/
│   ├── css/
│   │   └── style.css
│   ├── js/
│   │   ├── main.js
│   │   ├── queue.js
│   │   ├── library.js
│   │   ├── voice-manager.js
│   │   └── settings.js
│   ├── audio/            # Generated audio files (per-job subdirectories)
│   └── samples/          # Voice preview samples and manifest.json
├── data/
│   └── voice_prompts/    # Chatterbox voice recordings for cloning
└── templates/
    └── index.html        # Web interface
```

## API Endpoints

- `GET /` - Main web interface
- `GET /api/health` - Health check (TTS engine, Kokoro availability, CUDA status)
- `GET /api/voices` - Get available voices and preview sample status
- `POST /api/voices/samples` - Generate or regenerate voice preview samples
- `GET /api/settings` - Get current settings
- `POST /api/settings` - Update settings
- `POST /api/analyze` - Analyze text and return statistics/speakers
- `POST /api/gemini/sections` - Preview the sections (chapters/chunks) Gemini will process for a given input
- `POST /api/gemini/process-section` - Send a single section to Gemini (called in sequence by the frontend for live progress updates)
- `POST /api/gemini/process` - Process the entire text through Gemini in one backend call (used for scripted workflows)
- `POST /api/gemini/models` - Fetch available Gemini models after providing an API key
- `POST /api/generate` - Queue a new audio generation job
- `GET /api/status/<job_id>` - Check status of a specific job
- `POST /api/cancel/<job_id>` - Cancel a queued or running job
- `GET /api/queue` - Get all jobs, their status, and current queue size
- `GET /api/download/<job_id>` - Download generated audio file
- `GET /api/library` - List all completed audio files
- `DELETE /api/library/<job_id>` - Delete a specific library item
- `POST /api/library/clear` - Delete all library items
- `GET /api/custom-voices` - List custom voice blends (includes normalized metadata and component weights)
- `POST /api/custom-voices` - Create a new custom voice blend
- `GET /api/custom-voices/<voice_id>` - Retrieve a specific custom voice (ID or `custom_` code)
- `PUT /api/custom-voices/<voice_id>` - Update an existing custom voice blend
- `DELETE /api/custom-voices/<voice_id>` - Delete a custom voice and invalidate cached tensors
- `GET /api/chatterbox-voices` - List saved Chatterbox voice prompts
- `POST /api/chatterbox-voices` - Upload a new Chatterbox voice prompt
- `PUT /api/chatterbox-voices/<voice_id>` - Rename a Chatterbox voice
- `DELETE /api/chatterbox-voices/<voice_id>` - Delete a Chatterbox voice
- `GET /api/chatterbox-voices/<voice_id>/preview` - Preview a Chatterbox voice

## Performance

### Kokoro · Local GPU (NVIDIA RTX 3090)
- ~2 seconds per chunk (500 words)
- No API costs
- Full privacy

### Kokoro · Replicate
- ~2-3 seconds per chunk (varies by input)
- Cost varies by usage
- No GPU required
- Model: [jaaari/kokoro-82m](https://replicate.com/jaaari/kokoro-82m)

### Chatterbox · Local GPU
- Requires ~8GB VRAM
- Voice cloning from 10-15 second audio samples
- No API costs
- Full privacy

### Chatterbox · Replicate
- Voice cloning via cloud API
- Model: [resemble-ai/chatterbox-turbo](https://replicate.com/resemble-ai/chatterbox-turbo)
- Cost varies by usage
- No GPU required

## Troubleshooting

### espeak-ng not found
Make sure espeak-ng is installed and in your PATH.

### CUDA out of memory
Reduce chunk_size in settings or use a Replicate engine instead of local GPU.

### Audio quality issues
Adjust the speed parameter (0.5 - 2.0) in settings.

## License

Apache 2.0 - Same as Kokoro-82M

## Credits

- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad
- [Chatterbox](https://github.com/resemble-ai/chatterbox) by Resemble AI
- [StyleTTS2](https://github.com/yl4579/StyleTTS2) by yl4579
- [Replicate](https://replicate.com) for cloud API

## Support

For issues and questions, please open an issue on GitHub.


hle-eval-ollama

_Want to see how your favorite local LLMs fare against Humanity's Last Exam? What difference quantizations may make?_

This repo aims to **allow anyone to get up and running with [Humanity's Last Exam](https://lastexam.ai/) (or similar benchmarks!) and Ollama locally**.

The official repo with evaluation scripts by HLE is notoriously hard to use, only lightly documented and merely made to work with the OpenAI API. While Ollama exposes an OpenAI API compatible endpoint, this project aims for a two-way approach, featuring both a pure, Ollama-agnostic API implementation and an OpenAI API compatible backend to show what's possible.

> [!IMPORTANT]
> The whole quality of the benchmark results bases on how good the judge model does its job. If it judges poorly, good models might look worse and bad models better. Make sure to choose a strong model and verify results yourself.
>
> There are ongoing problems with the quality of the judge model's responses. Often, answers are still misjudged. Please exercise caution or manually review until cutting-edge models are able to correctly identify correct and wrong responses consistently.

## How to use it

It's luckily simple! First of all, make sure to have a [Hugging Face](https://huggingface.co/) account. The HLE dataset is **gated**, which means that you will need to authenticate in order to use it. You may want to visit the [HLE page](https://huggingface.co/datasets/cais/hle) on Hugging Face and agree to your information being submitted.

```bash
python3 -m venv .venv
. ./.venv/bin/activate
pip install -r ./requirements.txt
```

Generate a Hugging Face access token [here](https://huggingface.co/settings/tokens) and copy it to your clipboard.
Then, run

```
huggingface-cli login
```

Now you're all set! For example, run

```bash
python3 ./src/eval.py --model=gemma3 --judge=llama3:8b --num-questions=150
```

to begin the exam for the model! Results will also be written to an output file in the project root directory, ending in `.results.json`.

**Tip**: You can specify several models separated by commas in order to make them compete against each other. You can - and must - only specify one judge model (the model that will rate the answers), and it's highly recommended to choose a model that isn't part of the models taking the exam.

**Important**: Do not just perform separate runs with `--num-questions` specified, as this will pick different, random questions from the dataset for each run individually. If you want to compare models with a limited number of questions, use the tip described above.

**For text-only models** specify `--only-text` to only use the text subset of the HLE dataset.

> [!TIP]
> You can also use any OpenAI API compatible endpoint by providing the `--backend=openai` flag. Make sure to set the `HLE_EVAL_API_KEY` and `HLE_EVAL_ENDPOINT` environment variables.
>
> **PLEASE NOTE** that image input (vision) is still unstable for OpenAI endpoints - while it works, it consumes an absurd amount of tokens (which you may be billed for!) and is not recommended for use. You can use a lighter variant by setting `USE_EXPERIMENTAL_IMAGE_UPLOAD` to `True` in `src/constants.py`, but this does not work for every endpoint.

## Environment variables

- `HLE_EVAL_ENDPOINT`: specifies the host to connect to.
- `HLE_EVAL_API_KEY`: specifies the Bearer token to use for authentication.

## Thanks

Huge thanks to the [creators](https://github.com/centerforaisafety/hle/blob/main/citation.txt) of Humanity's last exam for the extraordinarily hard questions!

Also, huge thanks to the Ollama contributors and creators of all the [packages](./requirements.txt) used for this project!


# zclaw

The smallest possible AI personal assistant for ESP32.

zclaw is written in C and runs on ESP32 boards with a strict all-in firmware budget target of **<= 888 KiB** on the default build. It supports scheduled tasks, GPIO control, persistent memory, and custom tool composition through natural language.

The **888 KiB** cap is all-in firmware size, not just app code.
It includes `zclaw` logic plus ESP-IDF/FreeRTOS runtime, Wi-Fi/networking, TLS/crypto, and cert bundle overhead.

Fun to use, fun to hack on.

## Full Documentation

Use the docs site for complete guides and reference.

- [Full documentation](https://zclaw.dev)
- [Use cases: useful + fun](https://zclaw.dev/use-cases.html)
- [Changelog (web)](https://zclaw.dev/changelog.html)
- [Complete README (verbatim)](https://zclaw.dev/reference/README_COMPLETE.md)

## Quick Start

One-line bootstrap (macOS/Linux):

```bash
bash <(curl -fsSL https://raw.githubusercontent.com/tnm/zclaw/main/scripts/bootstrap.sh)
```

Already cloned?

```bash
./install.sh
```

Non-interactive install:

```bash
./install.sh -y
```

Setup notes

- `bootstrap.sh` clones/updates the repo and then runs `./install.sh`. You can inspect/verify the bootstrap flow first (including `ZCLAW_BOOTSTRAP_SHA256` integrity checks); see the [Getting Started docs](https://zclaw.dev/getting-started.html).
- For encrypted credentials in flash, use secure mode (`--flash-mode secure` in install flow, or `./scripts/flash-secure.sh` directly).
- After flashing, provision WiFi + LLM credentials with `./scripts/provision.sh`.
- You can re-run either `./scripts/provision.sh` or `./scripts/provision-dev.sh` at any time (no reflash required) to update runtime credentials: WiFi SSID/password, LLM backend/model/API key, and Telegram token/chat ID.
- Default LLM rate limits are `100/hour` and `1000/day`; change compile-time limits in `main/config.h` (`RATELIMIT_*`).
- Quick validation path: run `./scripts/web-relay.sh` and send a test message to confirm the device can answer.
- If serial port is busy, run `./scripts/release-port.sh` and retry.
- For repeat local reprovisioning without retyping secrets, use `./scripts/provision-dev.sh` with a local profile file (`provision-dev.sh` wraps `provision.sh --yes`).

## Highlights

- Chat via Telegram or hosted web relay
- Timezone-aware schedules (`daily`, `periodic`, and one-shot `once`)
- Built-in + user-defined tools
- GPIO read/write control with guardrails (including bulk `gpio_read_all`)
- Persistent memory across reboots
- Persona options: `neutral`, `friendly`, `technical`, `witty`
- Provider support for Anthropic, OpenAI, and OpenRouter

## Hardware

Tested targets: **ESP32-C3**, **ESP32-S3**, and **ESP32-C6**.
Other ESP32 variants should work fine (some may require manual ESP-IDF target setup).
Tests reports are very welcome!

Recommended starter board: [Seeed XIAO ESP32-C3](https://www.seeedstudio.com/Seeed-XIAO-ESP32C3-p-5431.html)

## Local Dev & Hacking

Typical fast loop:

```bash
./scripts/test.sh host
./scripts/build.sh
./scripts/flash.sh --kill-monitor /dev/cu.usbmodem1101
./scripts/provision-dev.sh --port /dev/cu.usbmodem1101
./scripts/monitor.sh /dev/cu.usbmodem1101
```

Profile setup once, then re-use:

```bash
./scripts/provision-dev.sh --write-template
# edit ~/.config/zclaw/dev.env
./scripts/provision-dev.sh --show-config
./scripts/provision-dev.sh

# if Telegram keeps replaying stale updates:
./scripts/telegram-clear-backlog.sh --show-config
```

More details in the [Local Dev & Hacking guide](https://zclaw.dev/local-dev.html).

### Other Useful Scripts

Show scripts

- `./scripts/flash-secure.sh` - Flash with encryption
- `./scripts/provision.sh` - Provision credentials to NVS
- `./scripts/provision-dev.sh` - Local profile wrapper for repeat provisioning
- `./scripts/telegram-clear-backlog.sh` - Clear queued Telegram updates
- `./scripts/erase.sh` - Erase NVS only (`--nvs`) or full flash (`--all`) with guardrails
- `./scripts/monitor.sh` - Serial monitor
- `./scripts/emulate.sh` - Run QEMU profile
- `./scripts/web-relay.sh` - Hosted relay + mobile chat UI
- `./scripts/benchmark.sh` - Benchmark relay/serial latency
- `./scripts/test.sh` - Run host/device test flows
- `./scripts/test-api.sh` - Run live provider API checks (manual/local)

## Size Breakdown

Current default `esp32s3` breakdown (grouped loadable image bytes from `idf.py -B build size-components`; rows sum to total image size):

| Segment | Bytes | Size | Share |
| --- | ---: | ---: | ---: |
| zclaw app logic (`libmain.a`) | `35742` | ~34.9 KiB | ~4.1% |
| Wi-Fi + networking stack | `397356` | ~388.0 KiB | ~45.7% |
| TLS/crypto stack | `112922` | ~110.3 KiB | ~13.0% |
| cert bundle + app metadata | `99722` | ~97.4 KiB | ~11.5% |
| other ESP-IDF/runtime/drivers/libc | `224096` | ~218.8 KiB | ~25.8% |

Total image size from this build is `869838` bytes; padded `zclaw.bin` is `869952` bytes (~849.6 KiB), still under the cap.

## Latency Benchmarking

Relay path benchmark (includes web relay processing + device round trip):

```bash
./scripts/benchmark.sh --mode relay --count 20 --message "ping"
```

Direct serial benchmark (host round trip + first response time). If firmware logs
`METRIC request ...` lines, the report also includes device-side timing:

```bash
./scripts/benchmark.sh --mode serial --serial-port /dev/cu.usbmodem1101 --count 20 --message "ping"
```

## License

MIT


# ASH-AutoSheets

Autodesk Build (ACC) sheet and document upload automation for ASH Enterprises.

## Features

- **Mobile-First Web UI** - Drag-and-drop PDF uploads from any device
- **SMB Share Monitoring** - Auto-detects new PDFs in `acc_upload/` folders
- **Fuzzy Project Matching** - Matches folder names to ACC projects automatically
- **Auto-Publish Sheets** - Uploads and publishes sheets in one step
- **Activity Log** - Real-time tracking of all uploads

## Quick Start

### Prerequisites

- Python 3.10+
- Access to ASH Enterprises SMB share (mounted at `/mnt/ash-shared`)
- Autodesk Construction Cloud (ACC) account with API access

### Installation

```bash
# Clone and enter directory
cd /home/crogers2287/accdrop

# Copy environment file
cp .env.example .env

# Edit .env with your credentials (see below)
nano .env

# Run setup helper (will guide you through ACC setup)
./run.sh setup_acc.py

# Start the application
./run.sh
```

### ACC Setup (One-Time)

The application uses Autodesk's 2-Legged OAuth (Client Credentials). You need to:

1. **Create an APS App** (already done)
   - Client ID and Secret are in `.env`

2. **Register in ACC Admin Console**
   - Go to: https://acc.autodesk.com/admin
   - Navigate to: Account Admin → Custom Integrations
   - Click "Add Custom Integration"
   - Enter Client ID: `eGJABtpxUP8Ey02MBKhxxWiH4IZn3qgSe8HGohGcZ9AzyBZ7`
   - Grant permissions: Document Management, Sheets API

3. **Get Your Account ID**
   - From the ACC Admin URL: `https://acc.autodesk.com/admin/accounts/{ACCOUNT_ID}/...`
   - Or run `python setup_acc.py` after registration

4. **Update .env**
   ```
   ACC_ACCOUNT_ID=your_account_id_here
   ```

## Usage

### Web Interface

Access at: http://acc.ashent.us (or http://localhost:8000)

1. Select a project from the dropdown
2. Choose upload type (Sheet or Document)
3. Drag-and-drop a PDF file
4. Monitor progress in the Activity Log

### SMB Folder Upload

Drop PDF files into project folders:

```
/mnt/ash-shared/Projects/
├── 25-028 BWW 2 Go Mulberry Fl/
│   └── acc_upload/
│       ├── sheets/    ← Drop sheet PDFs here
│       └── docs/      ← Drop document PDFs here
└── 25-029 Another Project/
    └── acc_upload/
        ├── sheets/
        └── docs/
```

Files are automatically:
1. Detected by the watchdog service
2. Matched to ACC projects (fuzzy matching)
3. Uploaded to Autodesk Build
4. Moved to `_Completed/` on success or `_Error/` on failure

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      ASH-AutoSheets                          │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │   FastAPI    │    │   Watchdog   │    │   Upload     │  │
│  │   Server     │    │   Service    │    │   Processor  │  │
│  │   (Web UI)   │    │   (SMB)      │    │   (Queue)    │  │
│  └──────┬───────┘    └──────┬───────┘    └──────┬───────┘  │
│         │                   │                   │          │
│         └───────────────────┴───────────────────┘          │
│                             │                               │
│                    ┌────────┴────────┐                     │
│                    │   ACC Client    │                     │
│                    │   (API + OAuth) │                     │
│                    └────────┬────────┘                     │
│                             │                               │
└─────────────────────────────┼───────────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │  Autodesk Build │
                    │  (ACC API)      │
                    └─────────────────┘
```

## API Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/` | GET | Web UI |
| `/health` | GET | Health check |
| `/api/projects` | GET | List ACC projects |
| `/api/upload/sheet` | POST | Upload PDF as sheet |
| `/api/upload/document` | POST | Upload PDF as document |
| `/api/activity` | GET | Get upload activity log |
| `/api/mappings` | GET/POST/DELETE | Manage project mappings |

## Configuration

Environment variables (`.env`):

| Variable | Description | Default |
|----------|-------------|---------|
| `ACC_CLIENT_ID` | APS application client ID | Required |
| `ACC_CLIENT_SECRET` | APS application client secret | Required |
| `ACC_ACCOUNT_ID` | ACC account ID | Required |
| `WATCH_DIR` | SMB share path | `/mnt/ash-shared/Projects` |
| `HOST` | Server bind address | `0.0.0.0` |
| `PORT` | Server port | `8000` |

## Troubleshooting

### "No hubs found" when running setup

The app isn't registered in ACC Admin Console yet. Follow the ACC Setup steps above.

### "OAuth authentication failed"

Check that `ACC_CLIENT_ID` and `ACC_CLIENT_SECRET` in `.env` are correct.

### Files not being detected from SMB

1. Check the watch directory exists: `ls /mnt/ash-shared/Projects`
2. Ensure files are in `acc_upload/sheets/` or `acc_upload/docs/` folders
3. Check logs: `journalctl -u ash-autosheets -f` (if running as service)

### Upload failing with 404

The project wasn't found. Check:
1. Project exists in ACC
2. Folder name matches project name closely enough (90% match required)
3. Add a manual mapping if needed via the web UI

## Production Deployment

### Systemd Service

```bash
sudo tee /etc/systemd/system/ash-autosheets.service << 'EOF'
[Unit]
Description=ASH-AutoSheets
After=network.target

[Service]
Type=simple
User=crogers2287
WorkingDirectory=/home/crogers2287/accdrop
ExecStart=/home/crogers2287/accdrop/.venv/bin/python main.py
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable ash-autosheets
sudo systemctl start ash-autosheets
```

### Nginx Reverse Proxy

```nginx
server {
    listen 80;
    server_name acc.ashent.us;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket support (for future real-time features)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
```

## License

Internal use only - ASH Enterprises


# Phone Agent Management Tools

Tools for managing and automating the Pixel 7 Pro phone agent running OpenClaw in Termux.

## Architecture

```
Fred (192.168.1.195)                    Pixel 7 Pro (100.94.190.4:8022)
├── fred/phone-monitor.sh  ──SSH──→     ├── pixel/pixel-health.sh
│   (monitors health, auto-fixes)       ├── pixel/adb-ui-helper.py
│                                       │   (Gemini vision → tap coordinates)
└── Clinker (OpenClaw agent)            ├── pixel/social-poster.py
    (orchestrates tasks)                │   (Reddit/LinkedIn/X posting)
                                        └── pixel/send-screenshot.sh
                                            (relay screenshots to Fred)
```

## Tools

### Fred-side (runs on the main server)

#### `fred/phone-monitor.sh`
Monitors the Pixel's health from Fred via SSH.

```bash
# Basic check
bash fred/phone-monitor.sh

# JSON output (for scripts/cron)
bash fred/phone-monitor.sh --json

# Auto-fix issues (restart gateway, reconnect ADB)
bash fred/phone-monitor.sh --fix

# Cron: check every 5 minutes
*/5 * * * * /home/crogers2287/phone-agent-tools/fred/phone-monitor.sh --fix >> /tmp/phone-monitor.log 2>&1
```

Checks: SSH connectivity, OpenClaw gateway, ADB connection, battery, gateway log errors, session file size.

### Pixel-side (runs on the phone in Termux)

#### `pixel/pixel-health.sh`
Self-check that runs on the phone itself.

```bash
bash pixel/pixel-health.sh          # Human-readable
bash pixel/pixel-health.sh --json   # JSON output
```

Checks: ADB, sshd (restarts if dead), battery, storage, OpenClaw gateway.

#### `pixel/adb-ui-helper.py`
Vision-based UI element finder. Uses Gemini API to analyze screenshots and find tap coordinates. Solves the Jetpack Compose problem where uiautomator XML dumps are empty.

```bash
# Find an element's coordinates
python3 pixel/adb-ui-helper.py find "the Post button"

# Find and tap an element
python3 pixel/adb-ui-helper.py tap "the comment input box"

# Type text using clipboard method (handles special chars)
python3 pixel/adb-ui-helper.py type "Hello world" --target "the text input field"

# Verify an action completed
python3 pixel/adb-ui-helper.py verify "a LinkedIn post was published"

# Take a screenshot
python3 pixel/adb-ui-helper.py screenshot

# Launch an app
python3 pixel/adb-ui-helper.py launch com.reddit.frontpage

# Wait for an element to appear
python3 pixel/adb-ui-helper.py wait "the subreddit feed loaded" --timeout 20

# Press a key
python3 pixel/adb-ui-helper.py key back
python3 pixel/adb-ui-helper.py key enter
```

#### `pixel/social-poster.py`
High-level social media posting with built-in verification.

```bash
# Comment on a Reddit post
python3 pixel/social-poster.py reddit-comment \
  --subreddit FLMedicalTrees \
  --text "Great recommendation!"

# Create a Reddit post
python3 pixel/social-poster.py reddit-post \
  --subreddit FLMedicalTrees \
  --title "My experience with..." \
  --body "Post body here"

# Create a LinkedIn post
python3 pixel/social-poster.py linkedin-post \
  --text "Check out our latest blog post" \
  --url "https://example.com/blog"

# Post a tweet
python3 pixel/social-poster.py twitter-post \
  --text "Exciting news from CannaDeals!"
```

Each command: launches native app → navigates → inputs text via clipboard → submits → verifies with screenshot + vision → reports success/failure with proof.

#### `pixel/send-screenshot.sh`
Take a screenshot and relay it to Fred for analysis.

```bash
bash pixel/send-screenshot.sh                 # Default name, sends to Fred
bash pixel/send-screenshot.sh my_screenshot   # Custom name
bash pixel/send-screenshot.sh --local-only    # Save locally, don't send
```

## Deployment

```bash
# From Fred, deploy tools to the Pixel:
scp -P 8022 -r pixel/* 100.94.190.4:~/phone-tools/

# On the Pixel, make scripts executable:
chmod +x ~/phone-tools/*.sh ~/phone-tools/*.py
```

## Dependencies

**Pixel (Termux):**
- `adb` (via `pkg install android-tools`)
- `python3` (via `pkg install python`)
- `termux-api` (via `pkg install termux-api` + Termux:API app)
- `openssh` (for sshd, via `pkg install openssh`)

**Fred:**
- `ssh` with key auth to Pixel on port 8022
- `bash`

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `GEMINI_API_KEY` | (built-in) | Gemini API key for vision analysis |

## Known Limitations

- **Jetpack Compose apps** (Reddit, LinkedIn, Twitter): uiautomator XML is sparse. The vision-based approach (screenshot → Gemini → coordinates) works but is slower (~2-3s per element lookup).
- **Text input**: Always uses clipboard method (`termux-clipboard-set` + `keyevent 279`). Direct `adb shell input text` breaks on special characters.
- **Rate limits**: Social media apps may rate-limit posting. Add delays between actions.
- **Gemini API**: Free tier has rate limits. If you hit them, add delays or switch to a paid key.


## 🎙 VibeVoice: A Frontier Long Conversational Text-to-Speech Model
[![Project Page](https://img.shields.io/badge/Project-Page-blue?logo=microsoft)](https://microsoft.github.io/VibeVoice)
[![Hugging Face](https://img.shields.io/badge/HuggingFace-Collection-orange?logo=huggingface)](https://huggingface.co/collections/microsoft/vibevoice-68a2ef24a875c44be47b034f)
[![Technical Report](https://img.shields.io/badge/Technical-Report-red?logo=adobeacrobatreader)](https://arxiv.org/pdf/2508.19205)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/VibeVoice_colab.ipynb)
[![Live Playground](https://img.shields.io/badge/Live-Playground-green?logo=gradio)](https://aka.ms/VibeVoice-Demo)
[![Colab](https://img.shields.io/badge/Run-Colab-orange?logo=googlecolab)](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/VibeVoice_colab.ipynb)

<picture>
  <source media="(prefers-color-scheme: dark)" srcset="Figures/VibeVoice_logo_white.png">

</picture>

VibeVoice is a novel framework designed for generating **expressive**, **long-form**, **multi-speaker** conversational audio, such as podcasts, from text. It addresses significant challenges in traditional Text-to-Speech (TTS) systems, particularly in scalability, speaker consistency, and natural turn-taking.

A core innovation of VibeVoice is its use of continuous speech tokenizers (Acoustic and Semantic) operating at an ultra-low frame rate of 7.5 Hz. These tokenizers efficiently preserve audio fidelity while significantly boosting computational efficiency for processing long sequences. VibeVoice employs a [next-token diffusion](https://arxiv.org/abs/2412.08635) framework, leveraging a Large Language Model (LLM) to understand textual context and dialogue flow, and a diffusion head to generate high-fidelity acoustic details.

The model can synthesize speech up to **90 minutes** long with up to **4 distinct speakers**, surpassing the typical 1-2 speaker limits of many prior models.

### 🔥 News

- **[2025-08-26] 🎉 We Open Source the [VibeVoice-7B-Preview](https://huggingface.co/WestZhang/VibeVoice-Large-pt) model weights!**
- **[2025-08-28] 🎉 We provide a [Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/VibeVoice_colab.ipynb) script for easy access to our model. Due to GPU memory limitations, only VibeVoice-1.5B is supported.**

### 📋 TODO

- [ ] Merge models into official Hugging Face repository
- [ ] Release example training code and documentation
- [ ] VibePod:  End-to-end solution that creates podcasts from documents, webpages, or even a simple topic.

### 🎵 Demo Examples

**Video Demo**

We produced this video with [Wan2.2](https://github.com/Wan-Video/Wan2.2). We sincerely appreciate the Wan-Video team for their great work.

**English**

https://github.com/user-attachments/assets/0967027c-141e-4909-bec8-091558b1b784

**Chinese**

https://github.com/user-attachments/assets/322280b7-3093-4c67-86e3-10be4746c88f

**Cross-Lingual**

https://github.com/user-attachments/assets/838d8ad9-a201-4dde-bb45-8cd3f59ce722

**Spontaneous Singing**

https://github.com/user-attachments/assets/6f27a8a5-0c60-4f57-87f3-7dea2e11c730

**Long Conversation with 4 people**

https://github.com/user-attachments/assets/a357c4b6-9768-495c-a576-1618f6275727

For more examples, see the [Project Page](https://microsoft.github.io/VibeVoice).

Try it on [Colab](https://colab.research.google.com/github/microsoft/VibeVoice/blob/main/demo/VibeVoice_colab.ipynb) or [Demo](https://aka.ms/VibeVoice-Demo).

## Models
| Model | Context Length | Generation Length |  Weight |
|-------|----------------|----------|----------|
| VibeVoice-0.5B-Streaming | - | - | On the way |
| VibeVoice-1.5B | 64K | ~90 min | [HF link](https://huggingface.co/microsoft/VibeVoice-1.5B) |
| VibeVoice-7B-Preview| 32K | ~45 min | [HF link](https://huggingface.co/WestZhang/VibeVoice-Large-pt) |

## Installation
We recommend to use NVIDIA Deep Learning Container to manage the CUDA environment.

1. Launch docker
```bash
# NVIDIA PyTorch Container 24.07 / 24.10 / 24.12 verified.
# Later versions are also compatible.
sudo docker run --privileged --net=host --ipc=host --ulimit memlock=-1:-1 --ulimit stack=-1:-1 --gpus all --rm -it  nvcr.io/nvidia/pytorch:24.07-py3

## If flash attention is not included in your docker environment, you need to install it manually
## Refer to https://github.com/Dao-AILab/flash-attention for installation instructions
# pip install flash-attn --no-build-isolation
```

2. Install from github
```bash
git clone https://github.com/microsoft/VibeVoice.git
cd VibeVoice/

pip install -e .
```

## Usages

### 🚨 Tips
We observed users may encounter occasional instability when synthesizing Chinese speech. We recommend:

- Using English punctuation even for Chinese text, preferably only commas and periods.
- Using the 7B model variant, which is considerably more stable.
- If you found the generated voice speak too fast. Please try to chunk your text with multiple speaker turns with same speaker label.

We'd like to thank [PsiPi](https://huggingface.co/PsiPi) for sharing an interesting way for emotion control. Detials can be found via [discussion12](https://huggingface.co/microsoft/VibeVoice-1.5B/discussions/12).

### Usage 1: Launch Gradio demo
```bash
apt update && apt install ffmpeg -y # for demo

# For 1.5B model
python demo/gradio_demo.py --model_path microsoft/VibeVoice-1.5B --share

# For 7B model
python demo/gradio_demo.py --model_path WestZhang/VibeVoice-Large-pt --share
```

### Usage 2: Inference from files directly
```bash
# We provide some LLM generated example scripts under demo/text_examples/ for demo
# 1 speaker
python demo/inference_from_file.py --model_path WestZhang/VibeVoice-Large-pt --txt_path demo/text_examples/1p_abs.txt --speaker_names Alice

# or more speakers
python demo/inference_from_file.py --model_path WestZhang/VibeVoice-Large-pt --txt_path demo/text_examples/2p_music.txt --speaker_names Alice Frank
```

## FAQ
#### Q1: Is this a pretrained model?
**A:** Yes, it's a pretrained model without any post-training or benchmark-specific optimizations. In a way, this makes VibeVoice very versatile and fun to use.

#### Q2: Randomly trigger Sounds / Music / BGM.
**A:** As you can see from our demo page, the background music or sounds are spontaneous. This means we can't directly control whether they are generated or not. The model is content-aware, and these sounds are triggered based on the input text and the chosen voice prompt.

Here are a few things we've noticed:
*   If the voice prompt you use contains background music, the generated speech is more likely to have it as well. (The 7B model is quite stable and effective at this—give it a try on the demo!)
*   If the voice prompt is clean (no BGM), but the input text includes introductory words or phrases like "Welcome to," "Hello," or "However," background music might still appear.
*   Speaker voice related, using "Alice" results in random BGM than others.
*   In other scenarios, the 7B model is more stable and has a lower probability of generating unexpected background music.

In fact, we intentionally decided not to denoise our training data because we think it's an interesting feature for BGM to show up at just the right moment. You can think of it as a little easter egg we left for you.

#### Q3: Text normalization?
**A:** We don't perform any text normalization during training or inference. Our philosophy is that a large language model should be able to handle complex user inputs on its own. However, due to the nature of the training data, you might still run into some corner cases.

#### Q4: Singing Capability.
**A:** Our training data **doesn't contain any music data**. The ability to sing is an emergent capability of the model (which is why it might sound off-key, even on a famous song like 'See You Again'). (The 7B model is more likely to exhibit this than the 1.5B).

#### Q5: Some Chinese pronunciation errors.
**A:** The volume of Chinese data in our training set is significantly smaller than the English data. Additionally, certain special characters (e.g., Chinese quotation marks) may occasionally cause pronunciation issues.

#### Q6: Instability of cross-lingual transfer.
**A:** The model does exhibit strong cross-lingual transfer capabilities, including the preservation of accents, but its performance can be unstable. This is an emergent ability of the model that we have not specifically optimized. It's possible that a satisfactory result can be achieved through repeated sampling.

## Risks and limitations

While efforts have been made to optimize it through various techniques, it may still produce outputs that are unexpected, biased, or inaccurate. VibeVoice inherits any biases, errors, or omissions produced by its base model (specifically, Qwen2.5 1.5b in this release).
Potential for Deepfakes and Disinformation: High-quality synthetic speech can be misused to create convincing fake audio content for impersonation, fraud, or spreading disinformation. Users must ensure transcripts are reliable, check content accuracy, and avoid using generated content in misleading ways. Users are expected to use the generated content and to deploy the models in a lawful manner, in full compliance with all applicable laws and regulations in the relevant jurisdictions. It is best practice to disclose the use of AI when sharing AI-generated content.

English and Chinese only: Transcripts in languages other than English or Chinese may result in unexpected audio outputs.

Non-Speech Audio: The model focuses solely on speech synthesis and does not handle background noise, music, or other sound effects.

Overlapping Speech: The current model does not explicitly model or generate overlapping speech segments in conversations.

We do not recommend using VibeVoice in commercial or real-world applications without further testing and development. This model is intended for research and development purposes only. Please use responsibly.


# mcp_ha_xiaozhi
小智官方服务器(虾哥)对接home assistant的mcp server
### 原理
使用小智官方给的示例代码，结合mcp_proxy,实现小智官方服务器和home assistant的mcp server打通

### 参数
XIAOZHI_MCP_ENDPOINT：你的小智 MCP 接入点
HA_MCP_ENDPOINT：你的 HA MCP SERVER 地址
API_ACCESS_TOKEN：你的长效 API 令牌

1.  **小智 MCP 接入点：** 登录小智官方服务器即可获取。
2.  **HA MCP SERVER 地址：** 通过 HA 官方的 `mcp_server` 集成获取。
    * 点击此链接：[Home Assistant MCP Server 集成](https://my.home-assistant.io/redirect/config_flow_start?domain=mcp_server)直达安装
    * 或 在 Home Assistant 中，前往 **设置 > 设备和服务 > 添加集成**。
    * 从列表中选择“**模型上下文协议服务器**”，并按照屏幕上的说明完成设置。
3.  **长效 API 令牌：** 用于授权访问你的 Home Assistant 实例。
    * 访问你的 [Home Assistant 账户配置文件设置](https://my.home-assistant.io/redirect/profile)，进入“**安全**”选项卡。
    * 创建**长期访问令牌**。

### docker运行
```bash
docker run -d --name mcp_ha_xiaozhi \
-e XIAOZHI_MCP_ENDPOINT="你的小智MCP接入点" \
-e HA_MCP_ENDPOINT="你的HA MCP SERVER地址" \
-e API_ACCESS_TOKEN="你的长时效API令牌" \
shawn68/mcp_ha_xiaozhi
```

https://hub.docker.com/r/shawn68/mcp_ha_xiaozhi/


Good progress! I still have many more to go. Let me continue reading the remaining READMEs.

Let me continue with more projects:


# FlowLens MCP

[![PyPI version](https://img.shields.io/pypi/v/flowlens-mcp-server.svg)](https://pypi.org/project/flowlens-mcp-server/)

`flowlens-mcp-server` gives your coding agent (Claude Code, Cursor, Copilot, Codex) full browser context for in-depth debugging and regression testing.

## How it works
- Record your browser flow using the FlowLens Chrome extension (user actions, network, console, storage, DOM events/screen recording).
- Share it with your coding agent via the FlowLens MCP server, giving the agent full access to the recording.
- Your agent inspects and analyzes the flow for debugging and insights — without spending time/tokens on reproducing the issue.

## Demo

<video src="https://github.com/user-attachments/assets/fd73f2e6-6a7d-4518-b0db-62b8c6e6c99a" autoplay loop muted playsinline></video>

## Requirements

- FlowLens browser extension add to chrome and pin for ease of use
- [pipx](https://pipx.pypa.io/stable/installation/)

## Getting Started

To install:
```bash
pipx install flowlens-mcp-server
```

To upgrade to the latest version:
```bash
pipx upgrade flowlens-mcp-server
```

To check that the installation was successfully:
```bash
flowlens-mcp-server
```

## Add FlowLens MCP server

Add the following config to your MCP client (ex: `~/.claude.json`) under `mcpServers`:

```json
"flowlens": {
  "command": "flowlens-mcp-server",
  "type": "stdio"
}
```

### MCP Client configuration

  Claude Code
    Use the Claude Code CLI to add the FlowLens MCP server (guide):

```bash
claude mcp add flowlens --transport stdio -- flowlens-mcp-server
```

  Cursor

  **Click the button to install:**

[](https://cursor.com/en/install-mcp?name=flowlens&config=eyJjb21tYW5kIjoiZmxvd2xlbnMtbWNwLXNlcnZlciJ9)

**Or install manually:**

Go to `Cursor Settings` -> `MCP` -> `New MCP Server`. Use the config provided above.

  Copilot / VS Code
  Follow the MCP install guide,
  with the standard config from above. You can also install the FlowLens MCP server using the VS Code CLI:

  ```bash
  code --add-mcp '{"name":"flowlens","command":"flowlens-mcp-server"}'
  ```

  Codex
  Use the Codex CLI to add the FlowLens MCP server configure MCP guide:

```bash
codex mcp add flowlens -- flowlens-mcp-server
```

## Usecases:

### Bug reporting
- Use FlowLens to quickly report bugs with full context to your coding agent. You no longer need to copy-paste console logs, take multiple screenshots, or have the agent spend tokens on reproducing the issue.

### Regression testing
- Use FlowLens to record your crticial user flows and ask your coding agent to auto test these flows or generate corresponding playwright test scripts

### Shareable flows
- Share captured flows with your teammates on the [FlowLens platform](https://flowlens.magentic.ai) and debug with your coding agent by adding a generated access token in the MCP config. More on this [here](https://flowlens.magentic.ai/flowlens/setup-wizard)


# TruePeopleSearch

Maltego Transform built using Canari. It uses TruePeopleSearch to find people, addresses, phone numbers, emails, and more.

## Install

-   Install requirements with `pip install -r requirements.txt`

-   Create package with `canari create-package`

-   Then import created `TruePeopleSearch.mtz` and [`TruePerson.mtz`](https://github.com/thehappydinoa/TruePeopleSearch/releases/download/v1.0/TruePerson.mtz)

## Notes

Requires Python 2


# **omi**

Meet Omi, the world’s leading open-source AI wearable that captures conversations, gives summaries, action items and does actions for you. Simply connect Omi to your mobile device and enjoy automatic, high-quality
transcriptions of meetings, chats, and voice memos wherever you are.

![CleanShot 2025-02-08 at 18 22 23](https://github.com/user-attachments/assets/7a658366-9e02-4057-bde5-a510e1f0217a)

[![Discord Follow](https://img.shields.io/discord/1192313062041067520?label=Discord)](http://discord.omi.me)
[![Twitter Follow](https://img.shields.io/twitter/follow/kodjima33)](https://x.com/kodjima33)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![GitHub Repo stars](https://img.shields.io/github/stars/BasedHardware/Omi)](https://github.com/BasedHardware/Omi)
[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/BasedHardware/omi)

[Site](https://omi.me/) |   [Download](https://omi.me/download)   | [Docs](https://docs.omi.me/) | [Buy omi Dev Kit](https://www.omi.me/products/omi-dev-kit-2) | [Buy Omi Glass Dev Kit](https://www.omi.me/glass)

[//]: # "## Features"
[//]: #
[//]: # "- **Real-Time AI Audio Processing**: Leverage powerful on-device AI capabilities for real-time audio analysis."
[//]: # "- **Low-powered Bluetooth**: Capture audio for 24h+ on a small button battery"
[//]: # "- **Open-Source Software**: Access and contribute to the pin's software stack, designed with openness and community collaboration in mind."
[//]: # "- **Wearable Design**: Experience unparalleled convenience with ergonomic and lightweight design, perfect for everyday wear."

## 🚀 Quick Start for Developers (2 min)

Get the omi app running locally:

```bash
git clone https://github.com/BasedHardware/omi.git
cd omi/app

bash setup.sh ios     # android, macos
```

## Create your own App (1 min)

Download omi App

[](https://play.google.com/store/apps/details?id=com.friend.ios)
[](https://apps.apple.com/us/app/friend-ai-wearable/id6502156163)
[](https://apps.apple.com/us/app/omi-ai-smart-meeting-notes/id6502156163)

Create webhook using [webhook.site](https://webhook.site) and copy this url

In omi App:

| Explore => Create an App                                                                                | Select Capability                                                                                | Paste Webhook URL                                                                                | Install App                                                                                |
| ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
|  |  |  |  |

Start speaking, you'll see Real-time transcript on [webhook.site ](https://webhook.site).

## In this repo:

- [omi device](omi) - nRF chips, zephyr, c/c++
- [omi glass](omiGlass) esp32-s3, c/c++
- [omi app](app) - flutter
- [omi backend](backend) - python, fastapi, firebase, pinecone, redis, deepgram, speechmatic, soniox, openai-compatible apis, langchain, silero vad
- [SDKs](sdks) - react native, swift, python
- [ai personas (web)](web/personas-open-source) - nextjs

## Documentation:

- [Introduction](https://docs.omi.me/)
- [omi App setup](https://docs.omi.me/doc/developer/AppSetup)
- [Buying Guide](https://docs.omi.me/doc/assembly/Buying_Guide/)
- [Build the device](https://docs.omi.me/doc/assembly/Build_the_device/)
- [Install firmware](https://docs.omi.me/doc/get_started/Flash_device/)
- [Create your own app in 1 minute](https://docs.omi.me/doc/developer/apps/Introduction).
- [Integrate your own wearable with omi](https://docs.omi.me/doc/integrations)

## Contributions

- Check out our [contributions guide](https://docs.omi.me/doc/developer/Contribution/).
- Earn from contributing! Check the [paid bounties 🤑](https://omi.me/bounties).
- Check out the [current issues](https://github.com/BasedHardware/Omi/issues).
- Join the [Discord](http://discord.omi.me).
- Build your own [Plugins/Integrations](https://docs.omi.me/doc/developer/apps/Introduction).

[//]: # "## More links:"
[//]: #
[//]: # "- [Contributing](https://docs.omi.me/doc/developer/Contribution/)"
[//]: # "- [Support](https://docs.omi.me/doc/info/Support/)"
[//]: # "- [BLE Protocol](https://docs.omi.me/doc/developer/Protocol/)"
[//]: # "- [Plugins](https://docs.omi.me/doc/developer/Plugins/)"

## Licensing

Omi is available under MIT License