feat: Add GEMINI.md and update README files
This commit is contained in:
parent
275bc9cc3b
commit
fe06ba9b1b
|
|
@ -0,0 +1,53 @@
|
|||
<!--
|
||||
-------------------------------------------------------------------------------
|
||||
AI Assistant Context (GEMINI.md)
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
This document provides the necessary context for the AI assistant (Gemini) to effectively collaborate on the "Vibe Coding" project.
|
||||
|
||||
-->
|
||||
|
||||
<div align="center">
|
||||
|
||||
# Vibe Coding Guide - AI Assistant Context
|
||||
|
||||
</div>
|
||||
|
||||
## 🚀 Project Overview
|
||||
|
||||
**Vibe Coding** is a comprehensive guide and workflow for AI-assisted pair programming. The goal is to provide a structured and efficient way to turn ideas into reality by leveraging the power of AI. This project is not about a specific programming language or framework, but rather a methodology that can be applied to any software development project.
|
||||
|
||||
The core of the project is a collection of documents, prompts, and tools that guide the developer and the AI assistant through the entire development process, from initial conception to final implementation.
|
||||
|
||||
## 的核心理念 (Core Philosophy)
|
||||
|
||||
The "Vibe Coding" methodology is based on the following key principles:
|
||||
|
||||
* **Planning is everything:** A detailed implementation plan is created before any code is written. This plan is then executed step-by-step, with each step being tested and verified before moving on to the next.
|
||||
* **Modularization:** The project is broken down into small, manageable modules that can be developed and tested independently.
|
||||
* **Context is king:** The AI assistant is provided with a "memory-bank" of all the relevant project documents, such as the game design document, tech stack, and implementation plan. This ensures that the AI has a deep understanding of the project's context and can provide accurate and relevant assistance.
|
||||
* **AI as a partner:** The AI assistant is not just a code generator, but a true partner in the development process. The developer and the AI work together, with the developer providing the high-level guidance and the AI providing the low-level implementation details.
|
||||
|
||||
## 📂 Folder Structure
|
||||
|
||||
The most important files and directories in this project are:
|
||||
|
||||
* `README.md`: The main entry point for the project, providing an overview and links to all the other resources.
|
||||
* `i18n/`: Contains the internationalization files for the project, with subdirectories for each supported language.
|
||||
* `i18n/en/`: The English version of the project documentation.
|
||||
* `i18n/zh/`: The Chinese version of the project documentation.
|
||||
* `libs/`: Contains common library code that can be used across different projects.
|
||||
* `prompts/`: A collection of prompts for different stages of the development process.
|
||||
* `skills/`: A collection of reusable skills that can be used to extend the functionality of the AI assistant.
|
||||
|
||||
## 🤖 AI Assistant's Role
|
||||
|
||||
As the AI assistant for this project, you are expected to:
|
||||
|
||||
* **Be a true partner:** Work collaboratively with the developer to achieve the project's goals.
|
||||
* **Be proactive:** Ask clarifying questions and provide suggestions to improve the project.
|
||||
* **Be a good communicator:** Clearly explain your reasoning and provide detailed explanations of your work.
|
||||
* **Be a good learner:** Continuously learn from your interactions with the developer and the project's context.
|
||||
* **Follow the "Vibe Coding" methodology:** Adhere to the principles of plan-driven development, modularization, and context-awareness.
|
||||
|
||||
By following these guidelines, you will be able to provide the best possible assistance to the developer and help them turn their ideas into reality.
|
||||
|
|
@ -303,6 +303,7 @@
|
|||
### 项目内部文档
|
||||
|
||||
* [**胶水编程 (Glue Coding)**](./i18n/zh/documents/胶水编程/): 软件工程的圣杯与银弹,Vibe Coding 的终极进化形态。
|
||||
* [**Chat Vault**](./libs/external/chat-vault/): AI 聊天记录保存工具,支持 Codex/Kiro/Gemini/Claude CLI。
|
||||
* [**prompts-library 工具说明**](./libs/external/prompts-library/): 支持 Excel 与 Markdown 格式互转,包含数百个精选提示词。
|
||||
* [**coding_prompts 集合**](./i18n/zh/prompts/coding_prompts/): 适用于 Vibe Coding 流程的专用提示词。
|
||||
* [**系统提示词构建原则**](./i18n/zh/documents/方法论与原则/系统提示词构建原则.md): 构建高效 AI 系统提示词的综合指南。
|
||||
|
|
|
|||
|
|
@ -221,6 +221,7 @@
|
|||
* [**在线提示词数据库**](https://docs.google.com/spreadsheets/d/1ngoQOhJqdguwNAilCl1joNwTje7FWWN9WiI2bo5VhpU/edit?gid=2093180351#gid=2093180351&range=A1): 包含数百个适用于各场景的用户及系统提示词的在线表格。
|
||||
* [**第三方系统提示词仓库**](https://github.com/x1xhlol/system-prompts-and-models-of-ai-tools): 汇集了多种 AI 工具的系统提示词。
|
||||
* **项目内部文档**:
|
||||
* [**Chat Vault**](../../libs/external/chat-vault/): AI 聊天记录保存工具,支持 Codex/Kiro/Gemini/Claude CLI。
|
||||
* [**prompts-library 工具说明**](./libs/external/prompts-library/): 该工具支持在 Excel 和 Markdown 格式之间转换提示词,并包含数百个精选提示词。
|
||||
* [**coding_prompts 集合**](./i18n/zh/prompts/coding_prompts/): 适用于 Vibe Coding 流程的专用提示词。
|
||||
* [**系统提示词构建原则**](./documents/方法论与原则/系统提示词构建原则.md): 关于如何构建高效、可靠的 AI 系统提示词的综合指南。
|
||||
|
|
|
|||
|
|
@ -27,8 +27,11 @@ libs/
|
|||
│ └── .gitkeep
|
||||
└── external/
|
||||
├── README.md
|
||||
├── chat-vault/
|
||||
├── prompts-library/
|
||||
├── l10n-tool/
|
||||
├── my-nvim/
|
||||
├── MCPlayerTransfer/
|
||||
├── XHS-image-to-PDF-conversion/
|
||||
└── .gitkeep
|
||||
```
|
||||
|
|
@ -57,6 +60,7 @@ libs/
|
|||
|
||||
## 常用入口
|
||||
|
||||
- AI 聊天记录保存:[`external/chat-vault/`](./external/chat-vault/)(支持 Codex/Kiro/Gemini/Claude CLI)
|
||||
- 提示词批量管理:[`external/prompts-library/`](./external/prompts-library/)(配合 `../prompts/` 使用)
|
||||
- 备份工具:优先使用仓库根目录的 `backups/`(当前与 `libs/common/utils/backups/` 内容一致)
|
||||
|
||||
|
|
|
|||
|
|
@ -11,16 +11,22 @@
|
|||
```
|
||||
libs/external/
|
||||
├── README.md
|
||||
├── chat-vault/ # AI 聊天记录保存工具
|
||||
├── prompts-library/ # 提示词库管理工具(Excel ↔ Markdown)
|
||||
├── l10n-tool/ # 多语言翻译脚本
|
||||
├── my-nvim/ # Neovim 配置(含 nvim-config/)
|
||||
├── MCPlayerTransfer/ # MC 玩家迁移工具
|
||||
├── XHS-image-to-PDF-conversion/ # 图片合并 PDF 工具
|
||||
└── .gitkeep
|
||||
```
|
||||
|
||||
## 工具清单(入口与文档)
|
||||
|
||||
- `chat-vault/`:AI 聊天记录保存工具,支持 Codex/Kiro/Gemini/Claude CLI(详见 [`chat-vault/README_CN.md`](./chat-vault/README_CN.md))
|
||||
- `prompts-library/`:提示词 Excel ↔ Markdown 批量互转与索引生成(详见 [`prompts-library/README.md`](./prompts-library/README.md))
|
||||
- `l10n-tool/`:多语言批量翻译脚本
|
||||
- `my-nvim/`:个人 Neovim 配置(详见 [`my-nvim/README.md`](./my-nvim/README.md))
|
||||
- `MCPlayerTransfer/`:MC 玩家迁移工具
|
||||
- `XHS-image-to-PDF-conversion/`:图片合并 PDF(详见 [`XHS-image-to-PDF-conversion/README.md`](./XHS-image-to-PDF-conversion/README.md))
|
||||
|
||||
## 新增外部工具(最小清单)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
# AI Chat Converter Configuration (Optional)
|
||||
# Default: Auto-detect paths, no configuration needed
|
||||
|
||||
# Custom paths (comma-separated for multiple)
|
||||
# CODEX_PATHS=~/.codex/sessions
|
||||
# KIRO_PATHS=~/.local/share/kiro-cli
|
||||
# GEMINI_PATHS=~/.gemini/tmp
|
||||
# CLAUDE_PATHS=~/.claude
|
||||
|
||||
# WSL paths also supported
|
||||
# CODEX_PATHS=\\wsl.localhost\Ubuntu\home\user\.codex\sessions
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.so
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
*.spec
|
||||
|
||||
# Output
|
||||
output/
|
||||
*.db
|
||||
*.sqlite3
|
||||
*.log
|
||||
|
||||
# Environment
|
||||
.env
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2024
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
|
@ -0,0 +1,318 @@
|
|||
<div align="center">
|
||||
|
||||
# 🔐 Chat Vault
|
||||
|
||||
**One tool to save ALL your AI chat history**
|
||||
|
||||
[](https://python.org)
|
||||
[](LICENSE)
|
||||
[]()
|
||||
[]()
|
||||
|
||||
[English](README.md) | [中文](README_CN.md)
|
||||
|
||||
[✨ Features](#-features) •
|
||||
[🚀 Quick Start](#-quick-start) •
|
||||
[📋 Commands](#-commands) •
|
||||
[📁 Project Structure](#-project-structure) •
|
||||
[❓ FAQ](#-faq)
|
||||
|
||||
[📞 Contact](#-contact) •
|
||||
[✨ Support](#-support) •
|
||||
[🤝 Contributing](#-contributing)
|
||||
|
||||
AI-powered docs: [zread.ai/tukuaiai/chat-vault](https://zread.ai/tukuaiai/chat-vault)
|
||||
|
||||
> 📦 This tool is part of [vibe-coding-cn](https://github.com/tukuaiai/vibe-coding-cn) - A comprehensive Vibe Coding guide
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## ✨ Features
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td>🔄 <b>Multi-CLI</b></td>
|
||||
<td>Codex, Kiro, Gemini, Claude - all supported</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚡ <b>Real-time</b></td>
|
||||
<td>Watch mode with system-level file monitoring</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🔢 <b>Token Stats</b></td>
|
||||
<td>Accurate counting using tiktoken (cl100k_base)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🔍 <b>Search</b></td>
|
||||
<td>Find any conversation instantly</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>📤 <b>Export</b></td>
|
||||
<td>JSON or CSV, your choice</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🚀 <b>Zero Config</b></td>
|
||||
<td>Auto-detects paths, just run it</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Sources
|
||||
A[~/.codex]
|
||||
B[~/.kiro]
|
||||
C[~/.gemini]
|
||||
D[~/.claude]
|
||||
end
|
||||
|
||||
subgraph Chat Vault
|
||||
E[Watcher]
|
||||
F[Parsers]
|
||||
G[Storage]
|
||||
end
|
||||
|
||||
subgraph Output
|
||||
H[(SQLite DB)]
|
||||
end
|
||||
|
||||
A --> E
|
||||
B --> E
|
||||
C --> E
|
||||
D --> E
|
||||
E --> F
|
||||
F --> G
|
||||
G --> H
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 How It Works
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant CLI as AI CLI (Codex/Kiro/...)
|
||||
participant Watcher
|
||||
participant Parser
|
||||
participant DB as SQLite
|
||||
|
||||
User->>CLI: Chat with AI
|
||||
CLI->>CLI: Save to local file
|
||||
Watcher->>Watcher: Detect file change
|
||||
Watcher->>Parser: Parse new content
|
||||
Parser->>DB: Upsert session
|
||||
DB-->>User: Query anytime
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 30 Seconds Setup
|
||||
|
||||
```bash
|
||||
# Clone
|
||||
git clone https://github.com/tukuaiai/vibe-coding-cn.git
|
||||
cd vibe-coding-cn/libs/external/chat-vault
|
||||
|
||||
# Run (auto-installs dependencies)
|
||||
./start.sh # Linux/macOS
|
||||
start.bat # Windows
|
||||
```
|
||||
|
||||
**That's it!** 🎉
|
||||
|
||||
---
|
||||
|
||||
## 📊 Example Output
|
||||
|
||||
```
|
||||
==================================================
|
||||
AI 聊天记录 → 集中存储
|
||||
==================================================
|
||||
数据库: ./output/chat_history.db
|
||||
|
||||
[Codex] 新增:1241 更新:0 跳过:0 错误:0
|
||||
[Kiro] 新增:21 更新:0 跳过:0 错误:0
|
||||
[Gemini] 新增:332 更新:0 跳过:0 错误:0
|
||||
[Claude] 新增:168 更新:0 跳过:0 错误:0
|
||||
|
||||
==================================================
|
||||
总计: 1762 会话, 40000+ 消息
|
||||
✓ 同步完成!
|
||||
|
||||
=== Token 统计 (tiktoken) ===
|
||||
codex: 11,659,952 tokens
|
||||
kiro: 26,337 tokens
|
||||
gemini: 3,195,821 tokens
|
||||
claude: 29,725 tokens
|
||||
总计: 14,911,835 tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `python src/main.py` | Sync once |
|
||||
| `python src/main.py -w` | Watch mode (real-time) |
|
||||
| `python src/main.py --stats` | Show statistics |
|
||||
| `python src/main.py --search "keyword"` | Search messages |
|
||||
| `python src/main.py --export json` | Export to JSON |
|
||||
| `python src/main.py --export csv --source codex` | Export specific source |
|
||||
| `python src/main.py --prune` | Clean orphaned records |
|
||||
|
||||
---
|
||||
|
||||
## 📁 Project Structure
|
||||
|
||||
```
|
||||
chat-vault/
|
||||
├── 🚀 start.sh / start.bat # One-click start
|
||||
├── 📦 build.py # Build standalone exe
|
||||
├── 📂 src/
|
||||
│ ├── main.py # CLI entry
|
||||
│ ├── config.py # Auto-detection
|
||||
│ ├── storage.py # SQLite + tiktoken
|
||||
│ ├── watcher.py # File monitoring
|
||||
│ └── parsers/ # CLI parsers
|
||||
├── 📂 docs/
|
||||
│ ├── AI_PROMPT.md # AI assistant guide
|
||||
│ └── schema.md # Database schema
|
||||
└── 📂 output/
|
||||
├── chat_history.db # Your database
|
||||
└── logs/ # Sync logs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ Database Schema
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
sessions {
|
||||
TEXT file_path PK
|
||||
TEXT session_id
|
||||
TEXT source
|
||||
TEXT cwd
|
||||
TEXT messages
|
||||
INTEGER file_mtime
|
||||
TEXT start_time
|
||||
INTEGER token_count
|
||||
}
|
||||
|
||||
meta {
|
||||
TEXT key PK
|
||||
TEXT value
|
||||
}
|
||||
|
||||
meta_codex {
|
||||
TEXT key PK
|
||||
TEXT value
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🤖 For AI Assistants
|
||||
|
||||
Send [docs/AI_PROMPT.md](docs/AI_PROMPT.md) to your AI assistant for:
|
||||
- SQL query examples
|
||||
- Python code snippets
|
||||
- Task guidance
|
||||
|
||||
---
|
||||
|
||||
## ❓ FAQ
|
||||
|
||||
<details>
|
||||
<summary><b>Do I need to configure anything?</b></summary>
|
||||
|
||||
No. Auto-detects `~/.codex`, `~/.kiro`, `~/.gemini`, `~/.claude`
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Does it work with WSL?</b></summary>
|
||||
|
||||
Yes! Paths like `\\wsl.localhost\Ubuntu\...` are supported
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>How do I view the database?</b></summary>
|
||||
|
||||
Use [DB Browser for SQLite](https://sqlitebrowser.org/) or any SQLite tool
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Is my data safe?</b></summary>
|
||||
|
||||
Yes. We only READ from AI tools, never modify original files
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 📞 Contact
|
||||
|
||||
- **GitHub**: [tukuaiai](https://github.com/tukuaiai)
|
||||
- **Twitter / X**: [123olp](https://x.com/123olp)
|
||||
- **Telegram**: [@desci0](https://t.me/desci0)
|
||||
- **Telegram Group**: [glue_coding](https://t.me/glue_coding)
|
||||
- **Telegram Channel**: [tradecat_ai_channel](https://t.me/tradecat_ai_channel)
|
||||
- **Email**: tukuai.ai@gmail.com
|
||||
|
||||
---
|
||||
|
||||
## ✨ Support
|
||||
|
||||
If this project helped you, consider supporting:
|
||||
|
||||
- **Binance UID**: `572155580`
|
||||
- **Tron (TRC20)**: `TQtBXCSTwLFHjBqTS4rNUp7ufiGx51BRey`
|
||||
- **Solana**: `HjYhozVf9AQmfv7yv79xSNs6uaEU5oUk2USasYQfUYau`
|
||||
- **Ethereum (ERC20)**: `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC`
|
||||
- **BNB Smart Chain (BEP20)**: `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC`
|
||||
- **Bitcoin**: `bc1plslluj3zq3snpnnczplu7ywf37h89dyudqua04pz4txwh8z5z5vsre7nlm`
|
||||
- **Sui**: `0xb720c98a48c77f2d49d375932b2867e793029e6337f1562522640e4f84203d2e`
|
||||
|
||||
---
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
We welcome all contributions! Feel free to open an [Issue](https://github.com/tukuaiai/vibe-coding-cn/issues) or submit a [Pull Request](https://github.com/tukuaiai/vibe-coding-cn/pulls).
|
||||
|
||||
---
|
||||
|
||||
## 📄 License
|
||||
|
||||
[MIT](LICENSE) - Do whatever you want with it.
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
**If this helped you, give it a ⭐!**
|
||||
|
||||
## Star History
|
||||
|
||||
<a href="https://www.star-history.com/#tukuaiai/vibe-coding-cn&type=Date">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
---
|
||||
|
||||
**Made with ❤️ by [tukuaiai](https://github.com/tukuaiai)**
|
||||
|
||||
[⬆ Back to Top](#-chat-vault)
|
||||
|
||||
</div>
|
||||
|
|
@ -0,0 +1,311 @@
|
|||
<div align="center">
|
||||
|
||||
# 🔐 Chat Vault
|
||||
|
||||
**一个工具保存你所有的 AI 聊天记录**
|
||||
|
||||
[](https://python.org)
|
||||
[](LICENSE)
|
||||
[]()
|
||||
[]()
|
||||
|
||||
[English](README.md) | [中文](README_CN.md)
|
||||
|
||||
[✨ 功能特性](#-功能特性) •
|
||||
[🚀 快速开始](#-30-秒快速开始) •
|
||||
[📋 命令一览](#-命令一览) •
|
||||
[📁 项目结构](#-项目结构) •
|
||||
[❓ 常见问题](#-常见问题)
|
||||
|
||||
[📞 联系方式](#-联系方式) •
|
||||
[✨ 支持项目](#-支持项目) •
|
||||
[🤝 参与贡献](#-参与贡献)
|
||||
|
||||
AI 解读文档: [zread.ai/tukuaiai/chat-vault](https://zread.ai/tukuaiai/chat-vault)
|
||||
|
||||
> 📦 本工具是 [vibe-coding-cn](https://github.com/tukuaiai/vibe-coding-cn) 的一部分 - 一份全面的 Vibe Coding 指南
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## ✨ 功能特性
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td>🔄 <b>多 CLI 支持</b></td>
|
||||
<td>Codex、Kiro、Gemini、Claude 全都行</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚡ <b>实时同步</b></td>
|
||||
<td>系统级文件监控,聊完自动保存</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🔢 <b>Token 统计</b></td>
|
||||
<td>tiktoken 精确计算,知道你用了多少</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🔍 <b>搜索</b></td>
|
||||
<td>秒找任何对话</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>📤 <b>导出</b></td>
|
||||
<td>JSON 或 CSV,随你选</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>🚀 <b>零配置</b></td>
|
||||
<td>自动检测路径,开箱即用</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ 架构图
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph 数据来源
|
||||
A[~/.codex]
|
||||
B[~/.kiro]
|
||||
C[~/.gemini]
|
||||
D[~/.claude]
|
||||
end
|
||||
|
||||
subgraph Chat Vault
|
||||
E[监控器]
|
||||
F[解析器]
|
||||
G[存储层]
|
||||
end
|
||||
|
||||
subgraph 输出
|
||||
H[(SQLite 数据库)]
|
||||
end
|
||||
|
||||
A --> E
|
||||
B --> E
|
||||
C --> E
|
||||
D --> E
|
||||
E --> F
|
||||
F --> G
|
||||
G --> H
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 工作流程
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant 用户
|
||||
participant CLI as AI CLI (Codex/Kiro/...)
|
||||
participant 监控器
|
||||
participant 解析器
|
||||
participant DB as SQLite
|
||||
|
||||
用户->>CLI: 和 AI 聊天
|
||||
CLI->>CLI: 保存到本地文件
|
||||
监控器->>监控器: 检测文件变化
|
||||
监控器->>解析器: 解析新内容
|
||||
解析器->>DB: 写入数据库
|
||||
DB-->>用户: 随时查询
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 30 秒快速开始
|
||||
|
||||
```bash
|
||||
# 下载
|
||||
git clone https://github.com/tukuaiai/vibe-coding-cn.git
|
||||
cd vibe-coding-cn/libs/external/chat-vault
|
||||
|
||||
# 运行(自动安装依赖)
|
||||
./start.sh # Linux/macOS
|
||||
start.bat # Windows(双击)
|
||||
```
|
||||
|
||||
**搞定!** 🎉
|
||||
|
||||
---
|
||||
|
||||
## 📊 运行效果
|
||||
|
||||
```
|
||||
==================================================
|
||||
AI 聊天记录 → 集中存储
|
||||
==================================================
|
||||
数据库: ./output/chat_history.db
|
||||
|
||||
[Codex] 新增:1241 更新:0 跳过:0 错误:0
|
||||
[Kiro] 新增:21 更新:0 跳过:0 错误:0
|
||||
[Gemini] 新增:332 更新:0 跳过:0 错误:0
|
||||
[Claude] 新增:168 更新:0 跳过:0 错误:0
|
||||
|
||||
==================================================
|
||||
总计: 1762 会话, 40000+ 消息
|
||||
✓ 同步完成!
|
||||
|
||||
=== Token 统计 (tiktoken) ===
|
||||
codex: 11,659,952 tokens
|
||||
kiro: 26,337 tokens
|
||||
gemini: 3,195,821 tokens
|
||||
claude: 29,725 tokens
|
||||
总计: 14,911,835 tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 命令一览
|
||||
|
||||
| 命令 | 说明 |
|
||||
|------|------|
|
||||
| `python src/main.py` | 同步一次 |
|
||||
| `python src/main.py -w` | 实时监控(推荐) |
|
||||
| `python src/main.py --stats` | 查看统计 |
|
||||
| `python src/main.py --search "关键词"` | 搜索消息 |
|
||||
| `python src/main.py --export json` | 导出 JSON |
|
||||
| `python src/main.py --export csv --source codex` | 导出指定来源 |
|
||||
| `python src/main.py --prune` | 清理孤立记录 |
|
||||
|
||||
---
|
||||
|
||||
## 📁 项目结构
|
||||
|
||||
```
|
||||
chat-vault/
|
||||
├── 🚀 start.sh / start.bat # 一键启动
|
||||
├── 📦 build.py # 打包脚本
|
||||
├── 📂 src/
|
||||
│ ├── main.py # 主程序
|
||||
│ ├── config.py # 配置检测
|
||||
│ ├── storage.py # SQLite + tiktoken
|
||||
│ ├── watcher.py # 文件监控
|
||||
│ └── parsers/ # 各 CLI 解析器
|
||||
├── 📂 docs/
|
||||
│ ├── AI_PROMPT.md # AI 助手指南
|
||||
│ └── schema.md # 数据库结构
|
||||
└── 📂 output/
|
||||
├── chat_history.db # 你的数据库
|
||||
└── logs/ # 日志
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ 数据库结构
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
sessions {
|
||||
TEXT file_path PK "文件路径"
|
||||
TEXT session_id "会话ID"
|
||||
TEXT source "来源"
|
||||
TEXT cwd "工作目录"
|
||||
TEXT messages "消息JSON"
|
||||
INTEGER file_mtime "修改时间"
|
||||
TEXT start_time "开始时间"
|
||||
INTEGER token_count "Token数"
|
||||
}
|
||||
|
||||
meta {
|
||||
TEXT key PK
|
||||
TEXT value
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🤖 让 AI 帮你查数据库
|
||||
|
||||
把 [docs/AI_PROMPT.md](docs/AI_PROMPT.md) 发给 AI 助手,它就知道:
|
||||
- 怎么写 SQL 查询
|
||||
- 怎么用 Python 分析
|
||||
- 怎么帮你找对话
|
||||
|
||||
---
|
||||
|
||||
## ❓ 常见问题
|
||||
|
||||
<details>
|
||||
<summary><b>需要配置什么吗?</b></summary>
|
||||
|
||||
不用。自动检测 `~/.codex`、`~/.kiro`、`~/.gemini`、`~/.claude`
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>WSL 能用吗?</b></summary>
|
||||
|
||||
能!`\\wsl.localhost\Ubuntu\...` 这种路径也支持
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>怎么看数据库?</b></summary>
|
||||
|
||||
用 [DB Browser for SQLite](https://sqlitebrowser.org/) 或任何 SQLite 工具
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>会不会搞坏我的数据?</b></summary>
|
||||
|
||||
不会。只读取,从不修改原始文件
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 📞 联系方式
|
||||
|
||||
- **GitHub**: [tukuaiai](https://github.com/tukuaiai)
|
||||
- **Twitter / X**: [123olp](https://x.com/123olp)
|
||||
- **Telegram**: [@desci0](https://t.me/desci0)
|
||||
- **Telegram 交流群**: [glue_coding](https://t.me/glue_coding)
|
||||
- **Telegram 频道**: [tradecat_ai_channel](https://t.me/tradecat_ai_channel)
|
||||
- **邮箱**: tukuai.ai@gmail.com
|
||||
|
||||
---
|
||||
|
||||
## ✨ 支持项目
|
||||
|
||||
如果这个项目帮到你了,考虑支持一下:
|
||||
|
||||
- **币安 UID**: `572155580`
|
||||
- **Tron (TRC20)**: `TQtBXCSTwLFHjBqTS4rNUp7ufiGx51BRey`
|
||||
- **Solana**: `HjYhozVf9AQmfv7yv79xSNs6uaEU5oUk2USasYQfUYau`
|
||||
- **Ethereum (ERC20)**: `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC`
|
||||
- **BNB Smart Chain (BEP20)**: `0xa396923a71ee7D9480b346a17dDeEb2c0C287BBC`
|
||||
- **Bitcoin**: `bc1plslluj3zq3snpnnczplu7ywf37h89dyudqua04pz4txwh8z5z5vsre7nlm`
|
||||
- **Sui**: `0xb720c98a48c77f2d49d375932b2867e793029e6337f1562522640e4f84203d2e`
|
||||
|
||||
---
|
||||
|
||||
## 🤝 参与贡献
|
||||
|
||||
欢迎各种形式的贡献!随时开启一个 [Issue](https://github.com/tukuaiai/vibe-coding-cn/issues) 或提交 [Pull Request](https://github.com/tukuaiai/vibe-coding-cn/pulls)。
|
||||
|
||||
---
|
||||
|
||||
## 📄 开源协议
|
||||
|
||||
[MIT](LICENSE) - 随便用,不用管我
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
**如果帮到你了,点个 ⭐ 呗!**
|
||||
|
||||
## Star History
|
||||
|
||||
<a href="https://www.star-history.com/#tukuaiai/vibe-coding-cn&type=Date">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date&theme=dark" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=tukuaiai/vibe-coding-cn&type=Date" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
---
|
||||
|
||||
**Made with ❤️ by [tukuaiai](https://github.com/tukuaiai)**
|
||||
|
||||
[⬆ 返回顶部](#-chat-vault)
|
||||
|
||||
</div>
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
@echo off
|
||||
echo 安装打包工具...
|
||||
pip install pyinstaller -q
|
||||
|
||||
echo 开始打包...
|
||||
pyinstaller --onefile --name ai-chat-converter ^
|
||||
--add-data "src;src" ^
|
||||
--hidden-import tiktoken_ext.openai_public ^
|
||||
--hidden-import tiktoken_ext ^
|
||||
--collect-data tiktoken ^
|
||||
src/main.py
|
||||
|
||||
echo.
|
||||
echo 完成! 输出: dist\ai-chat-converter.exe
|
||||
pause
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
#!/usr/bin/env python3
|
||||
"""打包脚本 - 生成独立可执行文件"""
|
||||
import subprocess
|
||||
import sys
|
||||
import os
|
||||
import shutil
|
||||
|
||||
def main():
|
||||
os.chdir(os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
for d in ['build', 'dist']:
|
||||
if os.path.exists(d):
|
||||
shutil.rmtree(d)
|
||||
|
||||
print("开始打包...")
|
||||
|
||||
sep = ";" if sys.platform == "win32" else ":"
|
||||
|
||||
cmd = [
|
||||
sys.executable, "-m", "PyInstaller",
|
||||
"--onefile",
|
||||
"--name", "ai-chat-converter",
|
||||
f"--add-data=src{sep}src",
|
||||
"--hidden-import", "tiktoken_ext.openai_public",
|
||||
"--hidden-import", "tiktoken_ext",
|
||||
"--hidden-import", "dotenv",
|
||||
"--collect-data", "tiktoken",
|
||||
"--collect-all", "watchdog",
|
||||
"--collect-all", "dotenv",
|
||||
"src/main.py"
|
||||
]
|
||||
|
||||
subprocess.run(cmd, check=True)
|
||||
|
||||
exe = "dist/ai-chat-converter.exe" if sys.platform == "win32" else "dist/ai-chat-converter"
|
||||
size = os.path.getsize(exe) / 1024 / 1024
|
||||
print(f"\n✓ 打包完成: {exe} ({size:.1f} MB)")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,231 @@
|
|||
# AI Chat Converter - AI 助手完全指南
|
||||
|
||||
> **把这个文档发给 AI 助手,它就知道怎么帮你用这个工具了**
|
||||
|
||||
---
|
||||
|
||||
## 🎯 这是什么?
|
||||
|
||||
一个把 Codex、Kiro、Gemini、Claude 的聊天记录全部存到一个 SQLite 数据库的工具。
|
||||
|
||||
**数据库位置**: `项目目录/output/chat_history.db`
|
||||
|
||||
---
|
||||
|
||||
## 🚀 怎么启动?
|
||||
|
||||
### 方式一:双击启动(推荐)
|
||||
```bash
|
||||
./start.sh # Linux/macOS
|
||||
start.bat # Windows(双击)
|
||||
```
|
||||
|
||||
### 方式二:命令行
|
||||
```bash
|
||||
cd ai-chat-converter
|
||||
python src/main.py --watch # 持续监控(推荐)
|
||||
python src/main.py # 同步一次就退出
|
||||
```
|
||||
|
||||
### 方式三:后台运行
|
||||
```bash
|
||||
nohup ./start.sh > /dev/null 2>&1 &
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 数据库长啥样?
|
||||
|
||||
### 主表:sessions
|
||||
|
||||
| 字段 | 说明 | 例子 |
|
||||
|------|------|------|
|
||||
| file_path | 主键,文件路径 | `/home/user/.codex/sessions/xxx.jsonl` |
|
||||
| session_id | 会话ID | `019b2164-168c-7133-9b1f-5d24fea1d3e1` |
|
||||
| source | 来源 | `codex` / `kiro` / `gemini` / `claude` |
|
||||
| cwd | 工作目录 | `/home/user/projects/myapp` |
|
||||
| messages | 消息内容(JSON) | `[{"time":"...", "role":"user", "content":"..."}]` |
|
||||
| start_time | 开始时间 | `2025-12-18T10:30:00` |
|
||||
| token_count | Token 数量 | `1234` |
|
||||
|
||||
---
|
||||
|
||||
## 🔍 常用查询(直接复制用)
|
||||
|
||||
### 1. 看看有多少数据
|
||||
|
||||
```sql
|
||||
SELECT source, COUNT(*) as 会话数, SUM(token_count) as Token总数
|
||||
FROM sessions
|
||||
GROUP BY source;
|
||||
```
|
||||
|
||||
### 2. 最近的 10 个会话
|
||||
|
||||
```sql
|
||||
SELECT session_id, source, cwd, start_time, token_count
|
||||
FROM sessions
|
||||
ORDER BY start_time DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
### 3. 搜索包含某个词的对话
|
||||
|
||||
```sql
|
||||
SELECT session_id, source, cwd, start_time
|
||||
FROM sessions
|
||||
WHERE messages LIKE '%要搜索的词%'
|
||||
ORDER BY start_time DESC
|
||||
LIMIT 20;
|
||||
```
|
||||
|
||||
### 4. 查某个项目的所有对话
|
||||
|
||||
```sql
|
||||
SELECT session_id, source, start_time, token_count
|
||||
FROM sessions
|
||||
WHERE cwd LIKE '%项目名%'
|
||||
ORDER BY start_time;
|
||||
```
|
||||
|
||||
### 5. 看某个会话的完整内容
|
||||
|
||||
```sql
|
||||
SELECT messages FROM sessions WHERE session_id = '会话ID';
|
||||
```
|
||||
|
||||
### 6. 统计每天用了多少 Token
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
date(start_time) as 日期,
|
||||
SUM(token_count) as Token数
|
||||
FROM sessions
|
||||
GROUP BY 日期
|
||||
ORDER BY 日期 DESC
|
||||
LIMIT 7;
|
||||
```
|
||||
|
||||
### 7. 统计每个来源的 Token
|
||||
|
||||
```sql
|
||||
SELECT source, SUM(token_count) as tokens
|
||||
FROM sessions
|
||||
GROUP BY source
|
||||
ORDER BY tokens DESC;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💻 命令行用法
|
||||
|
||||
| 命令 | 干啥的 |
|
||||
|------|--------|
|
||||
| `python src/main.py` | 同步一次 |
|
||||
| `python src/main.py -w` | 持续监控(推荐) |
|
||||
| `python src/main.py --stats` | 看统计信息 |
|
||||
| `python src/main.py --search "关键词"` | 搜索 |
|
||||
| `python src/main.py --export json` | 导出 JSON |
|
||||
| `python src/main.py --export csv` | 导出 CSV |
|
||||
| `python src/main.py --prune` | 清理已删除文件的记录 |
|
||||
|
||||
---
|
||||
|
||||
## 🐍 用 Python 查询
|
||||
|
||||
```python
|
||||
import sqlite3
|
||||
import json
|
||||
|
||||
# 连接数据库
|
||||
db = sqlite3.connect('output/chat_history.db')
|
||||
|
||||
# 查所有 Codex 会话
|
||||
for row in db.execute("SELECT session_id, cwd, token_count FROM sessions WHERE source='codex'"):
|
||||
print(f"{row[0]}: {row[2]} tokens - {row[1]}")
|
||||
|
||||
# 搜索包含 "python" 的对话
|
||||
for row in db.execute("SELECT session_id, source FROM sessions WHERE messages LIKE '%python%'"):
|
||||
print(f"[{row[1]}] {row[0]}")
|
||||
|
||||
# 获取某个会话的消息
|
||||
row = db.execute("SELECT messages FROM sessions WHERE session_id=?", ('会话ID',)).fetchone()
|
||||
if row:
|
||||
messages = json.loads(row[0])
|
||||
for msg in messages:
|
||||
print(f"{msg['role']}: {msg['content'][:100]}...")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 文件在哪?
|
||||
|
||||
```
|
||||
ai-chat-converter/
|
||||
├── start.sh ← 双击这个启动
|
||||
├── output/
|
||||
│ ├── chat_history.db ← 数据库在这
|
||||
│ └── logs/ ← 日志在这
|
||||
└── src/
|
||||
└── main.py ← 主程序
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ❓ AI 助手任务示例
|
||||
|
||||
当用户说这些话时,你应该这样做:
|
||||
|
||||
| 用户说 | 你做 |
|
||||
|--------|------|
|
||||
| "帮我查最近的对话" | 执行最近会话 SQL |
|
||||
| "搜索关于 Python 的讨论" | 用 `--search` 或 SQL 搜索 |
|
||||
| "这个月用了多少 Token" | 执行 Token 统计 SQL |
|
||||
| "导出所有 Codex 记录" | `python src/main.py --export json --source codex` |
|
||||
| "启动监控" | `./start.sh` 或 `python src/main.py -w` |
|
||||
| "数据库在哪" | `output/chat_history.db` |
|
||||
|
||||
---
|
||||
|
||||
## 🔧 出问题了?
|
||||
|
||||
### 问题:找不到数据库
|
||||
```bash
|
||||
# 先运行一次同步
|
||||
python src/main.py
|
||||
```
|
||||
|
||||
### 问题:依赖没装
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 问题:权限不够
|
||||
```bash
|
||||
chmod +x start.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 消息格式
|
||||
|
||||
数据库里的 `messages` 字段是 JSON 数组:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"time": "2025-12-18T10:30:00",
|
||||
"role": "user",
|
||||
"content": "帮我写个 Python 脚本"
|
||||
},
|
||||
{
|
||||
"time": "2025-12-18T10:30:05",
|
||||
"role": "ai",
|
||||
"content": "好的,这是一个简单的脚本..."
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
- `role`: `user`(用户)或 `ai`(AI 回复)
|
||||
- `time`: ISO 格式时间
|
||||
- `content`: 消息内容
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Roadmap
|
||||
|
||||
## v1.0 ✅ Core
|
||||
- [x] Multi-CLI support (Codex/Kiro/Gemini/Claude)
|
||||
- [x] Auto path detection
|
||||
- [x] SQLite storage
|
||||
- [x] Incremental sync
|
||||
- [x] Cross-platform watch mode (watchdog)
|
||||
- [x] Token counting (tiktoken)
|
||||
|
||||
## Future
|
||||
- [ ] Web UI
|
||||
- [ ] API server mode
|
||||
- [ ] Vector storage (RAG)
|
||||
- [ ] Cross-AI context sharing
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
# 数据库结构 (v5)
|
||||
|
||||
**位置**: `项目目录/output/chat_history.db`
|
||||
|
||||
## sessions 表(主表)
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| file_path | TEXT | 主键,源文件路径 |
|
||||
| session_id | TEXT | 会话 ID |
|
||||
| source | TEXT | 来源: codex/kiro/gemini/claude |
|
||||
| cwd | TEXT | 工作目录 |
|
||||
| messages | TEXT | JSON 数组 |
|
||||
| file_mtime | INTEGER | 文件修改时间戳 |
|
||||
| start_time | TEXT | 会话开始时间 |
|
||||
| token_count | INTEGER | Token 数量 |
|
||||
|
||||
**索引**: `idx_source`, `idx_session_id`, `idx_start_time`
|
||||
|
||||
## meta 表(全局统计)
|
||||
|
||||
| key | 说明 |
|
||||
|-----|------|
|
||||
| schema_version | 数据库版本 (5) |
|
||||
| total_sessions | 总会话数 |
|
||||
| total_messages | 总消息数 |
|
||||
| total_tokens | 总 Token 数 |
|
||||
| last_sync | 最后同步时间 |
|
||||
|
||||
## meta_{cli} 表(各 CLI 统计)
|
||||
|
||||
每个 CLI 独立的元信息表:`meta_codex`, `meta_kiro`, `meta_gemini`, `meta_claude`
|
||||
|
||||
| key | 说明 |
|
||||
|-----|------|
|
||||
| path | 监控路径 |
|
||||
| sessions | 会话数 |
|
||||
| messages | 消息数 |
|
||||
| total_tokens | Token 总数 |
|
||||
| last_sync | 最后同步时间 |
|
||||
|
||||
## 消息格式
|
||||
|
||||
```json
|
||||
[
|
||||
{"time": "2025-12-18T10:30:00", "role": "user", "content": "..."},
|
||||
{"time": "2025-12-18T10:30:05", "role": "ai", "content": "..."}
|
||||
]
|
||||
```
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
python-dotenv>=1.0.0
|
||||
watchdog>=3.0.0
|
||||
tiktoken>=0.5.0
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
#!/bin/bash
|
||||
cd "$(dirname "$0")/../src"
|
||||
python3 main.py
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
#!/bin/bash
|
||||
cd "$(dirname "$0")/../src"
|
||||
python3 main.py --watch
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
r"""
|
||||
配置模块 - 智能路径识别
|
||||
支持: Linux 原生路径、WSL 路径 (\\wsl.localhost\Ubuntu\...)
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# 项目目录
|
||||
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
OUTPUT_DIR = os.path.join(PROJECT_DIR, "output")
|
||||
|
||||
load_dotenv(os.path.join(PROJECT_DIR, ".env"))
|
||||
|
||||
def convert_wsl_path(path: str) -> str:
|
||||
match = re.match(r'^\\\\wsl[.\$]?[^\\]*\\[^\\]+\\(.+)$', path, re.IGNORECASE)
|
||||
if match:
|
||||
return '/' + match.group(1).replace('\\', '/')
|
||||
return path
|
||||
|
||||
def normalize_path(path: str) -> str:
|
||||
path = path.strip()
|
||||
path = convert_wsl_path(path)
|
||||
return os.path.expanduser(path)
|
||||
|
||||
def get_paths(env_key: str) -> list:
|
||||
val = os.getenv(env_key, "")
|
||||
if not val:
|
||||
return []
|
||||
return [normalize_path(p) for p in val.split(",") if p.strip()]
|
||||
|
||||
def auto_detect_paths() -> dict:
|
||||
home = os.path.expanduser("~")
|
||||
kiro_db = os.path.join(home, ".local", "share", "kiro-cli")
|
||||
candidates = {
|
||||
"codex_paths": [os.path.join(home, ".codex", "sessions"), os.path.join(home, ".codex")],
|
||||
"kiro_paths": [kiro_db] if os.path.exists(kiro_db) else [],
|
||||
"gemini_paths": [os.path.join(home, ".gemini", "tmp"), os.path.join(home, ".gemini")],
|
||||
"claude_paths": [os.path.join(home, ".claude")],
|
||||
}
|
||||
detected = {}
|
||||
for key, paths in candidates.items():
|
||||
for p in paths:
|
||||
if os.path.exists(p):
|
||||
detected[key] = [p]
|
||||
break
|
||||
if key not in detected:
|
||||
detected[key] = []
|
||||
return detected
|
||||
|
||||
def load_config() -> dict:
|
||||
auto = auto_detect_paths()
|
||||
|
||||
os.makedirs(OUTPUT_DIR, exist_ok=True)
|
||||
os.makedirs(os.path.join(OUTPUT_DIR, "logs"), exist_ok=True)
|
||||
|
||||
return {
|
||||
"codex_paths": get_paths("CODEX_PATHS") or auto.get("codex_paths", []),
|
||||
"kiro_paths": get_paths("KIRO_PATHS") or auto.get("kiro_paths", []),
|
||||
"gemini_paths": get_paths("GEMINI_PATHS") or auto.get("gemini_paths", []),
|
||||
"claude_paths": get_paths("CLAUDE_PATHS") or auto.get("claude_paths", []),
|
||||
"output_dir": OUTPUT_DIR,
|
||||
"log_dir": os.path.join(OUTPUT_DIR, "logs"),
|
||||
"db_path": os.path.join(OUTPUT_DIR, "chat_history.db"),
|
||||
}
|
||||
|
||||
CONFIG = load_config()
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""日志模块 - 同时输出到控制台和文件"""
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
|
||||
_logger = None
|
||||
|
||||
def setup_logger(log_dir: str = None) -> logging.Logger:
|
||||
global _logger
|
||||
if _logger:
|
||||
return _logger
|
||||
|
||||
_logger = logging.getLogger('ai_chat_converter')
|
||||
_logger.setLevel(logging.DEBUG)
|
||||
_logger.handlers.clear()
|
||||
|
||||
fmt = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
|
||||
|
||||
# 控制台
|
||||
ch = logging.StreamHandler()
|
||||
ch.setLevel(logging.INFO)
|
||||
ch.setFormatter(fmt)
|
||||
_logger.addHandler(ch)
|
||||
|
||||
# 文件
|
||||
if log_dir:
|
||||
os.makedirs(log_dir, exist_ok=True)
|
||||
log_file = os.path.join(log_dir, f"sync_{datetime.now().strftime('%Y%m%d')}.log")
|
||||
fh = logging.FileHandler(log_file, encoding='utf-8')
|
||||
fh.setLevel(logging.DEBUG)
|
||||
fh.setFormatter(fmt)
|
||||
_logger.addHandler(fh)
|
||||
|
||||
return _logger
|
||||
|
||||
def get_logger() -> logging.Logger:
|
||||
global _logger
|
||||
if not _logger:
|
||||
_logger = setup_logger()
|
||||
return _logger
|
||||
|
|
@ -0,0 +1,319 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
AI 聊天记录集中存储工具
|
||||
|
||||
命令:
|
||||
python main.py # 同步一次
|
||||
python main.py --watch # 持续监控
|
||||
python main.py --prune # 清理孤立记录
|
||||
python main.py --stats # 显示统计
|
||||
python main.py --search <keyword> # 搜索
|
||||
python main.py --export json|csv [--source codex|kiro|gemini|claude]
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import subprocess
|
||||
|
||||
# 项目根目录
|
||||
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
VENV_DIR = os.path.join(PROJECT_DIR, '.venv')
|
||||
REQUIREMENTS = os.path.join(PROJECT_DIR, 'requirements.txt')
|
||||
|
||||
def ensure_venv():
|
||||
"""检测并创建虚拟环境,安装依赖"""
|
||||
# 打包版本跳过
|
||||
if getattr(sys, 'frozen', False):
|
||||
return
|
||||
|
||||
# 已在虚拟环境中运行则跳过
|
||||
if sys.prefix != sys.base_prefix:
|
||||
return
|
||||
|
||||
# 检查 .venv 是否存在
|
||||
venv_python = os.path.join(VENV_DIR, 'bin', 'python') if os.name != 'nt' else os.path.join(VENV_DIR, 'Scripts', 'python.exe')
|
||||
|
||||
if not os.path.exists(venv_python):
|
||||
print("首次运行,创建虚拟环境...")
|
||||
subprocess.run([sys.executable, '-m', 'venv', VENV_DIR], check=True)
|
||||
print("安装依赖...")
|
||||
pip = os.path.join(VENV_DIR, 'bin', 'pip') if os.name != 'nt' else os.path.join(VENV_DIR, 'Scripts', 'pip.exe')
|
||||
subprocess.run([pip, 'install', '-r', REQUIREMENTS, '-q'], check=True)
|
||||
print("环境准备完成,重新启动...\n")
|
||||
|
||||
# 使用虚拟环境重新执行
|
||||
os.execv(venv_python, [venv_python] + sys.argv)
|
||||
|
||||
# 启动前检测虚拟环境
|
||||
ensure_venv()
|
||||
|
||||
# 支持 PyInstaller 打包
|
||||
if getattr(sys, 'frozen', False):
|
||||
BASE_DIR = sys._MEIPASS
|
||||
sys.path.insert(0, os.path.join(BASE_DIR, 'src'))
|
||||
else:
|
||||
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
import argparse
|
||||
from config import CONFIG
|
||||
from parsers import CodexParser, GeminiParser, ClaudeParser, KiroParser
|
||||
from storage import ChatStorage
|
||||
from logger import setup_logger, get_logger
|
||||
|
||||
storage: ChatStorage = None
|
||||
|
||||
def main():
|
||||
global storage
|
||||
|
||||
parser = argparse.ArgumentParser(description='AI Chat Converter')
|
||||
parser.add_argument('-w', '--watch', action='store_true', help='持续监控模式')
|
||||
parser.add_argument('--prune', action='store_true', help='清理孤立记录')
|
||||
parser.add_argument('--stats', action='store_true', help='显示统计信息')
|
||||
parser.add_argument('--search', type=str, help='搜索关键词')
|
||||
parser.add_argument('--export', choices=['json', 'csv'], help='导出格式')
|
||||
parser.add_argument('--source', choices=['codex', 'kiro', 'gemini', 'claude'], help='指定来源')
|
||||
parser.add_argument('--output', type=str, help='导出文件路径')
|
||||
args = parser.parse_args()
|
||||
|
||||
# 初始化
|
||||
setup_logger(CONFIG["log_dir"])
|
||||
log = get_logger()
|
||||
|
||||
storage = ChatStorage(CONFIG["db_path"])
|
||||
|
||||
# 命令分发
|
||||
if args.prune:
|
||||
cmd_prune()
|
||||
elif args.stats:
|
||||
cmd_stats()
|
||||
elif args.search:
|
||||
cmd_search(args.search, args.source)
|
||||
elif args.export:
|
||||
cmd_export(args.export, args.source, args.output)
|
||||
elif args.watch:
|
||||
cmd_sync()
|
||||
cmd_watch()
|
||||
else:
|
||||
cmd_sync()
|
||||
|
||||
def cmd_sync():
|
||||
log = get_logger()
|
||||
log.info("=" * 50)
|
||||
log.info("AI 聊天记录 → 集中存储")
|
||||
log.info("=" * 50)
|
||||
log.info(f"数据库: {CONFIG['db_path']}")
|
||||
|
||||
total_added, total_updated, total_skipped, total_errors = 0, 0, 0, 0
|
||||
|
||||
for cli, key, parser_cls in [
|
||||
('codex', 'codex_paths', lambda: CodexParser('codex')),
|
||||
('kiro', 'kiro_paths', KiroParser),
|
||||
('gemini', 'gemini_paths', GeminiParser),
|
||||
('claude', 'claude_paths', ClaudeParser),
|
||||
]:
|
||||
paths = CONFIG.get(key, [])
|
||||
if not paths:
|
||||
continue
|
||||
|
||||
parser = parser_cls()
|
||||
if cli in ('claude', 'kiro'):
|
||||
a, u, s, e = process_multi(parser, paths, cli)
|
||||
else:
|
||||
a, u, s, e = process(parser, paths)
|
||||
|
||||
log.info(f"[{cli.capitalize()}] 新增:{a} 更新:{u} 跳过:{s} 错误:{e}")
|
||||
update_cli_meta(cli)
|
||||
total_added += a
|
||||
total_updated += u
|
||||
total_skipped += s
|
||||
total_errors += e
|
||||
|
||||
total = storage.get_total_stats()
|
||||
storage.update_total_meta(total['sessions'], total['messages'], total['tokens'])
|
||||
|
||||
log.info("=" * 50)
|
||||
log.info(f"总计: {total['sessions']} 会话, {total['messages']} 消息")
|
||||
if total_errors > 0:
|
||||
log.warning(f"错误: {total_errors} 个文件解析失败")
|
||||
log.info("✓ 同步完成!")
|
||||
|
||||
print_token_stats()
|
||||
|
||||
def cmd_watch():
|
||||
from watcher import ChatWatcher
|
||||
from datetime import datetime
|
||||
|
||||
log = get_logger()
|
||||
log.info("")
|
||||
log.info("=" * 50)
|
||||
log.info("实时监听模式 (watchdog)")
|
||||
log.info("=" * 50)
|
||||
|
||||
watch_paths = []
|
||||
path_source_map = {}
|
||||
|
||||
for cli, key in [('codex', 'codex_paths'), ('kiro', 'kiro_paths'),
|
||||
('gemini', 'gemini_paths'), ('claude', 'claude_paths')]:
|
||||
for p in CONFIG.get(key, []):
|
||||
if os.path.isdir(p) or os.path.isfile(p):
|
||||
watch_paths.append(p)
|
||||
path_source_map[p] = cli
|
||||
|
||||
def on_change(file_path, event_type):
|
||||
now = datetime.now().strftime('%H:%M:%S')
|
||||
source = None
|
||||
for p, s in path_source_map.items():
|
||||
if file_path.startswith(p) or file_path == p:
|
||||
source = s
|
||||
break
|
||||
if not source:
|
||||
return
|
||||
|
||||
try:
|
||||
if source == 'kiro':
|
||||
parser = KiroParser()
|
||||
for sess in parser.parse_file(file_path):
|
||||
storage.upsert_session(sess.session_id, sess.source, sess.file_path, sess.cwd, sess.messages, int(sess.file_mtime))
|
||||
log.info(f"[{now}] kiro 更新")
|
||||
elif source == 'claude':
|
||||
parser = ClaudeParser()
|
||||
for sess in parser.parse_file(file_path):
|
||||
fp = f"claude:{sess.session_id}"
|
||||
storage.upsert_session(sess.session_id, sess.source, fp, sess.cwd, sess.messages, int(sess.file_mtime))
|
||||
log.info(f"[{now}] claude 更新")
|
||||
else:
|
||||
parser = CodexParser(source) if source == 'codex' else GeminiParser()
|
||||
sess = parser.parse_file(file_path)
|
||||
fp = os.path.abspath(sess.file_path)
|
||||
storage.upsert_session(sess.session_id, sess.source, fp, sess.cwd, sess.messages, int(sess.file_mtime))
|
||||
log.info(f"[{now}] {source} {event_type}: {os.path.basename(file_path)}")
|
||||
|
||||
update_cli_meta(source)
|
||||
total = storage.get_total_stats()
|
||||
storage.update_total_meta(total['sessions'], total['messages'], total['tokens'])
|
||||
except Exception as e:
|
||||
log.error(f"[{now}] 处理失败 {file_path}: {e}")
|
||||
|
||||
log.info(f"监听目录: {len(watch_paths)} 个")
|
||||
watcher = ChatWatcher(watch_paths, on_change)
|
||||
watcher.start()
|
||||
|
||||
def cmd_prune():
|
||||
log = get_logger()
|
||||
log.info("清理孤立记录...")
|
||||
removed = storage.prune()
|
||||
total = sum(removed.values())
|
||||
if total > 0:
|
||||
for cli, count in removed.items():
|
||||
if count > 0:
|
||||
log.info(f" {cli}: 删除 {count} 条")
|
||||
log.info(f"✓ 共清理 {total} 条孤立记录")
|
||||
else:
|
||||
log.info("✓ 无孤立记录")
|
||||
|
||||
def cmd_stats():
|
||||
log = get_logger()
|
||||
meta = storage.get_total_meta()
|
||||
tokens = storage.get_token_stats()
|
||||
|
||||
log.info("=" * 50)
|
||||
log.info("统计信息")
|
||||
log.info("=" * 50)
|
||||
log.info(f"数据库: {CONFIG['db_path']}")
|
||||
log.info(f"总会话: {meta['total_sessions']}")
|
||||
log.info(f"总消息: {meta['total_messages']}")
|
||||
log.info(f"最后同步: {meta['last_sync']}")
|
||||
log.info("")
|
||||
log.info("Token 统计 (tiktoken):")
|
||||
total_tokens = 0
|
||||
for source in ['codex', 'kiro', 'gemini', 'claude']:
|
||||
t = tokens.get(source, 0)
|
||||
if t > 0:
|
||||
log.info(f" {source}: {t:,}")
|
||||
total_tokens += t
|
||||
log.info(f" 总计: {total_tokens:,}")
|
||||
|
||||
def cmd_search(keyword: str, source: str = None):
|
||||
log = get_logger()
|
||||
results = storage.search(keyword, source)
|
||||
log.info(f"搜索 '{keyword}' 找到 {len(results)} 个会话:")
|
||||
for r in results[:20]:
|
||||
log.info(f" [{r['source']}] {r['session_id']} - {r['cwd'] or 'N/A'}")
|
||||
|
||||
def cmd_export(fmt: str, source: str = None, output: str = None):
|
||||
log = get_logger()
|
||||
if not output:
|
||||
output = os.path.join(CONFIG["output_dir"], f"export.{fmt}")
|
||||
|
||||
if fmt == 'json':
|
||||
count = storage.export_json(output, source)
|
||||
else:
|
||||
count = storage.export_csv(output, source)
|
||||
|
||||
log.info(f"✓ 导出 {count} 条到 {output}")
|
||||
|
||||
def print_token_stats():
|
||||
log = get_logger()
|
||||
tokens = storage.get_token_stats()
|
||||
log.info("")
|
||||
log.info("=== Token 统计 (tiktoken) ===")
|
||||
total = 0
|
||||
for source in ['codex', 'kiro', 'gemini', 'claude']:
|
||||
t = tokens.get(source, 0)
|
||||
if t > 0:
|
||||
log.info(f" {source}: {t:,} tokens")
|
||||
total += t
|
||||
log.info(f" 总计: {total:,} tokens")
|
||||
|
||||
def update_cli_meta(cli: str):
|
||||
stats = storage.get_cli_stats(cli)
|
||||
path = CONFIG.get(f"{cli}_paths", [""])[0] if CONFIG.get(f"{cli}_paths") else ""
|
||||
storage.update_cli_meta(cli, path, stats['sessions'], stats['messages'], stats['tokens'])
|
||||
|
||||
def process(parser, paths) -> tuple:
|
||||
log = get_logger()
|
||||
added, updated, skipped, errors = 0, 0, 0, 0
|
||||
for f in parser.find_files(paths):
|
||||
try:
|
||||
s = parser.parse_file(f)
|
||||
file_path = os.path.abspath(s.file_path)
|
||||
db_mtime = storage.get_file_mtime(file_path)
|
||||
file_mtime = int(s.file_mtime)
|
||||
if db_mtime == 0:
|
||||
storage.upsert_session(s.session_id, s.source, file_path, s.cwd, s.messages, file_mtime)
|
||||
added += 1
|
||||
elif file_mtime > db_mtime:
|
||||
storage.upsert_session(s.session_id, s.source, file_path, s.cwd, s.messages, file_mtime)
|
||||
updated += 1
|
||||
else:
|
||||
skipped += 1
|
||||
except Exception as e:
|
||||
log.debug(f"解析失败 {f}: {e}")
|
||||
errors += 1
|
||||
return added, updated, skipped, errors
|
||||
|
||||
def process_multi(parser, paths, source: str) -> tuple:
|
||||
"""处理返回多个会话的解析器(Claude/Kiro)"""
|
||||
log = get_logger()
|
||||
added, updated, skipped, errors = 0, 0, 0, 0
|
||||
for f in parser.find_files(paths):
|
||||
try:
|
||||
for s in parser.parse_file(f):
|
||||
file_path = s.file_path # kiro:xxx 或 claude:xxx
|
||||
db_mtime = storage.get_file_mtime(file_path)
|
||||
file_mtime = int(s.file_mtime)
|
||||
if db_mtime == 0:
|
||||
storage.upsert_session(s.session_id, s.source, file_path, s.cwd, s.messages, file_mtime)
|
||||
added += 1
|
||||
elif file_mtime > db_mtime:
|
||||
storage.upsert_session(s.session_id, s.source, file_path, s.cwd, s.messages, file_mtime)
|
||||
updated += 1
|
||||
else:
|
||||
skipped += 1
|
||||
except Exception as e:
|
||||
log.debug(f"解析失败 {f}: {e}")
|
||||
errors += 1
|
||||
return added, updated, skipped, errors
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
from .codex import CodexParser
|
||||
from .gemini import GeminiParser
|
||||
from .claude import ClaudeParser
|
||||
from .kiro import KiroParser
|
||||
from .base import SessionData
|
||||
|
||||
__all__ = ["CodexParser", "GeminiParser", "ClaudeParser", "KiroParser", "SessionData"]
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List, Dict
|
||||
|
||||
@dataclass
|
||||
class SessionData:
|
||||
"""会话数据"""
|
||||
session_id: str
|
||||
source: str
|
||||
file_path: str
|
||||
file_mtime: float = 0
|
||||
cwd: str = None
|
||||
messages: List[Dict] = field(default_factory=list) # [{"time", "role", "content"}]
|
||||
|
||||
class BaseParser(ABC):
|
||||
@abstractmethod
|
||||
def find_files(self, paths: list) -> list:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def parse_file(self, filepath: str) -> SessionData:
|
||||
pass
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
import os
|
||||
import json
|
||||
import hashlib
|
||||
from datetime import datetime
|
||||
from collections import defaultdict
|
||||
from .base import BaseParser, SessionData
|
||||
|
||||
class ClaudeParser(BaseParser):
|
||||
def find_files(self, paths: list) -> list:
|
||||
files = []
|
||||
for base in paths:
|
||||
history = os.path.join(base, "history.jsonl")
|
||||
if os.path.exists(history):
|
||||
files.append(history)
|
||||
return files
|
||||
|
||||
def parse_file(self, filepath: str) -> list:
|
||||
"""返回多个 SessionData(按 project 分组)"""
|
||||
projects = defaultdict(list)
|
||||
file_mtime = os.path.getmtime(filepath)
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
data = json.loads(line)
|
||||
content = data.get('display', '')
|
||||
if not content:
|
||||
continue
|
||||
|
||||
project = data.get('project', 'unknown')
|
||||
ts_ms = data.get('timestamp', 0)
|
||||
ts = datetime.fromtimestamp(ts_ms / 1000).isoformat() if ts_ms else ''
|
||||
|
||||
projects[project].append({
|
||||
'time': ts,
|
||||
'role': 'user',
|
||||
'content': content
|
||||
})
|
||||
|
||||
return [
|
||||
SessionData(
|
||||
session_id='claude-' + hashlib.md5(proj.encode()).hexdigest()[:12],
|
||||
source='claude',
|
||||
file_path=filepath,
|
||||
file_mtime=file_mtime,
|
||||
cwd=proj,
|
||||
messages=msgs
|
||||
)
|
||||
for proj, msgs in projects.items()
|
||||
]
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
import os
|
||||
import json
|
||||
import re
|
||||
from .base import BaseParser, SessionData
|
||||
|
||||
class CodexParser(BaseParser):
|
||||
def __init__(self, source: str = 'codex'):
|
||||
self.source = source
|
||||
|
||||
def find_files(self, paths: list) -> list:
|
||||
files = []
|
||||
for base in paths:
|
||||
if not os.path.exists(base):
|
||||
continue
|
||||
for root, _, names in os.walk(base):
|
||||
for f in names:
|
||||
if f.endswith('.jsonl') and f != 'history.jsonl':
|
||||
files.append(os.path.join(root, f))
|
||||
return files
|
||||
|
||||
def _extract_id(self, filepath: str) -> str:
|
||||
match = re.search(r'([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})',
|
||||
os.path.basename(filepath))
|
||||
return match.group(1) if match else os.path.basename(filepath).replace('.jsonl', '')
|
||||
|
||||
def parse_file(self, filepath: str) -> SessionData:
|
||||
s = SessionData(
|
||||
session_id=self._extract_id(filepath),
|
||||
source=self.source,
|
||||
file_path=filepath,
|
||||
file_mtime=os.path.getmtime(filepath)
|
||||
)
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
|
||||
for line in f:
|
||||
line = line.strip()
|
||||
if not line or line[0] != '{':
|
||||
continue
|
||||
try:
|
||||
data = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
if data.get('type') == 'session_meta':
|
||||
p = data.get('payload', {})
|
||||
s.cwd = p.get('cwd')
|
||||
s.session_id = p.get('id', s.session_id)
|
||||
continue
|
||||
|
||||
if data.get('type') != 'response_item':
|
||||
continue
|
||||
payload = data.get('payload', {})
|
||||
if payload.get('type') != 'message':
|
||||
continue
|
||||
role = payload.get('role')
|
||||
if role not in ('user', 'assistant'):
|
||||
continue
|
||||
|
||||
parts = [item.get('text', '') for item in payload.get('content', [])
|
||||
if isinstance(item, dict) and item.get('type') in ('input_text', 'output_text', 'text')]
|
||||
if parts:
|
||||
s.messages.append({
|
||||
'time': data.get('timestamp', ''),
|
||||
'role': 'user' if role == 'user' else 'ai',
|
||||
'content': ' '.join(parts)
|
||||
})
|
||||
|
||||
return s
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
import os
|
||||
import glob
|
||||
import json
|
||||
from .base import BaseParser, SessionData
|
||||
|
||||
class GeminiParser(BaseParser):
|
||||
def find_files(self, paths: list) -> list:
|
||||
files = []
|
||||
for base in paths:
|
||||
if os.path.exists(base):
|
||||
files.extend(glob.glob(os.path.join(base, "*", "chats", "*.json")))
|
||||
return files
|
||||
|
||||
def parse_file(self, filepath: str) -> SessionData:
|
||||
s = SessionData(
|
||||
session_id=os.path.basename(filepath).replace('.json', ''),
|
||||
source='gemini',
|
||||
file_path=filepath,
|
||||
file_mtime=os.path.getmtime(filepath)
|
||||
)
|
||||
|
||||
with open(filepath, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
|
||||
s.session_id = data.get('sessionId', s.session_id)
|
||||
|
||||
for msg in data.get('messages', []):
|
||||
if msg.get('type') not in ('user', 'gemini'):
|
||||
continue
|
||||
content = msg.get('content', '')
|
||||
if content:
|
||||
s.messages.append({
|
||||
'time': msg.get('timestamp', ''),
|
||||
'role': 'user' if msg.get('type') == 'user' else 'ai',
|
||||
'content': content
|
||||
})
|
||||
|
||||
return s
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Kiro CLI 解析器 - 从 SQLite 数据库读取"""
|
||||
import os
|
||||
import json
|
||||
import sqlite3
|
||||
import hashlib
|
||||
from datetime import datetime
|
||||
from .base import BaseParser, SessionData
|
||||
|
||||
KIRO_DB = os.path.expanduser("~/.local/share/kiro-cli/data.sqlite3")
|
||||
|
||||
class KiroParser(BaseParser):
|
||||
def find_files(self, paths: list) -> list:
|
||||
"""返回数据库路径(如果存在)"""
|
||||
if os.path.exists(KIRO_DB):
|
||||
return [KIRO_DB]
|
||||
return []
|
||||
|
||||
def parse_file(self, filepath: str) -> list:
|
||||
"""解析 Kiro SQLite 数据库,返回多个 SessionData"""
|
||||
sessions = []
|
||||
file_mtime = os.path.getmtime(filepath)
|
||||
|
||||
conn = sqlite3.connect(filepath)
|
||||
for row in conn.execute('SELECT key, value FROM conversations'):
|
||||
cwd, value = row
|
||||
try:
|
||||
data = json.loads(value)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
|
||||
conv_id = data.get('conversation_id', hashlib.md5(cwd.encode()).hexdigest()[:12])
|
||||
history = data.get('history', [])
|
||||
|
||||
messages = []
|
||||
for item in history:
|
||||
# 用户消息
|
||||
if 'user' in item:
|
||||
user = item['user']
|
||||
content = user.get('content', {})
|
||||
if isinstance(content, dict) and 'Prompt' in content:
|
||||
prompt = content['Prompt'].get('prompt', '')
|
||||
if prompt:
|
||||
messages.append({
|
||||
'time': '',
|
||||
'role': 'user',
|
||||
'content': prompt
|
||||
})
|
||||
|
||||
# AI 回复
|
||||
if 'assistant' in item:
|
||||
assistant = item['assistant']
|
||||
content = assistant.get('content', {})
|
||||
if isinstance(content, dict) and 'Message' in content:
|
||||
msg = content['Message'].get('message', '')
|
||||
if msg:
|
||||
messages.append({
|
||||
'time': '',
|
||||
'role': 'ai',
|
||||
'content': msg
|
||||
})
|
||||
|
||||
if messages:
|
||||
sessions.append(SessionData(
|
||||
session_id=f'kiro-{conv_id[:12]}',
|
||||
source='kiro',
|
||||
file_path=f'kiro:{conv_id}',
|
||||
file_mtime=file_mtime,
|
||||
cwd=cwd,
|
||||
messages=messages
|
||||
))
|
||||
|
||||
conn.close()
|
||||
return sessions
|
||||
|
|
@ -0,0 +1,246 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""SQLite 存储模块 - 完整版"""
|
||||
import sqlite3
|
||||
import json
|
||||
import os
|
||||
import datetime
|
||||
import tiktoken
|
||||
|
||||
SCHEMA_VERSION = 5
|
||||
CLIS = ('codex', 'kiro', 'gemini', 'claude')
|
||||
|
||||
_encoder = tiktoken.get_encoding("cl100k_base")
|
||||
|
||||
def count_tokens(text: str) -> int:
|
||||
return len(_encoder.encode(text)) if text else 0
|
||||
|
||||
class ChatStorage:
|
||||
def __init__(self, db_path: str):
|
||||
self.db_path = db_path
|
||||
os.makedirs(os.path.dirname(db_path) or '.', exist_ok=True)
|
||||
self._init_db()
|
||||
|
||||
def _conn(self):
|
||||
return sqlite3.connect(self.db_path)
|
||||
|
||||
def _init_db(self):
|
||||
with self._conn() as conn:
|
||||
conn.execute('''CREATE TABLE IF NOT EXISTS meta (key TEXT PRIMARY KEY, value TEXT)''')
|
||||
for cli in CLIS:
|
||||
conn.execute(f'''CREATE TABLE IF NOT EXISTS meta_{cli} (key TEXT PRIMARY KEY, value TEXT)''')
|
||||
conn.execute('''
|
||||
CREATE TABLE IF NOT EXISTS sessions (
|
||||
file_path TEXT PRIMARY KEY,
|
||||
session_id TEXT,
|
||||
source TEXT NOT NULL,
|
||||
cwd TEXT,
|
||||
messages TEXT,
|
||||
file_mtime INTEGER,
|
||||
start_time TEXT,
|
||||
token_count INTEGER DEFAULT 0
|
||||
)
|
||||
''')
|
||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_source ON sessions(source)')
|
||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_session_id ON sessions(session_id)')
|
||||
conn.execute('CREATE INDEX IF NOT EXISTS idx_start_time ON sessions(start_time)')
|
||||
self._set_meta('meta', 'schema_version', str(SCHEMA_VERSION))
|
||||
|
||||
def _set_meta(self, table: str, key: str, value: str):
|
||||
with self._conn() as conn:
|
||||
conn.execute(f'INSERT OR REPLACE INTO {table} (key, value) VALUES (?, ?)', (key, value))
|
||||
|
||||
def _get_meta(self, table: str, key: str) -> str:
|
||||
with self._conn() as conn:
|
||||
row = conn.execute(f'SELECT value FROM {table} WHERE key = ?', (key,)).fetchone()
|
||||
return row[0] if row else None
|
||||
|
||||
def update_cli_meta(self, cli: str, path: str, sessions: int, messages: int, tokens: int = None):
|
||||
table = f'meta_{cli}'
|
||||
now = datetime.datetime.now().isoformat()
|
||||
# 顺序: path, sessions, messages, total_tokens, last_sync
|
||||
self._set_meta(table, 'path', path)
|
||||
self._set_meta(table, 'sessions', str(sessions))
|
||||
self._set_meta(table, 'messages', str(messages))
|
||||
self._set_meta(table, 'total_tokens', str(tokens or 0))
|
||||
self._set_meta(table, 'last_sync', now)
|
||||
|
||||
def update_total_meta(self, sessions: int, messages: int, tokens: int = None):
|
||||
now = datetime.datetime.now().isoformat()
|
||||
self._set_meta('meta', 'total_sessions', str(sessions))
|
||||
self._set_meta('meta', 'total_messages', str(messages))
|
||||
if tokens is not None:
|
||||
self._set_meta('meta', 'total_tokens', str(tokens))
|
||||
self._set_meta('meta', 'last_sync', now)
|
||||
|
||||
def get_total_meta(self) -> dict:
|
||||
return {
|
||||
'schema_version': int(self._get_meta('meta', 'schema_version') or 0),
|
||||
'total_sessions': int(self._get_meta('meta', 'total_sessions') or 0),
|
||||
'total_messages': int(self._get_meta('meta', 'total_messages') or 0),
|
||||
'last_sync': self._get_meta('meta', 'last_sync'),
|
||||
}
|
||||
|
||||
def get_file_mtime(self, file_path: str) -> int:
|
||||
with self._conn() as conn:
|
||||
row = conn.execute('SELECT file_mtime FROM sessions WHERE file_path = ?', (file_path,)).fetchone()
|
||||
return row[0] if row else 0
|
||||
|
||||
def upsert_session(self, session_id: str, source: str, file_path: str,
|
||||
cwd: str, messages: list, file_mtime: int, start_time: str = None):
|
||||
if file_path and not file_path.startswith('claude:') and not os.path.isabs(file_path):
|
||||
file_path = os.path.abspath(file_path)
|
||||
|
||||
total_tokens = sum(count_tokens(msg.get('content', '')) for msg in messages)
|
||||
if not start_time and messages:
|
||||
start_time = messages[0].get('time')
|
||||
|
||||
messages_json = json.dumps(messages, ensure_ascii=False)
|
||||
with self._conn() as conn:
|
||||
conn.execute('''
|
||||
INSERT INTO sessions (file_path, session_id, source, cwd, messages, file_mtime, start_time, token_count)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
ON CONFLICT(file_path) DO UPDATE SET
|
||||
session_id=excluded.session_id, messages=excluded.messages,
|
||||
file_mtime=excluded.file_mtime, start_time=excluded.start_time, token_count=excluded.token_count
|
||||
''', (file_path, session_id, source, cwd, messages_json, file_mtime, start_time, total_tokens))
|
||||
|
||||
def get_cli_stats(self, cli: str) -> dict:
|
||||
with self._conn() as conn:
|
||||
sessions = conn.execute('SELECT COUNT(*) FROM sessions WHERE source = ?', (cli,)).fetchone()[0]
|
||||
row = conn.execute('SELECT SUM(json_array_length(messages)) FROM sessions WHERE source = ?', (cli,)).fetchone()
|
||||
messages = row[0] or 0
|
||||
tokens = conn.execute('SELECT SUM(token_count) FROM sessions WHERE source = ?', (cli,)).fetchone()[0] or 0
|
||||
return {'sessions': sessions, 'messages': messages, 'tokens': tokens}
|
||||
|
||||
def get_total_stats(self) -> dict:
|
||||
with self._conn() as conn:
|
||||
sessions = conn.execute('SELECT COUNT(*) FROM sessions').fetchone()[0]
|
||||
row = conn.execute('SELECT SUM(json_array_length(messages)) FROM sessions').fetchone()
|
||||
messages = row[0] or 0
|
||||
tokens = conn.execute('SELECT SUM(token_count) FROM sessions').fetchone()[0] or 0
|
||||
return {'sessions': sessions, 'messages': messages, 'tokens': tokens}
|
||||
|
||||
def get_token_stats(self) -> dict:
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute('SELECT source, SUM(token_count) FROM sessions GROUP BY source').fetchall()
|
||||
return {r[0]: r[1] or 0 for r in rows}
|
||||
|
||||
# === 清理孤立记录 ===
|
||||
def prune(self) -> dict:
|
||||
"""删除源文件已不存在的记录"""
|
||||
removed = {'codex': 0, 'kiro': 0, 'gemini': 0, 'claude': 0}
|
||||
with self._conn() as conn:
|
||||
rows = conn.execute('SELECT file_path, source FROM sessions').fetchall()
|
||||
for fp, source in rows:
|
||||
if fp.startswith('claude:'):
|
||||
continue # Claude 使用虚拟路径
|
||||
if not os.path.exists(fp):
|
||||
conn.execute('DELETE FROM sessions WHERE file_path = ?', (fp,))
|
||||
removed[source] = removed.get(source, 0) + 1
|
||||
return removed
|
||||
|
||||
# === 查询 ===
|
||||
def search(self, keyword: str, source: str = None, limit: int = 50) -> list:
|
||||
"""搜索消息内容"""
|
||||
sql = "SELECT file_path, session_id, source, cwd, messages, start_time FROM sessions WHERE messages LIKE ?"
|
||||
params = [f'%{keyword}%']
|
||||
if source:
|
||||
sql += " AND source = ?"
|
||||
params.append(source)
|
||||
sql += f" ORDER BY start_time DESC LIMIT {limit}"
|
||||
|
||||
results = []
|
||||
with self._conn() as conn:
|
||||
for row in conn.execute(sql, params):
|
||||
results.append({
|
||||
'file_path': row[0], 'session_id': row[1], 'source': row[2],
|
||||
'cwd': row[3], 'messages': json.loads(row[4]), 'start_time': row[5]
|
||||
})
|
||||
return results
|
||||
|
||||
def get_session(self, file_path: str) -> dict:
|
||||
"""获取单个会话"""
|
||||
with self._conn() as conn:
|
||||
row = conn.execute(
|
||||
'SELECT file_path, session_id, source, cwd, messages, start_time, token_count FROM sessions WHERE file_path = ?',
|
||||
(file_path,)
|
||||
).fetchone()
|
||||
if not row:
|
||||
return None
|
||||
return {
|
||||
'file_path': row[0], 'session_id': row[1], 'source': row[2], 'cwd': row[3],
|
||||
'messages': json.loads(row[4]), 'start_time': row[5], 'token_count': row[6]
|
||||
}
|
||||
|
||||
def list_sessions(self, source: str = None, limit: int = 100, offset: int = 0) -> list:
|
||||
"""列出会话"""
|
||||
sql = "SELECT file_path, session_id, source, cwd, start_time, token_count FROM sessions"
|
||||
params = []
|
||||
if source:
|
||||
sql += " WHERE source = ?"
|
||||
params.append(source)
|
||||
sql += f" ORDER BY start_time DESC LIMIT {limit} OFFSET {offset}"
|
||||
|
||||
results = []
|
||||
with self._conn() as conn:
|
||||
for row in conn.execute(sql, params):
|
||||
results.append({
|
||||
'file_path': row[0], 'session_id': row[1], 'source': row[2],
|
||||
'cwd': row[3], 'start_time': row[4], 'token_count': row[5]
|
||||
})
|
||||
return results
|
||||
|
||||
# === 导出 ===
|
||||
def export_json(self, output_path: str, source: str = None):
|
||||
"""导出为 JSON"""
|
||||
sql = "SELECT file_path, session_id, source, cwd, messages, start_time, token_count FROM sessions"
|
||||
params = []
|
||||
if source:
|
||||
sql += " WHERE source = ?"
|
||||
params.append(source)
|
||||
sql += " ORDER BY start_time"
|
||||
|
||||
data = []
|
||||
with self._conn() as conn:
|
||||
for row in conn.execute(sql, params):
|
||||
data.append({
|
||||
'file_path': row[0], 'session_id': row[1], 'source': row[2], 'cwd': row[3],
|
||||
'messages': json.loads(row[4]), 'start_time': row[5], 'token_count': row[6]
|
||||
})
|
||||
|
||||
with open(output_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||
return len(data)
|
||||
|
||||
def export_csv(self, output_path: str, source: str = None):
|
||||
"""导出为 CSV(扁平化消息)"""
|
||||
import csv
|
||||
sql = "SELECT session_id, source, cwd, messages, start_time FROM sessions"
|
||||
params = []
|
||||
if source:
|
||||
sql += " WHERE source = ?"
|
||||
params.append(source)
|
||||
sql += " ORDER BY start_time"
|
||||
|
||||
count = 0
|
||||
with open(output_path, 'w', encoding='utf-8', newline='') as f:
|
||||
writer = csv.writer(f)
|
||||
writer.writerow(['session_id', 'source', 'cwd', 'time', 'role', 'content'])
|
||||
with self._conn() as conn:
|
||||
for row in conn.execute(sql, params):
|
||||
session_id, src, cwd, msgs_json, _ = row
|
||||
for msg in json.loads(msgs_json):
|
||||
writer.writerow([session_id, src, cwd, msg.get('time', ''), msg.get('role', ''), msg.get('content', '')])
|
||||
count += 1
|
||||
return count
|
||||
|
||||
# === 获取所有文件路径(用于 prune 检查) ===
|
||||
def get_all_file_paths(self, source: str = None) -> set:
|
||||
sql = "SELECT file_path FROM sessions"
|
||||
params = []
|
||||
if source:
|
||||
sql += " WHERE source = ?"
|
||||
params.append(source)
|
||||
with self._conn() as conn:
|
||||
return {row[0] for row in conn.execute(sql, params)}
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""跨平台文件监控 (Linux/macOS/Windows)"""
|
||||
|
||||
from watchdog.observers import Observer
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
import time
|
||||
|
||||
class ChatFileHandler(FileSystemEventHandler):
|
||||
def __init__(self, callback, extensions):
|
||||
self.callback = callback
|
||||
self.extensions = extensions
|
||||
|
||||
def _check(self, event):
|
||||
if event.is_directory:
|
||||
return
|
||||
path = event.src_path
|
||||
if any(path.endswith(ext) for ext in self.extensions):
|
||||
self.callback(path, event.event_type)
|
||||
|
||||
def on_created(self, event):
|
||||
self._check(event)
|
||||
|
||||
def on_modified(self, event):
|
||||
self._check(event)
|
||||
|
||||
class ChatWatcher:
|
||||
def __init__(self, paths: list, callback, extensions=('.jsonl', '.json')):
|
||||
self.observer = Observer()
|
||||
handler = ChatFileHandler(callback, extensions)
|
||||
for path in paths:
|
||||
self.observer.schedule(handler, path, recursive=True)
|
||||
|
||||
def start(self):
|
||||
self.observer.start()
|
||||
try:
|
||||
while True:
|
||||
time.sleep(1)
|
||||
except KeyboardInterrupt:
|
||||
self.stop()
|
||||
|
||||
def stop(self):
|
||||
self.observer.stop()
|
||||
self.observer.join()
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
@echo off
|
||||
cd /d "%~dp0src"
|
||||
echo 正在启动 AI Chat Converter...
|
||||
echo 按 Ctrl+C 停止
|
||||
echo.
|
||||
python main.py --watch
|
||||
pause
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
#!/bin/bash
|
||||
# AI Chat Converter - 一键启动
|
||||
cd "$(dirname "$0")/src"
|
||||
echo "正在启动 AI Chat Converter..."
|
||||
echo "按 Ctrl+C 停止"
|
||||
echo ""
|
||||
python3 main.py --watch
|
||||
Loading…
Reference in New Issue