init: 初始化 AssetX 项目仓库
包含 webapp（Next.js 用户端）、webapp-back（Go 后端）、 antdesign（管理后台）、landingpage（营销落地页）、 数据库 SQL 和配置文件。
This commit is contained in:
454
webapp-back/holders/README.md
Normal file
454
webapp-back/holders/README.md
Normal file
@@ -0,0 +1,454 @@
|
||||
# Holders API - 持有者统计服务
|
||||
|
||||
## 📋 概述
|
||||
|
||||
这个模块提供了区块链代币持有者的统计和查询功能，包括：
|
||||
- **YT 代币持有者** (YT-A, YT-B, YT-C)
|
||||
- **ytLP 代币持有者**
|
||||
- **Lending 提供者**
|
||||
|
||||
## 🏗️ 架构
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌──────────────────┐ ┌──────────────┐
|
||||
│ Frontend │─────▶│ API Server │─────▶│ Database │
|
||||
│ (Ant Design) │ │ (Gin + Gorm) │ │ (MySQL/PG) │
|
||||
└─────────────────┘ └──────────────────┘ └──────────────┘
|
||||
│ ▲
|
||||
│ │
|
||||
▼ │
|
||||
┌──────────────────┐ │
|
||||
│ Scanner Service │────────────┘
|
||||
│ (Blockchain) │
|
||||
└──────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ Arbitrum RPC │
|
||||
│ (Sepolia) │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
## 📁 文件结构
|
||||
|
||||
```
|
||||
holders/
|
||||
├── routers.go # API 路由和处理函数
|
||||
├── scanner.go # 区块链扫描器核心逻辑
|
||||
└── README.md # 本文档
|
||||
|
||||
cmd/scanner/
|
||||
└── main.go # 扫描器服务启动入口
|
||||
|
||||
models/
|
||||
└── holder.go # 数据库模型定义
|
||||
```
|
||||
|
||||
## 🔧 安装依赖
|
||||
|
||||
### 1. 安装 Go 依赖
|
||||
|
||||
```bash
|
||||
cd /home/coder/myprojects/assetx/golang-gin-realworld
|
||||
|
||||
# 添加 ethereum 依赖
|
||||
go get github.com/ethereum/go-ethereum
|
||||
go get github.com/ethereum/go-ethereum/ethclient
|
||||
go get github.com/ethereum/go-ethereum/accounts/abi
|
||||
go get github.com/ethereum/go-ethereum/common
|
||||
go get github.com/ethereum/go-ethereum/core/types
|
||||
go get github.com/ethereum/go-ethereum/crypto
|
||||
|
||||
# 更新依赖
|
||||
go mod tidy
|
||||
```
|
||||
|
||||
### 2. 数据库迁移
|
||||
|
||||
数据库表会在启动时自动创建（通过 GORM AutoMigrate）：
|
||||
|
||||
```sql
|
||||
CREATE TABLE holder_snapshots (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
holder_address VARCHAR(42) NOT NULL,
|
||||
token_type VARCHAR(50) NOT NULL,
|
||||
token_address VARCHAR(42) NOT NULL,
|
||||
balance VARCHAR(78) NOT NULL,
|
||||
chain_id INTEGER NOT NULL,
|
||||
first_seen BIGINT NOT NULL,
|
||||
last_updated BIGINT NOT NULL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
|
||||
INDEX idx_holder_token (holder_address, token_type),
|
||||
INDEX idx_token_time (token_type, last_updated)
|
||||
);
|
||||
```
|
||||
|
||||
## 🚀 启动服务
|
||||
|
||||
### 方法一：分别启动（推荐开发环境）
|
||||
|
||||
**1. 启动 API 服务器**
|
||||
```bash
|
||||
cd /home/coder/myprojects/assetx/golang-gin-realworld
|
||||
|
||||
# 设置环境变量
|
||||
export DB_TYPE=mysql
|
||||
export DB_HOST=localhost
|
||||
export DB_PORT=3306
|
||||
export DB_USER=root
|
||||
export DB_PASSWORD=your_password
|
||||
export DB_NAME=assetx
|
||||
export GIN_MODE=release
|
||||
export PORT=8080
|
||||
|
||||
# 启动 API 服务
|
||||
go run main.go
|
||||
```
|
||||
|
||||
**2. 启动扫描器服务**
|
||||
```bash
|
||||
cd /home/coder/myprojects/assetx/golang-gin-realworld
|
||||
|
||||
# 设置 RPC URL（可选，有默认值）
|
||||
export RPC_URL=https://api.zan.top/node/v1/arb/sepolia/baf84c429d284bb5b676cb8c9ca21c07
|
||||
|
||||
# 启动扫描器
|
||||
go run cmd/scanner/main.go
|
||||
```
|
||||
|
||||
### 方法二：使用 PM2（推荐生产环境）
|
||||
|
||||
**1. 创建 PM2 配置文件**
|
||||
```json
|
||||
// ecosystem.holders.config.js
|
||||
module.exports = {
|
||||
apps: [
|
||||
{
|
||||
name: 'assetx-api',
|
||||
script: './golang-gin-realworld-example-app',
|
||||
env: {
|
||||
DB_TYPE: 'mysql',
|
||||
DB_HOST: 'localhost',
|
||||
DB_PORT: '3306',
|
||||
DB_USER: 'root',
|
||||
DB_PASSWORD: 'your_password',
|
||||
DB_NAME: 'assetx',
|
||||
GIN_MODE: 'release',
|
||||
PORT: '8080'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'holder-scanner',
|
||||
script: 'go',
|
||||
args: 'run cmd/scanner/main.go',
|
||||
env: {
|
||||
DB_TYPE: 'mysql',
|
||||
DB_HOST: 'localhost',
|
||||
DB_PORT: '3306',
|
||||
DB_USER: 'root',
|
||||
DB_PASSWORD: 'your_password',
|
||||
DB_NAME: 'assetx',
|
||||
RPC_URL: 'https://api.zan.top/node/v1/arb/sepolia/baf84c429d284bb5b676cb8c9ca21c07'
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
```
|
||||
|
||||
**2. 启动服务**
|
||||
```bash
|
||||
# 先编译 API 服务器
|
||||
go build -o golang-gin-realworld-example-app
|
||||
|
||||
# 使用 PM2 启动
|
||||
pm2 start ecosystem.holders.config.js
|
||||
pm2 logs
|
||||
pm2 status
|
||||
```
|
||||
|
||||
## 📡 API 接口
|
||||
|
||||
### 1. GET /api/holders/stats
|
||||
|
||||
获取所有代币类型的统计信息。
|
||||
|
||||
**请求**
|
||||
```bash
|
||||
curl http://localhost:8080/api/holders/stats
|
||||
```
|
||||
|
||||
**响应**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": [
|
||||
{
|
||||
"token_type": "YT-A",
|
||||
"holder_count": 156,
|
||||
"total_balance": "1250000000000000000000"
|
||||
},
|
||||
{
|
||||
"token_type": "YT-B",
|
||||
"holder_count": 89,
|
||||
"total_balance": "780000000000000000000"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 2. GET /api/holders/:tokenType
|
||||
|
||||
获取特定代币的持有者列表。
|
||||
|
||||
**参数**
|
||||
- `tokenType`: `YT-A` | `YT-B` | `YT-C` | `ytLP` | `Lending`
|
||||
|
||||
**请求**
|
||||
```bash
|
||||
curl http://localhost:8080/api/holders/YT-A
|
||||
```
|
||||
|
||||
**响应**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": [
|
||||
{
|
||||
"id": 1,
|
||||
"holder_address": "0x1234567890abcdef1234567890abcdef12345678",
|
||||
"token_type": "YT-A",
|
||||
"token_address": "0x97204190B35D9895a7a47aa7BaC61ac08De3cF05",
|
||||
"balance": "100000000000000000000",
|
||||
"chain_id": 421614,
|
||||
"first_seen": 1707800000,
|
||||
"last_updated": 1707900000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 3. POST /api/holders/update
|
||||
|
||||
手动触发数据更新（需要管理员权限）。
|
||||
|
||||
**请求**
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/api/holders/update \
|
||||
-H "Authorization: Bearer YOUR_ADMIN_TOKEN"
|
||||
```
|
||||
|
||||
**响应**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Update triggered successfully",
|
||||
"timestamp": 1707900000
|
||||
}
|
||||
```
|
||||
|
||||
## ⚙️ 配置说明
|
||||
|
||||
### 扫描器配置
|
||||
|
||||
在 `cmd/scanner/main.go` 中可以修改以下配置：
|
||||
|
||||
```go
|
||||
scannerConfig := holders.Config{
|
||||
RPCURL: "your_rpc_url", // RPC 节点地址
|
||||
PollInterval: 10 * time.Second, // 轮询间隔（10秒）
|
||||
BatchSize: 9999, // 每批次查询的区块数
|
||||
|
||||
// 代币合约地址
|
||||
YTVaults: []holders.VaultConfig{
|
||||
{Name: "YT-A", Address: "0x..."},
|
||||
{Name: "YT-B", Address: "0x..."},
|
||||
{Name: "YT-C", Address: "0x..."},
|
||||
},
|
||||
YTLPAddress: "0x...",
|
||||
LendingAddress: "0x...",
|
||||
|
||||
// 部署区块号（从这里开始扫描）
|
||||
DeploymentBlocks: holders.DeploymentBlocks{
|
||||
YTVaults: 227339300,
|
||||
YTLP: 227230270,
|
||||
Lending: 227746053,
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### 环境变量
|
||||
|
||||
| 变量名 | 说明 | 默认值 |
|
||||
|--------|------|--------|
|
||||
| `RPC_URL` | Arbitrum RPC 节点地址 | zan.top URL |
|
||||
| `DB_TYPE` | 数据库类型 | `sqlite` |
|
||||
| `DB_HOST` | 数据库主机 | `localhost` |
|
||||
| `DB_PORT` | 数据库端口 | `3306` |
|
||||
| `DB_USER` | 数据库用户名 | `root` |
|
||||
| `DB_PASSWORD` | 数据库密码 | - |
|
||||
| `DB_NAME` | 数据库名称 | `assetx` |
|
||||
|
||||
## 🔍 工作原理
|
||||
|
||||
### 扫描流程
|
||||
|
||||
1. **初始扫描**
|
||||
- 从合约部署区块开始
|
||||
- 扫描所有历史 Transfer/Supply 事件
|
||||
- 记录所有曾经持有代币的地址
|
||||
- 查询当前余额并保存到数据库
|
||||
|
||||
2. **增量扫描**
|
||||
- 每 10 秒检查一次新区块
|
||||
- 只扫描自上次扫描后的新区块
|
||||
- 更新新地址和余额变化
|
||||
- 持续追踪地址首次出现时间
|
||||
|
||||
3. **数据存储**
|
||||
- 使用 `holder_snapshots` 表存储快照
|
||||
- 余额以 wei 格式字符串存储（避免精度丢失）
|
||||
- 记录首次持有时间和最后更新时间
|
||||
- 支持唯一约束（address + token_type）
|
||||
|
||||
### 性能优化
|
||||
|
||||
- ✅ 批量查询区块事件（每次最多 9999 个区块）
|
||||
- ✅ 增量扫描（只查询新区块）
|
||||
- ✅ 地址缓存（减少重复查询）
|
||||
- ✅ 并发控制（防止重复扫描）
|
||||
- ✅ 速率限制（避免 RPC 限流）
|
||||
|
||||
## 📊 监控和日志
|
||||
|
||||
### 查看扫描器日志
|
||||
|
||||
```bash
|
||||
# 实时查看日志
|
||||
pm2 logs holder-scanner
|
||||
|
||||
# 或者直接运行时查看
|
||||
go run cmd/scanner/main.go
|
||||
```
|
||||
|
||||
### 日志示例
|
||||
|
||||
```
|
||||
=== Holder Scanner Service ===
|
||||
✓ Configuration loaded
|
||||
✓ Database tables checked
|
||||
🚀 Starting blockchain scanner...
|
||||
=== Holder Scanner Started ===
|
||||
RPC: https://api.zan.top/node/v1/arb/sepolia/...
|
||||
Poll Interval: 10s
|
||||
📊 Starting initial scan...
|
||||
Current block: 227950000
|
||||
|
||||
1. Scanning YT Vaults...
|
||||
Scanning YT-A (0x9720...)...
|
||||
Querying blocks 227339300 -> 227950000 (total: 610700 blocks)
|
||||
Querying blocks 227339300 - 227349299...
|
||||
✓ Got 45 events
|
||||
...
|
||||
Found 23 new addresses, total tracking: 156
|
||||
YT-A: 156 holders saved
|
||||
|
||||
2. Scanning ytLP...
|
||||
ytLP: 67 holders saved
|
||||
|
||||
3. Scanning Lending...
|
||||
Lending: 43 holders saved
|
||||
|
||||
📌 Last scanned block: 227950000
|
||||
✓ Initial scan completed in 2m34s
|
||||
|
||||
⏰ Starting polling every 10s...
|
||||
⏰ [15:30:45] No new blocks (current: 227950000)
|
||||
⏰ [15:30:55] Found new blocks
|
||||
🔄 Incremental scan: blocks 227950001 -> 227950015
|
||||
✓ Incremental scan completed in 1.2s
|
||||
```
|
||||
|
||||
## 🐛 故障排查
|
||||
|
||||
### 常见问题
|
||||
|
||||
**1. RPC 连接失败**
|
||||
```
|
||||
Error: failed to connect to Ethereum client
|
||||
```
|
||||
**解决**: 检查 `RPC_URL` 是否正确，网络是否通畅
|
||||
|
||||
**2. 数据库连接失败**
|
||||
```
|
||||
Error: failed to connect to database
|
||||
```
|
||||
**解决**: 检查数据库配置和权限
|
||||
|
||||
**3. 查询超时**
|
||||
```
|
||||
Error: context deadline exceeded
|
||||
```
|
||||
**解决**: 减小 `BatchSize` 参数（如改为 5000）
|
||||
|
||||
**4. 内存占用过高**
|
||||
```
|
||||
OOM or high memory usage
|
||||
```
|
||||
**解决**: 增加批次间隔时间或减小批次大小
|
||||
|
||||
### 数据验证
|
||||
|
||||
**检查数据库中的数据**
|
||||
```sql
|
||||
-- 查看所有代币类型的持有者数量
|
||||
SELECT token_type, COUNT(*) as count
|
||||
FROM holder_snapshots
|
||||
GROUP BY token_type;
|
||||
|
||||
-- 查看 YT-A 的前 10 名持有者
|
||||
SELECT holder_address, balance
|
||||
FROM holder_snapshots
|
||||
WHERE token_type = 'YT-A'
|
||||
ORDER BY CAST(balance AS DECIMAL) DESC
|
||||
LIMIT 10;
|
||||
|
||||
-- 查看最近更新的记录
|
||||
SELECT * FROM holder_snapshots
|
||||
ORDER BY last_updated DESC
|
||||
LIMIT 10;
|
||||
```
|
||||
|
||||
## 🔐 安全注意事项
|
||||
|
||||
1. **RPC 密钥保护**
|
||||
- 不要在代码中硬编码 RPC URL
|
||||
- 使用环境变量或配置文件
|
||||
- 限制 RPC API Key 的权限
|
||||
|
||||
2. **数据库安全**
|
||||
- 使用强密码
|
||||
- 限制数据库访问权限
|
||||
- 定期备份数据
|
||||
|
||||
3. **API 权限**
|
||||
- `/update` 接口需要管理员权限
|
||||
- 建议添加请求频率限制
|
||||
- 记录操作日志
|
||||
|
||||
## 📝 TODO
|
||||
|
||||
- [ ] 添加 WebSocket 支持（实时推送）
|
||||
- [ ] 实现手动触发扫描的实际逻辑
|
||||
- [ ] 添加 Prometheus 监控指标
|
||||
- [ ] 支持多链配置
|
||||
- [ ] 添加数据导出功能
|
||||
|
||||
## 📞 联系方式
|
||||
|
||||
如有问题请联系开发团队或查看项目文档。
|
||||
|
||||
---
|
||||
|
||||
**最后更新**: 2024-02-13
|
||||
Reference in New Issue
Block a user