360 Secure SkillHub
360 安全技能中心

surreal-sync

这是一款CLI工具，核心功能是将MongoDB、MySQL、PostgreSQL等7类数据源的数据同步至SurrealDB，支持全量迁移和基于CDC的增量同步。具备自动Schema推断、表创建、关系提取、断点续传、Kafka去重等特性。典型场景包括跨库迁移、实时同步、JSONL批量导入、Kafka数据流写入，通过CLI命令输入参数完成同步。

24601
24601
热度 70.50
最近更新 2026-05-11 17:26:52
分类：自动化依赖：代码执行应用场景：数据处理 / 自动化脚本 / DevOps / 工作流

安装技能

复制下面的话术发给你的 Agent，它会通过 secure-skills 检索并安装该技能。

执行命令安装此 skill： npx secure-skills-cli add surreal-sync
Metadata
name
surreal-sync
description
Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection.
license
MIT
metadata
{"version":"1.0.4","author":"24601","parent_skill":"surrealdb","snapshot_date":"2026-02-19","upstream":{"repo":"surrealdb/surreal-sync","release":"v0.3.4","sha":"8166b2b041b1"}}

Surreal-Sync -- Data Migration and Synchronization

Surreal-Sync is a CLI tool for migrating data from various database sources to SurrealDB with full and incremental synchronization via Change Data Capture (CDC).

Supported Sources

Source Full Sync Incremental CDC Method
MongoDB Yes Yes Change streams
MySQL Yes Yes Trigger-based CDC + sequence checkpoints
PostgreSQL (triggers) Yes Yes Trigger-based CDC + sequence checkpoints
PostgreSQL (wal2json) Yes Yes Logical replication with wal2json plugin
Neo4j Yes Yes Timestamp-based tracking
JSONL Files Yes N/A Batch import from JSON Lines
Apache Kafka Yes Yes Consumer subscriptions with deduplication

Quick Start

# Install surreal-sync (Rust binary)
cargo install surreal-sync

# Full sync from PostgreSQL (trigger-based)
surreal-sync from postgres trigger-full \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Incremental CDC from PostgreSQL (wal2json)
surreal-sync from postgres wal2json \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Full sync from MongoDB
surreal-sync from mongo full \
  --connection-string "mongodb://localhost:27017/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Batch import from JSONL
surreal-sync from jsonl import \
  --file data.jsonl \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Consume from Kafka
surreal-sync from kafka consume \
  --bootstrap-servers "localhost:9092" \
  --topic my-events \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

CLI Pattern

surreal-sync from <SOURCE> <COMMAND> \
  --connection-string [CONNECTION STRING] \
  --surreal-endpoint [SURREAL ENDPOINT] \
  --surreal-username [SURREAL USERNAME] \
  --surreal-password [SURREAL PASSWORD] \
  --to-namespace <NS> \
  --to-database <DB>

Key Features

  • Automatic schema inference and SurrealDB table creation
  • Record ID mapping from source primary keys
  • Relationship extraction and graph edge creation
  • Configurable batch sizes and parallelism
  • Resumable sync with checkpoint tracking
  • Deduplication for Kafka consumers

Full Documentation

See the main skill's rule file for complete guidance: