AI模型精度评估 | AI Model Accuracy Assessment
Performs a comprehensive accuracy assessment of classification/segmentation model predictions: confusion matrix, overall accuracy (OA), per-class Precision/Recall/F1, mean Intersection over Union (mIoU), Cohen's Kappa, and local accuracy maps that reveal the spatial distribution of errors.
This skill is an offline NumPy-equivalent implementation of the model evaluation pipeline: all metrics are computed directly with NumPy, and each one has unit tests against hand-computed baselines (cell-by-cell confusion matrix, exact OA/mIoU/Kappa values, spatial localization of error blocks).
Dependencies / 依赖
pip install numpy rasterio scipy scikit-learn geopandas shapely
Usage / 使用方法
Example 1 (Synthetic Data, Offline)
python geoskill-ai-accuracy-assessment.py --bbox 116.0 39.0 117.0 40.0 --synthetic --output-dir ./out
Example 2: Synthetic Evaluation Experiment (Offline, with Known Error Blocks)
python geoskill-ai-accuracy-assessment.py --bbox 116.0 39.0 117.0 40.0 --synthetic --window 7 --output-dir ./out
Example 3: Prediction + Ground Truth Dual Rasters
python geoskill-ai-accuracy-assessment.py --input pred.tif --truth ref.tif --output-dir ./out
示例 4：单文件双波段 [pred, truth]
python geoskill-ai-accuracy-assessment.py --input pair.tif --window 9 --output-dir ./out
Output / 输出
| File | Format | Description |
|---|---|---|
accuracy_report.json |
JSON | Confusion matrix/OA/mIoU/Kappa/per-class metrics |
spatial_accuracy.tif |
GeoTIFF | Windowed local accuracy map [0,1] |
output-manifest.json |
JSON | Run manifest (inputs/outputs/QA/exit code) |
Data Source / 数据源 / Source
Prediction/truth label rasters, or --synthetic (truth stripes + systematic error blocks + randomly scattered errors).
Privacy / 隐私声明 / Privacy
- Runs offline by default;
--syntheticmode requires no network at all. - All processing is done locally; no user data is ever uploaded.
License / License
MIT
name: geoskill-ai-accuracy-assessment description: '混淆矩阵+OA/mIoU/F1计算+空间精度图，输出精度评估报告（离线numpy等价实现）'
AI模型精度评估 | AI Model Accuracy Assessment
对分类/分割模型预测做全面精度评估：混淆矩阵、总体精度 OA、逐类 Precision/Recall/F1、平均交并比 mIoU、Cohen's Kappa，以及揭示误差空间分布的局部精度图。
本 skill 是模型评测流水线的离线 numpy 等价实现：所有指标由 numpy 直接计算，每一项都有手算基准的单元测试（混淆矩阵逐格、OA/mIoU/Kappa 精确值、误差块空间定位）。
依赖
pip install numpy rasterio scipy scikit-learn geopandas shapely
使用方法
示例 1（合成数据，离线）
python geoskill-ai-accuracy-assessment.py --bbox 116.0 39.0 117.0 40.0 --synthetic --output-dir ./out
示例 2：合成评估实验（离线，含已知误差块）
python geoskill-ai-accuracy-assessment.py --bbox 116.0 39.0 117.0 40.0 --synthetic --window 7 --output-dir ./out
示例 3：预测 + 真值双栅格
python geoskill-ai-accuracy-assessment.py --input pred.tif --truth ref.tif --output-dir ./out
示例 4：单文件双波段 [pred, truth]
python geoskill-ai-accuracy-assessment.py --input pair.tif --window 9 --output-dir ./out
输出
| 文件 | 格式 | 说明 |
|---|---|---|
accuracy_report.json |
JSON | 混淆矩阵/OA/mIoU/Kappa/逐类指标 |
spatial_accuracy.tif |
GeoTIFF | 窗口局部精度图 [0,1] |
output-manifest.json |
JSON | 运行清单（输入/输出/QA/退出码） |
数据源 / Source
预测/真值标签栅格，或 --synthetic（真值条带 + 系统性误差块 + 随机散布误差）。
隐私声明 / Privacy
- 默认离线运行，
--synthetic模式完全无网络。 - 所有处理在本地完成，不上传任何用户数据。
License
MIT