# 两项核心改进 — TOC 分页规则 · 搜索健康分 > 日期:2026-06-22 > 范围:仅移动端 Dart 侧(`sakura_mobile/lib/src/core/`) > 涉及文件: > > - `legado_models.dart` — 数据模型 > - `legado_rule_parser.dart` — 规则解析器 > - `legado_engine.dart` — 搜索引擎入口 > 原则:不改 Python 插件,不改 UI 层,只动 core 层。 --- ## 一、改进 1:TOC 章节目录分页规则完整支持 ### 1.1 问题 `fetchToc()` 只拉第一页目录。很多书源第一页只给 50~100 章,剩下的靠 **分页目录规则** 指向下一页。不支持分页规则 → 用户看到的目录永远不全。 ### 1.2 兼容的字段名 书源 JSON 中,下一页 URL 的字段名可能是以下任意一种,按优先级遍历取第一个非空值: ```dart // legado_models.dart — BookSource / TocRule 中新增字段解析 static const List _tocNextPageCandidates = [ 'tocNextPageRule', // Legado 标准名 'nextTocUrl', // 常见别名 'nextUrl', // 通用别名 'nextPage', // 页码式 'tocNextPage', // 变体 ]; String? get tocNextPageRule { for (final name in _tocNextPageCandidates) { final v = rawJson[name]; if (v is String && v.trim().isNotEmpty) return v.trim(); } return null; } ``` ### 1.3 分页规则语法全集 | 规则类型 | 语法示例 | 说明 | | --------- | ----------------------------------- | ------------------ | | JS 表达式 | `nextUrl` | 执行 JS 获取下一页 URL | | JS 缩写 | `@js:nextUrl` | 同上,简写形式 | | 模板拼接 | `https://host/list?page={{page}}` | `{{page}}` 从 1 自增 | | 偏移模板 | `{{page+1}}` / `{{page-1}}` | 页号偏移 | | CSS 选择器 | `text.下一页@href` | 从当前页 HTML 提取下一页链接 | | class 选择器 | `class.next@href` | 同上 | | id 选择器 | `id.pagenext@href` | 同上 | | 属性选择器 | `@href` | 取当前元素 href | | XPath | `//a[contains(text(),'下一页')]/@href` | XPath 提取 | | 字符串切片 | `!0:-1` | 对当前 URL 切片(去掉末尾字符) | | offset 翻页 | `offset=50` | 按 offset 递增翻页 | | 纯文本 URL | `https://...` | 直接跳到指定 URL | | 相对路径 | `/list/2.html` | 基于当前 URL 拼接 | ### 1.4 实现 — `TocPaginationResolver` ```dart // 放在 legado_engine.dart 或新建 legado_toc_pagination.dart class TocPaginationResolver { final LegadoRuleExecutor _ruleExecutor; final HttpClient _client; TocPaginationResolver(this._ruleExecutor, this._client); /// 拉取全部分页章节,合并去重。 Future> fetchAllChapters({ required String firstPageUrl, required String? tocNextPageRule, required String tocRule, required Map? headers, int maxPages = 20, }) async { final all = []; final seenPageUrls = {}; final seenChapterUrls = {}; String? url = firstPageUrl; int page = 0; while (url != null && page < maxPages) { if (!seenPageUrls.add(url)) break; // 死循环保护 final html = await _client.get(url, headers: headers); final batch = _ruleExecutor.selectList(html, url, rule: tocRule); int added = 0; for (final ch in batch) { if (seenChapterUrls.add(ch.url)) { all.add(ch); added++; } } if (added == 0) break; // 本页无新章节,终止翻页 url = _resolveNextPage( currentUrl: url, currentHtml: html, rule: tocNextPageRule, pageIndex: page, ); page++; } return all; } // ---------- 下一页 URL 解析 ---------- String? _resolveNextPage({ required String currentUrl, required String currentHtml, required String? rule, required int pageIndex, }) { if (rule == null || rule.trim().isEmpty) return null; final r = rule.trim(); // 1) JS: ... 或 @js:... if (r.startsWith('') || r.startsWith('@js:')) { final code = r .replaceFirst(RegExp(r'^'), '') .replaceFirst(RegExp(r'$'), '') .replaceFirst(RegExp(r'^@js:'), ''); return _ruleExecutor.evalJs(code, context: { 'result': currentHtml, 'baseUrl': currentUrl, 'page': pageIndex, }); } // 2) 切片: !start:end if (r.startsWith('!')) { return _applySlice(currentUrl, r.substring(1)); } // 3) CSS 选择器 / XPath if (_isSelector(r)) { return _ruleExecutor.select(currentHtml, currentUrl, rule: r); } // 4) 模板替换 {{page}} / {{page+1}} / {{page-1}} if (r.contains('{{page')) { return _renderPageTemplate(r, pageIndex); } // 5) offset / start 参数 if (RegExp(r'[?&](offset|start)=').hasMatch(r)) { return _renderOffsetTemplate(r, pageIndex); } // 6) 绝对 URL if (r.startsWith('http://') || r.startsWith('https://')) { return _renderPageTemplate(r, pageIndex); } // 7) 相对路径 return Uri.parse(currentUrl).resolve(r).toString(); } // ---------- 规则判定 ---------- bool _isSelector(String s) { return s.startsWith('text.') || s.startsWith('class.') || s.startsWith('id.') || s.startsWith('@') || s.startsWith('//'); // XPath } // ---------- 模板渲染 ---------- String _renderPageTemplate(String tpl, int page) { return tpl .replaceAll('{{page+1}}', '${page + 1}') .replaceAll('{{page-1}}', '${page - 1}') .replaceAll('{{page}}', '$page'); } String _renderOffsetTemplate(String tpl, int page) { return tpl .replaceAllMapped( RegExp(r'offset=(\d+)'), (m) => 'offset=${int.parse(m.group(1)!) * page}', ) .replaceAllMapped( RegExp(r'start=(\d+)'), (m) => 'start=${int.parse(m.group(1)!) * page}', ); } String? _applySlice(String url, String expr) { final parts = expr.split(':'); if (parts.length != 2) return null; final a = int.tryParse(parts[0]) ?? 0; final b = int.tryParse(parts[1]); final start = a < 0 ? url.length + a : a; if (b != null) { final end = b < 0 ? url.length + b : b; return url.substring(start, end); } return url.substring(start); } } ``` ### 1.5 集成到 `LegadoEngine.fetchToc` ```dart // legado_engine.dart — 修改 fetchToc Future> fetchToc(BookSource source, String tocUrl) async { final rule = source.tocRule; if (rule == null || rule.isEmpty) return []; final resolver = TocPaginationResolver(_ruleExecutor, _client); return resolver.fetchAllChapters( firstPageUrl: tocUrl, tocNextPageRule: source.tocNextPageRule, // ← 新增 tocRule: rule, headers: _buildHeaders(source), maxPages: 20, ); } ``` ### 1.6 死循环防护(三层) 1. **`maxPages = 20`** — 单书最多翻 20 页目录 2. **`seenPageUrls`** — 同一 URL 不重复请求 3. **`added == 0` 终止** — 当前页没有新增章节则停止 --- ## 二、改进 2:搜索源健康分 + 短超时 + 分阶段返回 ### 2.1 问题 当前 `search()` 对所有启用源并发请求,每个源 timeout 20~30 秒。10 个源一批,坏源多时整批拖到 20+ 秒才出结果。 ### 2.2 源健康分模型 ```dart // legado_models.dart — 新增 SourceHealth class SourceHealth { final String sourceId; int successCount = 0; int failCount = 0; int timeoutCount = 0; int totalLatencyMs = 0; DateTime? lastSuccessAt; DateTime? lastFailAt; int _consecutiveFails = 0; SourceHealth({required this.sourceId}); // ---- 判定 ---- /// 连续失败 ≥ 5 次 → 临时禁用 bool get isUsable { if (_consecutiveFails >= 5) return false; if (failCount >= 5 && successCount == 0) return false; return true; } int get consecutiveFails => _consecutiveFails; /// 健康分 0~100 double get score { final total = successCount + failCount + timeoutCount; if (total == 0) return 50.0; // 新源默认 50 double s = 0.0; // 成功率 60% s += 60.0 * (successCount / total); // 平均延迟 25% if (successCount > 0 && totalLatencyMs > 0) { final avg = totalLatencyMs / successCount; if (avg < 500) { s += 25.0; } else if (avg < 2000) { s += 25.0 * (1.0 - (avg - 500) / 1500); } } // 新鲜度 15%(24h 内有成功记录) if (lastSuccessAt != null) { final h = DateTime.now().difference(lastSuccessAt!).inHours; if (h < 1) { s += 15.0; } else if (h < 24) { s += 15.0 * (1.0 - h / 24); } } return s.clamp(0.0, 100.0); } // ---- 记录 ---- void recordSuccess(int latencyMs) { successCount++; totalLatencyMs += latencyMs; lastSuccessAt = DateTime.now(); _consecutiveFails = 0; } void recordFailure() { failCount++; lastFailAt = DateTime.now(); _consecutiveFails++; } void recordTimeout() { timeoutCount++; lastFailAt = DateTime.now(); _consecutiveFails++; } void resetConsecutiveFails() => _consecutiveFails = 0; // ---- 序列化(持久化到 SharedPreferences)---- Map toJson() => { 'sourceId': sourceId, 'successCount': successCount, 'failCount': failCount, 'timeoutCount': timeoutCount, 'totalLatencyMs': totalLatencyMs, 'lastSuccessAt': lastSuccessAt?.toIso8601String(), 'lastFailAt': lastFailAt?.toIso8601String(), 'consecutiveFails': _consecutiveFails, }; factory SourceHealth.fromJson(Map json) { final h = SourceHealth(sourceId: json['sourceId']); h.successCount = json['successCount'] ?? 0; h.failCount = json['failCount'] ?? 0; h.timeoutCount = json['timeoutCount'] ?? 0; h.totalLatencyMs = json['totalLatencyMs'] ?? 0; h.lastSuccessAt = json['lastSuccessAt'] != null ? DateTime.tryParse(json['lastSuccessAt']) : null; h.lastFailAt = json['lastFailAt'] != null ? DateTime.tryParse(json['lastFailAt']) : null; h._consecutiveFails = json['consecutiveFails'] ?? 0; return h; } } ``` ### 2.3 健康分持久化 ```dart // legado_engine.dart — SourceHealthStore class SourceHealthStore { static const _key = 'source_health_v1'; final Map _data = {}; Future load() async { final prefs = await SharedPreferences.getInstance(); final raw = prefs.getStringList(_key); if (raw == null) return; for (final s in raw) { try { final h = SourceHealth.fromJson(jsonDecode(s)); _data[h.sourceId] = h; } catch (_) {} } } Future save() async { final prefs = await SharedPreferences.getInstance(); final raw = _data.values.map((h) => jsonEncode(h.toJson())).toList(); await prefs.setStringList(_key, raw); } bool isUsable(String id) => _data[id]?.isUsable ?? true; double score(String id) => _data[id]?.score ?? 50.0; void _ensure(String id) { _data.putIfAbsent(id, () => SourceHealth(sourceId: id)); } void recordSuccess(String id, int latencyMs) { _ensure(id); _data[id]!.recordSuccess(latencyMs); } void recordFailure(String id) { _ensure(id); _data[id]!.recordFailure(); } void recordTimeout(String id) { _ensure(id); _data[id]!.recordTimeout(); } /// 用户手动重置所有源健康状态 void resetAll() { for (final h in _data.values) { h.resetConsecutiveFails(); } } } ``` ### 2.4 分阶段搜索调度器 ```dart // legado_engine.dart — SearchScheduler class SearchScheduler { static const fastTimeout = Duration(seconds: 5); static const slowTimeout = Duration(seconds: 8); static const maxConcurrent = 6; final SourceHealthStore _health; final LegadoEngine _engine; SearchScheduler(this._health, this._engine); /// 返回 Stream,UI 层逐阶段渲染结果 Stream search(String keyword, List allSources) async* { // 过滤 + 按健康分排序 final usable = allSources .where((s) => _health.isUsable(s.id)) .toList() ..sort((a, b) => _health.score(b.id).compareTo(_health.score(a.id))); if (usable.isEmpty) { yield SearchPhase(name: 'error', message: '没有可用的搜索源'); return; } // 分三档 final fast = usable.where((s) => _health.score(s.id) >= 70).toList(); final normal = usable.where((s) { final sc = _health.score(s.id); return sc >= 40 && sc < 70; }).toList(); final slow = usable.where((s) => _health.score(s.id) < 40).toList(); final results = []; final doneIds = {}; final seen = {}; // ---- 阶段 1:快速源,超时 5s ---- yield SearchPhase(name: 'searching', message: '正在搜索 ${usable.length} 个源...'); final r1 = await _batch(fast, keyword, timeout: fastTimeout, onDone: doneIds.add); _merge(results, r1, seen); yield SearchPhase( name: 'partial', items: List.of(results), doneCount: doneIds.length, totalCount: usable.length, ); // ---- 阶段 2:普通源,超时 8s ---- final r2 = await _batch(normal, keyword, timeout: slowTimeout, onDone: doneIds.add); _merge(results, r2, seen); yield SearchPhase( name: 'partial', items: List.of(results), doneCount: doneIds.length, totalCount: usable.length, ); // ---- 阶段 3:慢源后台继续,不阻塞 ---- if (slow.isNotEmpty) { // 放入后台队列,不 await _batch(slow, keyword, timeout: slowTimeout, onDone: doneIds.add) .then((r3) { _merge(results, r3, seen); // TODO: EventBus 通知 UI 追加 }); yield SearchPhase( name: 'complete_with_pending', items: List.of(results), doneCount: doneIds.length, totalCount: usable.length, message: '${slow.length} 个慢源将在后台继续搜索', ); } else { yield SearchPhase( name: 'complete', items: List.of(results), doneCount: doneIds.length, totalCount: usable.length, ); } } // ---- 内部分批并发 ---- Future> _batch( List sources, String keyword, { required Duration timeout, required void Function(String) onDone, }) async { final out = []; for (var i = 0; i < sources.length; i += maxConcurrent) { final chunk = sources.skip(i).take(maxConcurrent).toList(); final futures = chunk.map((s) => _searchOne(s, keyword, timeout)); final results = await Future.wait(futures); for (var j = 0; j < results.length; j++) { if (results[j] != null) out.addAll(results[j]!); onDone(chunk[j].id); } } return out; } Future?> _searchOne( BookSource source, String keyword, Duration timeout, ) async { final sw = Stopwatch()..start(); try { final items = await _engine.search(source, keyword).timeout(timeout); sw.stop(); _health.recordSuccess(source.id, sw.elapsedMilliseconds); return items; } on TimeoutException { _health.recordTimeout(source.id); return null; } catch (_) { _health.recordFailure(source.id); return null; } } void _merge(List all, List incoming, Set seen) { for (final item in incoming) { if (seen.add(item.id)) all.add(item); } } } // ---- 阶段结果 ---- class SearchPhase { final String name; // searching | partial | complete | complete_with_pending | error final List items; final int doneCount; final int totalCount; final String? message; SearchPhase({ required this.name, this.items = const [], this.doneCount = 0, this.totalCount = 0, this.message, }); } ``` ### 2.5 UI 增量渲染时序 ``` t=0s → 显示 "正在搜索 N 个源..." t=0~5s → 第 1 批结果到达 → 首屏渲染(最快的 5~10 条) t=5~8s → 第 2 批结果到达 → 列表追加 t=8s → 停止等待 → 提示"慢源已跳过,后台补充中" t=8~15s → 第 3 批结果到达 → 静默追加到列表末尾 ``` ### 2.6 健康分自动衰减 每次 `save()` 时做一次衰减: ```dart // SourceHealthStore.save() 追加衰减逻辑 Future save() async { // 超过 7 天无任何记录 → 衰减 successCount 和 failCount 各减半 for (final h in _data.values) { final latest = h.lastSuccessAt ?? h.lastFailAt; if (latest != null && DateTime.now().difference(latest).inDays >= 7) { h.successCount = (h.successCount * 0.5).round(); h.failCount = (h.failCount * 0.5).round(); h.timeoutCount = (h.timeoutCount * 0.5).round(); } } // ... 写入 SharedPreferences } ``` --- ## 三、两改进的协作关系 ``` 用户搜索一本书 │ ├─ SearchScheduler.search(keyword) │ ├─ 按健康分排序源 │ ├─ 快速源先搜(5s 超时) │ ├─ 普通源补搜(8s 超时) │ └─ 返回结果列表 │ ├─ 用户点击某条结果 → fetchBookInfo → fetchToc │ └─ TocPaginationResolver 拉全章节(支持分页规则) │ └─ 用户阅读某章 → fetchChapterContent(按需加载) ``` 改进 1 保证"搜到书后章节不断尾",改进 2 保证"搜索本身不卡"。 --- ## 四、实施顺序 ``` 第 1 步:改进 2 的 SourceHealth + SourceHealthStore → 只加数据模型 + 持久化,不影响现有搜索流程 → 可以先跑几天积累健康数据 第 2 步:改进 2 的 SearchScheduler + 短超时 → 替换现有 search() 的并发逻辑 → UI 层适配 Stream 第 3 步:改进 1 的 TocPaginationResolver → 依赖 RuleParser 链式语法的基础支持 → 见《Legado 规则引擎重构方案》第 1~3 步 第 4 步:联调 + 回归 ``` --- ## 五、风险控制 | 风险 | 应对 | | -------------------- | --------------------------------------------- | | TOC 分页死循环 | `maxPages=20` + `seenUrls` 去重 + `added==0` 终止 | | 健康分误杀快源 | 只降权不禁用;连续失败 ≥5 才标记不可用 + 手动可重置 | | 短超时漏结果 | 慢源后台继续,event bus 异步追加,不丢数据 | | 新源全部 50 分 | 第一轮搜索后就有真实分,冷启动无影响 | | SharedPreferences 过大 | 只存 String list,每个源约 200 字节,100 个源 ≈ 20KB |