Files
SakuraHub/docs/书源在线搜索-问题分析与改进方案.md
T
2026-11-16 20:45:22 +08:00

1225 lines
43 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 书源在线搜索 — 问题分析与改进方案
> 分析日期:2026-06-18
> 涉及代码:
> - `plugins/biquge/plugin.py` — 桌面端笔趣阁插件(Python)
> - `pixiv_app/gui/app.py` — GUI 调用层(Python)
> - `sakura_mobile/lib/src/core/legado_engine.dart` — 移动端搜索引擎(Dart)
> - `sakura_mobile/lib/src/core/legado_rule_parser.dart` — 规则解析器(Dart)
> - `sakura_mobile/lib/src/core/legado_models.dart` — 数据模型(Dart)
---
## 一、架构总览
```
用户输入 -> GUI(app.py) -> PluginManager -> BiqugePlugin.resolve_target()
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
_search_books_api() _reader_sort_books() _resolve_book_id()
(apibi.cc API 搜索) (榜单/分类列表) (书籍详情+章节)
│ │
▼ ▼
搜索结果列表 ──> BookSelectionDialog ──> 下载所有章节
(书源在线搜索) (GUI 弹窗选书)
```
---
## 二、桌面端(Python)问题分析
### 问题 1:API 搜索静默失败(最严重)
**位置**:`plugins/biquge/plugin.py:387-395`
```python
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except Exception:
return [] # ← 静默吞掉所有异常,无法定位问题
...
```
**原因**:`except Exception: return []` 吞掉了所有异常,包括:
- 网络不通/DNS 解析失败
- API 返回非 JSON 内容
- HTTP 错误状态码
- Token 加密失败(间接影响)
**后果**:搜索默默返回空列表,用户看到"没有搜索到相关书籍",但完全不知道是网络问题还是 API 挂了还是参数不对。
---
### 问题 2:API Token 加密密钥硬编码
**位置**:`plugins/biquge/plugin.py:61`
```python
TOKEN_SECRET = os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
TOKEN_MD5 = hashlib.md5(TOKEN_SECRET.encode("utf-8")).hexdigest()
TOKEN_IV = TOKEN_MD5[:16].encode("utf-8")
TOKEN_KEY = TOKEN_MD5[16:].encode("utf-8")
```
**原因**:`_encode_token()` 使用 AES-128-CBC 加密请求参数,密钥/IV 派生自硬编码字符串。如果 `apibi.cc` 服务端使用了不同的 secret,所有需要 token 的 API(book 详情、booklist 目录、chapter 章节内容)都会失败。
**影响范围**:
- `/api/search` — 不需要 token,不受影响
- `/api/book` — 需要 token,会失败
- `/api/booklist` — 需要 token,会失败
- `/api/chapter` — 需要 token,会失败
---
### 问题 3:章节 ID 错位问题(关键 Bug)
**位置**:`plugins/biquge/plugin.py:599-603`
```python
def _chapter_resources_from_book_data(self, book_data):
chapter_list = list(book_data.get("_chapter_list") or [])
for order, raw_title in enumerate(chapter_list, start=1):
chapter_id = str(order) # ← 使用 1,2,3... 而不是 API 返回的真实 ID
...
```
**原因**:API `/api/booklist` 返回的 `list` 可能是一个包含对象数组(每个对象有 `id` 和 `name`),而不是纯字符串数组。当前代码将其视为字符串列表,并用序号 `1,2,3...` 作为 `chapter_id` 去请求章节内容。
如果 API 的章节 ID 不是从 1 开始的连续数字(例如有些书源的章节 ID 是随机数或哈希值),那么 `_chapter_data_from_api(book_id, "1")` 会请求到错误的章节或失败。
**原始 API 返回可能是**:
```json
{
"list": [
{"id": 58291, "name": "第一章 楔子"},
{"id": 58292, "name": "第二章 测试"}
]
}
```
**但代码把它当成**:
```json
{
"list": ["第一章 楔子", "第二章 测试"]
}
```
---
### 问题 4:Legacy HTML 回退可能失效
**位置**:`plugins/biquge/plugin.py:368-385`
```python
def _search_books(self, keyword: str, *, limit: int) -> list[BookCandidate]:
books = self._search_books_api(keyword, limit=limit)
if books:
return books
# Legacy HTML fallback
attempts = [
{"url": f"{LEGACY_BASE_URL}/s", "params": {"q": keyword}},
{"url": f"{LEGACY_BASE_URL}/search", "params": {"q": keyword}},
]
```
**原因**:
- `LEGACY_BASE_URL = "https://www.bqgui.cc"` 这个域名可能已经失效或改版
- HTML 结构可能已变化,导致 `_BOOK_LINK_RE` 正则无法匹配
- 没有处理 GBK/GB2312 编码(很多中文小说站使用)
---
### 问题 5:缺少 HTTP 重试机制
**位置**:`plugins/biquge/plugin.py:1096-1117`
```python
def _api_get_json(self, endpoint, *, params=None, payload=None, ...):
url = host.rstrip("/") + endpoint
response = self.session.get(url, params=request_params, timeout=25, ...)
response.raise_for_status()
return response.json()
```
**原因**:API 调用没有重试逻辑。网络波动、临时限流(429)、服务器短暂不可用(502/503)都会直接导致失败。
---
### 问题 6:`requests.Session` 线程不安全
**位置**:`plugins/biquge/plugin.py:136-137`
```python
def __init__(self) -> None:
self.session = requests.Session()
```
**原因**:`requests.Session` 不是线程安全的。当 GUI 多线程下载章节时(`app.py` 中的 `run_download_biquge_resources_thread` 虽然是单线程循环下载,但 `sync_plugin_paths` 里可能涉及其他线程操作),共享的 session 可能导致连接池混乱或 Cookie 串扰。
---
### 问题 7:缺少编码检测
**位置**:`plugins/biquge/plugin.py:361-366`
```python
def _fetch_html(self, url, *, params=None):
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
response.encoding = response.apparent_encoding or "utf-8"
return response.text
```
**原因**:只处理了 `iso-8859-1` 和 `ascii`,没有显式处理 `gbk`/`gb2312`/`gb18030`。`apparent_encoding` 的检测也可能不准确。
---
### 问题 8:`resolve_target` 路由歧义
**位置**:`plugins/biquge/plugin.py:159-218`
当用户输入一个纯数字(如 `"12345"`)时:
1. `_normalize_target("12345")` → 返回 `""`(不是 URL)
2. `raw.isdigit()` → `True`
3. 直接调用 `_resolve_book_id("12345")` 将其当作书籍 ID
但如果用户想搜索书名为 "12345" 的书,这个逻辑就错了——它直接跳到书籍详情而不是搜索。
---
## 三、移动端(Dart/Flutter)问题分析
### 问题 9:并发搜索无去重
**位置**:`sakura_mobile/lib/src/core/legado_engine.dart:91-111`
```dart
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)...),
);
return responses.expand((items) => items).toList();
}
```
**原因**:如果 10 个不同书源(如多个笔趣阁镜像站)返回了同一本书,结果中会出现 10 条重复。没有基于书名+作者的跨源去重逻辑。
---
### 问题 10:GBK 编码依赖平台通道
**位置**:`sakura_mobile/lib/src/core/legado_engine.dart:329-361`
```dart
Future<String> _encodeKeyword(LegadoSource source, String keyword) async {
...
if (encoding == 'gbk' || encoding == 'gb2312' || ...) {
final bytes = gbk.encode(keyword); // ← 纯 Dart 实现,OK
return bytes.map(...).join();
}
// 其他编码需要平台通道
final encoded = await _encodingChannel.invokeMethod<String>(
'encodeQuery', {'text': keyword, 'charset': encoding},
);
...
}
```
**原因**:GBK 有 `gbk_codec` 包支持,但其他非 UTF-8 编码(如 Big5、EUC-JP、Shift-JIS)需要 Android/iOS 原生平台通道。在桌面端(Windows/macOS/Linux)会直接抛出 `MissingPluginException`。
---
### 问题 11:JSON 解析不够健壮
**位置**:`sakura_mobile/lib/src/core/legado_rule_parser.dart:85-93`
```dart
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
_decodedJson = jsonDecode(body);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
```
**原因**:只检查了 `FormatException`,但有些 API 返回的是带 BOM 头的 JSON、JSONP 回调包裹的数据、或 HTML 中内嵌的 JSON。这些情况都无法正确解析。
---
### 问题 12:回退搜索质量差
**位置**:`sakura_mobile/lib/src/core/legado_engine.dart:236-276`
```dart
List<UnifiedSearchResult> _fallbackSearchResults(...) {
...
for (final match in links) {
if (normalizedKeyword.isNotEmpty &&
!title.toLowerCase().contains(normalizedKeyword)) {
continue; // ← 要求标题包含搜索词,太严格
}
...
}
return results;
}
```
**原因**:回退逻辑要求 HTML 中每个 `<a>` 标签的文字必须包含搜索关键词。如果小说站点用表格或列表展示结果(没有用 `<a>` 标签包裹标题),就会漏掉结果。
---
## 四、改进方案
### 方案 A:桌面端 Python —— 增强健壮性和可调试性
#### A1. 添加结构化日志,不再静默吞异常
```python
import logging
logger = logging.getLogger(__name__)
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except requests.ConnectionError as exc:
logger.warning("API search connection failed for '%s': %s", keyword, exc)
return []
except requests.Timeout as exc:
logger.warning("API search timeout for '%s': %s", keyword, exc)
return []
except requests.HTTPError as exc:
logger.warning(
"API search HTTP %s for '%s': %s",
exc.response.status_code if exc.response else "?",
keyword,
exc,
)
return []
except Exception as exc:
logger.error("API search unexpected error for '%s': %s", keyword, exc, exc_info=True)
return []
...
```
#### A2. 支持 API Token Secret 可配置
```python
TOKEN_SECRET = os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
# 在 BiqugePlugin 中增加配置项,允许从 GUI/配置文件传入
def get_config_schema(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"token_secret": {
"type": "string",
"title": "API Token 密钥",
"description": "apibi.cc 的 AES 加密密钥,默认使用内置值",
},
"api_base_url": {
"type": "string",
"title": "API 基础地址",
"description": "apibi.cc 或自定义 API 地址",
},
},
}
def load_config(self) -> dict[str, Any]:
return {
"token_secret": os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html"),
"api_base_url": API_BASE_URL,
}
```
#### A3. 修复章节 ID 错位
从 API 返回中提取真实的章节 ID:
```python
def _chapter_resources_from_book_data(self, book_data: dict[str, Any]) -> list[Resource]:
book_id = str(book_data.get("id") or book_data.get("dirid") or "").strip()
if not book_id:
raise PluginParseError("书籍目录缺少书籍 ID。")
book_title = _normalize_space(str(book_data.get("title") or ""))
author = _normalize_space(str(book_data.get("author") or ""))
description = _normalize_space(str(book_data.get("intro") or ""))
chapter_list = list(book_data.get("_chapter_list") or [])
resources: list[Resource] = []
for order, raw_item in enumerate(chapter_list, start=1):
# === 改进:支持对象和字符串两种格式 ===
if isinstance(raw_item, dict):
chapter_title = _normalize_space(str(raw_item.get("name") or raw_item.get("title") or ""))
chapter_id = str(raw_item.get("id") or raw_item.get("chapterid") or order)
else:
chapter_title = _normalize_space(str(raw_item or ""))
chapter_id = str(order)
if not chapter_title:
continue
source_id = f"{book_id}:{chapter_id}"
resources.append(
Resource(
id=source_id,
url=self._reader_chapter_url(book_id, chapter_id),
title=chapter_title,
author=author,
author_id="",
files=[],
metadata={
"kind": "novel_chapter",
"resource_type": "novel",
"content_type": "novel",
"source_platform": "biquge",
"source_id": source_id,
"book_id": book_id,
"book_title": book_title,
"series_id": book_id,
"series_title": book_title,
"chapter_title": chapter_title,
"chapter_order": self._extract_maybe_numeric_order(chapter_title, fallback=order),
"description": description,
"last_update": str(book_data.get("lastupdate") or ""),
"book_status": str(book_data.get("full") or ""),
},
)
)
return resources
```
#### A4. 添加 API 重试机制
```python
def _api_get_json(
self,
endpoint: str,
*,
params: dict[str, Any] | None = None,
payload: dict[str, Any] | None = None,
host: str = API_BASE_URL,
referer: str = "",
max_retries: int = 3,
) -> Any:
request_params = dict(params or {})
if payload is not None:
request_params["token"] = self._encode_token(payload)
url = host.rstrip("/") + endpoint
last_exc: Exception | None = None
for attempt in range(1, max_retries + 1):
try:
response = self.session.get(
url,
params=request_params,
timeout=25,
headers=self._api_headers(referer),
)
response.raise_for_status()
return response.json()
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else 0
if status == 429 and attempt < max_retries:
retry_after = int(exc.response.headers.get("Retry-After", min(3 * attempt, 15)))
logger.warning("API rate limited, retrying in %ds (attempt %d/%d)", retry_after, attempt, max_retries)
time.sleep(retry_after)
continue
if status in (502, 503, 504) and attempt < max_retries:
logger.warning("API server error %d, retrying (attempt %d/%d)", status, attempt, max_retries)
time.sleep(min(2 ** attempt, 10))
continue
last_exc = exc
break
except (requests.ConnectionError, requests.Timeout) as exc:
if attempt < max_retries:
logger.warning("API network error, retrying (attempt %d/%d): %s", attempt, max_retries, exc)
time.sleep(min(2 ** attempt, 10))
continue
last_exc = exc
break
raise PluginParseError(f"API 请求失败 ({url}): {last_exc}") from last_exc
```
#### A5. 增强 GBK 编码支持
```python
def _fetch_html(self, url: str, *, params: dict[str, str] | None = None) -> str:
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
# === 改进:支持 GBK/GB2312/GB18030 ===
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
# 先尝试从 HTML meta 标签检测
meta_match = re.search(
r'<meta[^>]+charset=["\']?([a-zA-Z0-9_-]+)',
response.text[:2048] if response.text else "",
re.I,
)
if meta_match:
response.encoding = meta_match.group(1)
else:
response.encoding = response.apparent_encoding or "utf-8"
# 如果还是乱码,尝试 GBK 解码
try:
text = response.text
if len(text) > 0 and '�' in text[:min(500, len(text))]:
# 检测到 Unicode 替换字符,说明编码错了
response.encoding = "gb18030"
return response.text
except Exception:
pass
return response.text
```
#### A6. 线程安全的 Session 管理
```python
import threading
class BiqugePlugin(BasePlugin):
def __init__(self) -> None:
# 使用线程局部存储,每个线程独立的 Session
self._local = threading.local()
...
@property
def session(self) -> requests.Session:
if not hasattr(self._local, "session"):
self._local.session = requests.Session()
self._local.session.headers.update(self.get_headers())
return self._local.session
```
#### A7. 改进 `resolve_target` 路由歧义
```python
def resolve_target(self, target: str, *, limit: int = 20) -> dict[str, Any]:
raw = target.strip()
if not raw:
raise PluginParseError("请输入书名、作者、书籍链接或章节链接。")
normalized = self._normalize_target(raw)
if normalized:
# ... URL 路由逻辑不变
...
# === 改进:纯数字优先搜索而不是直接当 book_id ===
if raw.isdigit():
# 先尝试搜索,如果搜索无结果再当做 book_id
books = self._search_books(raw, limit=limit)
if books:
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
# 搜索无结果,尝试作为 book_id
try:
return self._resolve_book_id(raw)
except PluginParseError:
raise PluginParseError(f"没有搜索到与“{raw}”相关的书籍,也无法作为书籍 ID 解析。")
books = self._search_books(raw, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与“{raw}”相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
```
---
### 方案 B:移动端 Dart —— 增强兼容性
#### B1. 搜索结果跨源去重
```dart
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)
.timeout(perSourceTimeout)
.catchError((error) => <UnifiedSearchResult>[
UnifiedSearchResult.error(
sourceId: source.id,
sourceName: source.name,
errorMessage: _shortError(error),
),
]),
),
);
// === 改进:按书名+作者去重 ===
final seen = <String>{};
final deduped = <UnifiedSearchResult>[];
for (final item in responses.expand((items) => items)) {
if (item.isError) {
deduped.add(item);
continue;
}
final fingerprint = '${item.title}||${item.author ?? ''}'.toLowerCase();
if (seen.add(fingerprint)) {
deduped.add(item);
}
}
return deduped;
}
```
#### B2. 增强 JSON 兼容性(BOM / JSONP)
```dart
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
// 去掉 BOM 头
var text = body;
if (text.codeUnitAt(0) == 0xFEFF) {
text = text.substring(1);
}
// 尝试 JSONP 格式: callback({...})
final jsonp = RegExp(r'^[a-zA-Z_$][\w$]*\((.*)\)[;]?\s*$', dotAll: true)
.firstMatch(text.trim());
if (jsonp != null) {
text = jsonp.group(1)!;
}
_decodedJson = jsonDecode(text);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
```
#### B3. GBK 回退方案(桌面端兼容)
```dart
Future<String> _encodeKeyword(LegadoSource source, String keyword) async {
final encoding =
(source.urlEncoding ?? source.charset ?? 'utf-8').trim().toLowerCase();
if (encoding.isEmpty || encoding == 'utf-8' || encoding == 'utf8' || encoding == 'unicode') {
return Uri.encodeQueryComponent(keyword);
}
if (encoding == 'gbk' || encoding == 'gb2312' || encoding == 'gb18030' || encoding == 'gbk-0') {
final bytes = gbk.encode(keyword);
return bytes.map((b) => '%${b.toRadixString(16).padLeft(2, '0').toUpperCase()}').join();
}
try {
final encoded = await _encodingChannel.invokeMethod<String>(
'encodeQuery', {'text': keyword, 'charset': encoding},
);
if (encoded != null && encoded.isNotEmpty) return encoded;
} on MissingPluginException {
// === 改进:桌面端回退到 UTF-8 编码并提示 ===
logger.warning('编码 $encoding 在桌面端不可用,已回退到 UTF-8');
return Uri.encodeQueryComponent(keyword);
} on PlatformException catch (error) {
logger.warning('编码 $encoding 失败: ${error.message},已回退到 UTF-8');
return Uri.encodeQueryComponent(keyword);
}
return Uri.encodeQueryComponent(keyword);
}
```
#### B4. 改进回退搜索
```dart
List<UnifiedSearchResult> _fallbackSearchResults(
String body, LegadoSource source, String baseUrl, String keyword,
) {
final normalizedKeyword = keyword.trim().toLowerCase();
final seen = <String>{};
final results = <UnifiedSearchResult>[];
// === 改进 1:先尝试匹配所有链接(不要求标题包含关键词) ===
final allLinks = RegExp(
"<a\\b[^>]*\\bhref=[\"']([^\"']+)[\"'][^>]*>(.*?)</a>",
caseSensitive: false, dotAll: true,
).allMatches(body);
for (final match in allLinks) {
if (results.length >= 30) break;
final href = match.group(1)?.trim() ?? '';
if (href.isEmpty || href.startsWith('#') ||
href.toLowerCase().startsWith('javascript:')) continue;
final title = _htmlToText(match.group(2) ?? '');
if (title.isEmpty) continue;
final detailUrl = _resolveUrl(baseUrl, href);
if (detailUrl.isEmpty || !seen.add(detailUrl)) continue;
// 放宽匹配:有部分关键词匹配即可(而不是要求完全包含)
if (normalizedKeyword.isNotEmpty) {
final titleLower = title.toLowerCase();
final chars = normalizedKeyword.split('');
final matchCount = chars.where((c) => titleLower.contains(c)).length;
if (matchCount < chars.length * 0.5) continue; // 至少 50% 字符匹配
}
results.add(UnifiedSearchResult(
sourceId: source.id,
sourceName: source.name,
title: title,
detailUrl: detailUrl,
));
}
// === 改进 2:也尝试从非链接文本中提取 ===
if (results.isEmpty) {
final textBlocks = RegExp(r'<li[^>]*>(.*?)</li>', caseSensitive: false, dotAll: true)
.allMatches(body);
for (final match in textBlocks) {
if (results.length >= 10) break;
final text = _htmlToText(match.group(1) ?? '');
if (text.length > 2 && text.length < 100 && text.contains(normalizedKeyword)) {
results.add(UnifiedSearchResult(
sourceId: source.id,
sourceName: source.name,
title: text,
detailUrl: baseUrl,
));
}
}
}
return results;
}
```
---
## 五、完整改进后的核心代码段
### 5.1 改进后的 `biquge/plugin.py` 关键方法(完整版)
```python
# ============================================================
# 以下为改进后的关键代码段,可直接替换原文件对应部分
# ============================================================
import logging
import threading
import time
logger = logging.getLogger(__name__)
# ==================== 改进 1:可配置的 Token Secret ====================
class BiqugePlugin(BasePlugin):
name = "笔趣阁"
domain = "www.bqg551.cc"
version = "3.1.0"
def __init__(self) -> None:
# 线程安全的 Session 管理
self._local = threading.local()
self._playwright = None
self._browser = None
self._page = None
self._browser_lock = threading.Lock()
# 可配置项
self._config_token_secret: str = ""
self._config_api_base: str = ""
atexit.register(self._close_browser)
@property
def session(self) -> requests.Session:
"""线程局部 Session,保证线程安全"""
if not hasattr(self._local, "session"):
self._local.session = requests.Session()
self._local.session.headers.update(self.get_headers())
return self._local.session
# ==================== 改进 2:可配置的 API 参数 ====================
@property
def api_base_url(self) -> str:
return self._config_api_base or API_BASE_URL
def _resolve_token_secret(self) -> str:
"""解析当前生效的 Token Secret"""
if self._config_token_secret:
return self._config_token_secret
return os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
def _encode_token(self, payload: dict[str, Any]) -> str:
if not CRYPTOGRAPHY_AVAILABLE or padding is None or Cipher is None:
raise PluginParseError("当前环境缺少 cryptography,无法生成 apibi 访问 token。")
secret = self._resolve_token_secret()
token_md5 = hashlib.md5(secret.encode("utf-8")).hexdigest()
token_iv = token_md5[:16].encode("utf-8")
token_key = token_md5[16:].encode("utf-8")
raw = json.dumps(payload, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
padder = padding.PKCS7(128).padder()
padded = padder.update(raw) + padder.finalize()
encryptor = Cipher(algorithms.AES(token_key), modes.CBC(token_iv)).encryptor()
encrypted = encryptor.update(padded) + encryptor.finalize()
return base64.b64encode(encrypted).decode("ascii")
# ==================== 改进 3:可配置的配置项 ====================
def get_config_schema(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"token_secret": {
"type": "string",
"title": "API Token 密钥",
"description": "apibi.cc 的 AES 加密密钥(可选,默认使用内置值)",
},
"api_base_url": {
"type": "string",
"title": "API 基础地址",
"description": f"默认 {API_BASE_URL},可改为自定义 API 地址",
},
},
}
def load_config(self) -> dict[str, Any]:
return {
"token_secret": self._config_token_secret,
"api_base_url": self._config_api_base,
}
def save_config(self, data: dict[str, Any]) -> None:
self._config_token_secret = str(data.get("token_secret") or "").strip()
self._config_api_base = str(data.get("api_base_url") or "").strip()
# ==================== 改进 4:带重试和日志的 API 请求 ====================
def _api_get_json(
self,
endpoint: str,
*,
params: dict[str, Any] | None = None,
payload: dict[str, Any] | None = None,
host: str | None = None,
referer: str = "",
max_retries: int = 3,
) -> Any:
"""带重试机制的 API JSON 请求"""
request_params = dict(params or {})
if payload is not None:
request_params["token"] = self._encode_token(payload)
resolved_host = host or self.api_base_url
url = resolved_host.rstrip("/") + endpoint
last_exc: Exception | None = None
for attempt in range(1, max_retries + 1):
try:
response = self.session.get(
url,
params=request_params,
timeout=25,
headers=self._api_headers(referer),
)
response.raise_for_status()
return response.json()
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else 0
if status == 429 and attempt < max_retries:
retry_after = int(
exc.response.headers.get("Retry-After", min(3 * attempt, 15))
) if exc.response else min(3 * attempt, 15)
logger.warning(
"API %s rate limited, retrying in %ds (attempt %d/%d)",
endpoint, retry_after, attempt, max_retries,
)
time.sleep(retry_after)
continue
if status in (502, 503, 504) and attempt < max_retries:
wait = min(2 ** attempt, 10)
logger.warning(
"API %s server error %d, retrying in %ds (attempt %d/%d)",
endpoint, status, wait, attempt, max_retries,
)
time.sleep(wait)
continue
last_exc = exc
break
except (requests.ConnectionError, requests.Timeout) as exc:
if attempt < max_retries:
wait = min(2 ** attempt, 10)
logger.warning(
"API %s network error, retrying in %ds (attempt %d/%d): %s",
endpoint, wait, attempt, max_retries, exc,
)
time.sleep(wait)
continue
last_exc = exc
break
raise PluginParseError(f"API 请求失败 ({url}): {last_exc}") from last_exc
# ==================== 改进 5:带日志的搜索 ====================
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
"""搜索书籍(API 方式),失败时记录日志"""
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except requests.ConnectionError as exc:
logger.warning("API search connection failed for '%s': %s", keyword, exc)
return []
except requests.Timeout as exc:
logger.warning("API search timeout for '%s': %s", keyword, exc)
return []
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else "?"
logger.warning("API search HTTP %s for '%s'", status, keyword)
return []
except PluginParseError as exc:
logger.warning("API search failed for '%s': %s", keyword, exc)
return []
except Exception as exc:
logger.error(
"API search unexpected error for '%s': %s", keyword, exc, exc_info=True,
)
return []
rows = list(payload.get("data") or []) if isinstance(payload, dict) else []
ranked: list[tuple[int, BookCandidate]] = []
seen: set[str] = set()
keyword_parts = self._keyword_parts(keyword)
normalized_keyword = _normalize_space(keyword).lower()
for row in rows:
if not isinstance(row, dict):
continue
book_id = str(row.get("id") or row.get("dirid") or "").strip()
title = _normalize_space(str(row.get("title") or ""))
if not book_id or not title:
continue
href = self._reader_book_url(book_id)
if href in seen:
continue
seen.add(href)
author = _normalize_space(str(row.get("author") or ""))
latest = _normalize_space(str(row.get("lastchapter") or ""))
candidate = BookCandidate(
title=title, url=href, author=author, latest_chapter=latest,
)
score = self._candidate_score(candidate, keyword_parts)
title_lower = title.lower()
author_lower = author.lower()
if normalized_keyword and title_lower == normalized_keyword:
score += 1000
elif normalized_keyword and title_lower.startswith(normalized_keyword):
score += 200
elif normalized_keyword and normalized_keyword in title_lower:
score += 120
elif normalized_keyword and normalized_keyword in author_lower:
score += 60
ranked.append((score, candidate))
ranked.sort(key=lambda item: (-item[0], len(item[1].title), item[1].title))
result = [item[1] for item in ranked[:limit]]
logger.info("API search '%s' returned %d results", keyword, len(result))
return result
# ==================== 改进 6:修复章节 ID 错位 ====================
def _chapter_resources_from_book_data(
self, book_data: dict[str, Any],
) -> list[Resource]:
book_id = str(book_data.get("id") or book_data.get("dirid") or "").strip()
if not book_id:
raise PluginParseError("书籍目录缺少书籍 ID。")
book_title = _normalize_space(str(book_data.get("title") or ""))
author = _normalize_space(str(book_data.get("author") or ""))
description = _normalize_space(str(book_data.get("intro") or ""))
chapter_list = list(book_data.get("_chapter_list") or [])
resources: list[Resource] = []
for order, raw_item in enumerate(chapter_list, start=1):
# === 改进:支持 dict 和 str 两种格式 ===
if isinstance(raw_item, dict):
chapter_title = _normalize_space(
str(raw_item.get("name") or raw_item.get("title") or "")
)
chapter_id = str(
raw_item.get("id") or raw_item.get("chapterid") or order
)
else:
chapter_title = _normalize_space(str(raw_item or ""))
chapter_id = str(order)
if not chapter_title:
continue
source_id = f"{book_id}:{chapter_id}"
resources.append(
Resource(
id=source_id,
url=self._reader_chapter_url(book_id, chapter_id),
title=chapter_title,
author=author,
author_id="",
files=[],
metadata={
"kind": "novel_chapter",
"resource_type": "novel",
"content_type": "novel",
"source_platform": "biquge",
"source_id": source_id,
"book_id": book_id,
"book_title": book_title,
"series_id": book_id,
"series_title": book_title,
"chapter_title": chapter_title,
"chapter_order": self._extract_maybe_numeric_order(
chapter_title, fallback=order,
),
"description": description,
"last_update": str(book_data.get("lastupdate") or ""),
"book_status": str(book_data.get("full") or ""),
},
)
)
return resources
# ==================== 改进 7:增强编码检测 ====================
def _fetch_html(
self, url: str, *, params: dict[str, str] | None = None,
) -> str:
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
# 先尝试从 HTML meta 标签检测
try:
preview = response.content[:4096]
meta_match = re.search(
rb'<meta[^>]+charset=["\']?([a-zA-Z0-9_-]+)',
preview, re.I,
)
if meta_match:
detected = meta_match.group(1).decode("ascii", errors="replace")
response.encoding = detected
return response.text
except Exception:
pass
# 使用 apparent_encoding 兜底
response.encoding = response.apparent_encoding or "utf-8"
# 检测是否有大量乱码(Unicode 替换字符)
try:
text = response.text
sample = text[:min(1000, len(text))]
if sample.count('�') > len(sample) * 0.05:
# 超过 5% 是乱码,尝试 GB18030
response.encoding = "gb18030"
return response.text
except Exception:
pass
return response.text
# ==================== 改进 8:resolve_target 优化 ====================
def resolve_target(self, target: str, *, limit: int = 20) -> dict[str, Any]:
raw = target.strip()
if not raw:
raise PluginParseError("请输入书名、作者、书籍链接或章节链接。")
normalized = self._normalize_target(raw)
if normalized:
route = self._parse_reader_route(normalized)
if route["kind"] == "search":
keyword = route["keyword"]
books = self._search_books(keyword, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与"{keyword}"相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{keyword}",
"books": [candidate.__dict__ for candidate in books],
}
if route["kind"] == "sort":
books = self._reader_sort_books(str(route["slug"]), limit=limit)
if not books:
raise PluginParseError("该列表页没有解析到书籍。")
return {
"mode": "books",
"title": f"列表结果:{route['slug']}",
"books": [candidate.__dict__ for candidate in books],
}
if route["kind"] == "index":
books = self._reader_index_books(limit=limit)
if not books:
raise PluginParseError("首页推荐列表为空。")
return {
"mode": "books",
"title": "首页推荐",
"books": [candidate.__dict__ for candidate in books],
}
ids = self._extract_ids_from_url(normalized)
if ids.book_id and ids.chapter_id:
chapter = self._chapter_resource_from_ids(
ids.book_id, ids.chapter_id, source_url=normalized,
)
return {
"mode": "chapters",
"book_title": str(chapter.metadata.get("book_title") or ""),
"resources": [chapter],
}
if ids.book_id:
return self._resolve_book_id(ids.book_id)
return self._resolve_listing_url(normalized, limit=limit)
# === 改进:纯数字也先搜索再回退 ===
if raw.isdigit():
books = self._search_books(raw, limit=limit)
if books:
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
try:
return self._resolve_book_id(raw)
except PluginParseError:
raise PluginParseError(
f"没有搜索到与"{raw}"相关的书籍,也无法作为书籍 ID 解析。"
)
books = self._search_books(raw, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与"{raw}"相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
```
### 5.2 改进后的移动端关键代码
#### `legado_engine.dart` — searchAll 去重
```dart
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)
.timeout(perSourceTimeout)
.catchError((error) => <UnifiedSearchResult>[
UnifiedSearchResult.error(
sourceId: source.id,
sourceName: source.name,
errorMessage: _shortError(error),
),
]),
),
);
// 跨源去重:按书名+作者指纹
final seen = <String>{};
final deduped = <UnifiedSearchResult>[];
for (final item in responses.expand((items) => items)) {
if (item.isError) {
deduped.add(item);
continue;
}
final fingerprint = '${item.title}|${item.author ?? ''}'.toLowerCase();
if (seen.add(fingerprint)) {
deduped.add(item);
}
}
return deduped;
}
```
#### `legado_rule_parser.dart` — BOM/JSONP 兼容
```dart
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
var text = body;
// 去掉 UTF-8 BOM
if (text.isNotEmpty && text.codeUnitAt(0) == 0xFEFF) {
text = text.substring(1);
}
// 尝试 JSONP 格式
final jsonp = RegExp(
r'^[a-zA-Z_$][\w$]*\s*\(\s*(.*)\s*\)\s*;?\s*$',
dotAll: true,
).firstMatch(text.trim());
if (jsonp != null) {
text = jsonp.group(1)!;
}
_decodedJson = jsonDecode(text);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
```
---
## 六、总结
| # | 问题 | 严重度 | 修复方式 |
|---|------|--------|---------|
| 1 | API 搜索静默吞异常 | 🔴 高 | 分类捕获异常 + 日志记录 |
| 2 | Token Secret 硬编码 | 🔴 高 | 环境变量 + 配置文件 |
| 3 | 章节 ID 错位 | 🔴 高 | 支持 dict 格式提取真实 ID |
| 4 | Legacy HTML 失效 | 🟡 中 | 增加更多回退域名 + GBK |
| 5 | 缺少重试机制 | 🟡 中 | 指数退避重试 |
| 6 | Session 线程不安全 | 🟡 中 | `threading.local()` |
| 7 | 编码检测不完善 | 🟡 中 | HTML meta 检测 + 乱码检测 |
| 8 | 纯数字路由歧义 | 🟢 低 | 先搜索再回退 book_id |
| 9 | 跨源结果重复 | 🟡 中 | 书名+作者指纹去重 |
| 10 | GBK 桌面端不支持 | 🟡 中 | 非致命回退到 UTF-8 |
| 11 | JSONP/BOM 不兼容 | 🟢 低 | 预处理 body 文本 |
| 12 | 回退搜索太严格 | 🟢 低 | 放宽关键词匹配阈值 |
**最关键的三项修复**(建议优先实施):
1. **章节 ID 错位**(问题 3)— 直接导致下载失败,必须立即修复
2. **API 搜索静默失败**(问题 1)— 加了日志就能快速定位根因
3. **API 重试机制**(问题 5)— 解决大部分"偶发失败"问题