43 KiB
书源在线搜索 — 问题分析与改进方案
分析日期:2026-06-18
涉及代码:
plugins/biquge/plugin.py— 桌面端笔趣阁插件(Python)pixiv_app/gui/app.py— GUI 调用层(Python)sakura_mobile/lib/src/core/legado_engine.dart— 移动端搜索引擎(Dart)sakura_mobile/lib/src/core/legado_rule_parser.dart— 规则解析器(Dart)sakura_mobile/lib/src/core/legado_models.dart— 数据模型(Dart)
一、架构总览
用户输入 -> GUI(app.py) -> PluginManager -> BiqugePlugin.resolve_target()
│
┌───────────────────────┼───────────────────────┐
▼ ▼ ▼
_search_books_api() _reader_sort_books() _resolve_book_id()
(apibi.cc API 搜索) (榜单/分类列表) (书籍详情+章节)
│ │
▼ ▼
搜索结果列表 ──> BookSelectionDialog ──> 下载所有章节
(书源在线搜索) (GUI 弹窗选书)
二、桌面端(Python)问题分析
问题 1:API 搜索静默失败(最严重)
位置:plugins/biquge/plugin.py:387-395
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except Exception:
return [] # ← 静默吞掉所有异常,无法定位问题
...
原因:except Exception: return [] 吞掉了所有异常,包括:
- 网络不通/DNS 解析失败
- API 返回非 JSON 内容
- HTTP 错误状态码
- Token 加密失败(间接影响)
后果:搜索默默返回空列表,用户看到"没有搜索到相关书籍",但完全不知道是网络问题还是 API 挂了还是参数不对。
问题 2:API Token 加密密钥硬编码
位置:plugins/biquge/plugin.py:61
TOKEN_SECRET = os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
TOKEN_MD5 = hashlib.md5(TOKEN_SECRET.encode("utf-8")).hexdigest()
TOKEN_IV = TOKEN_MD5[:16].encode("utf-8")
TOKEN_KEY = TOKEN_MD5[16:].encode("utf-8")
原因:_encode_token() 使用 AES-128-CBC 加密请求参数,密钥/IV 派生自硬编码字符串。如果 apibi.cc 服务端使用了不同的 secret,所有需要 token 的 API(book 详情、booklist 目录、chapter 章节内容)都会失败。
影响范围:
/api/search— 不需要 token,不受影响/api/book— 需要 token,会失败/api/booklist— 需要 token,会失败/api/chapter— 需要 token,会失败
问题 3:章节 ID 错位问题(关键 Bug)
位置:plugins/biquge/plugin.py:599-603
def _chapter_resources_from_book_data(self, book_data):
chapter_list = list(book_data.get("_chapter_list") or [])
for order, raw_title in enumerate(chapter_list, start=1):
chapter_id = str(order) # ← 使用 1,2,3... 而不是 API 返回的真实 ID
...
原因:API /api/booklist 返回的 list 可能是一个包含对象数组(每个对象有 id 和 name),而不是纯字符串数组。当前代码将其视为字符串列表,并用序号 1,2,3... 作为 chapter_id 去请求章节内容。
如果 API 的章节 ID 不是从 1 开始的连续数字(例如有些书源的章节 ID 是随机数或哈希值),那么 _chapter_data_from_api(book_id, "1") 会请求到错误的章节或失败。
原始 API 返回可能是:
{
"list": [
{"id": 58291, "name": "第一章 楔子"},
{"id": 58292, "name": "第二章 测试"}
]
}
但代码把它当成:
{
"list": ["第一章 楔子", "第二章 测试"]
}
问题 4:Legacy HTML 回退可能失效
位置:plugins/biquge/plugin.py:368-385
def _search_books(self, keyword: str, *, limit: int) -> list[BookCandidate]:
books = self._search_books_api(keyword, limit=limit)
if books:
return books
# Legacy HTML fallback
attempts = [
{"url": f"{LEGACY_BASE_URL}/s", "params": {"q": keyword}},
{"url": f"{LEGACY_BASE_URL}/search", "params": {"q": keyword}},
]
原因:
LEGACY_BASE_URL = "https://www.bqgui.cc"这个域名可能已经失效或改版- HTML 结构可能已变化,导致
_BOOK_LINK_RE正则无法匹配 - 没有处理 GBK/GB2312 编码(很多中文小说站使用)
问题 5:缺少 HTTP 重试机制
位置:plugins/biquge/plugin.py:1096-1117
def _api_get_json(self, endpoint, *, params=None, payload=None, ...):
url = host.rstrip("/") + endpoint
response = self.session.get(url, params=request_params, timeout=25, ...)
response.raise_for_status()
return response.json()
原因:API 调用没有重试逻辑。网络波动、临时限流(429)、服务器短暂不可用(502/503)都会直接导致失败。
问题 6:requests.Session 线程不安全
位置:plugins/biquge/plugin.py:136-137
def __init__(self) -> None:
self.session = requests.Session()
原因:requests.Session 不是线程安全的。当 GUI 多线程下载章节时(app.py 中的 run_download_biquge_resources_thread 虽然是单线程循环下载,但 sync_plugin_paths 里可能涉及其他线程操作),共享的 session 可能导致连接池混乱或 Cookie 串扰。
问题 7:缺少编码检测
位置:plugins/biquge/plugin.py:361-366
def _fetch_html(self, url, *, params=None):
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
response.encoding = response.apparent_encoding or "utf-8"
return response.text
原因:只处理了 iso-8859-1 和 ascii,没有显式处理 gbk/gb2312/gb18030。apparent_encoding 的检测也可能不准确。
问题 8:resolve_target 路由歧义
位置:plugins/biquge/plugin.py:159-218
当用户输入一个纯数字(如 "12345")时:
_normalize_target("12345")→ 返回""(不是 URL)raw.isdigit()→True- 直接调用
_resolve_book_id("12345")将其当作书籍 ID
但如果用户想搜索书名为 "12345" 的书,这个逻辑就错了——它直接跳到书籍详情而不是搜索。
三、移动端(Dart/Flutter)问题分析
问题 9:并发搜索无去重
位置:sakura_mobile/lib/src/core/legado_engine.dart:91-111
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)...),
);
return responses.expand((items) => items).toList();
}
原因:如果 10 个不同书源(如多个笔趣阁镜像站)返回了同一本书,结果中会出现 10 条重复。没有基于书名+作者的跨源去重逻辑。
问题 10:GBK 编码依赖平台通道
位置:sakura_mobile/lib/src/core/legado_engine.dart:329-361
Future<String> _encodeKeyword(LegadoSource source, String keyword) async {
...
if (encoding == 'gbk' || encoding == 'gb2312' || ...) {
final bytes = gbk.encode(keyword); // ← 纯 Dart 实现,OK
return bytes.map(...).join();
}
// 其他编码需要平台通道
final encoded = await _encodingChannel.invokeMethod<String>(
'encodeQuery', {'text': keyword, 'charset': encoding},
);
...
}
原因:GBK 有 gbk_codec 包支持,但其他非 UTF-8 编码(如 Big5、EUC-JP、Shift-JIS)需要 Android/iOS 原生平台通道。在桌面端(Windows/macOS/Linux)会直接抛出 MissingPluginException。
问题 11:JSON 解析不够健壮
位置:sakura_mobile/lib/src/core/legado_rule_parser.dart:85-93
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
_decodedJson = jsonDecode(body);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
原因:只检查了 FormatException,但有些 API 返回的是带 BOM 头的 JSON、JSONP 回调包裹的数据、或 HTML 中内嵌的 JSON。这些情况都无法正确解析。
问题 12:回退搜索质量差
位置:sakura_mobile/lib/src/core/legado_engine.dart:236-276
List<UnifiedSearchResult> _fallbackSearchResults(...) {
...
for (final match in links) {
if (normalizedKeyword.isNotEmpty &&
!title.toLowerCase().contains(normalizedKeyword)) {
continue; // ← 要求标题包含搜索词,太严格
}
...
}
return results;
}
原因:回退逻辑要求 HTML 中每个 <a> 标签的文字必须包含搜索关键词。如果小说站点用表格或列表展示结果(没有用 <a> 标签包裹标题),就会漏掉结果。
四、改进方案
方案 A:桌面端 Python —— 增强健壮性和可调试性
A1. 添加结构化日志,不再静默吞异常
import logging
logger = logging.getLogger(__name__)
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except requests.ConnectionError as exc:
logger.warning("API search connection failed for '%s': %s", keyword, exc)
return []
except requests.Timeout as exc:
logger.warning("API search timeout for '%s': %s", keyword, exc)
return []
except requests.HTTPError as exc:
logger.warning(
"API search HTTP %s for '%s': %s",
exc.response.status_code if exc.response else "?",
keyword,
exc,
)
return []
except Exception as exc:
logger.error("API search unexpected error for '%s': %s", keyword, exc, exc_info=True)
return []
...
A2. 支持 API Token Secret 可配置
TOKEN_SECRET = os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
# 在 BiqugePlugin 中增加配置项,允许从 GUI/配置文件传入
def get_config_schema(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"token_secret": {
"type": "string",
"title": "API Token 密钥",
"description": "apibi.cc 的 AES 加密密钥,默认使用内置值",
},
"api_base_url": {
"type": "string",
"title": "API 基础地址",
"description": "apibi.cc 或自定义 API 地址",
},
},
}
def load_config(self) -> dict[str, Any]:
return {
"token_secret": os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html"),
"api_base_url": API_BASE_URL,
}
A3. 修复章节 ID 错位
从 API 返回中提取真实的章节 ID:
def _chapter_resources_from_book_data(self, book_data: dict[str, Any]) -> list[Resource]:
book_id = str(book_data.get("id") or book_data.get("dirid") or "").strip()
if not book_id:
raise PluginParseError("书籍目录缺少书籍 ID。")
book_title = _normalize_space(str(book_data.get("title") or ""))
author = _normalize_space(str(book_data.get("author") or ""))
description = _normalize_space(str(book_data.get("intro") or ""))
chapter_list = list(book_data.get("_chapter_list") or [])
resources: list[Resource] = []
for order, raw_item in enumerate(chapter_list, start=1):
# === 改进:支持对象和字符串两种格式 ===
if isinstance(raw_item, dict):
chapter_title = _normalize_space(str(raw_item.get("name") or raw_item.get("title") or ""))
chapter_id = str(raw_item.get("id") or raw_item.get("chapterid") or order)
else:
chapter_title = _normalize_space(str(raw_item or ""))
chapter_id = str(order)
if not chapter_title:
continue
source_id = f"{book_id}:{chapter_id}"
resources.append(
Resource(
id=source_id,
url=self._reader_chapter_url(book_id, chapter_id),
title=chapter_title,
author=author,
author_id="",
files=[],
metadata={
"kind": "novel_chapter",
"resource_type": "novel",
"content_type": "novel",
"source_platform": "biquge",
"source_id": source_id,
"book_id": book_id,
"book_title": book_title,
"series_id": book_id,
"series_title": book_title,
"chapter_title": chapter_title,
"chapter_order": self._extract_maybe_numeric_order(chapter_title, fallback=order),
"description": description,
"last_update": str(book_data.get("lastupdate") or ""),
"book_status": str(book_data.get("full") or ""),
},
)
)
return resources
A4. 添加 API 重试机制
def _api_get_json(
self,
endpoint: str,
*,
params: dict[str, Any] | None = None,
payload: dict[str, Any] | None = None,
host: str = API_BASE_URL,
referer: str = "",
max_retries: int = 3,
) -> Any:
request_params = dict(params or {})
if payload is not None:
request_params["token"] = self._encode_token(payload)
url = host.rstrip("/") + endpoint
last_exc: Exception | None = None
for attempt in range(1, max_retries + 1):
try:
response = self.session.get(
url,
params=request_params,
timeout=25,
headers=self._api_headers(referer),
)
response.raise_for_status()
return response.json()
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else 0
if status == 429 and attempt < max_retries:
retry_after = int(exc.response.headers.get("Retry-After", min(3 * attempt, 15)))
logger.warning("API rate limited, retrying in %ds (attempt %d/%d)", retry_after, attempt, max_retries)
time.sleep(retry_after)
continue
if status in (502, 503, 504) and attempt < max_retries:
logger.warning("API server error %d, retrying (attempt %d/%d)", status, attempt, max_retries)
time.sleep(min(2 ** attempt, 10))
continue
last_exc = exc
break
except (requests.ConnectionError, requests.Timeout) as exc:
if attempt < max_retries:
logger.warning("API network error, retrying (attempt %d/%d): %s", attempt, max_retries, exc)
time.sleep(min(2 ** attempt, 10))
continue
last_exc = exc
break
raise PluginParseError(f"API 请求失败 ({url}): {last_exc}") from last_exc
A5. 增强 GBK 编码支持
def _fetch_html(self, url: str, *, params: dict[str, str] | None = None) -> str:
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
# === 改进:支持 GBK/GB2312/GB18030 ===
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
# 先尝试从 HTML meta 标签检测
meta_match = re.search(
r'<meta[^>]+charset=["\']?([a-zA-Z0-9_-]+)',
response.text[:2048] if response.text else "",
re.I,
)
if meta_match:
response.encoding = meta_match.group(1)
else:
response.encoding = response.apparent_encoding or "utf-8"
# 如果还是乱码,尝试 GBK 解码
try:
text = response.text
if len(text) > 0 and '�' in text[:min(500, len(text))]:
# 检测到 Unicode 替换字符,说明编码错了
response.encoding = "gb18030"
return response.text
except Exception:
pass
return response.text
A6. 线程安全的 Session 管理
import threading
class BiqugePlugin(BasePlugin):
def __init__(self) -> None:
# 使用线程局部存储,每个线程独立的 Session
self._local = threading.local()
...
@property
def session(self) -> requests.Session:
if not hasattr(self._local, "session"):
self._local.session = requests.Session()
self._local.session.headers.update(self.get_headers())
return self._local.session
A7. 改进 resolve_target 路由歧义
def resolve_target(self, target: str, *, limit: int = 20) -> dict[str, Any]:
raw = target.strip()
if not raw:
raise PluginParseError("请输入书名、作者、书籍链接或章节链接。")
normalized = self._normalize_target(raw)
if normalized:
# ... URL 路由逻辑不变
...
# === 改进:纯数字优先搜索而不是直接当 book_id ===
if raw.isdigit():
# 先尝试搜索,如果搜索无结果再当做 book_id
books = self._search_books(raw, limit=limit)
if books:
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
# 搜索无结果,尝试作为 book_id
try:
return self._resolve_book_id(raw)
except PluginParseError:
raise PluginParseError(f"没有搜索到与“{raw}”相关的书籍,也无法作为书籍 ID 解析。")
books = self._search_books(raw, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与“{raw}”相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
方案 B:移动端 Dart —— 增强兼容性
B1. 搜索结果跨源去重
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)
.timeout(perSourceTimeout)
.catchError((error) => <UnifiedSearchResult>[
UnifiedSearchResult.error(
sourceId: source.id,
sourceName: source.name,
errorMessage: _shortError(error),
),
]),
),
);
// === 改进:按书名+作者去重 ===
final seen = <String>{};
final deduped = <UnifiedSearchResult>[];
for (final item in responses.expand((items) => items)) {
if (item.isError) {
deduped.add(item);
continue;
}
final fingerprint = '${item.title}||${item.author ?? ''}'.toLowerCase();
if (seen.add(fingerprint)) {
deduped.add(item);
}
}
return deduped;
}
B2. 增强 JSON 兼容性(BOM / JSONP)
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
// 去掉 BOM 头
var text = body;
if (text.codeUnitAt(0) == 0xFEFF) {
text = text.substring(1);
}
// 尝试 JSONP 格式: callback({...})
final jsonp = RegExp(r'^[a-zA-Z_$][\w$]*\((.*)\)[;]?\s*$', dotAll: true)
.firstMatch(text.trim());
if (jsonp != null) {
text = jsonp.group(1)!;
}
_decodedJson = jsonDecode(text);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
B3. GBK 回退方案(桌面端兼容)
Future<String> _encodeKeyword(LegadoSource source, String keyword) async {
final encoding =
(source.urlEncoding ?? source.charset ?? 'utf-8').trim().toLowerCase();
if (encoding.isEmpty || encoding == 'utf-8' || encoding == 'utf8' || encoding == 'unicode') {
return Uri.encodeQueryComponent(keyword);
}
if (encoding == 'gbk' || encoding == 'gb2312' || encoding == 'gb18030' || encoding == 'gbk-0') {
final bytes = gbk.encode(keyword);
return bytes.map((b) => '%${b.toRadixString(16).padLeft(2, '0').toUpperCase()}').join();
}
try {
final encoded = await _encodingChannel.invokeMethod<String>(
'encodeQuery', {'text': keyword, 'charset': encoding},
);
if (encoded != null && encoded.isNotEmpty) return encoded;
} on MissingPluginException {
// === 改进:桌面端回退到 UTF-8 编码并提示 ===
logger.warning('编码 $encoding 在桌面端不可用,已回退到 UTF-8');
return Uri.encodeQueryComponent(keyword);
} on PlatformException catch (error) {
logger.warning('编码 $encoding 失败: ${error.message},已回退到 UTF-8');
return Uri.encodeQueryComponent(keyword);
}
return Uri.encodeQueryComponent(keyword);
}
B4. 改进回退搜索
List<UnifiedSearchResult> _fallbackSearchResults(
String body, LegadoSource source, String baseUrl, String keyword,
) {
final normalizedKeyword = keyword.trim().toLowerCase();
final seen = <String>{};
final results = <UnifiedSearchResult>[];
// === 改进 1:先尝试匹配所有链接(不要求标题包含关键词) ===
final allLinks = RegExp(
"<a\\b[^>]*\\bhref=[\"']([^\"']+)[\"'][^>]*>(.*?)</a>",
caseSensitive: false, dotAll: true,
).allMatches(body);
for (final match in allLinks) {
if (results.length >= 30) break;
final href = match.group(1)?.trim() ?? '';
if (href.isEmpty || href.startsWith('#') ||
href.toLowerCase().startsWith('javascript:')) continue;
final title = _htmlToText(match.group(2) ?? '');
if (title.isEmpty) continue;
final detailUrl = _resolveUrl(baseUrl, href);
if (detailUrl.isEmpty || !seen.add(detailUrl)) continue;
// 放宽匹配:有部分关键词匹配即可(而不是要求完全包含)
if (normalizedKeyword.isNotEmpty) {
final titleLower = title.toLowerCase();
final chars = normalizedKeyword.split('');
final matchCount = chars.where((c) => titleLower.contains(c)).length;
if (matchCount < chars.length * 0.5) continue; // 至少 50% 字符匹配
}
results.add(UnifiedSearchResult(
sourceId: source.id,
sourceName: source.name,
title: title,
detailUrl: detailUrl,
));
}
// === 改进 2:也尝试从非链接文本中提取 ===
if (results.isEmpty) {
final textBlocks = RegExp(r'<li[^>]*>(.*?)</li>', caseSensitive: false, dotAll: true)
.allMatches(body);
for (final match in textBlocks) {
if (results.length >= 10) break;
final text = _htmlToText(match.group(1) ?? '');
if (text.length > 2 && text.length < 100 && text.contains(normalizedKeyword)) {
results.add(UnifiedSearchResult(
sourceId: source.id,
sourceName: source.name,
title: text,
detailUrl: baseUrl,
));
}
}
}
return results;
}
五、完整改进后的核心代码段
5.1 改进后的 biquge/plugin.py 关键方法(完整版)
# ============================================================
# 以下为改进后的关键代码段,可直接替换原文件对应部分
# ============================================================
import logging
import threading
import time
logger = logging.getLogger(__name__)
# ==================== 改进 1:可配置的 Token Secret ====================
class BiqugePlugin(BasePlugin):
name = "笔趣阁"
domain = "www.bqg551.cc"
version = "3.1.0"
def __init__(self) -> None:
# 线程安全的 Session 管理
self._local = threading.local()
self._playwright = None
self._browser = None
self._page = None
self._browser_lock = threading.Lock()
# 可配置项
self._config_token_secret: str = ""
self._config_api_base: str = ""
atexit.register(self._close_browser)
@property
def session(self) -> requests.Session:
"""线程局部 Session,保证线程安全"""
if not hasattr(self._local, "session"):
self._local.session = requests.Session()
self._local.session.headers.update(self.get_headers())
return self._local.session
# ==================== 改进 2:可配置的 API 参数 ====================
@property
def api_base_url(self) -> str:
return self._config_api_base or API_BASE_URL
def _resolve_token_secret(self) -> str:
"""解析当前生效的 Token Secret"""
if self._config_token_secret:
return self._config_token_secret
return os.environ.get("SAKURA_BIQUGE_TOKEN_SECRET", "book@token.html")
def _encode_token(self, payload: dict[str, Any]) -> str:
if not CRYPTOGRAPHY_AVAILABLE or padding is None or Cipher is None:
raise PluginParseError("当前环境缺少 cryptography,无法生成 apibi 访问 token。")
secret = self._resolve_token_secret()
token_md5 = hashlib.md5(secret.encode("utf-8")).hexdigest()
token_iv = token_md5[:16].encode("utf-8")
token_key = token_md5[16:].encode("utf-8")
raw = json.dumps(payload, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
padder = padding.PKCS7(128).padder()
padded = padder.update(raw) + padder.finalize()
encryptor = Cipher(algorithms.AES(token_key), modes.CBC(token_iv)).encryptor()
encrypted = encryptor.update(padded) + encryptor.finalize()
return base64.b64encode(encrypted).decode("ascii")
# ==================== 改进 3:可配置的配置项 ====================
def get_config_schema(self) -> dict[str, Any]:
return {
"type": "object",
"properties": {
"token_secret": {
"type": "string",
"title": "API Token 密钥",
"description": "apibi.cc 的 AES 加密密钥(可选,默认使用内置值)",
},
"api_base_url": {
"type": "string",
"title": "API 基础地址",
"description": f"默认 {API_BASE_URL},可改为自定义 API 地址",
},
},
}
def load_config(self) -> dict[str, Any]:
return {
"token_secret": self._config_token_secret,
"api_base_url": self._config_api_base,
}
def save_config(self, data: dict[str, Any]) -> None:
self._config_token_secret = str(data.get("token_secret") or "").strip()
self._config_api_base = str(data.get("api_base_url") or "").strip()
# ==================== 改进 4:带重试和日志的 API 请求 ====================
def _api_get_json(
self,
endpoint: str,
*,
params: dict[str, Any] | None = None,
payload: dict[str, Any] | None = None,
host: str | None = None,
referer: str = "",
max_retries: int = 3,
) -> Any:
"""带重试机制的 API JSON 请求"""
request_params = dict(params or {})
if payload is not None:
request_params["token"] = self._encode_token(payload)
resolved_host = host or self.api_base_url
url = resolved_host.rstrip("/") + endpoint
last_exc: Exception | None = None
for attempt in range(1, max_retries + 1):
try:
response = self.session.get(
url,
params=request_params,
timeout=25,
headers=self._api_headers(referer),
)
response.raise_for_status()
return response.json()
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else 0
if status == 429 and attempt < max_retries:
retry_after = int(
exc.response.headers.get("Retry-After", min(3 * attempt, 15))
) if exc.response else min(3 * attempt, 15)
logger.warning(
"API %s rate limited, retrying in %ds (attempt %d/%d)",
endpoint, retry_after, attempt, max_retries,
)
time.sleep(retry_after)
continue
if status in (502, 503, 504) and attempt < max_retries:
wait = min(2 ** attempt, 10)
logger.warning(
"API %s server error %d, retrying in %ds (attempt %d/%d)",
endpoint, status, wait, attempt, max_retries,
)
time.sleep(wait)
continue
last_exc = exc
break
except (requests.ConnectionError, requests.Timeout) as exc:
if attempt < max_retries:
wait = min(2 ** attempt, 10)
logger.warning(
"API %s network error, retrying in %ds (attempt %d/%d): %s",
endpoint, wait, attempt, max_retries, exc,
)
time.sleep(wait)
continue
last_exc = exc
break
raise PluginParseError(f"API 请求失败 ({url}): {last_exc}") from last_exc
# ==================== 改进 5:带日志的搜索 ====================
def _search_books_api(self, keyword: str, *, limit: int) -> list[BookCandidate]:
"""搜索书籍(API 方式),失败时记录日志"""
try:
payload = self._api_get_json(
"/api/search",
params={"q": keyword},
referer=f"{READER_BASE_URL}/#/search?q={quote(keyword)}",
)
except requests.ConnectionError as exc:
logger.warning("API search connection failed for '%s': %s", keyword, exc)
return []
except requests.Timeout as exc:
logger.warning("API search timeout for '%s': %s", keyword, exc)
return []
except requests.HTTPError as exc:
status = exc.response.status_code if exc.response else "?"
logger.warning("API search HTTP %s for '%s'", status, keyword)
return []
except PluginParseError as exc:
logger.warning("API search failed for '%s': %s", keyword, exc)
return []
except Exception as exc:
logger.error(
"API search unexpected error for '%s': %s", keyword, exc, exc_info=True,
)
return []
rows = list(payload.get("data") or []) if isinstance(payload, dict) else []
ranked: list[tuple[int, BookCandidate]] = []
seen: set[str] = set()
keyword_parts = self._keyword_parts(keyword)
normalized_keyword = _normalize_space(keyword).lower()
for row in rows:
if not isinstance(row, dict):
continue
book_id = str(row.get("id") or row.get("dirid") or "").strip()
title = _normalize_space(str(row.get("title") or ""))
if not book_id or not title:
continue
href = self._reader_book_url(book_id)
if href in seen:
continue
seen.add(href)
author = _normalize_space(str(row.get("author") or ""))
latest = _normalize_space(str(row.get("lastchapter") or ""))
candidate = BookCandidate(
title=title, url=href, author=author, latest_chapter=latest,
)
score = self._candidate_score(candidate, keyword_parts)
title_lower = title.lower()
author_lower = author.lower()
if normalized_keyword and title_lower == normalized_keyword:
score += 1000
elif normalized_keyword and title_lower.startswith(normalized_keyword):
score += 200
elif normalized_keyword and normalized_keyword in title_lower:
score += 120
elif normalized_keyword and normalized_keyword in author_lower:
score += 60
ranked.append((score, candidate))
ranked.sort(key=lambda item: (-item[0], len(item[1].title), item[1].title))
result = [item[1] for item in ranked[:limit]]
logger.info("API search '%s' returned %d results", keyword, len(result))
return result
# ==================== 改进 6:修复章节 ID 错位 ====================
def _chapter_resources_from_book_data(
self, book_data: dict[str, Any],
) -> list[Resource]:
book_id = str(book_data.get("id") or book_data.get("dirid") or "").strip()
if not book_id:
raise PluginParseError("书籍目录缺少书籍 ID。")
book_title = _normalize_space(str(book_data.get("title") or ""))
author = _normalize_space(str(book_data.get("author") or ""))
description = _normalize_space(str(book_data.get("intro") or ""))
chapter_list = list(book_data.get("_chapter_list") or [])
resources: list[Resource] = []
for order, raw_item in enumerate(chapter_list, start=1):
# === 改进:支持 dict 和 str 两种格式 ===
if isinstance(raw_item, dict):
chapter_title = _normalize_space(
str(raw_item.get("name") or raw_item.get("title") or "")
)
chapter_id = str(
raw_item.get("id") or raw_item.get("chapterid") or order
)
else:
chapter_title = _normalize_space(str(raw_item or ""))
chapter_id = str(order)
if not chapter_title:
continue
source_id = f"{book_id}:{chapter_id}"
resources.append(
Resource(
id=source_id,
url=self._reader_chapter_url(book_id, chapter_id),
title=chapter_title,
author=author,
author_id="",
files=[],
metadata={
"kind": "novel_chapter",
"resource_type": "novel",
"content_type": "novel",
"source_platform": "biquge",
"source_id": source_id,
"book_id": book_id,
"book_title": book_title,
"series_id": book_id,
"series_title": book_title,
"chapter_title": chapter_title,
"chapter_order": self._extract_maybe_numeric_order(
chapter_title, fallback=order,
),
"description": description,
"last_update": str(book_data.get("lastupdate") or ""),
"book_status": str(book_data.get("full") or ""),
},
)
)
return resources
# ==================== 改进 7:增强编码检测 ====================
def _fetch_html(
self, url: str, *, params: dict[str, str] | None = None,
) -> str:
response = self.session.get(url, params=params, timeout=25)
response.raise_for_status()
if not response.encoding or response.encoding.lower() in {"iso-8859-1", "ascii"}:
# 先尝试从 HTML meta 标签检测
try:
preview = response.content[:4096]
meta_match = re.search(
rb'<meta[^>]+charset=["\']?([a-zA-Z0-9_-]+)',
preview, re.I,
)
if meta_match:
detected = meta_match.group(1).decode("ascii", errors="replace")
response.encoding = detected
return response.text
except Exception:
pass
# 使用 apparent_encoding 兜底
response.encoding = response.apparent_encoding or "utf-8"
# 检测是否有大量乱码(Unicode 替换字符)
try:
text = response.text
sample = text[:min(1000, len(text))]
if sample.count('�') > len(sample) * 0.05:
# 超过 5% 是乱码,尝试 GB18030
response.encoding = "gb18030"
return response.text
except Exception:
pass
return response.text
# ==================== 改进 8:resolve_target 优化 ====================
def resolve_target(self, target: str, *, limit: int = 20) -> dict[str, Any]:
raw = target.strip()
if not raw:
raise PluginParseError("请输入书名、作者、书籍链接或章节链接。")
normalized = self._normalize_target(raw)
if normalized:
route = self._parse_reader_route(normalized)
if route["kind"] == "search":
keyword = route["keyword"]
books = self._search_books(keyword, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与"{keyword}"相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{keyword}",
"books": [candidate.__dict__ for candidate in books],
}
if route["kind"] == "sort":
books = self._reader_sort_books(str(route["slug"]), limit=limit)
if not books:
raise PluginParseError("该列表页没有解析到书籍。")
return {
"mode": "books",
"title": f"列表结果:{route['slug']}",
"books": [candidate.__dict__ for candidate in books],
}
if route["kind"] == "index":
books = self._reader_index_books(limit=limit)
if not books:
raise PluginParseError("首页推荐列表为空。")
return {
"mode": "books",
"title": "首页推荐",
"books": [candidate.__dict__ for candidate in books],
}
ids = self._extract_ids_from_url(normalized)
if ids.book_id and ids.chapter_id:
chapter = self._chapter_resource_from_ids(
ids.book_id, ids.chapter_id, source_url=normalized,
)
return {
"mode": "chapters",
"book_title": str(chapter.metadata.get("book_title") or ""),
"resources": [chapter],
}
if ids.book_id:
return self._resolve_book_id(ids.book_id)
return self._resolve_listing_url(normalized, limit=limit)
# === 改进:纯数字也先搜索再回退 ===
if raw.isdigit():
books = self._search_books(raw, limit=limit)
if books:
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
try:
return self._resolve_book_id(raw)
except PluginParseError:
raise PluginParseError(
f"没有搜索到与"{raw}"相关的书籍,也无法作为书籍 ID 解析。"
)
books = self._search_books(raw, limit=limit)
if not books:
raise PluginParseError(f"没有搜索到与"{raw}"相关的书籍。")
return {
"mode": "books",
"title": f"搜索结果:{raw}",
"books": [candidate.__dict__ for candidate in books],
}
5.2 改进后的移动端关键代码
legado_engine.dart — searchAll 去重
Future<List<UnifiedSearchResult>> searchAll(
List<LegadoSource> enabledSources,
String keyword, {int page = 1,}
) async {
final batch = enabledSources.take(maxConcurrency).toList();
final responses = await Future.wait(
batch.map((source) => search(source, keyword, page: page)
.timeout(perSourceTimeout)
.catchError((error) => <UnifiedSearchResult>[
UnifiedSearchResult.error(
sourceId: source.id,
sourceName: source.name,
errorMessage: _shortError(error),
),
]),
),
);
// 跨源去重:按书名+作者指纹
final seen = <String>{};
final deduped = <UnifiedSearchResult>[];
for (final item in responses.expand((items) => items)) {
if (item.isError) {
deduped.add(item);
continue;
}
final fingerprint = '${item.title}|${item.author ?? ''}'.toLowerCase();
if (seen.add(fingerprint)) {
deduped.add(item);
}
}
return deduped;
}
legado_rule_parser.dart — BOM/JSONP 兼容
Object? _jsonBody() {
if (_decodedJson != null) return _decodedJson;
try {
var text = body;
// 去掉 UTF-8 BOM
if (text.isNotEmpty && text.codeUnitAt(0) == 0xFEFF) {
text = text.substring(1);
}
// 尝试 JSONP 格式
final jsonp = RegExp(
r'^[a-zA-Z_$][\w$]*\s*\(\s*(.*)\s*\)\s*;?\s*$',
dotAll: true,
).firstMatch(text.trim());
if (jsonp != null) {
text = jsonp.group(1)!;
}
_decodedJson = jsonDecode(text);
} on FormatException {
_decodedJson = null;
}
return _decodedJson;
}
六、总结
| # | 问题 | 严重度 | 修复方式 |
|---|---|---|---|
| 1 | API 搜索静默吞异常 | 🔴 高 | 分类捕获异常 + 日志记录 |
| 2 | Token Secret 硬编码 | 🔴 高 | 环境变量 + 配置文件 |
| 3 | 章节 ID 错位 | 🔴 高 | 支持 dict 格式提取真实 ID |
| 4 | Legacy HTML 失效 | 🟡 中 | 增加更多回退域名 + GBK |
| 5 | 缺少重试机制 | 🟡 中 | 指数退避重试 |
| 6 | Session 线程不安全 | 🟡 中 | threading.local() |
| 7 | 编码检测不完善 | 🟡 中 | HTML meta 检测 + 乱码检测 |
| 8 | 纯数字路由歧义 | 🟢 低 | 先搜索再回退 book_id |
| 9 | 跨源结果重复 | 🟡 中 | 书名+作者指纹去重 |
| 10 | GBK 桌面端不支持 | 🟡 中 | 非致命回退到 UTF-8 |
| 11 | JSONP/BOM 不兼容 | 🟢 低 | 预处理 body 文本 |
| 12 | 回退搜索太严格 | 🟢 低 | 放宽关键词匹配阈值 |
最关键的三项修复(建议优先实施):
- 章节 ID 错位(问题 3)— 直接导致下载失败,必须立即修复
- API 搜索静默失败(问题 1)— 加了日志就能快速定位根因
- API 重试机制(问题 5)— 解决大部分"偶发失败"问题