first commit

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
kunkunxiaomaoandClaude committed 2026-11-16 20:45:22 +08:00
commit 9c5fef1a4d
412 files changed
+94582

No files matched your search

+95
View File
@@ -0,0 +1,95 @@
# AO3 插件测试与补丁报告
日期:2026-06-14
## 新增测试文件
- `tests/test_ao3_plugin.py`
## 覆盖的测试项
- AO3 URL 识别测试:只接受 `archiveofourown.org` / `ao3.org` 的 `/works/{id}` 与 `/works/{id}/chapters/{id}` 链接,拒绝伪域名和非作品页。
- `work_id` / `chapter_id` 提取测试。
- 无 Cookie 情况下访问公开作品的解析逻辑测试。
- Cookie 注入 headers 测试。
- 元数据解析 mock 测试,覆盖标题、作者、作者 ID、章节数、状态、字数、标签、更新时间。
- EPUB 下载入口 mock 测试。
- EPUB 失败后的 HTML 下载兜底入口 mock 测试。
- AO3 章节页 `workskin` 正文提取回归测试,确认不会把 Summary 或评论区当正文。
- 更新检测逻辑测试。
- 云端同步 payload 生成测试。
- 插件注册 / 加载测试,覆盖 `PluginManager.load_plugin()` 和 URL 到插件匹配。
- 错误处理测试:无效链接、登录限制、网络失败。
所有 AO3 网络请求均使用 mock,没有真实访问 AO3。
## 修复的问题
- 修复 AO3 URL 识别过宽的问题,避免 `evilarchiveofourown.org` 这类伪域名被误识别。
- 新增 `extract_ao3_ids()`,统一提取 `work_id` 与 `chapter_id`。
- `parse()` 增加严格 URL 校验,非 AO3 作品链接直接抛出 `PluginParseError`。
- Cookie 请求头注入路径补测试,确认不会把 Cookie 误漏到无 Cookie 请求。
- 修复 AO3 `<dt>/<dd>` 元数据解析对 `Archive Warnings*`、`Fandoms*` 等带星号标签不兼容的问题。
- EPUB 下载失败时增加 HTML 全文页面兜底下载,公开作品不依赖 Cookie 也可生成 txt。
- 修复下载成功判断依赖中文提示文本的问题,改为检查 `save_text_file()` 返回值与文件是否存在。
- 清理 AO3 插件中的乱码异常提示和注释,避免日志/异常不可读。
- 修复 `https://archiveofourown.org/works/86702241/chapters/229471946` 这类章节页正文提取不完整的问题:优先读取 `role="article"` 正文块,并在 `<!--/main-->`、评论区、afterword、notes 前截断,避免退回 Summary。
- 修复手机端接收内容与桌面端下载 txt 不一致的问题:AO3 会同时产出 `.txt` 和 `.epub`,通用云同步曾把两者都按同一个小说同步,后同步的 EPUB 会覆盖云端 Novel 的 `file_id`。现在小说资源只同步 `.txt` 正文给手机端,EPUB 仅本地归档。
## 测试命令
Python / AO3:
```powershell
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m py_compile plugins\ao3\plugin.py tests\test_ao3_plugin.py
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m py_compile pixiv_app\gui\app.py plugins\ao3\plugin.py
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m unittest tests.test_ao3_plugin
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m unittest tests.test_ao3_plugin tests.test_plugin_manager
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m unittest discover tests
```
当前仓库未发现统一的一键全项目测试命令。本次重点运行与 AO3 插件和插件加载机制直接相关的 Python unittest,并额外尝试了 `unittest discover tests`。
可选的其他模块测试方式:
```powershell
# Python 其他测试
& 'C:\Users\LENOVO\AppData\Local\Python\pythoncore-3.14-64\python.exe' -m unittest discover tests
# Flutter 移动端
cd sakura_mobile
flutter analyze
flutter test
# Web 前端(如存在 package.json)
npm install
npm run build
```
## 测试结果
- `py_compile plugins\ao3\plugin.py tests\test_ao3_plugin.py`:通过。
- `py_compile pixiv_app\gui\app.py plugins\ao3\plugin.py`:通过。
- `python -m unittest tests.test_ao3_plugin`:通过,12 个测试 OK。
- `python -m unittest tests.test_ao3_plugin tests.test_plugin_manager`:通过,13 个测试 OK。
- 使用用户提供的 AO3 页面源码本地验证 `_extract_html_work_text()`:提取 5971 个字符、57 段;开头为正文“手机的铃声在深夜响起…”,不包含 `Summary:` 与评论区内容。
- `python -m unittest discover tests`:未全量通过,44 个测试中 1 个既有测试失败。
全量 discover 失败项:
- `tests.test_origin_follow_sync.OriginFollowSyncLocalTest.test_gallery_sync_creates_cloud_alert_for_new_pixiv_work`
- 失败原因:测试期望 `target_url` 包含作品 ID `67890`,当前实现返回作者页 `https://www.pixiv.net/users/12345`。这与 Pixiv 更新提醒已改为“按作者聚合 alert”的行为一致,和 AO3 插件补丁无直接关系。
测试期间出现环境依赖警告:
- `RequestsDependencyWarning: urllib3/chardet/charset_normalizer doesn't match a supported version`
该警告来自当前 Python 环境的 requests 依赖组合,不是 AO3 测试失败。
## 仍然存在的风险
- AO3 页面结构如果后续大改,纯 HTML 正则解析可能需要更新。
- EPUB 内容提取目前是轻量实现,复杂 EPUB spine 顺序或特殊 XHTML 结构可能需要更完整的 EPUB 解析。
- 更新检测当前返回 `has_update=True`,实际“是否比本地更新”的判定仍由调用方结合本地记录完成。
- 未在本轮运行 Flutter、Web 前端、云端服务集成测试;本次补丁范围限定在 AO3 插件框架与插件加载测试。
- Python 全量 discover 存在 1 个既有 Pixiv 关注同步断言与当前“作者聚合提醒”行为不一致的问题,后续应单独确认该测试期望是否需要更新。
+42
View File
@@ -0,0 +1,42 @@
from __future__ import annotations
import json
import os
from pathlib import Path
from plugins.douyin.plugin import DouyinPlugin
URL = "https://www.douyin.com/jingxuan?modal_id=7635574968763288867"
def main() -> None:
plugin = DouyinPlugin()
resource = plugin.parse(URL)[0]
downloaded: list[str] = []
if os.environ.get("SAKURA_DOUYIN_SMOKE_DOWNLOAD") == "1":
original_files = resource.files
resource.files = resource.files[:1]
downloaded = [str(path) for path in plugin.download(resource, Path("runtime/douyin_smoke_downloads"))]
resource.files = original_files
print(
json.dumps(
{
"ok": True,
"id": resource.id,
"title": resource.title[:80],
"author": resource.author,
"file_count": len(resource.files),
"file_types": [file.get("type") for file in resource.files],
"downloaded": downloaded,
},
ensure_ascii=False,
)
)
if os.environ.get("SAKURA_DOUYIN_SMOKE_VERBOSE") == "1":
for index, file in enumerate(resource.files[:30]):
print(f"{index}: {file.get('type')} {str(file.get('url') or '')[:180]}")
if __name__ == "__main__":
main()
+70
View File
@@ -0,0 +1,70 @@
from __future__ import annotations
import json
import os
from pathlib import Path
from plugins.x.plugin import XPlugin
URLS = {
"normal": "https://x.com/VMAX_IS_MY_Life/status/2079538532822179896",
"image": "https://x.com/kapipi_media/status/2069324792621420865",
"video": "https://x.com/moonlight_ruri/status/2079132957797646448",
}
def main() -> None:
plugin = XPlugin()
cookie_file = os.environ.get("SAKURA_X_COOKIE_FILE", "").strip()
if cookie_file:
plugin.config.cookie_file = cookie_file
cache_file = os.environ.get("SAKURA_X_SMOKE_CACHE_FILE", "").strip()
if cache_file:
plugin._cache_file = Path(cache_file)
plugin.config.headless = True
plugin.config.safety_mode = False
plugin.config.min_action_delay = 0.3
plugin.config.max_action_delay = 0.8
plugin.config.parse_cooldown_seconds = 0
plugin.config.wait_timeout = 30000
download_root = Path(os.environ.get("SAKURA_X_SMOKE_DOWNLOAD_ROOT", "runtime/x_smoke_downloads"))
do_download = os.environ.get("SAKURA_X_SMOKE_DOWNLOAD") == "1"
selected_kind = os.environ.get("SAKURA_X_SMOKE_KIND", "").strip().lower()
targets = URLS.items() if not selected_kind else [(selected_kind, URLS[selected_kind])]
for kind, url in targets:
try:
resources = plugin.parse(url)
downloaded: list[str] = []
if do_download and kind in {"image", "video"} and resources:
target = next(
(
resource
for resource in resources
if any(file.get("type") == kind for file in resource.files)
),
resources[0],
)
target.files = target.files[:1]
downloaded = [str(path) for path in plugin.download(target, download_root / kind)]
print(
json.dumps(
{
"kind": kind,
"ok": True,
"count": len(resources),
"file_types": [file.get("type") for resource in resources for file in resource.files],
"titles": [(resource.title or "")[:60] for resource in resources[:2]],
"downloaded": downloaded,
},
ensure_ascii=False,
)
)
except Exception as exc:
print(json.dumps({"kind": kind, "ok": False, "error": str(exc)[:500]}, ensure_ascii=False))
if __name__ == "__main__":
main()
+225
View File
@@ -0,0 +1,225 @@
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest import TestCase
from fastapi.testclient import TestClient
from jose import jwt
from sqlalchemy import create_engine, select
from sqlalchemy.orm import sessionmaker
from sakura_cloud.core.config import settings
from sakura_cloud.core.security import create_access_token, create_refresh_token, hash_password
from sakura_cloud.db.base import Base
from sakura_cloud.db.session import get_db
from sakura_cloud.main import create_app
from sakura_cloud.models.admin import AdminAuditLog
from sakura_cloud.models.user import User, UserRole
from sakura_cloud.core.logging import logger
class AdminConsoleSecurityTest(TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.root = Path(self.temp_dir.name)
self.db_path = self.root / "cloud.db"
self.engine = create_engine(f"sqlite:///{self.db_path}", connect_args={"check_same_thread": False})
self.SessionLocal = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)
Base.metadata.create_all(self.engine)
self.old_cors = settings.cors_origins
self.old_log_dir = settings.log_dir
settings.cors_origins = ["http://admin.example.test"]
settings.log_dir = str(self.root / "logs")
Path(settings.log_dir).mkdir(parents=True, exist_ok=True)
self.app = create_app()
self.app.dependency_overrides[get_db] = self._override_db
self.client = TestClient(self.app)
self.user_id = "user"
self.admin_id = "admin"
self.super_id = "super"
with self.SessionLocal() as db:
db.add_all(
[
User(
id=self.user_id,
username="user",
email="user@example.test",
password_hash=hash_password("password123"),
role=UserRole.USER,
),
User(
id=self.admin_id,
username="admin",
email="admin@example.test",
password_hash=hash_password("password123"),
role=UserRole.ADMIN,
),
User(
id=self.super_id,
username="root",
email="root@example.test",
password_hash=hash_password("password123"),
role=UserRole.SUPER_ADMIN,
),
]
)
db.commit()
self.user_auth = {"Authorization": f"Bearer {create_access_token(self.user_id)}"}
self.admin_auth = {"Authorization": f"Bearer {create_access_token(self.admin_id)}"}
self.super_auth = {"Authorization": f"Bearer {create_access_token(self.super_id)}"}
def tearDown(self) -> None:
self.client.close()
self.engine.dispose()
logger.remove()
settings.cors_origins = self.old_cors
settings.log_dir = self.old_log_dir
try:
self.temp_dir.cleanup()
except PermissionError:
pass
def _override_db(self):
db = self.SessionLocal()
try:
yield db
finally:
db.close()
def test_admin_requires_login_and_role(self) -> None:
self.assertEqual(self.client.get("/api/v1/admin/dashboard").status_code, 401)
self.assertEqual(self.client.get("/api/v1/admin/dashboard", headers=self.user_auth).status_code, 403)
self.assertEqual(self.client.get("/api/v1/admin/dashboard", headers=self.admin_auth).status_code, 200)
def test_admin_cannot_disable_super_admin(self) -> None:
response = self.client.patch(
f"/api/v1/admin/users/{self.super_id}/status",
headers=self.admin_auth,
json={"is_active": False},
)
self.assertEqual(response.status_code, 403)
def test_invalid_query_params_return_422(self) -> None:
response = self.client.get("/api/v1/admin/users?sort=password_hash", headers=self.admin_auth)
self.assertEqual(response.status_code, 422)
def test_sensitive_fields_are_not_returned(self) -> None:
response = self.client.get("/api/v1/admin/users", headers=self.admin_auth)
self.assertEqual(response.status_code, 200)
text = response.text
self.assertNotIn("password_hash", text)
self.assertNotIn("password123", text)
self.assertIn("admin@example.test", text)
def test_audit_log_is_written_for_disable_user(self) -> None:
response = self.client.patch(
f"/api/v1/admin/users/{self.user_id}/status",
headers=self.admin_auth,
json={"is_active": False},
)
self.assertEqual(response.status_code, 200)
with self.SessionLocal() as db:
row = db.scalar(
select(AdminAuditLog).where(
AdminAuditLog.admin_user_id == self.admin_id,
AdminAuditLog.action == "disable_user",
AdminAuditLog.target_id == self.user_id,
)
)
self.assertIsNotNone(row)
self.assertEqual(row.detail_json["is_active"], False)
def test_admin_login_writes_audit_log(self) -> None:
response = self.client.post(
"/api/v1/admin/login",
json={"username_or_email": "admin", "password": "password123"},
)
self.assertEqual(response.status_code, 200)
self.assertIn("access_token", response.json()["data"])
with self.SessionLocal() as db:
row = db.scalar(select(AdminAuditLog).where(AdminAuditLog.action == "admin_login"))
self.assertIsNotNone(row)
def test_login_does_not_revoke_another_device_access_token(self) -> None:
response = self.client.post(
"/api/v1/auth/login",
json={"username_or_email": "user", "password": "password123"},
)
self.assertEqual(response.status_code, 200)
with self.SessionLocal() as db:
self.assertEqual(db.get(User, self.user_id).token_version, 0)
def test_refresh_does_not_revoke_other_device_tokens(self) -> None:
first_refresh_token = create_refresh_token(self.user_id)
second_refresh_token = create_refresh_token(self.user_id)
first_response = self.client.post(
"/api/v1/auth/refresh",
json={"refresh_token": first_refresh_token},
)
self.assertEqual(first_response.status_code, 200)
with self.SessionLocal() as db:
self.assertEqual(db.get(User, self.user_id).token_version, 0)
second_response = self.client.post(
"/api/v1/auth/refresh",
json={"refresh_token": second_refresh_token},
)
self.assertEqual(second_response.status_code, 200)
def test_security_headers_and_cors_are_present(self) -> None:
response = self.client.get(
"/api/v1/admin/dashboard",
headers={**self.admin_auth, "Origin": "http://admin.example.test"},
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.headers["X-Content-Type-Options"], "nosniff")
self.assertEqual(response.headers["X-Frame-Options"], "DENY")
self.assertIn("Referrer-Policy", response.headers)
self.assertIn("Content-Security-Policy", response.headers)
self.assertEqual(response.headers.get("Access-Control-Allow-Origin"), "http://admin.example.test")
self.assertEqual(response.headers["Referrer-Policy"], "no-referrer")
self.assertIn("Permissions-Policy", response.headers)
def test_disabled_user_access_and_refresh_are_rejected(self) -> None:
refresh_token = create_refresh_token(self.user_id)
with self.SessionLocal() as db:
user = db.get(User, self.user_id)
user.is_active = False
user.token_version += 1
db.commit()
response = self.client.get("/api/v1/me", headers=self.user_auth)
self.assertEqual(response.status_code, 401)
refresh_response = self.client.post("/api/v1/auth/refresh", json={"refresh_token": refresh_token})
self.assertEqual(refresh_response.status_code, 401)
def test_jwt_missing_exp_wrong_type_and_tampered_are_rejected(self) -> None:
no_exp = jwt.encode({"sub": self.user_id, "type": "access"}, settings.jwt_secret_key, algorithm=settings.jwt_algorithm)
wrong_type = jwt.encode({"sub": self.user_id, "type": "reset", "exp": 4102444800}, settings.jwt_secret_key, algorithm=settings.jwt_algorithm)
valid = create_access_token(self.user_id)
tampered = valid[:-1] + ("a" if valid[-1] != "a" else "b")
for token in (no_exp, wrong_type, tampered):
response = self.client.get("/api/v1/me", headers={"Authorization": f"Bearer {token}"})
self.assertEqual(response.status_code, 401)
def test_admin_role_changes_require_super_admin(self) -> None:
response = self.client.patch(
f"/api/v1/admin/users/{self.user_id}/role",
headers=self.admin_auth,
json={"role": "admin"},
)
self.assertEqual(response.status_code, 403)
response = self.client.patch(
f"/api/v1/admin/users/{self.user_id}/role",
headers=self.super_auth,
json={"role": "admin"},
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json()["data"]["role"], "admin")
+296
View File
@@ -0,0 +1,296 @@
from __future__ import annotations
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import requests
from pixiv_app.core.plugin.base import PluginAuthError, PluginDownloadError, PluginError, PluginParseError, Resource
from pixiv_app.core.plugin.manager import PluginManager
from plugins.ao3.plugin import (
AO3Plugin,
AO3_BASE,
AO3WorkMeta,
_ao3_request,
_extract_html_work_text,
build_ao3_sync_metadata,
extract_ao3_ids,
parse_work_meta,
)
PUBLIC_WORK_HTML = """
<html>
<head><meta property="og:title" content="Fallback Title"></head>
<body>
<h2 class="title heading">The Sakura Work</h2>
<a rel="author" href="/users/sakura_user/pseuds/Sakura">Sakura</a>
<blockquote class="userstuff"><p>A gentle summary.</p></blockquote>
<dl class="work meta group">
<dt>Rating:</dt><dd><a class="tag">Teen And Up Audiences</a></dd>
<dt>Archive Warnings*:</dt><dd><a class="tag">No Archive Warnings Apply</a></dd>
<dt>Fandoms:</dt><dd><a class="tag">Original Work</a></dd>
<dt>Relationships:</dt><dd><a class="tag">A/B</a></dd>
<dt>Characters:</dt><dd><a class="tag">A</a><a class="tag">B</a></dd>
<dt>Additional Tags:</dt><dd><a class="tag">Fluff</a><a class="tag">Tea</a></dd>
<dt>Language:</dt><dd>English</dd>
<dt>Chapters:</dt><dd>3/5</dd>
<dt>Words:</dt><dd>12,345</dd>
<dt>Published:</dt><dd>2024-01-02</dd>
<dt>Updated:</dt><dd>2024-02-03</dd>
</dl>
<div class="userstuff module" role="article">
<h3 class="landmark heading" id="work">Chapter One</h3>
<p>Hello AO3.</p>
<p>Second paragraph.</p>
</div>
</body>
</html>
"""
AO3_CHAPTER_PAGE_WITH_SUMMARY_HTML = """
<html>
<body>
<div id="workskin">
<div class="preface group">
<div class="summary module">
<h3 class="heading">Summary:</h3>
<blockquote class="userstuff">
<p>Summary should not be treated as body.</p>
</blockquote>
</div>
</div>
<div id="chapters">
<div class="chapter" id="chapter-1">
<div class="chapter preface group">
<h3 class="title">
<a href="/works/86702241/chapters/229471946">Chapter 1</a>
</h3>
</div>
<!--main content-->
<div class="userstuff module" role="article">
<h3 class="landmark heading" id="work">Chapter Text</h3>
<p>The phone rang in the deep night.</p>
<p>Only this person had that ringtone.</p>
</div>
<!--/main-->
</div>
</div>
</div>
<!-- BEGIN comment section -->
<div id="feedback"><p>Comment UI should not be body.</p></div>
</body>
</html>
"""
class FakeResponse:
def __init__(
self,
*,
status_code: int = 200,
text: str = PUBLIC_WORK_HTML,
url: str = f"{AO3_BASE}/works/12345",
content: bytes = b"",
headers: dict[str, str] | None = None,
) -> None:
self.status_code = status_code
self.text = text
self.url = url
self.content = content
self.headers = headers or {}
self.ok = 200 <= status_code < 400
def raise_for_status(self) -> None:
if not self.ok:
raise requests.HTTPError(f"HTTP {self.status_code}")
def iter_content(self, chunk_size: int = 1024):
yield self.content or b"x" * 2048
class AO3PluginTests(unittest.TestCase):
def test_url_recognition_is_strict(self) -> None:
plugin = AO3Plugin()
self.assertTrue(plugin.can_handle("https://archiveofourown.org/works/12345"))
self.assertTrue(plugin.can_handle("https://archiveofourown.org/works/12345/chapters/67890"))
self.assertTrue(plugin.can_handle("https://ao3.org/works/12345"))
self.assertFalse(plugin.can_handle("https://evilarchiveofourown.org/works/12345"))
self.assertFalse(plugin.can_handle("https://archiveofourown.org/users/sakura"))
self.assertFalse(plugin.can_handle(""))
def test_extract_work_and_chapter_ids(self) -> None:
self.assertEqual(extract_ao3_ids("https://archiveofourown.org/works/12345"), ("12345", None))
self.assertEqual(
extract_ao3_ids("https://archiveofourown.org/works/12345/chapters/67890"),
("12345", "67890"),
)
with self.assertRaises(PluginParseError):
extract_ao3_ids("https://archiveofourown.org/users/sakura")
def test_public_work_parse_without_cookie(self) -> None:
plugin = AO3Plugin()
with patch("plugins.ao3.plugin.requests.get", return_value=FakeResponse()) as mocked:
resources = plugin.parse("https://archiveofourown.org/works/12345")
self.assertEqual(mocked.call_args.kwargs["headers"].get("Cookie"), None)
self.assertEqual(len(resources), 1)
self.assertEqual(resources[0].id, "12345")
self.assertEqual(resources[0].title, "The Sakura Work")
self.assertEqual(resources[0].author, "Sakura")
self.assertEqual(resources[0].metadata["source_platform"], "ao3")
def test_cookie_injected_into_headers_and_requests(self) -> None:
plugin = AO3Plugin(cookie="session=abc")
self.assertEqual(plugin.get_headers()["Cookie"], "session=abc")
with patch("plugins.ao3.plugin.requests.get", return_value=FakeResponse()) as mocked:
_ao3_request("https://archiveofourown.org/works/12345", cookie="session=abc")
self.assertEqual(mocked.call_args.kwargs["headers"]["Cookie"], "session=abc")
def test_metadata_parse(self) -> None:
meta = parse_work_meta(PUBLIC_WORK_HTML, "https://archiveofourown.org/works/12345")
self.assertEqual(meta.work_id, "12345")
self.assertEqual(meta.title, "The Sakura Work")
self.assertEqual(meta.author, "Sakura")
self.assertEqual(meta.author_id, "sakura_user")
self.assertEqual(meta.chapter_count, 3)
self.assertEqual(meta.status, "In Progress")
self.assertEqual(meta.word_count, 12345)
self.assertIn("Fluff", meta.tags)
self.assertEqual(meta.updated_at, "2024-02-03T00:00:00+00:00")
def test_html_primary_download_with_epub_secondary(self) -> None:
"""HTML full-work page is the primary download; EPUB is a secondary bonus."""
plugin = AO3Plugin(cookie="session=abc")
resource = _resource()
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
epub_url = f"{AO3_BASE}/downloads/12345/Title.epub"
def fake_download(_url: str, save_path: Path, **_kwargs) -> Path:
save_path.write_bytes(b"PK" + b"x" * 2048)
return save_path
with (
patch(
"plugins.ao3.plugin._ao3_request",
return_value=FakeResponse(text=PUBLIC_WORK_HTML),
) as mocked_html_request,
patch(
"plugins.ao3.plugin._resolve_ao3_epub_url",
return_value=epub_url,
),
patch(
"plugins.ao3.plugin._ao3_download_file",
side_effect=fake_download,
) as mocked_download,
):
paths = plugin.download(resource, root)
# Primary: HTML → .txt
self.assertTrue(any(path.suffix == ".txt" for path in paths),
f"Expected .txt file, got: {[p.suffix for p in paths]}")
html_url = mocked_html_request.call_args.args[0]
self.assertIn("view_full_work=true", html_url)
# Secondary: EPUB → .epub
self.assertTrue(any(path.suffix == ".epub" for path in paths),
f"Expected .epub file, got: {[p.suffix for p in paths]}")
self.assertIn(epub_url, mocked_download.call_args.args[0])
def test_html_only_download_when_epub_not_available(self) -> None:
"""When EPUB resolution returns None, the .txt alone still succeeds."""
plugin = AO3Plugin()
resource = _resource()
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
with (
patch(
"plugins.ao3.plugin._ao3_request",
return_value=FakeResponse(text=PUBLIC_WORK_HTML),
) as mocked_request,
patch(
"plugins.ao3.plugin._resolve_ao3_epub_url",
return_value=None,
),
):
paths = plugin.download(resource, root)
self.assertTrue(any(path.suffix == ".txt" for path in paths),
f"Expected .txt file, got: {[p.suffix for p in paths]}")
self.assertFalse(any(path.suffix == ".epub" for path in paths))
self.assertIn("view_full_work=true", mocked_request.call_args.args[0])
def test_chapter_page_body_extraction_ignores_summary_and_feedback(self) -> None:
text = _extract_html_work_text(AO3_CHAPTER_PAGE_WITH_SUMMARY_HTML)
self.assertIn("The phone rang in the deep night.", text)
self.assertIn("Only this person had that ringtone.", text)
self.assertNotIn("Summary should not be treated as body.", text)
self.assertNotIn("Comment UI should not be body.", text)
self.assertNotIn("Chapter Text", text)
def test_update_check_returns_metadata(self) -> None:
plugin = AO3Plugin()
with patch("plugins.ao3.plugin._ao3_request", return_value=FakeResponse(text=PUBLIC_WORK_HTML)):
update = plugin.check_update("12345")
assert update is not None
self.assertEqual(update["work_id"], "12345")
self.assertEqual(update["chapter_count"], 3)
self.assertTrue(update["has_update"])
def test_cloud_sync_payload_generation(self) -> None:
meta = parse_work_meta(PUBLIC_WORK_HTML, "https://archiveofourown.org/works/12345")
payload = build_ao3_sync_metadata(meta, Path("work.txt"))
self.assertEqual(payload.resource_type, "novel")
self.assertEqual(payload.source_platform, "ao3")
self.assertEqual(payload.source_id, "12345")
self.assertEqual(payload.series_id, "12345")
self.assertEqual(payload.word_count, 12345)
self.assertIn("Fluff", payload.tags)
def test_plugin_registration_and_loading(self) -> None:
root = Path(__file__).resolve().parents[1]
if str(root) not in sys.path:
sys.path.insert(0, str(root))
manager = PluginManager([root / "plugins"])
plugin = manager.load_plugin(root / "plugins" / "ao3" / "plugin.py")
self.assertIsNotNone(plugin)
assert plugin is not None
self.assertEqual(plugin.name, "AO3")
self.assertIs(manager.get_plugin_for_url("https://archiveofourown.org/works/12345"), plugin)
def test_error_handling_invalid_link_login_wall_and_network_failure(self) -> None:
plugin = AO3Plugin()
with self.assertRaises(PluginParseError):
plugin.parse("https://archiveofourown.org/users/sakura")
with (
patch("plugins.ao3.plugin.requests.get", return_value=FakeResponse(text="Please log in")),
self.assertRaises(PluginAuthError),
):
_ao3_request("https://archiveofourown.org/works/12345", retries=1)
with (
patch("plugins.ao3.plugin.requests.get", side_effect=requests.RequestException("offline")),
self.assertRaises(PluginError),
):
_ao3_request("https://archiveofourown.org/works/12345", retries=1)
def _resource() -> Resource:
return Resource(
id="12345",
url="https://archiveofourown.org/works/12345",
title="The Sakura Work",
author="Sakura",
author_id="sakura_user",
files=[],
metadata={"work_id": "12345", "summary": "A gentle summary."},
)
if __name__ == "__main__":
unittest.main()
+88
View File
@@ -0,0 +1,88 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
from unittest.mock import AsyncMock, patch
from pixiv_app.core.plugin.base import Resource
from pixiv_app.core.plugin.manager import PluginManager
from plugins.pixiv.plugin import PixivPlugin
from plugins.x.plugin import XPlugin
class BundledPluginRegressionTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.root = Path(__file__).resolve().parents[1]
if str(cls.root) not in sys.path:
sys.path.insert(0, str(cls.root))
def test_all_bundled_plugins_load_without_errors(self) -> None:
manager = PluginManager([self.root / "plugins"])
plugins = manager.load_all()
self.assertTrue({"Pixiv", "X", "Douyin", "AO3"}.issubset(plugins))
self.assertEqual(manager.load_errors, {})
for plugin in plugins.values():
self.assertIsInstance(plugin.validate(), bool)
def test_manager_routes_x_and_pixiv_urls(self) -> None:
manager = PluginManager([self.root / "plugins"])
manager.load_all()
self.assertEqual(
manager.get_plugin_for_url("https://x.com/example/status/123456789").name,
"X",
)
self.assertEqual(
manager.get_plugin_for_url("https://www.pixiv.net/artworks/123456789").name,
"Pixiv",
)
def test_pixiv_novel_url_creates_lazy_novel_resource(self) -> None:
plugin = PixivPlugin()
resources = plugin.parse("https://www.pixiv.net/novel/show.php?id=123456")
self.assertEqual(len(resources), 1)
self.assertEqual(resources[0].id, "123456")
self.assertEqual(resources[0].metadata["kind"], "novel")
def test_x_parse_wrapper_returns_resources_from_parser(self) -> None:
plugin = XPlugin()
plugin.config.safety_mode = False
expected = [
Resource(
id="123_0",
url="https://x.com/example/status/123",
title="example",
author="Example",
author_id="example",
files=[{"url": "https://video.twimg.com/example.mp4", "type": "video"}],
)
]
with patch.object(plugin, "_parse_async", AsyncMock(return_value=expected)):
resources = plugin.parse("https://x.com/example/status/123")
self.assertEqual(resources, expected)
self.assertEqual(resources[0].files[0]["type"], "video")
def test_x_url_recognition_rejects_unrelated_hosts(self) -> None:
plugin = XPlugin()
self.assertTrue(plugin.can_handle("https://x.com/example/media"))
self.assertTrue(plugin.can_handle("https://twitter.com/example/status/123"))
self.assertFalse(plugin.can_handle("https://example.com/x.com/example/status/123"))
def test_pixiv_url_recognition_rejects_unrelated_hosts(self) -> None:
plugin = PixivPlugin()
self.assertTrue(plugin.can_handle("https://www.pixiv.net/artworks/123"))
self.assertTrue(plugin.can_handle("https://i.pximg.net/img-original/example.jpg"))
self.assertFalse(plugin.can_handle("https://example.com/pixiv.net/artworks/123"))
if __name__ == "__main__":
unittest.main()
+35
View File
@@ -0,0 +1,35 @@
from __future__ import annotations
from datetime import datetime, timezone
from types import SimpleNamespace
from unittest import TestCase
from sqlalchemy import MetaData
from sakura_cloud.schemas.follow import CreatorFollowOut
class CreatorFollowSchemaTest(TestCase):
def test_creator_follow_out_uses_metadata_json_not_sqlalchemy_metadata(self) -> None:
now = datetime.now(timezone.utc)
row = SimpleNamespace(
id="follow-1",
platform="pixiv",
creator_id="123",
creator_name="Alice",
creator_handle=None,
profile_url=None,
avatar_url=None,
enabled=True,
last_checked_at=None,
latest_published_at=None,
metadata=MetaData(),
metadata_json={"source": "mobile"},
created_at=now,
updated_at=now,
)
output = CreatorFollowOut.from_row(row).model_dump(by_alias=True)
self.assertEqual(output["metadata"], {"source": "mobile"})
self.assertNotIn("metadata_json", output)
+65
View File
@@ -0,0 +1,65 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from pixiv_app.core.plugin.base import Resource
from pixiv_app.core.x_media_service import XMediaService
PROJECT_ROOT = Path(__file__).resolve().parents[1]
class _LocalOnlyPlugin:
def download(self, resource: Resource, save_root: Path) -> list[Path]:
save_root.mkdir(parents=True, exist_ok=True)
target = save_root / f"{resource.id}.jpg"
target.write_bytes(b"local-only")
return [target]
class CloudUploadRemovedTests(unittest.TestCase):
def test_desktop_download_entrypoints_do_not_call_cloud_upload(self) -> None:
callers = (
PROJECT_ROOT / "pixiv_app" / "gui" / "app.py",
PROJECT_ROOT / "pixiv_app" / "services" / "gallery_api.py",
PROJECT_ROOT / "pixiv_app" / "core" / "x_media_service.py",
PROJECT_ROOT / "pixiv_app" / "core" / "gallery_service.py",
PROJECT_ROOT / "pixiv_app" / "core" / "origin_follow_sync.py",
)
for path in callers:
source = path.read_text(encoding="utf-8")
self.assertNotIn(".sync_file(", source, path.name)
self.assertNotIn("sync_plugin_paths(", source, path.name)
self.assertNotIn("sync_pixiv_result(", source, path.name)
self.assertNotIn("SakuraCloudClient(", source, path.name)
self.assertNotIn(".upsert_follow(", source, path.name)
self.assertNotIn(".create_alert(", source, path.name)
self.assertNotIn(".bulk_upsert_follows(", source, path.name)
def test_x_preview_download_is_local_only(self) -> None:
with tempfile.TemporaryDirectory() as directory:
service = XMediaService(save_root=Path(directory))
service._manager = type("Manager", (), {"plugins": {"X": _LocalOnlyPlugin()}})()
service._preview_cache["preview"] = [
Resource(
id="tweet-1",
url="https://x.com/example/status/1",
title="example",
author="Example",
author_id="example",
files=[{"type": "image", "url": "https://example.invalid/image.jpg"}],
)
]
result = service.download_preview("preview")
self.assertEqual(result["downloaded_files"], 1)
self.assertEqual(result["synced_files"], 0)
self.assertEqual(result["items"][0]["synced_files"], 0)
self.assertTrue(Path(result["items"][0]["paths"][0]).is_file())
if __name__ == "__main__":
unittest.main()
+94
View File
@@ -0,0 +1,94 @@
from __future__ import annotations
import tempfile
from datetime import datetime, timedelta, timezone
from pathlib import Path
from unittest import TestCase
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sakura_cloud.db.base import Base
from sakura_cloud.models.follow import UpdateAlert
from sakura_cloud.models.user import User
from sakura_cloud.services.reminders import run_daily_reminder_digest
class DailyReminderDigestTest(TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.db_path = Path(self.temp_dir.name) / "cloud.db"
self.engine = create_engine(f"sqlite:///{self.db_path}", connect_args={"check_same_thread": False})
self.SessionLocal = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)
Base.metadata.create_all(self.engine)
def tearDown(self) -> None:
self.engine.dispose()
self.temp_dir.cleanup()
def test_digest_is_created_even_when_no_updates(self) -> None:
with self.SessionLocal() as db:
user = User(
id="user-a",
username="alice",
email="alice@example.test",
password_hash="unused",
reminder_enabled=True,
reminder_hour=0,
reminder_minute=0,
reminder_timezone="UTC",
)
db.add(user)
db.commit()
self.assertEqual(run_daily_reminder_digest(db), 1)
digest = db.query(UpdateAlert).filter(UpdateAlert.platform == "system").one()
self.assertIn("未更新", digest.title)
self.assertEqual(digest.payload_json["new_alert_count"], 0)
self.assertEqual(digest.payload_json["status"], "not_updated")
def test_digest_counts_new_update_records(self) -> None:
now = datetime.now(timezone.utc)
with self.SessionLocal() as db:
user = User(
id="user-a",
username="alice",
email="alice@example.test",
password_hash="unused",
reminder_enabled=True,
reminder_hour=0,
reminder_minute=0,
reminder_timezone="UTC",
last_reminder_sent_at=now - timedelta(days=1),
)
db.add(user)
db.add_all(
[
UpdateAlert(
user_id="user-a",
platform="pixiv",
creator_id="123",
title="new 1",
body="new",
payload_json={"creator_name": "Sakura"},
),
UpdateAlert(
user_id="user-a",
platform="x",
creator_id="sakura",
title="new 2",
body="new",
payload_json={"creator_handle": "sakura"},
),
]
)
db.commit()
self.assertEqual(run_daily_reminder_digest(db), 1)
digest = db.query(UpdateAlert).filter(UpdateAlert.platform == "system").one()
self.assertIn("更新 2 条记录", digest.title)
self.assertEqual(digest.payload_json["new_alert_count"], 2)
self.assertEqual(digest.payload_json["status"], "updated")
+110
View File
@@ -0,0 +1,110 @@
from __future__ import annotations
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
from pixiv_app.core.plugin.manager import PluginManager
from plugins.douyin.plugin import DouyinPlugin
VIDEO_HTML = """
<html><body>
<script id="RENDER_DATA" type="application/json">
%7B%22aweme%22%3A%7B%22aweme_id%22%3A%227321234567890123456%22%2C%22desc%22%3A%22test%20video%22%2C%22author%22%3A%7B%22nickname%22%3A%22sakura%22%2C%22uid%22%3A%22u1%22%7D%2C%22video%22%3A%7B%22play_addr%22%3A%7B%22url_list%22%3A%5B%22https%3A%5C%2F%5C%2Fv3.douyinvod.com%5C%2Fvideo.mp4%22%5D%7D%7D%7D%7D
</script>
</body></html>
"""
IMAGE_HTML = """
<html><body>
<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__" type="application/json">
{"aweme":{"aweme_id":"7321234567890123457","desc":"test images","author":{"nickname":"hana","uid":"u2"},"images":[{"url_list":["https:\\/\\/p3-pc.douyinpic.com\\/a.jpg"]},{"url_list":["https:\\/\\/p3-pc.douyinpic.com\\/b.webp"]}]}}
</script>
</body></html>
"""
class FakeResponse:
def __init__(self, url: str, text: str = "", content: bytes = b"video-bytes" * 80, content_type: str = "video/mp4") -> None:
self.url = url
self.text = text
self.content = content
self.headers = {"content-type": content_type}
def raise_for_status(self) -> None:
return None
class FakeClient:
def __init__(self, *args, **kwargs) -> None:
self.requests: list[str] = []
def __enter__(self) -> "FakeClient":
return self
def __exit__(self, *args) -> None:
return None
def get(self, url: str, *args, **kwargs) -> FakeResponse:
self.requests.append(url)
if "note" in url:
return FakeResponse("https://www.douyin.com/note/7321234567890123457", IMAGE_HTML)
if "v.douyin.com" in url:
return FakeResponse("https://www.douyin.com/video/7321234567890123456", VIDEO_HTML)
if url.endswith(".jpg") or url.endswith(".webp"):
return FakeResponse(url, content=b"image-bytes" * 80, content_type="image/jpeg")
return FakeResponse(url, VIDEO_HTML)
class DouyinPluginTests(unittest.TestCase):
def test_can_handle_douyin_urls(self) -> None:
plugin = DouyinPlugin()
self.assertTrue(plugin.can_handle("https://v.douyin.com/abc123/"))
self.assertTrue(plugin.can_handle("https://www.douyin.com/video/7321234567890123456"))
self.assertTrue(plugin.can_handle("https://www.douyin.com/note/7321234567890123457"))
self.assertFalse(plugin.can_handle("https://x.com/test/status/1"))
@patch("plugins.douyin.plugin.httpx.Client", FakeClient)
def test_parse_video_from_render_data(self) -> None:
plugin = DouyinPlugin()
resource = plugin.parse("https://v.douyin.com/mock/")[0]
self.assertEqual(resource.id, "7321234567890123456")
self.assertEqual(resource.title, "test video")
self.assertEqual(resource.author, "sakura")
self.assertEqual(resource.files[0]["type"], "video")
self.assertIn("douyinvod.com/video.mp4", resource.files[0]["url"])
@patch("plugins.douyin.plugin.httpx.Client", FakeClient)
def test_parse_image_note(self) -> None:
plugin = DouyinPlugin()
resource = plugin.parse("https://www.douyin.com/note/7321234567890123457")[0]
self.assertEqual(resource.id, "7321234567890123457")
self.assertEqual(resource.title, "test images")
self.assertEqual([item["type"] for item in resource.files], ["image", "image"])
@patch("plugins.douyin.plugin.httpx.Client", FakeClient)
def test_download_video_entry(self) -> None:
plugin = DouyinPlugin()
resource = plugin.parse("https://v.douyin.com/mock/")[0]
with tempfile.TemporaryDirectory() as tmp:
paths = plugin.download(resource, Path(tmp))
self.assertEqual(len(paths), 1)
self.assertTrue(paths[0].is_file())
self.assertEqual(paths[0].suffix, ".mp4")
def test_plugin_registration(self) -> None:
root = Path(__file__).resolve().parents[1]
if str(root) not in sys.path:
sys.path.insert(0, str(root))
manager = PluginManager([root / "plugins"])
plugin = manager.load_plugin(root / "plugins" / "douyin" / "plugin.py")
self.assertIsNotNone(plugin)
assert plugin is not None
self.assertIs(manager.get_plugin_for_url("https://www.douyin.com/video/7321234567890123456"), plugin)
if __name__ == "__main__":
unittest.main()
+40
View File
@@ -0,0 +1,40 @@
from __future__ import annotations
import json
import tempfile
from pathlib import Path
from unittest import TestCase
from pixiv_app.core.library import PixivLibrary
class PixivLibraryArtistRepairTest(TestCase):
def test_setup_repairs_missing_artwork_artist_from_metadata(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "library.db"
library = PixivLibrary(db_path)
try:
with library.connection:
library.connection.execute(
"""
INSERT INTO artworks (artwork_id, artist_id, type, title, metadata_json, updated_at)
VALUES (?, NULL, 'illust', 'work', ?, CURRENT_TIMESTAMP)
""",
(123, json.dumps({"userId": 456, "userName": "Alice"})),
)
finally:
library.close()
repaired = PixivLibrary(db_path)
try:
row = repaired.connection.execute(
"SELECT artist_id FROM artworks WHERE artwork_id = 123"
).fetchone()
artist = repaired.connection.execute(
"SELECT name FROM artists WHERE artist_id = 456"
).fetchone()
finally:
repaired.close()
self.assertEqual(row["artist_id"], 456)
self.assertEqual(artist["name"], "Alice")
+74
View File
@@ -0,0 +1,74 @@
from __future__ import annotations
import json
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from PIL import Image
from pixiv_app.core.library import PixivLibrary
from pixiv_app.core.gallery_service import LocalGalleryService
from pixiv_app.core.thumbnails import ThumbnailCache
from pixiv_app.gui.app import SakuraDownloaderGUI
class _HealthResponse:
status = 200
def __enter__(self) -> "_HealthResponse":
return self
def __exit__(self, *args) -> None:
return None
def read(self, _limit: int) -> bytes:
return json.dumps({"status": "ok"}).encode("utf-8")
class LocalGalleryCreationTests(unittest.TestCase):
def test_gallery_readiness_uses_health_endpoint(self) -> None:
app = object.__new__(SakuraDownloaderGUI)
with patch("pixiv_app.gui.app.urllib.request.urlopen", return_value=_HealthResponse()) as mocked:
self.assertTrue(app.is_gallery_wall_ready("http://127.0.0.1:8765"))
self.assertEqual(mocked.call_args.args[0], "http://127.0.0.1:8765/api/health")
def test_thumbnail_cache_creates_local_parent_directories(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
db_path = root / "data" / "library.db"
cache_dir = root / "data" / "runtime" / "thumbnails"
source = root / "source.png"
Image.new("RGB", (640, 480), "#f4aac6").save(source)
library = PixivLibrary(db_path)
try:
library.connection.execute(
"INSERT INTO artworks (artwork_id, type, title, created_at, updated_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
(1001, "illust", "local thumbnail"),
)
library.connection.execute(
"INSERT INTO artwork_files (artwork_id, page_index, file_path, source_url, status, updated_at) VALUES (?, ?, ?, ?, ?, CURRENT_TIMESTAMP)",
(1001, 0, str(source), "", "downloaded"),
)
library.connection.commit()
finally:
library.close()
results = ThumbnailCache(db_path=db_path, cache_dir=cache_dir).ensure_for_artwork(1001)
self.assertEqual(len(results), 1)
self.assertTrue(Path(results[0].thumbnail_path).is_file())
self.assertTrue(db_path.is_file())
def test_gallery_service_creates_database_immediately(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
db_path = root / "data" / "library.db"
LocalGalleryService(db_path=db_path, thumbnail_dir=root / "data" / "thumbs")
self.assertTrue(db_path.is_file())
if __name__ == "__main__":
unittest.main()
+183
View File
@@ -0,0 +1,183 @@
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest import TestCase
from fastapi import FastAPI
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, select
from sqlalchemy.orm import sessionmaker
from sakura_cloud.api.v1.router import api_router
from sakura_cloud.core.config import settings
from sakura_cloud.core.security import create_access_token
from sakura_cloud.db.base import Base
from sakura_cloud.db.session import get_db
from sakura_cloud.models.artwork import Artwork
from sakura_cloud.models.ocr import ArtworkOcrText
from sakura_cloud.models.user import User, UserRole
from sakura_cloud.services.ocr import run_ocr_for_artwork
class OcrGallerySearchTest(TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.root = Path(self.temp_dir.name)
self.db_path = self.root / "cloud.db"
self.image_path = self.root / "image.jpg"
self.text_path = self.root / "note.txt"
self.image_path.write_bytes(b"fake image")
self.text_path.write_text("not image", encoding="utf-8")
self.engine = create_engine(f"sqlite:///{self.db_path}", connect_args={"check_same_thread": False})
self.SessionLocal = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)
Base.metadata.create_all(self.engine)
self.old_ocr_enabled = settings.ocr_enabled
settings.ocr_enabled = True
self.app = FastAPI()
self.app.include_router(api_router, prefix=settings.api_prefix)
self.app.dependency_overrides[get_db] = self._override_db
self.client = TestClient(self.app)
self.user_a_id = "user-a"
self.user_b_id = "user-b"
self.art_a_id = "art-a"
self.art_b_id = "art-b"
self.text_art_id = "text-art"
with self.SessionLocal() as db:
db.add_all(
[
User(
id=self.user_a_id,
username="alice",
email="alice@example.test",
password_hash="unused",
role=UserRole.USER,
),
User(
id=self.user_b_id,
username="bob",
email="bob@example.test",
password_hash="unused",
role=UserRole.USER,
),
Artwork(
id=self.art_a_id,
user_id=self.user_a_id,
source_platform="pixiv",
source_id="100",
content_type="illust",
title="Sakura",
file_path=str(self.image_path),
file_size=10,
mime_type="image/jpeg",
tags=["flower"],
artwork_metadata={},
),
Artwork(
id=self.art_b_id,
user_id=self.user_b_id,
source_platform="pixiv",
source_id="200",
content_type="illust",
title="Other",
file_path=str(self.image_path),
file_size=10,
mime_type="image/jpeg",
tags=[],
artwork_metadata={},
),
Artwork(
id=self.text_art_id,
user_id=self.user_a_id,
source_platform="pixiv",
content_type="illust",
title="Text file",
file_path=str(self.text_path),
file_size=8,
mime_type="text/plain",
tags=[],
artwork_metadata={},
),
]
)
db.commit()
self.auth_a = {"Authorization": f"Bearer {create_access_token(self.user_a_id)}"}
self.auth_b = {"Authorization": f"Bearer {create_access_token(self.user_b_id)}"}
def tearDown(self) -> None:
self.client.close()
self.engine.dispose()
settings.ocr_enabled = self.old_ocr_enabled
self.temp_dir.cleanup()
def _override_db(self):
db = self.SessionLocal()
try:
yield db
finally:
db.close()
def test_ocr_result_is_saved_and_searchable_by_scope(self) -> None:
with self.SessionLocal() as db:
artwork = db.get(Artwork, self.art_a_id)
self.assertIsNotNone(artwork)
row = run_ocr_for_artwork(
db,
artwork,
extractor=lambda _path: ("舰娘 远征记录", 0.92, "ch"),
)
self.assertEqual(row.status, "done")
self.assertIn("舰娘", row.ocr_text)
ocr_search = self.client.get("/api/v1/gallery?query=舰娘&search_scope=ocr", headers=self.auth_a)
self.assertEqual(ocr_search.status_code, 200)
items = ocr_search.json()["data"]["items"]
self.assertEqual([item["id"] for item in items], [self.art_a_id])
self.assertEqual(items[0]["matched_by"], "ocr")
title_search = self.client.get("/api/v1/gallery?query=舰娘&search_scope=title", headers=self.auth_a)
self.assertEqual(title_search.status_code, 200)
self.assertEqual(title_search.json()["data"]["items"], [])
def test_ocr_data_is_isolated_by_user(self) -> None:
with self.SessionLocal() as db:
artwork = db.get(Artwork, self.art_a_id)
run_ocr_for_artwork(db, artwork, extractor=lambda _path: ("舰娘", 0.9, "ch"))
other_user = self.client.get("/api/v1/gallery?query=舰娘&search_scope=ocr", headers=self.auth_b)
self.assertEqual(other_user.status_code, 200)
self.assertEqual(other_user.json()["data"]["items"], [])
self.assertEqual(self.client.get(f"/api/v1/gallery/{self.art_a_id}/ocr", headers=self.auth_b).status_code, 404)
def test_non_image_file_is_skipped(self) -> None:
with self.SessionLocal() as db:
artwork = db.get(Artwork, self.text_art_id)
row = run_ocr_for_artwork(db, artwork, extractor=lambda _path: ("should not run", 1.0, "ch"))
self.assertEqual(row.status, "failed")
self.assertEqual(row.error, "unsupported_file_type")
stored = db.scalar(select(ArtworkOcrText).where(ArtworkOcrText.artwork_id == self.text_art_id))
self.assertIsNotNone(stored)
self.assertEqual(stored.ocr_text, "")
def test_ocr_failure_does_not_block_gallery_create(self) -> None:
settings.ocr_enabled = False
response = self.client.post(
"/api/v1/gallery",
headers=self.auth_a,
json={
"source_platform": "pixiv",
"content_type": "illust",
"title": "New upload",
"file_path": str(self.image_path),
"file_size": 10,
"mime_type": "image/jpeg",
"tags": [],
"metadata": {},
},
)
self.assertEqual(response.status_code, 201)
+231
View File
@@ -0,0 +1,231 @@
from __future__ import annotations
import unittest
from unittest.mock import Mock, patch
import requests
from sakura_cloud.services.online_search_engine import (
OnlineSearchEngine,
RuleConfig,
RuleManager,
build_search_request,
build_search_url,
parse_comic,
parse_novel,
resolve_resource_url,
title_match_score,
)
class OnlineSearchEngineTest(unittest.TestCase):
def test_resolve_resource_url_handles_relative_and_scheme_relative(self) -> None:
self.assertEqual(
resolve_resource_url("https://example.com/books/index.html", "../read/1.html"),
"https://example.com/read/1.html",
)
self.assertEqual(
resolve_resource_url("https://example.com", "//cdn.example.com/a.webp"),
"https://cdn.example.com/a.webp",
)
def test_build_search_url_replaces_common_legado_placeholders(self) -> None:
keyword = "十宗罪"
template = "/search?q={{key}}&k={keyword}&s=%s&x=searchKey"
result = build_search_url(template, keyword)
self.assertNotIn("{{key}}", result)
self.assertNotIn("{keyword}", result)
self.assertNotIn("%s", result)
self.assertNotIn("searchKey", result)
self.assertIn("%E5%8D%81%E5%AE%97%E7%BD%AA", result)
def test_build_search_request_supports_post_body(self) -> None:
url, data = build_search_request("POST /search@keyword={{key}}&page=1", "魔法", "https://novel.example.com")
self.assertEqual(url, "https://novel.example.com/search")
self.assertEqual(data, {"keyword": "%E9%AD%94%E6%B3%95", "page": "1"})
def test_title_match_score_uses_noise_words_and_fuzzy_match(self) -> None:
self.assertGreater(title_match_score("十宗罪 最新章节 免费阅读", "十宗罪"), 0.75)
self.assertGreater(title_match_score("千分之一的魔法", "千分之一魔法"), 0.55)
def test_rule_manager_can_disable_rules(self) -> None:
manager = RuleManager([RuleConfig(rule_id="a", name="A", base_url="https://a.example", search_url="/s?q={{key}}")])
self.assertEqual(len(manager.get_rules("all")), 1)
manager.disable_rule("a")
self.assertEqual(manager.get_rules("all"), [])
def test_search_rule_parses_json_payload_without_network(self) -> None:
response = requests.Response()
response.status_code = 200
response._content = (
b'{"data":[{"bookName":"Sakura Magic","bookUrl":"/book/1","author":"Nana","cover":"/c.jpg"}]}'
)
response.headers["content-type"] = "application/json; charset=utf-8"
session = Mock()
session.request.return_value = response
manager = RuleManager(
[
RuleConfig(
rule_id="json",
name="JSON Source",
base_url="https://novel.example.com",
resource_type="novel",
search_url="/search?q={{key}}",
rule_search={
"bookList": "$.data[*]",
"name": "$.bookName",
"bookUrl": "$.bookUrl",
"author": "$.author",
"cover": "$.cover",
},
)
]
)
engine = OnlineSearchEngine(manager, session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
rows = engine.search_resource("Sakura", "novel")
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0].url, "https://novel.example.com/book/1")
self.assertEqual(rows[0].cover, "https://novel.example.com/c.jpg")
def test_parse_novel_public_page_without_cookie(self) -> None:
html = """
<html><head><meta property="og:image" content="/cover.jpg"></head>
<body><h1>神秘复苏</h1><div id="content">第一章\n论坛里的故事。\n这里是正文。</div></body></html>
"""
response = requests.Response()
response.status_code = 200
response._content = html.encode("utf-8")
response.headers["content-type"] = "text/html; charset=utf-8"
session = Mock()
session.request.return_value = response
engine = OnlineSearchEngine(session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
parsed = engine.parse_novel("https://novel.example.com/read/1.html")
self.assertEqual(parsed.title, "神秘复苏")
self.assertIn("论坛里的故事", parsed.chapters[0].content or "")
self.assertEqual(parsed.cover, "https://novel.example.com/cover.jpg")
def test_parse_comic_reads_lazy_loaded_images(self) -> None:
html = """
<html><body><h1>魔法漫画</h1>
<div class="chapter-img"><img data-original="/manga/001.webp" width="800" height="1200"></div>
<img src="/static/logo.png" width="40" height="40">
</body></html>
"""
response = requests.Response()
response.status_code = 200
response._content = html.encode("utf-8")
response.headers["content-type"] = "text/html; charset=utf-8"
session = Mock()
session.request.return_value = response
engine = OnlineSearchEngine(session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
parsed = engine.parse_comic("https://manga.example.com/c/1")
self.assertEqual(parsed.title, "魔法漫画")
self.assertEqual(parsed.chapters[0].images, ("https://manga.example.com/manga/001.webp",))
def test_http_444_is_reported_as_site_rejected(self) -> None:
response = requests.Response()
response.status_code = 444
response._content = b""
session = Mock()
session.request.return_value = response
engine = OnlineSearchEngine(session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
with self.assertRaisesRegex(Exception, "拒绝访问"):
engine.parse_comic("https://manga.example.com/c/1")
def test_popular_manga_titles_can_be_found_from_html_cards(self) -> None:
html = """
<ul>
<li class="comic-card"><a href="/comic/yirenzhixia" title="一人之下 漫画全集"><img data-src="/cover/yirenzhixia.webp"></a></li>
<li class="comic-card"><a href="/comic/naruto"><img alt="火影忍者" src="/cover/naruto.jpg"></a></li>
<li class="comic-card"><a href="/comic/fox" title="狐妖小红娘在线阅读"></a></li>
</ul>
"""
response = requests.Response()
response.status_code = 200
response._content = html.encode("utf-8")
response.headers["content-type"] = "text/html; charset=utf-8"
session = Mock()
session.request.return_value = response
manager = RuleManager(
[RuleConfig(rule_id="manga-html", name="漫画源", base_url="https://manga.example.com", resource_type="manga", search_url="/search?q={{key}}")]
)
engine = OnlineSearchEngine(manager, session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
one = engine.search_resource("一人之下", "manga")
naruto = engine.search_resource("火影忍者", "manga")
fox = engine.search_resource("狐妖小红娘", "manga")
self.assertEqual(one[0].url, "https://manga.example.com/comic/yirenzhixia")
self.assertEqual(naruto[0].cover, "https://manga.example.com/cover/naruto.jpg")
self.assertEqual(fox[0].title, "狐妖小红娘在线阅读")
def test_popular_novel_titles_can_be_found_from_json(self) -> None:
response = requests.Response()
response.status_code = 200
response._content = (
'{"data":[{"bookName":"龙族","id":"dragon-raja","author":"江南"},'
'{"bookName":"乐可","bookUrl":"/book/leke","author":"匿名"}]}'
).encode("utf-8")
response.headers["content-type"] = "application/json; charset=utf-8"
session = Mock()
session.request.return_value = response
manager = RuleManager(
[RuleConfig(rule_id="novel-json", name="小说源", base_url="https://novel.example.com", resource_type="novel", search_url="/api/search?key={{keyword}}")]
)
engine = OnlineSearchEngine(manager, session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
dragon = engine.search_resource("龙族", "novel")
leke = engine.search_resource("乐可", "novel")
self.assertEqual(dragon[0].url, "https://novel.example.com/book/dragon-raja/")
self.assertEqual(leke[0].url, "https://novel.example.com/book/leke")
def test_manga_script_images_can_be_viewed(self) -> None:
html = """
<html><body><h1>一人之下 第1话</h1>
<script>const chapterImages = ["/pages/001.webp", {"url": "//cdn.example.com/pages/002.avif"}];</script>
</body></html>
"""
response = requests.Response()
response.status_code = 200
response._content = html.encode("utf-8")
response.headers["content-type"] = "text/html; charset=utf-8"
session = Mock()
session.request.return_value = response
engine = OnlineSearchEngine(session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
parsed = engine.parse_comic("https://manga.example.com/comic/yirenzhixia/1")
self.assertEqual(parsed.chapters[0].images[0], "https://manga.example.com/pages/001.webp")
self.assertEqual(parsed.chapters[0].images[1], "https://cdn.example.com/pages/002.avif")
def test_novel_body_can_be_read_for_dragon_and_leke_like_pages(self) -> None:
html = """
<html><body><h1>龙族</h1>
<main><p>第一章 卡塞尔之门。</p><p>路明非站在雨里,故事从这里开始。</p></main>
<a href="/read/2.html">下一章</a>
</body></html>
"""
response = requests.Response()
response.status_code = 200
response._content = html.encode("utf-8")
response.headers["content-type"] = "text/html; charset=utf-8"
session = Mock()
session.request.return_value = response
engine = OnlineSearchEngine(session=session)
with patch("sakura_cloud.services.online_search_engine.assert_public_url"):
parsed = engine.parse_novel("https://novel.example.com/read/1.html")
self.assertEqual(parsed.title, "龙族")
self.assertIn("路明非", parsed.chapters[0].content or "")
self.assertEqual(parsed.chapters[1].url, "https://novel.example.com/read/2.html")
class TopLevelFunctionTest(unittest.TestCase):
def test_top_level_parse_functions_are_available(self) -> None:
self.assertTrue(callable(parse_novel))
self.assertTrue(callable(parse_comic))
if __name__ == "__main__":
unittest.main()
+78
View File
@@ -0,0 +1,78 @@
from __future__ import annotations
import unittest
from unittest.mock import patch
from sakura_cloud.services.online_search import _assert_public_url, search_sources
class OnlineSearchServiceTest(unittest.TestCase):
def test_novel_json_rule_search(self) -> None:
sources = [
{
"bookSourceName": "Novel JSON",
"bookSourceUrl": "https://novel.example.com",
"bookSourceType": 0,
"searchUrl": "/search?q={{key}}",
"ruleSearch": {
"bookList": "$.data[*]",
"name": "$.name",
"bookUrl": "$.url",
"author": "$.author",
},
}
]
body = '{"data":[{"name":"十宗罪","url":"/book/1","author":"蜘蛛"}]}'
with patch("sakura_cloud.services.online_search._assert_public_url"), patch(
"sakura_cloud.services.online_search._fetch_text",
return_value=body,
):
rows = search_sources(sources, "十宗罪", mode="novel")
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0].title, "十宗罪")
self.assertEqual(rows[0].detail_url, "https://novel.example.com/book/1")
def test_manga_json_fallback_search(self) -> None:
sources = [
{
"bookSourceName": "Manga JSON",
"bookSourceUrl": "https://manga.example.com",
"bookSourceType": 2,
"searchUrl": "/search?q={{key}}",
"ruleSearch": {"bookList": "$.missing[*]"},
}
]
body = '{"data":[{"comic_name":"樱花漫画","comic_url":"/comic/1","cover_url":"/cover.jpg"}]}'
with patch("sakura_cloud.services.online_search._assert_public_url"), patch(
"sakura_cloud.services.online_search._fetch_text",
return_value=body,
):
rows = search_sources(sources, "樱花", mode="manga")
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0].cover_url, "https://manga.example.com/cover.jpg")
def test_manga_html_card_fallback_search(self) -> None:
sources = [
{
"bookSourceName": "Manga HTML",
"bookSourceUrl": "https://manga.example.com",
"bookSourceType": 2,
"searchUrl": "/search?q={{key}}",
}
]
body = '<a href="/manga/1" title="魔法漫画"><img data-src="/cover.webp"></a>'
with patch("sakura_cloud.services.online_search._assert_public_url"), patch(
"sakura_cloud.services.online_search._fetch_text",
return_value=body,
):
rows = search_sources(sources, "魔法", mode="manga")
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0].detail_url, "https://manga.example.com/manga/1")
def test_private_url_is_rejected(self) -> None:
with self.assertRaises(Exception):
_assert_public_url("http://127.0.0.1:8000/search")
if __name__ == "__main__":
unittest.main()
+100
View File
@@ -0,0 +1,100 @@
from __future__ import annotations
import tempfile
import unittest
from unittest.mock import patch
import os
from pathlib import Path
from pixiv_app.core.auth import SessionStore
from pixiv_app.core.gallery_service import LocalGalleryService
from pixiv_app.core.library import PixivLibrary
from pixiv_app.core.origin_follow_sync import OriginFollow, OriginFollowSyncResult, OriginFollowSyncService
from pixiv_app.core.paths import app_session_file
class OriginFollowSyncLocalTest(unittest.TestCase):
def test_pixiv_origin_follow_upsert_is_idempotent(self) -> None:
with tempfile.TemporaryDirectory() as tmp:
db_path = Path(tmp) / "library.db"
service = OriginFollowSyncService(db_path=str(db_path))
result = OriginFollowSyncResult(platform="pixiv")
follows = [
OriginFollow(
platform="pixiv",
external_id="12345",
name="Sakura Artist",
handle="sakura_artist",
avatar_url="https://i.pximg.net/user-profile/img.jpg",
profile_url="https://www.pixiv.net/users/12345",
)
]
service._upsert_pixiv_local(follows, result)
second = OriginFollowSyncResult(platform="pixiv")
service._upsert_pixiv_local(follows, second)
self.assertEqual(result.created, 1)
self.assertEqual(second.skipped, 1)
library = PixivLibrary(db_path)
try:
rows = library.connection.execute("SELECT artist_id, enabled FROM followed_artists").fetchall()
self.assertEqual(len(rows), 1)
self.assertEqual(int(rows[0]["artist_id"]), 12345)
self.assertEqual(int(rows[0]["enabled"]), 1)
finally:
library.close()
def test_gallery_sync_reads_shared_app_session_cookie(self) -> None:
old_data_dir = os.environ.get("SAKURA_APP_DATA_DIR")
try:
with tempfile.TemporaryDirectory() as tmp:
os.environ["SAKURA_APP_DATA_DIR"] = tmp
db_path = Path(tmp) / "library.db"
SessionStore(app_session_file()).save({"cookie": "PHPSESSID=fake"})
library = PixivLibrary(db_path)
try:
library.set_followed_artist(12345, True, "Sakura Artist")
finally:
library.close()
service = LocalGalleryService(db_path=db_path)
with patch("pixiv_app.core.gallery_service.fetch_user_work_ids", return_value=(12345, [])) as mocked:
result = service.sync_followed_artists(download=False)
self.assertEqual(result["artists"][0]["artist_id"], 12345)
self.assertEqual(mocked.call_args.kwargs["cookie"], "PHPSESSID=fake")
finally:
if old_data_dir is None:
os.environ.pop("SAKURA_APP_DATA_DIR", None)
else:
os.environ["SAKURA_APP_DATA_DIR"] = old_data_dir
def test_gallery_sync_keeps_new_pixiv_work_local(self) -> None:
old_data_dir = os.environ.get("SAKURA_APP_DATA_DIR")
try:
with tempfile.TemporaryDirectory() as tmp:
os.environ["SAKURA_APP_DATA_DIR"] = tmp
db_path = Path(tmp) / "library.db"
SessionStore(app_session_file()).save({"cookie": "PHPSESSID=fake"})
library = PixivLibrary(db_path)
try:
library.set_followed_artist(12345, True, "Sakura Artist")
finally:
library.close()
service = LocalGalleryService(db_path=db_path)
with patch("pixiv_app.core.gallery_service.fetch_user_work_ids", return_value=(12345, [67890])):
result = service.sync_followed_artists(download=False)
self.assertEqual(result["artists"][0]["new_count"], 1)
self.assertEqual(result["artists"][0]["alerts"], 0)
self.assertFalse(result["artists"][0]["cloud_synced"])
finally:
if old_data_dir is None:
os.environ.pop("SAKURA_APP_DATA_DIR", None)
else:
os.environ["SAKURA_APP_DATA_DIR"] = old_data_dir
if __name__ == "__main__":
unittest.main()
+33
View File
@@ -0,0 +1,33 @@
from __future__ import annotations
import sys
import unittest
from pathlib import Path
from pixiv_app.core.plugin.manager import PluginManager
class PluginManagerTests(unittest.TestCase):
def test_discover_and_load_pixiv(self) -> None:
root = Path(__file__).resolve().parents[1]
plugins_dir = root / "plugins"
plugin_py = plugins_dir / "pixiv" / "plugin.py"
if not plugin_py.is_file():
self.skipTest("bundled plugins/pixiv not present")
if str(root) not in sys.path:
sys.path.insert(0, str(root))
mgr = PluginManager([plugins_dir])
paths = mgr.discover()
self.assertTrue(any(p.parent.name == "pixiv" for p in paths))
plugin = mgr.load_plugin(plugin_py)
self.assertIsNotNone(plugin)
assert plugin is not None
self.assertTrue(plugin.can_handle("https://www.pixiv.net/artworks/12345"))
mgr.unload_plugin(plugin.name)
if __name__ == "__main__":
unittest.main()
+169
View File
@@ -0,0 +1,169 @@
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest import TestCase
from fastapi import FastAPI
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, select
from sqlalchemy.orm import sessionmaker
from sakura_cloud.api.v1.router import api_router
from sakura_cloud.core.config import settings
from sakura_cloud.core.security import create_access_token
from sakura_cloud.db.base import Base
from sakura_cloud.db.session import get_db
from sakura_cloud.models.follow import UpdateAlert
from sakura_cloud.models.push import PushDeviceToken
from sakura_cloud.models.user import User, UserRole
from sakura_cloud.services.push import send_update_alert_push
class PushNotificationTest(TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.db_path = Path(self.temp_dir.name) / "cloud.db"
self.engine = create_engine(f"sqlite:///{self.db_path}", connect_args={"check_same_thread": False})
self.SessionLocal = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)
Base.metadata.create_all(self.engine)
self.app = FastAPI()
self.app.include_router(api_router, prefix=settings.api_prefix)
self.app.dependency_overrides[get_db] = self._override_db
self.client = TestClient(self.app)
self.user_a_id = "user-a"
self.user_b_id = "user-b"
with self.SessionLocal() as db:
db.add_all(
[
User(
id=self.user_a_id,
username="alice",
email="alice@example.test",
password_hash="unused",
role=UserRole.USER,
),
User(
id=self.user_b_id,
username="bob",
email="bob@example.test",
password_hash="unused",
role=UserRole.USER,
),
]
)
db.commit()
self.auth_a = {"Authorization": f"Bearer {create_access_token(self.user_a_id)}"}
self.auth_b = {"Authorization": f"Bearer {create_access_token(self.user_b_id)}"}
def tearDown(self) -> None:
self.client.close()
self.engine.dispose()
self.temp_dir.cleanup()
def _override_db(self):
db = self.SessionLocal()
try:
yield db
finally:
db.close()
def test_push_endpoints_require_login(self) -> None:
cases = [
("GET", "/api/v1/push/devices", None),
(
"POST",
"/api/v1/push/register-token",
{"device_id": "android-test", "platform": "android", "fcm_token": "token_1234567890:abcdef"},
),
("POST", "/api/v1/push/unregister-token", {"device_id": "android-test"}),
]
for method, path, body in cases:
with self.subTest(method=method, path=path):
response = self.client.request(method, path, json=body)
self.assertEqual(response.status_code, 401)
def test_register_update_and_unregister_token(self) -> None:
payload = {
"device_id": "android-test",
"platform": "android",
"fcm_token": "token_1234567890:abcdef",
}
created = self.client.post("/api/v1/push/register-token", headers=self.auth_a, json=payload)
self.assertEqual(created.status_code, 201)
self.assertEqual(created.json()["data"]["device_id"], "android-test")
updated = self.client.post(
"/api/v1/push/register-token",
headers=self.auth_a,
json={**payload, "fcm_token": "token_abcdef1234567890:updated"},
)
self.assertEqual(updated.status_code, 201)
devices = self.client.get("/api/v1/push/devices", headers=self.auth_a)
self.assertEqual(devices.status_code, 200)
self.assertEqual(len(devices.json()["data"]), 1)
self.assertTrue(devices.json()["data"][0]["is_active"])
unregistered = self.client.post(
"/api/v1/push/unregister-token",
headers=self.auth_a,
json={"device_id": "android-test"},
)
self.assertEqual(unregistered.status_code, 200)
self.assertTrue(unregistered.json()["data"]["ok"])
devices_after = self.client.get("/api/v1/push/devices", headers=self.auth_a).json()["data"]
self.assertFalse(devices_after[0]["is_active"])
def test_users_cannot_list_or_unregister_other_users_devices(self) -> None:
self.client.post(
"/api/v1/push/register-token",
headers=self.auth_b,
json={
"device_id": "android-b",
"platform": "android",
"fcm_token": "token_1234567890:bbbbbb",
},
)
devices_a = self.client.get("/api/v1/push/devices", headers=self.auth_a)
self.assertEqual(devices_a.status_code, 200)
self.assertEqual(devices_a.json()["data"], [])
self.client.post("/api/v1/push/unregister-token", headers=self.auth_a, json={"device_id": "android-b"})
devices_b = self.client.get("/api/v1/push/devices", headers=self.auth_b).json()["data"]
self.assertEqual(len(devices_b), 1)
self.assertTrue(devices_b[0]["is_active"])
def test_invalid_token_is_disabled_after_push_failure(self) -> None:
with self.SessionLocal() as db:
token = PushDeviceToken(
user_id=self.user_a_id,
device_id="android-bad",
platform="android",
fcm_token="token_1234567890:invalid",
is_active=True,
)
alert = UpdateAlert(
user_id=self.user_a_id,
platform="pixiv",
creator_id="42",
title="New Work",
body="body",
content_type="illust",
payload_json={"creator_name": "Sakura"},
)
db.add_all([token, alert])
db.commit()
db.refresh(alert)
summary = send_update_alert_push(db, alert, sender=lambda *_args: False)
self.assertEqual(summary.attempted, 1)
self.assertEqual(summary.disabled, 1)
stored = db.scalar(select(PushDeviceToken).where(PushDeviceToken.device_id == "android-bad"))
self.assertIsNotNone(stored)
self.assertFalse(stored.is_active)
+301
View File
@@ -0,0 +1,301 @@
from __future__ import annotations
import tempfile
from pathlib import Path
from unittest import TestCase
from fastapi import APIRouter, FastAPI
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sakura_cloud.api.deps import AdminUser
from sakura_cloud.api.v1.router import api_router
from sakura_cloud.core.config import settings
from sakura_cloud.core.safety import is_allowed_external_url
from sakura_cloud.core.security import create_access_token
from sakura_cloud.db.base import Base
from sakura_cloud.db.session import get_db
from sakura_cloud.models import Artwork, CreatorFollow, FileBlob, Novel, User, UserFile
from sakura_cloud.models.user import UserRole
class SakuraCloudSecurityTest(TestCase):
def setUp(self) -> None:
self.temp_dir = tempfile.TemporaryDirectory()
self.root = Path(self.temp_dir.name)
self.db_path = self.root / "cloud.db"
self.engine = create_engine(f"sqlite:///{self.db_path}", connect_args={"check_same_thread": False})
self.SessionLocal = sessionmaker(bind=self.engine, autoflush=False, autocommit=False)
Base.metadata.create_all(self.engine)
self.old_storage_root = settings.storage_root
self.old_max_upload_size_mb = settings.max_upload_size_mb
settings.storage_root = str(self.root / "storage")
settings.max_upload_size_mb = 1
admin_router = APIRouter()
@admin_router.get("/admin/probe")
def admin_probe(_: AdminUser) -> dict[str, bool]:
return {"ok": True}
self.app = FastAPI()
self.app.include_router(api_router, prefix=settings.api_prefix)
self.app.include_router(admin_router, prefix=settings.api_prefix)
self.app.dependency_overrides[get_db] = self._override_db
self.client = TestClient(self.app)
self.user_a_id = "user-a"
self.user_b_id = "user-b"
self.admin_id = "admin"
self._seed_data()
self.auth_a = {"Authorization": f"Bearer {create_access_token(self.user_a_id)}"}
self.auth_b = {"Authorization": f"Bearer {create_access_token(self.user_b_id)}"}
self.auth_admin = {"Authorization": f"Bearer {create_access_token(self.admin_id)}"}
def tearDown(self) -> None:
self.client.close()
self.engine.dispose()
settings.storage_root = self.old_storage_root
settings.max_upload_size_mb = self.old_max_upload_size_mb
self.temp_dir.cleanup()
def _override_db(self):
db = self.SessionLocal()
try:
yield db
finally:
db.close()
def _seed_data(self) -> None:
storage_file = self.root / "b-file.txt"
storage_file.write_text("secret", encoding="utf-8")
with self.SessionLocal() as db:
user_a = User(
id=self.user_a_id,
username="alice",
email="alice@example.test",
password_hash="unused",
role=UserRole.USER,
)
user_b = User(
id=self.user_b_id,
username="bob",
email="bob@example.test",
password_hash="unused",
role=UserRole.USER,
)
admin = User(
id=self.admin_id,
username="admin",
email="admin@example.test",
password_hash="unused",
role=UserRole.ADMIN,
)
blob = FileBlob(
sha256="b" * 64,
storage_path=str(storage_file),
file_size=storage_file.stat().st_size,
mime_type="text/plain",
)
user_file = UserFile(id="file-b", user_id=self.user_b_id, file_sha256=blob.sha256, original_name="b-file.txt")
artwork = Artwork(
id="art-b",
user_id=self.user_b_id,
file_id=user_file.id,
source_platform="pixiv",
content_type="illust",
title="bob artwork",
file_path=str(storage_file),
file_size=storage_file.stat().st_size,
tags=[],
artwork_metadata={},
)
novel = Novel(
id="novel-b",
user_id=self.user_b_id,
file_id=user_file.id,
source_platform="pixiv",
title="bob novel",
file_path=str(storage_file),
file_size=storage_file.stat().st_size,
tags=[],
)
series_chapter = Novel(
id="novel-series-a",
user_id=self.user_a_id,
source_platform="biquge",
title="sakura series",
series_id="series-a",
series_title="Sakura Series",
chapter_title="Chapter One",
chapter_order=1,
file_path=str(storage_file),
file_size=storage_file.stat().st_size,
tags=[],
)
follow = CreatorFollow(
id="follow-b",
user_id=self.user_b_id,
platform="pixiv",
creator_id="creator-b",
creator_name="Bob Creator",
enabled=True,
metadata_json={},
)
db.add_all([user_a, user_b, admin, blob, user_file, artwork, novel, series_chapter, follow])
db.commit()
def test_protected_resource_groups_return_401_without_login(self) -> None:
cases = [
("GET", "/api/v1/gallery"),
("GET", "/api/v1/gallery/feed"),
("GET", "/api/v1/gallery/stat"),
("POST", "/api/v1/gallery"),
("GET", "/api/v1/novels"),
("GET", "/api/v1/novels/recent"),
("GET", "/api/v1/files/file-b/content"),
("POST", "/api/v1/upload/init"),
("GET", "/api/v1/upload/status/unknown"),
("GET", "/api/v1/follows"),
("POST", "/api/v1/follows"),
("POST", "/api/v1/follows/bulk-upsert"),
("GET", "/api/v1/follows/alerts"),
("GET", "/api/v1/push/devices"),
("POST", "/api/v1/push/register-token"),
("POST", "/api/v1/push/unregister-token"),
]
for method, path in cases:
with self.subTest(method=method, path=path):
response = self.client.request(method, path, json={})
self.assertEqual(response.status_code, 401)
def test_user_a_cannot_access_user_b_gallery_novel_file_or_follow_data(self) -> None:
gallery = self.client.get("/api/v1/gallery", headers=self.auth_a)
self.assertEqual(gallery.status_code, 200)
self.assertEqual(gallery.json()["data"]["items"], [])
self.assertEqual(self.client.get("/api/v1/gallery/art-b", headers=self.auth_a).status_code, 404)
novels = self.client.get("/api/v1/novels", headers=self.auth_a)
self.assertEqual(novels.status_code, 200)
self.assertNotIn("novel-b", {item["id"] for item in novels.json()["data"]["items"]})
self.assertEqual(self.client.get("/api/v1/novels/novel-b", headers=self.auth_a).status_code, 404)
self.assertEqual(self.client.get("/api/v1/novels/novel-b/content", headers=self.auth_a).status_code, 404)
self.assertEqual(self.client.get("/api/v1/files/file-b/content", headers=self.auth_a).status_code, 404)
self.assertEqual(self.client.get("/api/v1/files/file-b/thumbnail", headers=self.auth_a).status_code, 404)
follows = self.client.get("/api/v1/follows", headers=self.auth_a)
self.assertEqual(follows.status_code, 200)
self.assertEqual(follows.json()["data"], [])
self.assertEqual(self.client.delete("/api/v1/follows/follow-b", headers=self.auth_a).status_code, 404)
self.assertEqual(self.client.get("/api/v1/gallery/art-b", headers=self.auth_b).status_code, 200)
self.assertEqual(self.client.get("/api/v1/novels/novel-b", headers=self.auth_b).status_code, 200)
self.assertEqual(self.client.get("/api/v1/files/file-b/content", headers=self.auth_b).status_code, 200)
def test_follow_bulk_upsert_uses_current_user_and_deduplicates(self) -> None:
payload = {
"items": [
{
"platform": "pixiv",
"creator_id": "origin-1",
"creator_name": "Origin One",
"creator_handle": "origin_one",
"profile_url": "https://www.pixiv.net/users/origin-1",
"metadata": {"user_id": "user-b"},
},
{
"platform": "pixiv",
"creator_id": "origin-1",
"creator_name": "Origin One Duplicate",
},
{
"platform": "x",
"creator_id": "sakura",
"creator_handle": "sakura",
"profile_url": "https://x.com/sakura",
},
]
}
response = self.client.post("/api/v1/follows/bulk-upsert", headers=self.auth_a, json=payload)
self.assertEqual(response.status_code, 200)
data = response.json()["data"]
self.assertEqual(data["created"], 2)
self.assertEqual(data["updated"], 0)
self.assertEqual(data["skipped"], 1)
follows_a = self.client.get("/api/v1/follows", headers=self.auth_a).json()["data"]
follows_b = self.client.get("/api/v1/follows", headers=self.auth_b).json()["data"]
self.assertEqual({item["creator_id"] for item in follows_a}, {"origin-1", "sakura"})
self.assertEqual({item["creator_id"] for item in follows_b}, {"creator-b"})
def test_novel_series_summary_defers_chapter_loading(self) -> None:
summary = self.client.get("/api/v1/novels/series?include_chapters=false", headers=self.auth_a)
self.assertEqual(summary.status_code, 200)
item = summary.json()["data"][0]
self.assertEqual(item["series_id"], "series-a")
self.assertEqual(item["chapter_count"], 1)
self.assertEqual(item["chapters"], [])
detail = self.client.get("/api/v1/novels/series/series-a", headers=self.auth_a)
self.assertEqual(detail.status_code, 200)
self.assertEqual(len(detail.json()["data"]["chapters"]), 1)
self.assertEqual(self.client.get("/api/v1/novels/series/series-a", headers=self.auth_b).status_code, 404)
def test_batch_delete_novels_by_series_id(self) -> None:
response = self.client.post(
"/api/v1/novels/batch-delete",
headers=self.auth_a,
json={"ids": [], "series_ids": ["series-a"]},
)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.json()["data"]["deleted"], 1)
self.assertEqual(self.client.get("/api/v1/novels/series/series-a", headers=self.auth_a).status_code, 404)
def test_upload_init_rejects_large_path_traversal_and_disallowed_mime(self) -> None:
base_payload = {
"sha256": "a" * 64,
"file_name": "safe.png",
"file_size": 10,
"mime_type": "image/png",
"resource_type": "image",
}
too_large = {**base_payload, "file_size": 2 * 1024 * 1024}
traversal = {**base_payload, "file_name": "../secret.png"}
windows_traversal = {**base_payload, "file_name": r"..\\secret.png"}
bad_mime = {**base_payload, "mime_type": "application/x-msdownload"}
self.assertEqual(self.client.post("/api/v1/upload/init", headers=self.auth_a, json=too_large).status_code, 413)
self.assertEqual(self.client.post("/api/v1/upload/init", headers=self.auth_a, json=traversal).status_code, 400)
self.assertEqual(self.client.post("/api/v1/upload/init", headers=self.auth_a, json=windows_traversal).status_code, 400)
self.assertEqual(self.client.post("/api/v1/upload/init", headers=self.auth_a, json=bad_mime).status_code, 415)
ok = self.client.post("/api/v1/upload/init", headers=self.auth_a, json=base_payload)
self.assertEqual(ok.status_code, 200)
self.assertFalse(ok.json()["data"]["exists"])
def test_external_url_collection_allows_only_whitelisted_public_domains(self) -> None:
self.assertTrue(is_allowed_external_url("https://www.pixiv.net/artworks/123"))
self.assertTrue(is_allowed_external_url("https://i.pximg.net/img-original/example.jpg"))
blocked = [
"http://127.0.0.1:8000/internal",
"http://localhost/admin",
"http://10.0.0.5/private",
"http://172.16.0.1/private",
"http://192.168.1.1/private",
"http://169.254.169.254/latest/meta-data",
"file:///etc/passwd",
"https://evil.example.test/image.jpg",
]
for url in blocked:
with self.subTest(url=url):
self.assertFalse(is_allowed_external_url(url))
def test_admin_dependency_requires_admin_role(self) -> None:
self.assertEqual(self.client.get("/api/v1/admin/probe").status_code, 401)
self.assertEqual(self.client.get("/api/v1/admin/probe", headers=self.auth_a).status_code, 403)
self.assertEqual(self.client.get("/api/v1/admin/probe", headers=self.auth_admin).status_code, 200)
+124
View File
@@ -0,0 +1,124 @@
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from pixiv_app.core.plugin.base import PluginParseError
from pixiv_app.core.plugin.manager import PluginManager
from pixiv_app.core import paths as app_paths
from plugins.taobao.plugin import TaobaoPlugin, _extract_item_id
PRODUCT_HTML = """
<html><head>
<meta property="og:title" content="测试商品 - 淘宝网" />
<meta property="og:image" content="https://img.alicdn.com/main.jpg" />
<meta name="description" content="商品描述" />
<script id="__NEXT_DATA__" type="application/json">
{"item":{"itemId":"123456789012","title":"测试商品","price":"99.00","images":[
"https://img.alicdn.com/a.jpg","//img.alicdn.com/b.webp"],"videoUrl":"https://cloud.video.taobao.com/demo.mp4"},
"seller":{"shopName":"樱花小店"}}
</script>
</head><body><img data-src="//img.alicdn.com/detail.jpg" /></body></html>
"""
class FakeResponse:
def __init__(self, url: str, *, text: str = "", content: bytes = b"media" * 100, content_type: str = "image/jpeg") -> None:
self.url = url
self.text = text
self.content = content
self.headers = {"content-type": content_type}
def raise_for_status(self) -> None:
return None
class FakeClient:
def __init__(self, *args, **kwargs) -> None:
self.headers = kwargs.get("headers", {})
def __enter__(self) -> "FakeClient":
return self
def __exit__(self, *args) -> None:
return None
def get(self, url: str, *args, **kwargs) -> FakeResponse:
if "item.taobao.com" in url:
return FakeResponse("https://item.taobao.com/item.htm?id=123456789012", text=PRODUCT_HTML, content_type="text/html")
content_type = "video/mp4" if url.endswith(".mp4") else "image/jpeg"
return FakeResponse(url, content_type=content_type)
class TaobaoPluginTests(unittest.TestCase):
def test_url_recognition_is_strict(self) -> None:
plugin = TaobaoPlugin()
self.assertTrue(plugin.can_handle("https://item.taobao.com/item.htm?id=123456789012"))
self.assertTrue(plugin.can_handle("https://detail.tmall.com/item.htm?id=123456789012"))
self.assertTrue(plugin.can_handle("https://m.tb.cn/h.example"))
self.assertFalse(plugin.can_handle("https://example.com/item.htm?id=123456789012"))
self.assertFalse(plugin.can_handle("https://taobao.com.evil.example/item.htm?id=1"))
def test_item_id_extraction(self) -> None:
self.assertEqual(_extract_item_id("https://item.taobao.com/item.htm?id=123456789012"), "123456789012")
def test_cookie_is_injected_from_local_file(self) -> None:
plugin = TaobaoPlugin()
with tempfile.TemporaryDirectory() as directory:
cookie_file = Path(directory) / "cookies.txt"
cookie_file.write_text(".taobao.com\tTRUE\t/\tTRUE\t0\ttest_cookie\ttest_value\n", encoding="utf-8")
plugin.config.cookie_file = str(cookie_file)
self.assertIn("test_cookie=test_value", plugin.get_headers().get("Cookie", ""))
@patch("plugins.taobao.plugin.httpx.Client", FakeClient)
def test_parse_product_metadata_and_media(self) -> None:
resource = TaobaoPlugin().parse("https://item.taobao.com/item.htm?id=123456789012")[0]
self.assertEqual(resource.id, "123456789012")
self.assertEqual(resource.title, "测试商品")
self.assertEqual(resource.author, "樱花小店")
self.assertEqual(resource.metadata["price"], "99.00")
self.assertGreaterEqual(len(resource.files), 4)
self.assertIn("video", {item["type"] for item in resource.files})
@patch("plugins.taobao.plugin.httpx.Client", FakeClient)
def test_download_entry(self) -> None:
plugin = TaobaoPlugin()
resource = plugin.parse("https://item.taobao.com/item.htm?id=123456789012")[0]
with tempfile.TemporaryDirectory() as directory:
paths = plugin.download(resource, Path(directory))
self.assertEqual(len(paths), len(resource.files))
self.assertTrue(all(path.is_file() for path in paths))
def test_login_challenge_is_reported(self) -> None:
plugin = TaobaoPlugin()
with patch.object(plugin, "_fetch_page", return_value=("https://item.taobao.com/item.htm?id=123456789012", "请登录并完成滑动验证")):
with self.assertRaisesRegex(PluginParseError, "登录或安全验证"):
plugin.parse("https://item.taobao.com/item.htm?id=123456789012")
def test_plugin_registration(self) -> None:
root = Path(__file__).resolve().parents[1]
manager = PluginManager([root / "plugins"])
plugin = manager.load_plugin(root / "plugins" / "taobao" / "plugin.py")
self.assertIsNotNone(plugin)
assert plugin is not None
self.assertIs(manager.get_plugin_for_url("https://detail.tmall.com/item.htm?id=123456789012"), plugin)
def test_external_plugin_root_is_scanned(self) -> None:
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
builtin = root / "bundle" / "plugins"
external = root / "dist" / "plugins"
user = root / "user" / "plugins"
with (
patch.object(app_paths, "plugins_root", return_value=builtin),
patch.object(app_paths, "external_plugins_root", return_value=external),
patch.object(app_paths, "user_plugins_root", return_value=user),
):
self.assertEqual(app_paths.plugin_roots(), [builtin, external, user])
if __name__ == "__main__":
unittest.main()
+44
View File
@@ -0,0 +1,44 @@
from __future__ import annotations
import unittest
from pixiv_app.core.cookie_import import cookies_to_playwright, parse_cookie_text
class XCookieImportTests(unittest.TestCase):
def test_netscape_x_cookie_export_is_playwright_compatible(self) -> None:
text = """# Netscape HTTP Cookie File
.x.com\tTRUE\t/\tTRUE\t1819195756\tguest_id\tv1%3Aexample
x.com\tFALSE\t/\tFALSE\t0\tlang\tzh-CN
.x.com\tTRUE\t/\tTRUE\t1809869781\tauth_token\tsecret-auth-token
.x.com\tTRUE\t/\tTRUE\t1809869782\tct0\tsecret-csrf-token
"""
cookies = parse_cookie_text(text)
playwright_cookies = cookies_to_playwright(cookies, ("x.com", "twitter.com"))
names = {item["name"] for item in playwright_cookies}
self.assertIn("auth_token", names)
self.assertIn("ct0", names)
self.assertIn("guest_id", names)
self.assertIn("lang", names)
lang_cookie = next(item for item in playwright_cookies if item["name"] == "lang")
self.assertNotIn("expires", lang_cookie)
def test_x_cookie_conversion_drops_invalid_items_and_session_expiry(self) -> None:
prepared = cookies_to_playwright(
[
{"name": "", "value": "bad", "domain": ".x.com"},
{"name": "lang", "value": "zh-CN", "domain": "x.com", "path": "/", "expires": 0},
{"name": "ct0", "value": "token", "domain": ".x.com", "path": "/", "expires": "1809869782"},
],
("x.com",),
)
self.assertEqual({item["name"] for item in prepared}, {"lang", "ct0"})
self.assertNotIn("expires", next(item for item in prepared if item["name"] == "lang"))
self.assertEqual(next(item for item in prepared if item["name"] == "ct0")["expires"], 1809869782)
if __name__ == "__main__":
unittest.main()