```
feat: 添加飞书用户模块和订阅功能支持 - 新增feishu_users模块用于处理飞书用户身份验证和权限管理 - 新增subscriptions模块用于处理订阅相关功能 - 新增personalization模块用于个性化服务 - 在alembic迁移配置中注册新的模型模块 - 在API路由器中添加feishu_users和subscriptions路由 - 实现事件调度服务的改进,包括错误处理和状态更新优化 - 添加飞书命令处理的权限检查机制 - 实现飞书应用票据事件处理 - 改进审计日志记录功能 ```
This commit is contained in:
251
.claude/specs/feishu-account-personalization/design.md
Normal file
251
.claude/specs/feishu-account-personalization/design.md
Normal file
@@ -0,0 +1,251 @@
|
|||||||
|
# 设计文档
|
||||||
|
|
||||||
|
## 概述
|
||||||
|
|
||||||
|
本设计在现有 FastAPI 模块化单体内增加飞书终端用户身份、个人画像与持久化订阅三个
|
||||||
|
业务域。内部 API 继续使用服务 API Key;飞书终端权限只能来自已经通过验签或 SDK
|
||||||
|
验证的事件,身份主键固定为 `(tenant_key, open_id)`。
|
||||||
|
|
||||||
|
平台只写自身数据库。旧业务 MySQL 保持 SELECT-only。个人定时提示词不允许调用工具、
|
||||||
|
读取公司报表或写入问答历史;群订阅也不加载创建者个人画像。
|
||||||
|
|
||||||
|
## 架构设计
|
||||||
|
|
||||||
|
### 系统架构图
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
F["飞书 webhook / 长连接"] --> V["事件验证与解密边界"]
|
||||||
|
V --> I["feishu_users<br/>身份与权限"]
|
||||||
|
I --> C["飞书命令编排"]
|
||||||
|
C --> P["personalization<br/>规则/偏好/会话"]
|
||||||
|
C --> S["subscriptions<br/>计划/投递"]
|
||||||
|
C --> A["ai_agent<br/>受限问答"]
|
||||||
|
API["内部 API Key"] --> I
|
||||||
|
API --> S
|
||||||
|
DB[("平台数据库")] --- I
|
||||||
|
DB --- P
|
||||||
|
DB --- S
|
||||||
|
T["每分钟持久化扫描"] --> S
|
||||||
|
S --> A
|
||||||
|
S --> FC["FeishuClient"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### 数据流图
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
participant F as 飞书
|
||||||
|
participant E as 事件服务
|
||||||
|
participant U as 用户服务
|
||||||
|
participant C as 命令服务
|
||||||
|
participant P as 个性化服务
|
||||||
|
participant A as AI
|
||||||
|
|
||||||
|
F->>E: 已签名消息事件
|
||||||
|
E->>E: 验签/解密、事件去重
|
||||||
|
E->>U: tenant_key + sender open_id
|
||||||
|
U->>U: 查找或自动注册普通用户
|
||||||
|
U-->>E: FeishuPrincipal
|
||||||
|
E->>C: principal + chat context + mentions
|
||||||
|
C->>P: 仅加载 principal 所有的数据
|
||||||
|
P->>A: 安全规则→公司规则→个人规则→请求→偏好/记忆→历史
|
||||||
|
A-->>C: 回答或明确不可用
|
||||||
|
C-->>F: 原会话回复
|
||||||
|
```
|
||||||
|
|
||||||
|
## 组件与接口
|
||||||
|
|
||||||
|
### `app/modules/feishu_users`
|
||||||
|
|
||||||
|
- `models.py` 定义 `FeishuUser`,公开随机 `code`,内部使用整数主键。
|
||||||
|
- `principal.py` 定义不可伪造的 `FeishuPrincipal`,包含用户编号、租户、open_id、角色、
|
||||||
|
状态、当前 chat_id/chat_type 和结构化 mentions。
|
||||||
|
- `services/identity.py` 只接收验证边界传入的身份,负责首次注册、初始管理员引导、
|
||||||
|
最后活跃时间和旧自选认领。
|
||||||
|
- `services/management.py` 负责列表、角色/状态修改、最后管理员保护和审计。
|
||||||
|
- `routes.py` 暴露服务 API Key 保护的用户管理接口;正文中的 actor/open_id 不参与授权。
|
||||||
|
|
||||||
|
### `app/modules/personalization`
|
||||||
|
|
||||||
|
- `UserPreference` 保存白名单类别:`language`、`tone`、`detail`、`topic`、`interest`。
|
||||||
|
- `AIConversation` 唯一标识“用户 + 私聊/群聊”;`AIConversationMessage` 保存消息,
|
||||||
|
每次读写时清理 30 天外记录并裁剪到最近 20 轮。
|
||||||
|
- `services/preferences.py` 提供显式偏好管理和真实 AI 可用时的结构化提取;敏感类别及
|
||||||
|
密钥样式在入库前拒绝。
|
||||||
|
- `services/conversations.py` 提供历史加载、追加、重置。
|
||||||
|
- `services/context.py` 按固定优先级组装问答上下文。
|
||||||
|
- `services/erasure.py` 实现一次性确认码和事务性“忘记我”,审计只保留随机匿名主体。
|
||||||
|
|
||||||
|
### `app/modules/subscriptions`
|
||||||
|
|
||||||
|
- `services/schedule_parser.py` 实现受控中文语法,不使用不确定的自由文本推断。
|
||||||
|
- `services/subscriptions.py` 管理创建、列表、暂停、恢复、退订、时区和安静时段。
|
||||||
|
- `services/scanner.py` 每分钟领取到期订阅,使用行锁(支持时)与唯一投递键创建投递,
|
||||||
|
并推进订阅的下一执行时间。
|
||||||
|
- `services/delivery.py` 生成受限 AI 内容并发送;个人订阅仅使用个人上下文,群订阅仅使用
|
||||||
|
系统/公司规则。HTTP 错误、429、5xx 和飞书业务 `code != 0` 都是失败。
|
||||||
|
- `tasks/` 和 `task_queue/` 仅作为执行适配层;关闭 Celery 时由扫描器直接处理持久化投递。
|
||||||
|
|
||||||
|
### 共享集成
|
||||||
|
|
||||||
|
- `FeishuEventService` 在验证后、注册事件前解析 `tenant_key/open_id`;缺失身份不创建数据。
|
||||||
|
- `FeishuCommandService.handle_text` 接收 `FeishuPrincipal | None`。内部预览接口不构造终端
|
||||||
|
用户身份,因此只能预览不涉及个人数据的安全路径。
|
||||||
|
- 管理员能力由统一权限守卫保护:公司规则、公司/财务/风险/考勤命令、用户管理、群订阅。
|
||||||
|
- `FeishuClient.send_message` 接收租户键和可选稳定 `uuid`,并把非零业务码转换为可分类失败。
|
||||||
|
- 自建应用使用 `/auth/v3/tenant_access_token/internal`;商店应用先使用最近一次已验证的
|
||||||
|
`app_ticket` 获取 `app_access_token`,再按目标 `tenant_key` 获取
|
||||||
|
`tenant_access_token`。应用令牌按应用缓存,租户令牌按 `(app_id, tenant_key)` 隔离缓存。
|
||||||
|
- 只有通过 webhook 验真或长连接 SDK 验证的 `app_ticket` 事件可以轮换持久化票据;
|
||||||
|
环境变量票据仅作为启动兜底,票据和访问令牌不得进入日志、审计或响应。
|
||||||
|
- 命令回复、卡片、图片和个人/群订阅都显式携带事件主体的 `tenant_key`;固定默认群报表
|
||||||
|
使用 `FEISHU_DEFAULT_TENANT_KEY`,不得复用另一租户的令牌。
|
||||||
|
- AI adapter context 增加每用户 session id;`noop` 明确返回不可用且不写会话、偏好或记忆。
|
||||||
|
|
||||||
|
## 数据模型
|
||||||
|
|
||||||
|
### `FeishuUser`
|
||||||
|
|
||||||
|
- `id`, `code`, `tenant_key`, `open_id`, `union_id`, `user_id`
|
||||||
|
- `role` (`user|admin`), `status` (`active|disabled`)
|
||||||
|
- `timezone`, `quiet_hours_start`, `quiet_hours_end`
|
||||||
|
- `last_active_at`, `created_at`, `updated_at`
|
||||||
|
- 唯一约束:`(tenant_key, open_id)`;`code` 全局唯一。
|
||||||
|
|
||||||
|
### `FeishuAdminBootstrapTombstone`
|
||||||
|
|
||||||
|
- 仅保存带域隔离的 `tenant_key + open_id` 不可逆摘要和创建时间。
|
||||||
|
- 不保存原始飞书身份、owner、角色或其他画像;仅用于阻止已执行“忘记我”的
|
||||||
|
配置初始管理员在重新联系时被自动重授管理员。
|
||||||
|
|
||||||
|
### `UserPreference`
|
||||||
|
|
||||||
|
- `id`, `code`, `owner_id`, `category`, `value`, `source`, `created_at`, `updated_at`
|
||||||
|
- 唯一约束:`(owner_id, category, normalized_value)`。
|
||||||
|
|
||||||
|
### `AIConversation` / `AIConversationMessage`
|
||||||
|
|
||||||
|
- 会话:`id`, `code`, `owner_id`, `chat_type`, `chat_key`, `created_at`, `updated_at`
|
||||||
|
- 消息:`id`, `conversation_id`, `role`, `content`, `created_at`
|
||||||
|
- 唯一约束:`(owner_id, chat_type, chat_key)`。
|
||||||
|
|
||||||
|
### 现有模型扩展
|
||||||
|
|
||||||
|
- `AIMemoryEntry.owner_id` 可空;公司规则 `owner_id=NULL`,个人规则/记忆必须有 owner。
|
||||||
|
- `AIMemoryEntry.kind` 区分 `company_rule|personal_rule|memory`,旧显式规则迁为
|
||||||
|
`company_rule/legacy_company`,旧无所有者自动记忆归档。
|
||||||
|
- 指纹唯一性改为 `(owner_id, fingerprint)`,公司数据使用空 owner 的独立范围。
|
||||||
|
- `MarketWatchlist.owner_id` 可空;旧 actor 暂存为 legacy claim key,首次注册时认领,
|
||||||
|
认领前个人查询不可见。
|
||||||
|
|
||||||
|
### `PushSubscription`
|
||||||
|
|
||||||
|
- `id`, `code`, `owner_id`, `target_type` (`user|chat`), `target_id`
|
||||||
|
- `prompt`, `schedule_type`, `schedule_config`, `timezone`, `next_run_at`
|
||||||
|
- `status`, `consented_at`, `last_run_at`, `created_at`, `updated_at`
|
||||||
|
- 群目标仅从当前已验证群事件写入。
|
||||||
|
|
||||||
|
### `PushDelivery`
|
||||||
|
|
||||||
|
- `id`, `code`, `subscription_id`, `scheduled_for`, `idempotency_key`, `message_uuid`
|
||||||
|
- `status`, `attempt_count`, `next_attempt_at`, `provider_message_id`
|
||||||
|
- `last_error`, `sent_at`, `created_at`, `updated_at`
|
||||||
|
- `idempotency_key` 和 `message_uuid` 全局唯一。
|
||||||
|
|
||||||
|
### `FeishuAppTicket`
|
||||||
|
|
||||||
|
- `id`, `app_id`, `app_ticket`, `received_at`, `updated_at`
|
||||||
|
- `app_id` 全局唯一;只保留当前有效票据,不保留票据历史。
|
||||||
|
|
||||||
|
## 业务流程
|
||||||
|
|
||||||
|
### 身份与权限
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
E["收到事件"] --> V{"来源已验证?"}
|
||||||
|
V -- 否 --> R["拒绝且不创建个人数据"]
|
||||||
|
V -- 是 --> K{"tenant_key/open_id 完整?"}
|
||||||
|
K -- 否 --> R
|
||||||
|
K -- 是 --> U["查找或注册普通用户"]
|
||||||
|
U --> B["应用 FEISHU_ADMIN_IDENTITIES 引导"]
|
||||||
|
B --> S{"用户有效且有权限?"}
|
||||||
|
S -- 否 --> D["拒绝并审计"]
|
||||||
|
S -- 是 --> C["执行命令"]
|
||||||
|
```
|
||||||
|
|
||||||
|
初始管理员配置按 `tenant_key:open_id` 精确匹配,多条配置均可各自完成首次引导。若该
|
||||||
|
配置身份曾执行“忘记我”,系统在同一删除事务内保留不可逆 bootstrap tombstone;再次
|
||||||
|
联系时只注册为普通用户。飞书管理员命令只读取事件中的结构化 mention open_id;内部
|
||||||
|
API 修改由服务 principal 审计。降级或停用前锁定目标并统计有效管理员,禁止移除最后
|
||||||
|
一个。
|
||||||
|
|
||||||
|
### 问答与偏好
|
||||||
|
|
||||||
|
1. 验证 principal 并按用户 + chat 建立会话。
|
||||||
|
2. 加载启用的公司规则、当前用户个人规则、偏好/兴趣、相关个人记忆和最近历史。
|
||||||
|
3. 按固定顺序传给 AI;使用由用户与会话派生的 provider session id。
|
||||||
|
4. 真实 AI 成功后保存本轮消息,再做一次受限偏好提取;定时任务跳过两步。
|
||||||
|
5. 每次读写裁剪到最近 20 轮并删除 30 天前消息。
|
||||||
|
|
||||||
|
### 计划解析
|
||||||
|
|
||||||
|
支持:
|
||||||
|
|
||||||
|
- 单次:今天/明天 HH:mm,`YYYY-MM-DD HH:mm`
|
||||||
|
- 周期:每天、工作日、每周一至周日、每月 1-31 号
|
||||||
|
- 间隔:每隔 N 分钟/小时,折算后不得短于 15 分钟
|
||||||
|
|
||||||
|
解析输出 `schedule_type + schedule_config + timezone + next_run_at(UTC)`。日期不存在、时间
|
||||||
|
已过、模糊表达、无效 IANA 时区均拒绝。月末没有目标日期时跳到下个有效月份。
|
||||||
|
|
||||||
|
### 扫描、投递与重试
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
S["每分钟扫描 next_run_at<=now"] --> L["领取订阅并创建唯一投递"]
|
||||||
|
L --> Q{"用户/订阅/频控/安静时段允许?"}
|
||||||
|
Q -- 否 --> P["跳过或延后,并推进计划"]
|
||||||
|
Q -- 是 --> G["生成受限内容"]
|
||||||
|
G --> F["以 delivery UUID 发送飞书"]
|
||||||
|
F --> O{"HTTP 与业务 code 成功?"}
|
||||||
|
O -- 是 --> X["标记 sent"]
|
||||||
|
O -- 可重试 --> B["1/5/15 分钟后 retry"]
|
||||||
|
O -- 最终失败 --> Z["标记 failed"]
|
||||||
|
```
|
||||||
|
|
||||||
|
投递尝试总计最多四次(初次 + 三次重试)。扫描器重启后继续处理
|
||||||
|
`pending/retry` 状态。唯一投递键由 `subscription_id + scheduled_for` 派生;同一键重复任务
|
||||||
|
返回已有结果,不再次调用飞书。
|
||||||
|
|
||||||
|
### 忘记我
|
||||||
|
|
||||||
|
首次命令仅保存哈希确认码与短期过期时间。确认后在一个事务内删除个人规则/记忆、偏好、
|
||||||
|
兴趣、会话、订阅及投递中的个人内容,删除身份映射;相关审计 actor/target 替换为随机
|
||||||
|
匿名标识,并清空目标、请求、响应和请求 ID。若被删除身份仍属于初始管理员配置,仅保留
|
||||||
|
不可逆 bootstrap tombstone 防止重新授予;用户再次联系时按新普通用户注册。删除成功
|
||||||
|
确认消息不再额外写入包含目标摘要或提供方响应的发送审计。
|
||||||
|
|
||||||
|
## 错误处理
|
||||||
|
|
||||||
|
- 未验证事件、缺身份、停用用户、权限不足:拒绝且只审计最小元数据。
|
||||||
|
- AI 未配置或 `noop`:返回“AI 当前不可用”,不创建虚假回答、历史、偏好或记忆。
|
||||||
|
- 计划解析失败、额度超限、手填群 id:使用可操作示例响应且不部分写入。
|
||||||
|
- 飞书 429/5xx/超时及业务非零码:分类为可重试;其他 4xx 为最终失败。
|
||||||
|
- 缺少飞书凭据且存在启用订阅:readiness 返回 degraded,不影响基础 health。
|
||||||
|
- 商店应用缺少可用 `app_ticket` 或默认租户时 readiness 返回 degraded;自建应用若启用订阅
|
||||||
|
涉及多个租户时 readiness 返回 degraded,避免把单租户令牌错误用于其他租户。
|
||||||
|
- 数据库竞争:依赖唯一约束兜底;冲突后回滚到保存点并读取已存在投递。
|
||||||
|
|
||||||
|
## 测试策略
|
||||||
|
|
||||||
|
- 单元测试:身份解析、权限矩阵、敏感偏好过滤、上下文顺序、会话裁剪、计划解析、
|
||||||
|
安静时段、下一执行时间和重试分类。
|
||||||
|
- 服务测试:两租户/两用户隔离、最后管理员保护、群订阅绑定、忘记我级联与匿名审计。
|
||||||
|
- 并发/幂等测试:多扫描器只产生一条投递、相同 UUID 不重复发送、业务非零码不成功。
|
||||||
|
- API/事件测试:验签失败不注册、内部 API Key 保护、正文 actor 不参与权限。
|
||||||
|
- 多租户认证测试:A/B 租户令牌缓存隔离、`app_ticket` 只能由已验证事件更新、文本/卡片/
|
||||||
|
图片/订阅均使用目标租户、固定群任务使用默认租户。
|
||||||
|
- 迁移测试:Alembic head 与元数据一致,旧规则/记忆/自选按既定策略迁移。
|
||||||
|
- 回归测试:现有固定群报表调度与现有内部接口继续工作。
|
||||||
213
.claude/specs/feishu-account-personalization/requirements.md
Normal file
213
.claude/specs/feishu-account-personalization/requirements.md
Normal file
@@ -0,0 +1,213 @@
|
|||||||
|
# 需求文档
|
||||||
|
|
||||||
|
## 简介
|
||||||
|
|
||||||
|
本功能为飞书用户提供基于飞书账号身份的安全问答、个人规则、长期记忆、
|
||||||
|
兴趣偏好和定时推送能力。飞书账号是终端用户权限的唯一来源;内部 API Key
|
||||||
|
继续用于服务间认证,不得代表或伪造终端用户。
|
||||||
|
|
||||||
|
本功能只允许写入平台自身的身份、权限、偏好、订阅、记忆、审计和投递状态。
|
||||||
|
现有业务 MySQL 必须保持 SELECT-only,不得增加业务数据回写、审批、付款、
|
||||||
|
绩效定级、交易或其他越权动作。
|
||||||
|
|
||||||
|
## 已确认边界
|
||||||
|
|
||||||
|
- 单个飞书应用可以服务一个或多个飞书租户。
|
||||||
|
- 飞书用户身份使用 `tenant_key + open_id` 唯一确定。
|
||||||
|
- `union_id` 和飞书 `user_id` 可以作为关联标识,但不得单独作为默认权限主键。
|
||||||
|
- `chat_id` 只表示会话或群推送目标,不表示用户身份。
|
||||||
|
- 个人数据默认不共享;公司级规则必须经过管理员权限控制。
|
||||||
|
- 用户规则和兴趣属于低信任数据,不能覆盖系统安全、只读、权限和审计规则。
|
||||||
|
- 本阶段不建设终端用户 Web 页面,不同步修改飞书通讯录,也不回写任何业务系统。
|
||||||
|
|
||||||
|
## 需求列表
|
||||||
|
|
||||||
|
### 需求 1:飞书账号身份认证
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望系统准确识别我的飞书账号,以便所有问答、
|
||||||
|
规则和订阅都归属于我本人。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 系统收到已经通过飞书验证的 webhook 或长连接消息事件 THEN 系统 SHALL
|
||||||
|
从事件中提取 `tenant_key` 和发送者 `open_id`,并解析为唯一用户身份。
|
||||||
|
2. IF 消息事件缺少 `tenant_key`、`open_id` 或未通过飞书来源验证 THEN 系统 SHALL
|
||||||
|
拒绝执行用户命令,且不得创建任何个人数据。
|
||||||
|
3. WHEN 一个合法飞书账号首次与机器人交互 THEN 系统 SHALL 为其建立默认普通用户身份,
|
||||||
|
且不得自动授予管理权限。
|
||||||
|
4. WHERE 不同租户出现相同 `open_id` THEN 系统 SHALL 将其识别为不同用户。
|
||||||
|
5. WHEN 系统处理群聊消息 THEN 系统 SHALL 以消息发送者账号作为用户身份,而不是以
|
||||||
|
`chat_id` 或群成员身份作为用户身份。
|
||||||
|
6. IF 飞书用户被停用 THEN 系统 SHALL 拒绝其所有受保护命令并记录拒绝审计。
|
||||||
|
|
||||||
|
### 需求 2:基于飞书账号的角色与权限
|
||||||
|
|
||||||
|
**用户故事:** 作为管理员,我希望权限绑定到飞书账号,以便普通用户只能操作自己的数据,
|
||||||
|
而受信任人员才能使用公司级能力。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 新飞书用户首次注册 THEN 系统 SHALL 仅授予普通用户角色。
|
||||||
|
2. WHEN 用户执行命令 THEN 系统 SHALL 根据该飞书身份的有效角色和权限决定是否允许执行。
|
||||||
|
3. IF 普通用户尝试创建、修改或停用公司级规则 THEN 系统 SHALL 拒绝操作并记录审计。
|
||||||
|
4. IF 用户尝试访问未授权的公司、项目、财务、风险或审计能力 THEN 系统 SHALL
|
||||||
|
返回不泄露受保护数据的拒绝响应。
|
||||||
|
5. WHEN 内部服务管理用户角色或状态 THEN 系统 SHALL 要求有效的内部服务认证,
|
||||||
|
且角色变更 SHALL 记录操作者、目标飞书用户、变更前后状态和时间。
|
||||||
|
6. IF 请求正文提供 `actor`、`open_id` 或其他可伪造身份字段 THEN 系统 SHALL
|
||||||
|
忽略这些字段作为权限依据。
|
||||||
|
|
||||||
|
### 需求 3:个人规则和记忆隔离
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望我的规则和历史记忆只影响我自己的回答,
|
||||||
|
以免其他用户看到或受其影响。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 用户创建规则、偏好或 AI 记忆 THEN 系统 SHALL 将记录关联到当前飞书用户所有者。
|
||||||
|
2. WHEN 系统列出、召回、更新、启停或删除个人规则与记忆 THEN 系统 SHALL
|
||||||
|
强制按当前用户所有者过滤。
|
||||||
|
3. IF 用户提交其他用户的数据编号 THEN 系统 SHALL 返回未找到或无权限,
|
||||||
|
且不得泄露该记录是否存在。
|
||||||
|
4. WHEN 两个用户保存相同内容 THEN 系统 SHALL 分别保存或去重到各自所有者范围内,
|
||||||
|
不得复用另一用户的记录。
|
||||||
|
5. WHEN 用户在群聊中创建个人规则或产生记忆 THEN 系统 SHALL 仍将其仅归属于发送者。
|
||||||
|
6. WHERE 公司级规则被应用 THEN 系统 SHALL 仅使用已经启用且具有管理员来源的公司规则。
|
||||||
|
7. WHEN 迁移现有无所有者的规则和记忆 THEN 系统 SHALL 不得把旧数据自动暴露给任意个人;
|
||||||
|
无法安全归属的数据 SHALL 进入停用或待审状态。
|
||||||
|
|
||||||
|
### 需求 4:个人规则、偏好和兴趣管理
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望通过飞书告诉机器人我的规则、表达偏好和兴趣,
|
||||||
|
并能随时查看或撤销,以便后续交流符合我的要求。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 普通用户发送学习规则命令 THEN 系统 SHALL 默认创建个人规则,而不是公司规则。
|
||||||
|
2. WHEN 有权限的管理员明确发送公司规则命令 THEN 系统 SHALL 创建公司级规则并记录审计。
|
||||||
|
3. WHEN 用户设置语言、语气、内容详略、关注主题或其他偏好 THEN 系统 SHALL
|
||||||
|
以结构化个人偏好保存并关联当前用户。
|
||||||
|
4. WHEN 用户添加或移除兴趣 THEN 系统 SHALL 更新当前用户的兴趣数据,
|
||||||
|
且现有个人股票自选 SHALL 作为市场兴趣来源之一。
|
||||||
|
5. WHEN 普通问答中出现语言、语气、详略或内容主题等白名单偏好线索且真实 AI
|
||||||
|
提供方可用 THEN 系统 SHALL 静默提取并永久保存非敏感偏好;IF 内容涉及密钥、
|
||||||
|
健康、宗教、政治、性取向、绩效或财务秘密 THEN 系统 SHALL 拒绝保存。
|
||||||
|
6. WHEN 用户请求查看、启停、修改或删除个人规则与偏好 THEN 系统 SHALL
|
||||||
|
仅操作当前用户的数据并返回明确结果。
|
||||||
|
7. IF 用户规则与安全、权限、只读或公司规则冲突 THEN 系统 SHALL 忽略冲突部分并保留审计。
|
||||||
|
|
||||||
|
### 需求 5:个性化 AI 问答
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望 AI 根据我的规则、偏好、兴趣和相关记忆回答,
|
||||||
|
同时不混入其他用户的信息。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 用户发起 AI 问答 THEN 系统 SHALL 按当前飞书身份加载公司规则、个人规则、
|
||||||
|
个人偏好、个人兴趣和当前用户的相关记忆。
|
||||||
|
2. WHEN 系统组装 AI 上下文 THEN 系统 SHALL 按照“系统安全与只读规则、公司规则、
|
||||||
|
个人明确规则、当前请求、偏好与兴趣、相关记忆”的优先顺序处理。
|
||||||
|
3. IF AI 提供方支持会话标识 THEN 系统 SHALL 为不同飞书用户使用隔离的会话标识,
|
||||||
|
不得复用全局用户会话。
|
||||||
|
4. IF AI 提供方不可用或仍为占位提供方 THEN 系统 SHALL 返回明确的不可用响应,
|
||||||
|
不得把占位内容当作真实个性化答案。
|
||||||
|
5. WHEN 问答来自群聊 THEN 系统 SHALL 回复原会话,但个人规则、偏好和记忆仍只属于发送者。
|
||||||
|
6. IF 个人规则试图要求调用未授权工具、修改业务数据或绕过安全控制 THEN 系统 SHALL
|
||||||
|
拒绝执行该要求。
|
||||||
|
|
||||||
|
### 需求 6:个人推送订阅
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望按自己的兴趣和时间订阅消息,并能暂停或退订,
|
||||||
|
以便只收到需要的内容。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 用户创建订阅 THEN 系统 SHALL 保存订阅主题、飞书接收身份、计划时间、时区、
|
||||||
|
安静时段、启用状态和用户同意时间。
|
||||||
|
2. WHEN 用户未明确创建或同意订阅 THEN 系统 SHALL 不得主动向该用户发送个人推送。
|
||||||
|
3. WHEN 用户查看订阅 THEN 系统 SHALL 仅返回当前用户的订阅。
|
||||||
|
4. WHEN 用户暂停、恢复或退订 THEN 系统 SHALL 立即更新当前用户订阅状态并记录审计。
|
||||||
|
5. IF 用户尝试修改其他用户订阅 THEN 系统 SHALL 拒绝且不泄露目标订阅信息。
|
||||||
|
6. WHEN 订阅主题支持个人兴趣过滤 THEN 系统 SHALL 使用当前用户明确保存的兴趣生成内容。
|
||||||
|
7. WHILE 当前时间位于用户安静时段内 THEN 系统 SHALL 延后非紧急推送,不得直接发送。
|
||||||
|
8. IF 订阅时间、时区、主题或接收目标无效 THEN 系统 SHALL 拒绝创建并返回可操作的错误说明。
|
||||||
|
9. WHEN 用户以受控中文时间语法创建订阅且解析成功 THEN 系统 SHALL 立即启用订阅,
|
||||||
|
返回标准化计划、下次执行时间和暂停命令。
|
||||||
|
10. IF 订阅间隔短于 15 分钟、用户已有 50 个启用订阅或当日投递将超过 96 条
|
||||||
|
THEN 系统 SHALL 拒绝创建或跳过投递并说明限制。
|
||||||
|
11. WHEN 普通用户创建订阅 THEN 系统 SHALL 固定绑定其本人 `open_id`;
|
||||||
|
WHEN 管理员在群内创建群订阅 THEN 系统 SHALL 仅绑定当前事件的 `chat_id`,
|
||||||
|
且 SHALL 拒绝用户手工提供原始 `chat_id`。
|
||||||
|
|
||||||
|
### 需求 7:可靠的个性化定时投递
|
||||||
|
|
||||||
|
**用户故事:** 作为订阅用户,我希望消息按时且不重复地送达,并在临时失败后安全重试。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 个人订阅到期 THEN 系统 SHALL 通过持久化订阅状态发现任务并生成投递请求,
|
||||||
|
不得仅依赖进程内临时任务。
|
||||||
|
2. WHEN 系统发送个人消息 THEN 系统 SHALL 默认使用当前用户的飞书 `open_id`
|
||||||
|
作为接收目标。
|
||||||
|
3. WHEN 系统处理同一用户、同一订阅和同一时间窗口 THEN 系统 SHALL 只创建一次有效投递。
|
||||||
|
4. IF 飞书返回 HTTP 错误、限流响应或非零业务状态码 THEN 系统 SHALL 将投递标记为失败,
|
||||||
|
不得记录为成功。
|
||||||
|
5. IF 投递失败属于可重试错误 THEN 系统 SHALL 按受限退避策略重试;
|
||||||
|
超过最大次数后 SHALL 进入最终失败状态。
|
||||||
|
6. WHEN 投递成功 THEN 系统 SHALL 保存飞书响应标识、发送时间和最终状态。
|
||||||
|
7. IF 多个调度器或工作进程同时扫描同一订阅 THEN 系统 SHALL 防止重复生成或重复发送消息。
|
||||||
|
8. WHILE `READ_ONLY_MODE=true` THEN 系统 SHALL 允许平台自身的订阅与投递状态写入,
|
||||||
|
但 SHALL 保持旧业务数据库只读。
|
||||||
|
|
||||||
|
### 需求 8:飞书自助命令
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户,我希望直接通过机器人管理个人规则、偏好和订阅,
|
||||||
|
无需访问后台页面。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 用户请求帮助 THEN 系统 SHALL 返回其当前权限允许使用的命令说明。
|
||||||
|
2. WHEN 用户管理个人规则 THEN 系统 SHALL 支持创建、查看、修改、启停和删除。
|
||||||
|
3. WHEN 用户管理偏好或兴趣 THEN 系统 SHALL 支持设置、查看和删除。
|
||||||
|
4. WHEN 用户管理订阅 THEN 系统 SHALL 支持创建、查看、暂停、恢复和退订。
|
||||||
|
5. WHEN 管理员执行公司级命令 THEN 系统 SHALL 在执行前验证其飞书账号权限。
|
||||||
|
6. IF 命令格式错误 THEN 系统 SHALL 返回示例格式,且不得产生部分写入。
|
||||||
|
7. IF 命令涉及敏感信息、密钥或禁止内容 THEN 系统 SHALL 拒绝保存并给出安全提示。
|
||||||
|
8. WHEN 管理员通过飞书管理用户角色或状态 THEN 系统 SHALL 只接受当前验证事件中的
|
||||||
|
结构化 `@用户` 身份,不得接受文本伪造的 `open_id`。
|
||||||
|
9. IF 操作会停用或降级最后一个有效管理员 THEN 系统 SHALL 拒绝操作。
|
||||||
|
|
||||||
|
### 需求 9:隐私、审计和用户控制
|
||||||
|
|
||||||
|
**用户故事:** 作为飞书用户和审计人员,我希望个性化数据受到保护且关键操作可追踪。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 系统认证用户、拒绝权限、修改规则、修改偏好、修改订阅或执行投递 THEN 系统 SHALL
|
||||||
|
记录必要的审计元数据。
|
||||||
|
2. WHEN 系统记录审计或错误 THEN 系统 SHALL 避免保存访问令牌、密钥和不必要的完整个人问答内容。
|
||||||
|
3. WHEN 用户请求删除个人画像和记忆 THEN 系统 SHALL 删除或匿名化可删除的个人数据,
|
||||||
|
停止其订阅,并保留满足审计要求的最小不可逆证据。
|
||||||
|
4. WHEN 用户请求查看自己的个性化数据 THEN 系统 SHALL 返回其规则、偏好、兴趣和订阅摘要。
|
||||||
|
5. IF 内部服务查询个人数据 THEN 系统 SHALL 根据服务权限限制数据范围并记录访问审计。
|
||||||
|
6. WHEN 用户发送“忘记我” THEN 系统 SHALL 返回短时有效的一次性确认码;
|
||||||
|
WHEN 同一用户发送“确认忘记我 <码>”且验证码有效 THEN 系统 SHALL 在一个事务中
|
||||||
|
删除其个人数据、停用订阅、移除身份映射并将审计主体替换为随机匿名标识。
|
||||||
|
|
||||||
|
### 需求 10:兼容性、迁移与验证
|
||||||
|
|
||||||
|
**用户故事:** 作为维护人员,我希望现有固定群推送和内部 API 保持可用,
|
||||||
|
同时安全升级到飞书用户权限体系。
|
||||||
|
|
||||||
|
#### 验收条件
|
||||||
|
|
||||||
|
1. WHEN 数据库升级 THEN 系统 SHALL 通过 Alembic 创建身份、权限、偏好、订阅和投递所需结构。
|
||||||
|
2. WHEN 升级现有数据库 THEN 系统 SHALL 保留已有审计、报表推送和业务只读数据。
|
||||||
|
3. WHEN 现有固定群定时任务运行 THEN 系统 SHALL 保持原有功能,且不得被误认为个人订阅。
|
||||||
|
4. WHEN 内部 API 使用 API Key 调用 THEN 系统 SHALL 保持服务级认证,
|
||||||
|
但不得通过请求参数伪造飞书终端用户权限。
|
||||||
|
5. WHEN 自动化测试运行 THEN 系统 SHALL 覆盖两个不同飞书用户的数据隔离、
|
||||||
|
越权拒绝、角色权限、个人问答上下文、订阅生命周期、安静时段、幂等投递、
|
||||||
|
飞书非零业务码和数据库迁移一致性。
|
||||||
|
6. WHEN 完整验证运行 THEN Ruff、Python 编译检查、Alembic 元数据一致性测试和全部 pytest
|
||||||
|
测试 SHALL 通过。
|
||||||
70
.claude/specs/feishu-account-personalization/tasks.md
Normal file
70
.claude/specs/feishu-account-personalization/tasks.md
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
# 实现任务
|
||||||
|
|
||||||
|
- [x] 1. 建立飞书用户身份与权限域
|
||||||
|
- 新增 `app/modules/feishu_users/` 的模型、schema、principal、身份与管理服务。
|
||||||
|
- 实现首次注册、初始管理员引导、角色/状态修改、最后管理员保护与审计。
|
||||||
|
- 增加用户管理 API 和身份/权限测试。
|
||||||
|
- _Requirements: 1, 2, 8.5, 8.8, 8.9, 9.1, 10.4_
|
||||||
|
|
||||||
|
- [x] 2. 将验证后的飞书事件接入用户主体
|
||||||
|
- 扩展 webhook/长连接事件提取 `tenant_key`、sender ids、chat type 和 mentions。
|
||||||
|
- 只有验证后的事件才能解析或创建 `FeishuPrincipal`;群聊始终使用发送者身份。
|
||||||
|
- 把 principal 传入命令处理并在停用/缺身份时安全拒绝。
|
||||||
|
- _Requirements: 1, 2.2, 2.4, 8.1, 9.1_
|
||||||
|
|
||||||
|
- [x] 3. 实现个人规则、偏好、兴趣和记忆隔离
|
||||||
|
- 新增 `app/modules/personalization/` 偏好模型与服务。
|
||||||
|
- 为 AI 规则/记忆和市场自选增加 owner,按 owner 查询、去重和安全认领。
|
||||||
|
- 将现有 AI rule API 固定为公司规则入口;飞书个人/公司规则命令分别授权。
|
||||||
|
- 实现个人规则、偏好、兴趣 CRUD 命令及跨用户隔离测试。
|
||||||
|
- _Requirements: 3, 4, 8.2, 8.3, 8.5, 8.7, 9.4_
|
||||||
|
|
||||||
|
- [x] 4. 实现会话与个性化 AI 问答
|
||||||
|
- 新增会话和消息模型、20 轮裁剪、30 天清理与重置命令。
|
||||||
|
- 按既定优先级组装公司规则、个人规则、请求、偏好/兴趣、记忆和会话历史。
|
||||||
|
- 隔离 provider session;`noop`/AI 不可用时不写历史、偏好或记忆。
|
||||||
|
- 实现白名单自动偏好提取与敏感画像拒绝。
|
||||||
|
- _Requirements: 3, 4.5, 5, 8.3, 8.7, 9.2_
|
||||||
|
|
||||||
|
- [x] 5. 实现订阅与受控中文计划解析
|
||||||
|
- 新增订阅/投递模型、schema 和服务。
|
||||||
|
- 支持单次、每天、工作日、每周、每月和间隔计划以及 IANA 时区。
|
||||||
|
- 实现 15 分钟下限、50 个启用订阅、每日 96 条、安静时段。
|
||||||
|
- 实现私聊固定 open_id、管理员当前群绑定以及订阅生命周期命令/API。
|
||||||
|
- _Requirements: 6, 8.4, 8.6, 8.7, 10.4_
|
||||||
|
|
||||||
|
- [x] 6. 实现持久化扫描、幂等投递和重试
|
||||||
|
- 每分钟扫描到期订阅,通过行锁和唯一投递键领取并推进下一执行时间。
|
||||||
|
- 为 Feishu client 增加 `open_id`、消息 UUID 和业务状态校验。
|
||||||
|
- 实现个人/群订阅上下文边界、1/5/15 分钟重试和无 Celery 持久化恢复。
|
||||||
|
- 接入 scheduler、Celery task/queue helper 并保持固定群报表任务不变。
|
||||||
|
- _Requirements: 6.7, 7, 10.3_
|
||||||
|
|
||||||
|
- [x] 7. 实现用户数据查看、忘记我和运行可观测性
|
||||||
|
- 实现 `我的数据`、二次确认、事务删除和审计匿名化。
|
||||||
|
- 增加用户/订阅/投递查询 API、活跃用户和投递状态指标。
|
||||||
|
- 启用订阅但缺少飞书凭据时 readiness 返回 degraded。
|
||||||
|
- _Requirements: 6.2, 9, 10.2, 10.4_
|
||||||
|
|
||||||
|
- [x] 8. 完成配置、Alembic 迁移与全量验证
|
||||||
|
- 增加安全关闭的功能开关、管理员身份配置并更新 `.env.example`。
|
||||||
|
- 创建迁移并处理旧公司规则、旧自动记忆和旧自选延迟认领。
|
||||||
|
- 补齐身份、隔离、权限、计划、并发、投递、删除和迁移回归测试。
|
||||||
|
- 运行 Ruff、compileall、完整 pytest 和迁移一致性检查。
|
||||||
|
- _Requirements: 3.7, 10_
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
T1["1 身份权限"] --> T2["2 事件主体"]
|
||||||
|
T1 --> T3["3 个人数据隔离"]
|
||||||
|
T2 --> T4["4 个性化问答"]
|
||||||
|
T3 --> T4
|
||||||
|
T1 --> T5["5 订阅与计划"]
|
||||||
|
T5 --> T6["6 扫描投递"]
|
||||||
|
T3 --> T7["7 删除与可观测性"]
|
||||||
|
T5 --> T7
|
||||||
|
T2 --> T8["8 迁移与验证"]
|
||||||
|
T4 --> T8
|
||||||
|
T6 --> T8
|
||||||
|
T7 --> T8
|
||||||
|
```
|
||||||
@@ -10,7 +10,10 @@ from app.modules.audit import models as audit_models
|
|||||||
from app.modules.business import models as business_models
|
from app.modules.business import models as business_models
|
||||||
from app.modules.events import models as event_models
|
from app.modules.events import models as event_models
|
||||||
from app.modules.feishu import models as feishu_models
|
from app.modules.feishu import models as feishu_models
|
||||||
|
from app.modules.feishu_users import models as feishu_user_models
|
||||||
from app.modules.observability import models as observability_models
|
from app.modules.observability import models as observability_models
|
||||||
|
from app.modules.personalization import models as personalization_models
|
||||||
|
from app.modules.subscriptions import models as subscription_models
|
||||||
from app.modules.workflows import models as workflow_models
|
from app.modules.workflows import models as workflow_models
|
||||||
|
|
||||||
config = context.config
|
config = context.config
|
||||||
@@ -28,7 +31,10 @@ _REGISTERED_MODEL_MODULES = (
|
|||||||
business_models,
|
business_models,
|
||||||
event_models,
|
event_models,
|
||||||
feishu_models,
|
feishu_models,
|
||||||
|
feishu_user_models,
|
||||||
observability_models,
|
observability_models,
|
||||||
|
personalization_models,
|
||||||
|
subscription_models,
|
||||||
workflow_models,
|
workflow_models,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
51
alembic/versions/202607260001_v3_runtime_reliability.py
Normal file
51
alembic/versions/202607260001_v3_runtime_reliability.py
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
"""Harden V3 heartbeat identity and concurrent updates.
|
||||||
|
|
||||||
|
Revision ID: 202607260001
|
||||||
|
Revises: 202607150001
|
||||||
|
Create Date: 2026-07-26
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "202607260001"
|
||||||
|
down_revision = "202607150001"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
HEARTBEAT_TABLE = "system_heartbeats"
|
||||||
|
HEARTBEAT_IDENTITY_CONSTRAINT = "uq_system_heartbeat_component_instance"
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"""
|
||||||
|
DELETE FROM system_heartbeats
|
||||||
|
WHERE id IN (
|
||||||
|
SELECT id
|
||||||
|
FROM (
|
||||||
|
SELECT
|
||||||
|
id,
|
||||||
|
ROW_NUMBER() OVER (
|
||||||
|
PARTITION BY component, instance_id
|
||||||
|
ORDER BY last_seen_at DESC, id DESC
|
||||||
|
) AS duplicate_rank
|
||||||
|
FROM system_heartbeats
|
||||||
|
) ranked
|
||||||
|
WHERE duplicate_rank > 1
|
||||||
|
)
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with op.batch_alter_table(HEARTBEAT_TABLE) as batch_op:
|
||||||
|
batch_op.create_unique_constraint(
|
||||||
|
HEARTBEAT_IDENTITY_CONSTRAINT,
|
||||||
|
["component", "instance_id"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table(HEARTBEAT_TABLE) as batch_op:
|
||||||
|
batch_op.drop_constraint(HEARTBEAT_IDENTITY_CONSTRAINT, type_="unique")
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
"""Add V3 workflow-event and AI-memory idempotency keys.
|
||||||
|
|
||||||
|
Revision ID: 202607260002
|
||||||
|
Revises: 202607260001
|
||||||
|
Create Date: 2026-07-26
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "202607260002"
|
||||||
|
down_revision = "202607260001"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
AI_MEMORY_TABLE = "ai_memory_entries"
|
||||||
|
AI_MEMORY_FINGERPRINT_INDEX = "ix_ai_memory_entries_fingerprint"
|
||||||
|
WORKFLOW_ACTION_TABLE = "workflow_actions"
|
||||||
|
WORKFLOW_SOURCE_EVENT_INDEX = "ix_workflow_actions_source_event_id"
|
||||||
|
WORKFLOW_SOURCE_EVENT_FOREIGN_KEY = "fk_workflow_actions_source_event_id"
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
with op.batch_alter_table(AI_MEMORY_TABLE) as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column("fingerprint", sa.String(length=64), nullable=True)
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
AI_MEMORY_FINGERPRINT_INDEX,
|
||||||
|
["fingerprint"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
with op.batch_alter_table(WORKFLOW_ACTION_TABLE) as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column("source_event_id", sa.String(length=64), nullable=True)
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
WORKFLOW_SOURCE_EVENT_INDEX,
|
||||||
|
["source_event_id"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
WORKFLOW_SOURCE_EVENT_FOREIGN_KEY,
|
||||||
|
"domain_events",
|
||||||
|
["source_event_id"],
|
||||||
|
["event_id"],
|
||||||
|
ondelete="RESTRICT",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table(WORKFLOW_ACTION_TABLE) as batch_op:
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
WORKFLOW_SOURCE_EVENT_FOREIGN_KEY,
|
||||||
|
type_="foreignkey",
|
||||||
|
)
|
||||||
|
batch_op.drop_index(WORKFLOW_SOURCE_EVENT_INDEX)
|
||||||
|
batch_op.drop_column("source_event_id")
|
||||||
|
|
||||||
|
with op.batch_alter_table(AI_MEMORY_TABLE) as batch_op:
|
||||||
|
batch_op.drop_index(AI_MEMORY_FINGERPRINT_INDEX)
|
||||||
|
batch_op.drop_column("fingerprint")
|
||||||
@@ -0,0 +1,481 @@
|
|||||||
|
"""Add Feishu identities, personalization, and durable subscriptions.
|
||||||
|
|
||||||
|
Revision ID: 202607260003
|
||||||
|
Revises: 202607260002
|
||||||
|
Create Date: 2026-07-26
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "202607260003"
|
||||||
|
down_revision = "202607260002"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
AI_MEMORY_TABLE = "ai_memory_entries"
|
||||||
|
MARKET_WATCHLIST_TABLE = "market_watchlists"
|
||||||
|
|
||||||
|
|
||||||
|
def _create_indexes(
|
||||||
|
table_name: str,
|
||||||
|
definitions: tuple[tuple[str, tuple[str, ...], bool], ...],
|
||||||
|
) -> None:
|
||||||
|
for name, columns, unique in definitions:
|
||||||
|
op.create_index(op.f(name), table_name, list(columns), unique=unique)
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
_create_feishu_users()
|
||||||
|
_upgrade_ai_memory()
|
||||||
|
_upgrade_market_watchlists()
|
||||||
|
_create_personalization_tables()
|
||||||
|
_create_subscription_tables()
|
||||||
|
|
||||||
|
|
||||||
|
def _create_feishu_users() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"feishu_users",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("code", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("tenant_key", sa.String(length=128), nullable=False),
|
||||||
|
sa.Column("open_id", sa.String(length=128), nullable=False),
|
||||||
|
sa.Column("union_id", sa.String(length=128), nullable=True),
|
||||||
|
sa.Column("user_id", sa.String(length=128), nullable=True),
|
||||||
|
sa.Column("role", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("timezone", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("quiet_hours_start", sa.Time(), nullable=True),
|
||||||
|
sa.Column("quiet_hours_end", sa.Time(), nullable=True),
|
||||||
|
sa.Column("last_active_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"tenant_key",
|
||||||
|
"open_id",
|
||||||
|
name="uq_feishu_user_tenant_open_id",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"feishu_users",
|
||||||
|
(
|
||||||
|
("ix_feishu_users_code", ("code",), True),
|
||||||
|
("ix_feishu_users_created_at", ("created_at",), False),
|
||||||
|
("ix_feishu_users_last_active_at", ("last_active_at",), False),
|
||||||
|
("ix_feishu_users_open_id", ("open_id",), False),
|
||||||
|
("ix_feishu_users_role", ("role",), False),
|
||||||
|
("ix_feishu_users_status", ("status",), False),
|
||||||
|
("ix_feishu_users_tenant_key", ("tenant_key",), False),
|
||||||
|
("ix_feishu_users_union_id", ("union_id",), False),
|
||||||
|
("ix_feishu_users_user_id", ("user_id",), False),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _upgrade_ai_memory() -> None:
|
||||||
|
op.drop_index(
|
||||||
|
"ix_ai_memory_entries_fingerprint",
|
||||||
|
table_name=AI_MEMORY_TABLE,
|
||||||
|
)
|
||||||
|
with op.batch_alter_table(AI_MEMORY_TABLE) as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column("owner_id", sa.Integer(), nullable=True)
|
||||||
|
)
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column(
|
||||||
|
"kind",
|
||||||
|
sa.String(length=32),
|
||||||
|
nullable=False,
|
||||||
|
server_default="memory",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
"fk_ai_memory_entries_owner_id",
|
||||||
|
"feishu_users",
|
||||||
|
["owner_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="CASCADE",
|
||||||
|
)
|
||||||
|
batch_op.create_unique_constraint(
|
||||||
|
"uq_ai_memory_owner_fingerprint",
|
||||||
|
["owner_id", "fingerprint"],
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
"ix_ai_memory_entries_fingerprint",
|
||||||
|
["fingerprint"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
"ix_ai_memory_entries_owner_id",
|
||||||
|
["owner_id"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
"ix_ai_memory_entries_kind",
|
||||||
|
["kind"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
connection = op.get_bind()
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"""
|
||||||
|
UPDATE ai_memory_entries
|
||||||
|
SET
|
||||||
|
owner_id = NULL,
|
||||||
|
kind = 'company_rule',
|
||||||
|
source = 'legacy_company',
|
||||||
|
status = 'active'
|
||||||
|
WHERE source = 'user_rule'
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"""
|
||||||
|
UPDATE ai_memory_entries
|
||||||
|
SET kind = 'memory', status = 'archived'
|
||||||
|
WHERE owner_id IS NULL AND source IN ('auto', 'hermes')
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with op.batch_alter_table(AI_MEMORY_TABLE) as batch_op:
|
||||||
|
batch_op.alter_column(
|
||||||
|
"kind",
|
||||||
|
existing_type=sa.String(length=32),
|
||||||
|
nullable=False,
|
||||||
|
server_default=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _upgrade_market_watchlists() -> None:
|
||||||
|
with op.batch_alter_table(MARKET_WATCHLIST_TABLE) as batch_op:
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
"uq_market_watchlist_actor_symbol",
|
||||||
|
type_="unique",
|
||||||
|
)
|
||||||
|
batch_op.add_column(sa.Column("owner_id", sa.Integer(), nullable=True))
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
"fk_market_watchlists_owner_id",
|
||||||
|
"feishu_users",
|
||||||
|
["owner_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="CASCADE",
|
||||||
|
)
|
||||||
|
batch_op.create_unique_constraint(
|
||||||
|
"uq_market_watchlist_owner_symbol",
|
||||||
|
["owner_id", "symbol"],
|
||||||
|
)
|
||||||
|
batch_op.create_index(
|
||||||
|
"ix_market_watchlists_owner_id",
|
||||||
|
["owner_id"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_personalization_tables() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"user_preferences",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("code", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("owner_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("category", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("value", sa.Text(), nullable=False),
|
||||||
|
sa.Column("normalized_value", sa.String(length=1000), nullable=False),
|
||||||
|
sa.Column("source", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["owner_id"],
|
||||||
|
["feishu_users.id"],
|
||||||
|
name="fk_user_preferences_owner_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"owner_id",
|
||||||
|
"category",
|
||||||
|
"normalized_value",
|
||||||
|
name="uq_user_preference_owner_category_value",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"user_preferences",
|
||||||
|
(
|
||||||
|
("ix_user_preferences_category", ("category",), False),
|
||||||
|
("ix_user_preferences_code", ("code",), True),
|
||||||
|
("ix_user_preferences_created_at", ("created_at",), False),
|
||||||
|
("ix_user_preferences_owner_id", ("owner_id",), False),
|
||||||
|
("ix_user_preferences_source", ("source",), False),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"ai_conversations",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("code", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("owner_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("chat_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("chat_key", sa.String(length=256), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["owner_id"],
|
||||||
|
["feishu_users.id"],
|
||||||
|
name="fk_ai_conversations_owner_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"owner_id",
|
||||||
|
"chat_type",
|
||||||
|
"chat_key",
|
||||||
|
name="uq_ai_conversation_owner_chat",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"ai_conversations",
|
||||||
|
(
|
||||||
|
("ix_ai_conversations_chat_key", ("chat_key",), False),
|
||||||
|
("ix_ai_conversations_chat_type", ("chat_type",), False),
|
||||||
|
("ix_ai_conversations_code", ("code",), True),
|
||||||
|
("ix_ai_conversations_created_at", ("created_at",), False),
|
||||||
|
("ix_ai_conversations_owner_id", ("owner_id",), False),
|
||||||
|
("ix_ai_conversations_updated_at", ("updated_at",), False),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"ai_conversation_messages",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("conversation_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("role", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("content", sa.Text(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["conversation_id"],
|
||||||
|
["ai_conversations.id"],
|
||||||
|
name="fk_ai_conversation_messages_conversation_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"ai_conversation_messages",
|
||||||
|
(
|
||||||
|
(
|
||||||
|
"ix_ai_conversation_messages_conversation_id",
|
||||||
|
("conversation_id",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
("ix_ai_conversation_messages_created_at", ("created_at",), False),
|
||||||
|
("ix_ai_conversation_messages_role", ("role",), False),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"personal_data_erasure_requests",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("owner_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("token_hash", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("expires_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["owner_id"],
|
||||||
|
["feishu_users.id"],
|
||||||
|
name="fk_personal_data_erasure_requests_owner_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"personal_data_erasure_requests",
|
||||||
|
(
|
||||||
|
(
|
||||||
|
"ix_personal_data_erasure_requests_created_at",
|
||||||
|
("created_at",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ix_personal_data_erasure_requests_expires_at",
|
||||||
|
("expires_at",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ix_personal_data_erasure_requests_owner_id",
|
||||||
|
("owner_id",),
|
||||||
|
True,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _create_subscription_tables() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"push_subscriptions",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("code", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("owner_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("target_type", sa.String(length=16), nullable=False),
|
||||||
|
sa.Column("target_id", sa.String(length=256), nullable=False),
|
||||||
|
sa.Column("prompt", sa.Text(), nullable=False),
|
||||||
|
sa.Column("schedule_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("schedule_config", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("timezone", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("next_run_at", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("status", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("consented_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("last_run_at", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("locked_by", sa.String(length=128), nullable=True),
|
||||||
|
sa.Column("locked_until", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["owner_id"],
|
||||||
|
["feishu_users.id"],
|
||||||
|
name="fk_push_subscriptions_owner_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"push_subscriptions",
|
||||||
|
(
|
||||||
|
("ix_push_subscriptions_code", ("code",), True),
|
||||||
|
("ix_push_subscriptions_created_at", ("created_at",), False),
|
||||||
|
("ix_push_subscriptions_locked_by", ("locked_by",), False),
|
||||||
|
("ix_push_subscriptions_locked_until", ("locked_until",), False),
|
||||||
|
("ix_push_subscriptions_next_run_at", ("next_run_at",), False),
|
||||||
|
("ix_push_subscriptions_owner_id", ("owner_id",), False),
|
||||||
|
("ix_push_subscriptions_schedule_type", ("schedule_type",), False),
|
||||||
|
("ix_push_subscriptions_status", ("status",), False),
|
||||||
|
("ix_push_subscriptions_target_type", ("target_type",), False),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"push_deliveries",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("code", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("subscription_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("scheduled_for", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("idempotency_key", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("message_uuid", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("attempt_count", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("next_attempt_at", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("rendered_content", sa.Text(), nullable=True),
|
||||||
|
sa.Column("provider_message_id", sa.String(length=256), nullable=True),
|
||||||
|
sa.Column("last_error", sa.Text(), nullable=True),
|
||||||
|
sa.Column("locked_by", sa.String(length=128), nullable=True),
|
||||||
|
sa.Column("locked_until", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("sent_at", sa.DateTime(), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["subscription_id"],
|
||||||
|
["push_subscriptions.id"],
|
||||||
|
name="fk_push_deliveries_subscription_id",
|
||||||
|
ondelete="CASCADE",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"subscription_id",
|
||||||
|
"scheduled_for",
|
||||||
|
name="uq_push_delivery_subscription_schedule",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
_create_indexes(
|
||||||
|
"push_deliveries",
|
||||||
|
(
|
||||||
|
("ix_push_deliveries_code", ("code",), True),
|
||||||
|
("ix_push_deliveries_created_at", ("created_at",), False),
|
||||||
|
(
|
||||||
|
"ix_push_deliveries_idempotency_key",
|
||||||
|
("idempotency_key",),
|
||||||
|
True,
|
||||||
|
),
|
||||||
|
("ix_push_deliveries_locked_by", ("locked_by",), False),
|
||||||
|
("ix_push_deliveries_locked_until", ("locked_until",), False),
|
||||||
|
("ix_push_deliveries_message_uuid", ("message_uuid",), True),
|
||||||
|
(
|
||||||
|
"ix_push_deliveries_next_attempt_at",
|
||||||
|
("next_attempt_at",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"ix_push_deliveries_provider_message_id",
|
||||||
|
("provider_message_id",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
("ix_push_deliveries_scheduled_for", ("scheduled_for",), False),
|
||||||
|
("ix_push_deliveries_sent_at", ("sent_at",), False),
|
||||||
|
("ix_push_deliveries_status", ("status",), False),
|
||||||
|
(
|
||||||
|
"ix_push_deliveries_subscription_id",
|
||||||
|
("subscription_id",),
|
||||||
|
False,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("push_deliveries")
|
||||||
|
op.drop_table("push_subscriptions")
|
||||||
|
op.drop_table("personal_data_erasure_requests")
|
||||||
|
op.drop_table("ai_conversation_messages")
|
||||||
|
op.drop_table("ai_conversations")
|
||||||
|
op.drop_table("user_preferences")
|
||||||
|
|
||||||
|
with op.batch_alter_table(MARKET_WATCHLIST_TABLE) as batch_op:
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
"uq_market_watchlist_owner_symbol",
|
||||||
|
type_="unique",
|
||||||
|
)
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
"fk_market_watchlists_owner_id",
|
||||||
|
type_="foreignkey",
|
||||||
|
)
|
||||||
|
batch_op.drop_index("ix_market_watchlists_owner_id")
|
||||||
|
batch_op.drop_column("owner_id")
|
||||||
|
batch_op.create_unique_constraint(
|
||||||
|
"uq_market_watchlist_actor_symbol",
|
||||||
|
["actor", "symbol"],
|
||||||
|
)
|
||||||
|
|
||||||
|
op.execute(
|
||||||
|
sa.text(
|
||||||
|
"""
|
||||||
|
UPDATE ai_memory_entries
|
||||||
|
SET source = 'user_rule'
|
||||||
|
WHERE owner_id IS NULL
|
||||||
|
AND kind = 'company_rule'
|
||||||
|
AND source = 'legacy_company'
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with op.batch_alter_table(AI_MEMORY_TABLE) as batch_op:
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
"uq_ai_memory_owner_fingerprint",
|
||||||
|
type_="unique",
|
||||||
|
)
|
||||||
|
batch_op.drop_constraint(
|
||||||
|
"fk_ai_memory_entries_owner_id",
|
||||||
|
type_="foreignkey",
|
||||||
|
)
|
||||||
|
batch_op.drop_index("ix_ai_memory_entries_fingerprint")
|
||||||
|
batch_op.drop_index("ix_ai_memory_entries_kind")
|
||||||
|
batch_op.drop_index("ix_ai_memory_entries_owner_id")
|
||||||
|
batch_op.drop_column("kind")
|
||||||
|
batch_op.drop_column("owner_id")
|
||||||
|
batch_op.create_index(
|
||||||
|
"ix_ai_memory_entries_fingerprint",
|
||||||
|
["fingerprint"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
op.drop_table("feishu_users")
|
||||||
43
alembic/versions/202607260004_feishu_app_tickets.py
Normal file
43
alembic/versions/202607260004_feishu_app_tickets.py
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
"""Persist verified Feishu app tickets.
|
||||||
|
|
||||||
|
Revision ID: 202607260004
|
||||||
|
Revises: 202607260003
|
||||||
|
Create Date: 2026-07-26
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "202607260004"
|
||||||
|
down_revision = "202607260003"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"feishu_app_tickets",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("app_id", sa.String(length=128), nullable=False),
|
||||||
|
sa.Column("app_ticket", sa.Text(), nullable=False),
|
||||||
|
sa.Column("received_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_feishu_app_tickets_app_id"),
|
||||||
|
"feishu_app_tickets",
|
||||||
|
["app_id"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_feishu_app_tickets_received_at"),
|
||||||
|
"feishu_app_tickets",
|
||||||
|
["received_at"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("feishu_app_tickets")
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
"""Prevent erased initial administrators from being bootstrapped again.
|
||||||
|
|
||||||
|
Revision ID: 202607260005
|
||||||
|
Revises: 202607260004
|
||||||
|
Create Date: 2026-07-27
|
||||||
|
"""
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "202607260005"
|
||||||
|
down_revision = "202607260004"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"feishu_admin_bootstrap_tombstones",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("identity_hash", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("created_at", sa.DateTime(), nullable=False),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_feishu_admin_bootstrap_tombstones_created_at"),
|
||||||
|
"feishu_admin_bootstrap_tombstones",
|
||||||
|
["created_at"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_feishu_admin_bootstrap_tombstones_identity_hash"),
|
||||||
|
"feishu_admin_bootstrap_tombstones",
|
||||||
|
["identity_hash"],
|
||||||
|
unique=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_table("feishu_admin_bootstrap_tombstones")
|
||||||
@@ -8,11 +8,13 @@ from app.modules.business.routes import router as business_router
|
|||||||
from app.modules.dashboard.routes import router as dashboard_router
|
from app.modules.dashboard.routes import router as dashboard_router
|
||||||
from app.modules.events.routes import router as events_router
|
from app.modules.events.routes import router as events_router
|
||||||
from app.modules.feishu.routes import router as feishu_router
|
from app.modules.feishu.routes import router as feishu_router
|
||||||
|
from app.modules.feishu_users.routes import router as feishu_users_router
|
||||||
from app.modules.legacy_mysql.routes import router as legacy_mysql_router
|
from app.modules.legacy_mysql.routes import router as legacy_mysql_router
|
||||||
from app.modules.market.routes import router as market_router
|
from app.modules.market.routes import router as market_router
|
||||||
from app.modules.observability.routes import router as observability_router
|
from app.modules.observability.routes import router as observability_router
|
||||||
from app.modules.reports.routes import router as reports_router
|
from app.modules.reports.routes import router as reports_router
|
||||||
from app.modules.risk.routes import router as risk_router
|
from app.modules.risk.routes import router as risk_router
|
||||||
|
from app.modules.subscriptions.routes import router as subscriptions_router
|
||||||
from app.modules.workflows.routes import router as workflows_router
|
from app.modules.workflows.routes import router as workflows_router
|
||||||
|
|
||||||
api_router = APIRouter()
|
api_router = APIRouter()
|
||||||
@@ -29,11 +31,21 @@ api_router.include_router(business_router, prefix="/business", tags=["business"]
|
|||||||
api_router.include_router(dashboard_router, prefix="/dashboard", tags=["dashboard"])
|
api_router.include_router(dashboard_router, prefix="/dashboard", tags=["dashboard"])
|
||||||
api_router.include_router(legacy_mysql_router, prefix="/integrations/mysql", tags=["mysql"])
|
api_router.include_router(legacy_mysql_router, prefix="/integrations/mysql", tags=["mysql"])
|
||||||
api_router.include_router(feishu_router, prefix="/integrations/feishu", tags=["feishu"])
|
api_router.include_router(feishu_router, prefix="/integrations/feishu", tags=["feishu"])
|
||||||
|
api_router.include_router(
|
||||||
|
feishu_users_router,
|
||||||
|
prefix="/integrations/feishu",
|
||||||
|
tags=["feishu-users"],
|
||||||
|
)
|
||||||
api_router.include_router(ai_router, prefix="/ai", tags=["ai"])
|
api_router.include_router(ai_router, prefix="/ai", tags=["ai"])
|
||||||
api_router.include_router(ai_memory_router, prefix="/ai", tags=["ai-memory"])
|
api_router.include_router(ai_memory_router, prefix="/ai", tags=["ai-memory"])
|
||||||
api_router.include_router(reports_router, prefix="/reports", tags=["reports"])
|
api_router.include_router(reports_router, prefix="/reports", tags=["reports"])
|
||||||
api_router.include_router(market_router, prefix="/market", tags=["market"])
|
api_router.include_router(market_router, prefix="/market", tags=["market"])
|
||||||
api_router.include_router(risk_router, prefix="/risks", tags=["risks"])
|
api_router.include_router(risk_router, prefix="/risks", tags=["risks"])
|
||||||
|
api_router.include_router(
|
||||||
|
subscriptions_router,
|
||||||
|
prefix="/subscriptions",
|
||||||
|
tags=["subscriptions"],
|
||||||
|
)
|
||||||
api_router.include_router(audit_router, prefix="/audit", tags=["audit"])
|
api_router.include_router(audit_router, prefix="/audit", tags=["audit"])
|
||||||
api_router.include_router(events_router, prefix="/events", tags=["events"])
|
api_router.include_router(events_router, prefix="/events", tags=["events"])
|
||||||
api_router.include_router(workflows_router, prefix="/workflows", tags=["workflows"])
|
api_router.include_router(workflows_router, prefix="/workflows", tags=["workflows"])
|
||||||
|
|||||||
76
app/application/delivery/subscriptions.py
Normal file
76
app/application/delivery/subscriptions.py
Normal file
@@ -0,0 +1,76 @@
|
|||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from app.core.constants import ActorValue
|
||||||
|
from app.core.database import SessionLocal
|
||||||
|
from app.modules.ai_agent.constants import AIResponseKey
|
||||||
|
from app.modules.ai_agent.service import AIService
|
||||||
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.subscriptions.services import (
|
||||||
|
DeliveryGenerationRequest,
|
||||||
|
DeliverySendRequest,
|
||||||
|
DeliveryService,
|
||||||
|
RetryableDeliveryError,
|
||||||
|
SubscriptionScanner,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AISubscriptionGenerator:
|
||||||
|
"""Generate side-effect-free subscription content through the configured AI."""
|
||||||
|
|
||||||
|
def __init__(self, ai: AIService):
|
||||||
|
self.ai = ai
|
||||||
|
|
||||||
|
def generate(self, request: DeliveryGenerationRequest) -> str:
|
||||||
|
result = self.ai.generate_scheduled(
|
||||||
|
request.prompt,
|
||||||
|
owner_id=request.owner_id,
|
||||||
|
group=request.use_company_rules and not request.use_personal_context,
|
||||||
|
actor=ActorValue.SCHEDULER,
|
||||||
|
)
|
||||||
|
if not result.get(AIResponseKey.OK):
|
||||||
|
raise RetryableDeliveryError("AI provider is unavailable")
|
||||||
|
return str(result[AIResponseKey.ANSWER])
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuSubscriptionSender:
|
||||||
|
"""Send a delivery with the stable Feishu UUID supplied by durable state."""
|
||||||
|
|
||||||
|
def __init__(self, feishu: FeishuService):
|
||||||
|
self.feishu = feishu
|
||||||
|
|
||||||
|
def send(self, request: DeliverySendRequest) -> dict[str, Any]:
|
||||||
|
return self.feishu.send_text(
|
||||||
|
request.text,
|
||||||
|
receive_id=request.receive_id,
|
||||||
|
receive_id_type=request.receive_id_type,
|
||||||
|
actor=ActorValue.SCHEDULER,
|
||||||
|
uuid=request.uuid,
|
||||||
|
tenant_key=request.tenant_key,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run_subscription_cycle(
|
||||||
|
*,
|
||||||
|
actor: str = ActorValue.SCHEDULER,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Materialize due windows and process pending/retry deliveries."""
|
||||||
|
|
||||||
|
_ = actor
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
created = SubscriptionScanner(db).scan_due()
|
||||||
|
delivery_service = DeliveryService(
|
||||||
|
db,
|
||||||
|
generator=AISubscriptionGenerator(AIService(db)),
|
||||||
|
sender=FeishuSubscriptionSender(FeishuService(db)),
|
||||||
|
)
|
||||||
|
processed = delivery_service.process_due()
|
||||||
|
return {
|
||||||
|
"created": [item.code for item in created],
|
||||||
|
"processed": [
|
||||||
|
{"code": item.code, "status": item.status}
|
||||||
|
for item in processed
|
||||||
|
],
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
@@ -3,9 +3,12 @@ from typing import Any
|
|||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
from sqlalchemy import or_, select
|
from sqlalchemy import or_, select, update
|
||||||
|
|
||||||
from app.application.events.handlers import EventHandlerMixin
|
from app.application.events.handlers import (
|
||||||
|
EventHandlerMixin,
|
||||||
|
UnsupportedEventTypeError,
|
||||||
|
)
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.http.pagination import bounded_limit
|
from app.core.http.pagination import bounded_limit
|
||||||
@@ -44,12 +47,11 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
worker_id: str | None = None,
|
worker_id: str | None = None,
|
||||||
preclaimed: bool = False,
|
preclaimed: bool = False,
|
||||||
) -> DomainEvent:
|
) -> DomainEvent:
|
||||||
lock_owner = worker_id or f"api:{uuid4().hex}"
|
if preclaimed:
|
||||||
record = (
|
lock_owner = worker_id or ""
|
||||||
self.get_event(event_id)
|
else:
|
||||||
if preclaimed
|
lock_owner = f"{worker_id or 'api'}:{uuid4().hex}"
|
||||||
else self._claim_event(event_id, lock_owner)
|
record = self.get_event(event_id) if preclaimed else self._claim_event(event_id, lock_owner)
|
||||||
)
|
|
||||||
if record.status == EventStatus.PROCESSED:
|
if record.status == EventStatus.PROCESSED:
|
||||||
return record
|
return record
|
||||||
if preclaimed and record.locked_by != lock_owner:
|
if preclaimed and record.locked_by != lock_owner:
|
||||||
@@ -63,30 +65,31 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
try:
|
try:
|
||||||
self._handle_event(record)
|
self._handle_event(record)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
retryable = record.attempts < self._max_attempts(record)
|
self.db.rollback()
|
||||||
record.status = EventStatus.PENDING if retryable else EventStatus.FAILED
|
record = self.get_event(event_id)
|
||||||
record.last_error = str(exc)
|
retryable = not isinstance(
|
||||||
record.next_attempt_at = (
|
exc, UnsupportedEventTypeError
|
||||||
utc_now() + timedelta(seconds=settings.event_dispatch_retry_delay_seconds)
|
) and record.attempts < self._max_attempts(record)
|
||||||
if retryable
|
return self._finalize_event(
|
||||||
else None
|
event_id,
|
||||||
|
lock_owner,
|
||||||
|
status_value=(EventStatus.PENDING if retryable else EventStatus.FAILED),
|
||||||
|
last_error=f"{type(exc).__name__}: {exc}"[:2000],
|
||||||
|
next_attempt_at=(
|
||||||
|
utc_now() + timedelta(seconds=settings.event_dispatch_retry_delay_seconds)
|
||||||
|
if retryable
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
processed_at=None,
|
||||||
)
|
)
|
||||||
record.locked_by = None
|
return self._finalize_event(
|
||||||
record.locked_until = None
|
event_id,
|
||||||
self.db.commit()
|
lock_owner,
|
||||||
self.db.refresh(record)
|
status_value=EventStatus.PROCESSED,
|
||||||
self._audit_dispatch(record)
|
last_error=None,
|
||||||
return record
|
next_attempt_at=None,
|
||||||
record.status = EventStatus.PROCESSED
|
processed_at=utc_now(),
|
||||||
record.last_error = None
|
)
|
||||||
record.processed_at = utc_now()
|
|
||||||
record.next_attempt_at = None
|
|
||||||
record.locked_by = None
|
|
||||||
record.locked_until = None
|
|
||||||
self.db.commit()
|
|
||||||
self.db.refresh(record)
|
|
||||||
self._audit_dispatch(record)
|
|
||||||
return record
|
|
||||||
|
|
||||||
def dispatch_pending(
|
def dispatch_pending(
|
||||||
self,
|
self,
|
||||||
@@ -95,7 +98,7 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
now = utc_now()
|
now = utc_now()
|
||||||
stmt = (
|
stmt = (
|
||||||
select(DomainEvent)
|
select(DomainEvent.event_id)
|
||||||
.where(
|
.where(
|
||||||
DomainEvent.status == EventStatus.PENDING,
|
DomainEvent.status == EventStatus.PENDING,
|
||||||
or_(
|
or_(
|
||||||
@@ -113,38 +116,51 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
)
|
)
|
||||||
.order_by(DomainEvent.id.asc())
|
.order_by(DomainEvent.id.asc())
|
||||||
.limit(bounded_limit(limit))
|
.limit(bounded_limit(limit))
|
||||||
.with_for_update(skip_locked=True)
|
|
||||||
)
|
)
|
||||||
records = list(self.db.execute(stmt).scalars())
|
event_ids = list(self.db.execute(stmt).scalars())
|
||||||
lock_owner = worker_id or f"worker:{uuid4().hex}"
|
lock_owner = f"{worker_id or 'worker'}:{uuid4().hex}"
|
||||||
locked_until = now + timedelta(
|
dispatched: list[dict[str, Any]] = []
|
||||||
seconds=get_settings().event_dispatch_lock_seconds
|
for event_id in event_ids:
|
||||||
)
|
try:
|
||||||
for record in records:
|
record = self.dispatch_event(
|
||||||
record.locked_by = lock_owner
|
event_id,
|
||||||
record.locked_until = locked_until
|
|
||||||
record.attempts += 1
|
|
||||||
self.db.commit()
|
|
||||||
return [
|
|
||||||
_serialize_event(
|
|
||||||
self.dispatch_event(
|
|
||||||
record.event_id,
|
|
||||||
worker_id=lock_owner,
|
worker_id=lock_owner,
|
||||||
preclaimed=True,
|
|
||||||
)
|
)
|
||||||
)
|
except HTTPException as exc:
|
||||||
for record in records
|
if exc.status_code == status.HTTP_409_CONFLICT:
|
||||||
]
|
continue
|
||||||
|
raise
|
||||||
|
dispatched.append(_serialize_event(record))
|
||||||
|
return dispatched
|
||||||
|
|
||||||
def retry_event(self, event_id: str, actor: str = ActorValue.API) -> DomainEvent:
|
def retry_event(self, event_id: str, actor: str = ActorValue.API) -> DomainEvent:
|
||||||
record = self.get_event(event_id)
|
record = self.db.execute(
|
||||||
|
select(DomainEvent).where(DomainEvent.event_id == event_id).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
self.db.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=EventErrorDetail.EVENT_NOT_FOUND,
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
if (
|
||||||
|
record.locked_by is not None
|
||||||
|
and record.locked_until is not None
|
||||||
|
and record.locked_until > now
|
||||||
|
):
|
||||||
|
self.db.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail=EventErrorDetail.EVENT_LOCKED,
|
||||||
|
)
|
||||||
if record.status == EventStatus.PROCESSED:
|
if record.status == EventStatus.PROCESSED:
|
||||||
|
self.db.rollback()
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_409_CONFLICT,
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
detail=EventErrorDetail.EVENT_NOT_RETRYABLE,
|
detail=EventErrorDetail.EVENT_NOT_RETRYABLE,
|
||||||
)
|
)
|
||||||
record.status = EventStatus.PENDING
|
record.status = EventStatus.PENDING
|
||||||
record.actor = actor
|
|
||||||
record.attempts = 0
|
record.attempts = 0
|
||||||
record.max_attempts = record.max_attempts or get_settings().event_dispatch_max_attempts
|
record.max_attempts = record.max_attempts or get_settings().event_dispatch_max_attempts
|
||||||
record.last_error = None
|
record.last_error = None
|
||||||
@@ -155,8 +171,48 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
self.db.refresh(record)
|
self.db.refresh(record)
|
||||||
return record
|
return record
|
||||||
|
|
||||||
def _audit_dispatch(self, record: DomainEvent) -> None:
|
def _finalize_event(
|
||||||
AuditService(self.db).log(
|
self,
|
||||||
|
event_id: str,
|
||||||
|
lock_owner: str,
|
||||||
|
*,
|
||||||
|
status_value: str,
|
||||||
|
last_error: str | None,
|
||||||
|
next_attempt_at: Any,
|
||||||
|
processed_at: Any,
|
||||||
|
) -> DomainEvent:
|
||||||
|
result = self.db.execute(
|
||||||
|
update(DomainEvent)
|
||||||
|
.where(
|
||||||
|
DomainEvent.event_id == event_id,
|
||||||
|
DomainEvent.locked_by == lock_owner,
|
||||||
|
DomainEvent.status == EventStatus.PENDING,
|
||||||
|
)
|
||||||
|
.values(
|
||||||
|
status=status_value,
|
||||||
|
last_error=last_error,
|
||||||
|
next_attempt_at=next_attempt_at,
|
||||||
|
processed_at=processed_at,
|
||||||
|
locked_by=None,
|
||||||
|
locked_until=None,
|
||||||
|
)
|
||||||
|
.execution_options(synchronize_session=False)
|
||||||
|
)
|
||||||
|
if result.rowcount != 1:
|
||||||
|
self.db.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail=EventErrorDetail.EVENT_LOCKED,
|
||||||
|
)
|
||||||
|
self.db.expire_all()
|
||||||
|
record = self.get_event(event_id)
|
||||||
|
self._stage_dispatch_audit(record)
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return record
|
||||||
|
|
||||||
|
def _stage_dispatch_audit(self, record: DomainEvent) -> None:
|
||||||
|
AuditService(self.db).record(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=record.actor,
|
actor=record.actor,
|
||||||
source=AuditSource.EVENTS,
|
source=AuditSource.EVENTS,
|
||||||
@@ -198,9 +254,7 @@ class EventDispatchService(EventHandlerMixin):
|
|||||||
detail=EventErrorDetail.EVENT_LOCKED,
|
detail=EventErrorDetail.EVENT_LOCKED,
|
||||||
)
|
)
|
||||||
record.locked_by = lock_owner
|
record.locked_by = lock_owner
|
||||||
record.locked_until = now + timedelta(
|
record.locked_until = now + timedelta(seconds=get_settings().event_dispatch_lock_seconds)
|
||||||
seconds=get_settings().event_dispatch_lock_seconds
|
|
||||||
)
|
|
||||||
record.status = EventStatus.PENDING
|
record.status = EventStatus.PENDING
|
||||||
record.attempts += 1
|
record.attempts += 1
|
||||||
self.db.commit()
|
self.db.commit()
|
||||||
|
|||||||
@@ -6,6 +6,10 @@ from app.modules.events.constants import (
|
|||||||
from app.modules.events.models import DomainEvent
|
from app.modules.events.models import DomainEvent
|
||||||
|
|
||||||
|
|
||||||
|
class UnsupportedEventTypeError(ValueError):
|
||||||
|
"""Raised when no application handler is registered for an event type."""
|
||||||
|
|
||||||
|
|
||||||
class EventHandlerMixin:
|
class EventHandlerMixin:
|
||||||
def _handle_event(self, record: DomainEvent) -> None:
|
def _handle_event(self, record: DomainEvent) -> None:
|
||||||
if record.event_type == EventType.RISK_ACTION_RECORDED:
|
if record.event_type == EventType.RISK_ACTION_RECORDED:
|
||||||
@@ -30,6 +34,9 @@ class EventHandlerMixin:
|
|||||||
if record.event_type == EventType.ENTERPRISE_ANALYTICS_GENERATED:
|
if record.event_type == EventType.ENTERPRISE_ANALYTICS_GENERATED:
|
||||||
self._handle_enterprise_analytics_event(record)
|
self._handle_enterprise_analytics_event(record)
|
||||||
return
|
return
|
||||||
|
raise UnsupportedEventTypeError(
|
||||||
|
f"Unsupported domain event type: {record.event_type}"
|
||||||
|
)
|
||||||
|
|
||||||
def _handle_risk_action(self, record: DomainEvent) -> None:
|
def _handle_risk_action(self, record: DomainEvent) -> None:
|
||||||
from app.modules.risk.constants import RiskEventActionValue
|
from app.modules.risk.constants import RiskEventActionValue
|
||||||
@@ -53,6 +60,7 @@ class EventHandlerMixin:
|
|||||||
actor=record.actor,
|
actor=record.actor,
|
||||||
payload=payload,
|
payload=payload,
|
||||||
commit=False,
|
commit=False,
|
||||||
|
source_event_id=record.event_id,
|
||||||
)
|
)
|
||||||
|
|
||||||
def _handle_report_event(self, record: DomainEvent) -> None:
|
def _handle_report_event(self, record: DomainEvent) -> None:
|
||||||
@@ -125,4 +133,5 @@ class EventHandlerMixin:
|
|||||||
actor=record.actor,
|
actor=record.actor,
|
||||||
payload=record.payload or {},
|
payload=record.payload or {},
|
||||||
commit=False,
|
commit=False,
|
||||||
|
source_event_id=record.event_id,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -6,15 +6,23 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from app.application.feishu.delivery import send_card_if_configured, send_text_if_configured
|
from app.application.feishu.delivery import send_card_if_configured, send_text_if_configured
|
||||||
from app.application.feishu.handlers import (
|
from app.application.feishu.handlers import (
|
||||||
|
handle_admin_command,
|
||||||
handle_finance_command,
|
handle_finance_command,
|
||||||
handle_market_command,
|
handle_market_command,
|
||||||
|
handle_personal_data_command,
|
||||||
|
handle_personalization_command,
|
||||||
handle_rule_command,
|
handle_rule_command,
|
||||||
|
handle_subscription_command,
|
||||||
|
is_admin_command,
|
||||||
|
is_company_rule_command,
|
||||||
)
|
)
|
||||||
from app.application.feishu.results import command_result
|
from app.application.feishu.results import command_result
|
||||||
|
from app.core.config import get_settings
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.modules.ai_agent.constants import AIResponseKey
|
from app.modules.ai_agent.constants import AIResponseKey
|
||||||
from app.modules.ai_agent.service import AIService
|
from app.modules.ai_agent.service import AIService
|
||||||
from app.modules.audit.constants import AuditSource
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
from app.modules.feishu.constants import (
|
from app.modules.feishu.constants import (
|
||||||
FEISHU_AI_REPLY_TITLE,
|
FEISHU_AI_REPLY_TITLE,
|
||||||
FEISHU_MENTION_PATTERN,
|
FEISHU_MENTION_PATTERN,
|
||||||
@@ -25,6 +33,12 @@ from app.modules.feishu.constants import (
|
|||||||
FeishuReplyType,
|
FeishuReplyType,
|
||||||
)
|
)
|
||||||
from app.modules.feishu.service import FeishuService
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FEISHU_USER_TARGET_TYPE,
|
||||||
|
FeishuCapability,
|
||||||
|
FeishuUserAuditAction,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.principal import FeishuMention, FeishuPrincipal
|
||||||
from app.modules.reports.constants import ReportResponseKey
|
from app.modules.reports.constants import ReportResponseKey
|
||||||
from app.modules.reports.services import ReportService
|
from app.modules.reports.services import ReportService
|
||||||
|
|
||||||
@@ -34,6 +48,18 @@ ATTENDANCE_KEYWORDS = ("打卡", "考勤", "attendance")
|
|||||||
RISK_KEYWORDS = ("风险", "预警", "risk")
|
RISK_KEYWORDS = ("风险", "预警", "risk")
|
||||||
AI_COMMAND_PREFIXES = ("问 ", "ai ", "AI ", "/ask ")
|
AI_COMMAND_PREFIXES = ("问 ", "ai ", "AI ", "/ask ")
|
||||||
DEFAULT_AI_PROMPT = "请说明你能做什么。"
|
DEFAULT_AI_PROMPT = "请说明你能做什么。"
|
||||||
|
PERMISSION_DENIED_TITLE = "权限不足"
|
||||||
|
COMPANY_RULE_COMMAND_PREFIXES = (
|
||||||
|
"学习公司规则",
|
||||||
|
"查看公司规则",
|
||||||
|
"修改公司规则",
|
||||||
|
"启用公司规则",
|
||||||
|
"停用公司规则",
|
||||||
|
"删除公司规则",
|
||||||
|
"学习公司市场规则",
|
||||||
|
"查看公司市场规则",
|
||||||
|
)
|
||||||
|
FINANCE_COMMAND_PREFIXES = ("资金需求", "未来30天资金需求", "项目资金 ")
|
||||||
|
|
||||||
|
|
||||||
def _parse_content_text(content: Any) -> str:
|
def _parse_content_text(content: Any) -> str:
|
||||||
@@ -70,12 +96,14 @@ class FeishuCommandService:
|
|||||||
self.feishu = FeishuService(db)
|
self.feishu = FeishuService(db)
|
||||||
|
|
||||||
def extract_event_command(self, payload: dict[str, Any]) -> dict[str, Any] | None:
|
def extract_event_command(self, payload: dict[str, Any]) -> dict[str, Any] | None:
|
||||||
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
event = payload.get(FeishuPayloadKey.EVENT) or {}
|
event = payload.get(FeishuPayloadKey.EVENT) or {}
|
||||||
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
||||||
if not message:
|
if not message:
|
||||||
return None
|
return None
|
||||||
text = _clean_command_text(_parse_content_text(message.get(FeishuPayloadKey.CONTENT)))
|
raw_text = _parse_content_text(message.get(FeishuPayloadKey.CONTENT))
|
||||||
if not text:
|
command_text = _clean_command_text(raw_text)
|
||||||
|
if not command_text:
|
||||||
return None
|
return None
|
||||||
sender = event.get(FeishuPayloadKey.SENDER) or {}
|
sender = event.get(FeishuPayloadKey.SENDER) or {}
|
||||||
sender_id = sender.get(FeishuPayloadKey.SENDER_ID) or {}
|
sender_id = sender.get(FeishuPayloadKey.SENDER_ID) or {}
|
||||||
@@ -85,9 +113,18 @@ class FeishuCommandService:
|
|||||||
or ActorValue.FEISHU
|
or ActorValue.FEISHU
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
FeishuCommandKey.TEXT: text,
|
FeishuCommandKey.TEXT: (
|
||||||
|
raw_text
|
||||||
|
if get_settings().feishu_user_features_enabled
|
||||||
|
else command_text
|
||||||
|
),
|
||||||
FeishuCommandKey.CHAT_ID: message.get(FeishuCommandKey.CHAT_ID),
|
FeishuCommandKey.CHAT_ID: message.get(FeishuCommandKey.CHAT_ID),
|
||||||
|
FeishuCommandKey.CHAT_TYPE: message.get(FeishuPayloadKey.CHAT_TYPE),
|
||||||
FeishuCommandKey.ACTOR: actor,
|
FeishuCommandKey.ACTOR: actor,
|
||||||
|
FeishuCommandKey.MENTIONS: _parse_mentions(
|
||||||
|
message.get(FeishuPayloadKey.MENTIONS),
|
||||||
|
str(header.get(FeishuPayloadKey.TENANT_KEY) or ""),
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
def handle_text(
|
def handle_text(
|
||||||
@@ -96,14 +133,99 @@ class FeishuCommandService:
|
|||||||
chat_id: str | None = None,
|
chat_id: str | None = None,
|
||||||
actor: str = ActorValue.FEISHU,
|
actor: str = ActorValue.FEISHU,
|
||||||
auto_reply: bool = True,
|
auto_reply: bool = True,
|
||||||
|
principal: FeishuPrincipal | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
if tenant_key is not None:
|
||||||
|
self.feishu.set_tenant_key(tenant_key)
|
||||||
|
raw_text = text or ""
|
||||||
command_text = _clean_command_text(text)
|
command_text = _clean_command_text(text)
|
||||||
lowered = command_text.lower()
|
lowered = command_text.lower()
|
||||||
|
if get_settings().feishu_user_features_enabled:
|
||||||
|
if principal is None:
|
||||||
|
return self._permission_denied(
|
||||||
|
chat_id=chat_id,
|
||||||
|
actor=actor,
|
||||||
|
auto_reply=False,
|
||||||
|
principal=None,
|
||||||
|
reason="missing_verified_identity",
|
||||||
|
content="无法确认飞书账号身份,已拒绝执行该命令。",
|
||||||
|
)
|
||||||
|
self.feishu.set_tenant_key(principal.tenant_key)
|
||||||
|
chat_id = principal.chat_id or chat_id
|
||||||
|
actor = principal.user_code
|
||||||
|
if not principal.is_active:
|
||||||
|
return self._permission_denied(
|
||||||
|
chat_id=chat_id,
|
||||||
|
actor=actor,
|
||||||
|
auto_reply=auto_reply,
|
||||||
|
principal=principal,
|
||||||
|
reason="disabled_user",
|
||||||
|
content="当前飞书账号已停用,请联系管理员。",
|
||||||
|
)
|
||||||
|
required_capability = _required_capability(command_text)
|
||||||
|
if (
|
||||||
|
required_capability is not None
|
||||||
|
and not principal.has_capability(required_capability)
|
||||||
|
):
|
||||||
|
return self._permission_denied(
|
||||||
|
chat_id=chat_id,
|
||||||
|
actor=actor,
|
||||||
|
auto_reply=auto_reply,
|
||||||
|
principal=principal,
|
||||||
|
reason=f"missing_capability:{required_capability}",
|
||||||
|
content="当前飞书账号无权使用该公司级功能。",
|
||||||
|
)
|
||||||
|
admin_result = handle_admin_command(
|
||||||
|
self.db,
|
||||||
|
self.feishu,
|
||||||
|
raw_text=raw_text,
|
||||||
|
command_text=command_text,
|
||||||
|
principal=principal,
|
||||||
|
auto_reply=auto_reply,
|
||||||
|
)
|
||||||
|
if admin_result is not None:
|
||||||
|
return admin_result
|
||||||
|
for handler in (
|
||||||
|
handle_subscription_command,
|
||||||
|
handle_personal_data_command,
|
||||||
|
handle_personalization_command,
|
||||||
|
):
|
||||||
|
result = handler(
|
||||||
|
self.db,
|
||||||
|
self.feishu,
|
||||||
|
command_text,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
|
||||||
|
rule_result = handle_rule_command(
|
||||||
|
self.db,
|
||||||
|
self.feishu,
|
||||||
|
command_text,
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
principal=principal,
|
||||||
|
)
|
||||||
|
if rule_result is not None:
|
||||||
|
return rule_result
|
||||||
|
market_result = handle_market_command(
|
||||||
|
self.db,
|
||||||
|
self.feishu,
|
||||||
|
command_text,
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
principal=principal,
|
||||||
|
)
|
||||||
|
if market_result is not None:
|
||||||
|
return market_result
|
||||||
|
|
||||||
for handler in (
|
for handler in (
|
||||||
handle_rule_command,
|
|
||||||
handle_finance_command,
|
handle_finance_command,
|
||||||
handle_market_command,
|
|
||||||
):
|
):
|
||||||
result = handler(
|
result = handler(
|
||||||
self.db,
|
self.db,
|
||||||
@@ -115,11 +237,65 @@ class FeishuCommandService:
|
|||||||
)
|
)
|
||||||
if result is not None:
|
if result is not None:
|
||||||
return result
|
return result
|
||||||
|
if not get_settings().feishu_user_features_enabled:
|
||||||
|
for handler in (handle_rule_command, handle_market_command):
|
||||||
|
result = handler(
|
||||||
|
self.db,
|
||||||
|
self.feishu,
|
||||||
|
command_text,
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if result is not None:
|
||||||
|
return result
|
||||||
|
|
||||||
report_result = self._handle_report_command(command_text, chat_id, actor, auto_reply)
|
report_result = self._handle_report_command(command_text, chat_id, actor, auto_reply)
|
||||||
if report_result is not None:
|
if report_result is not None:
|
||||||
return report_result
|
return report_result
|
||||||
return self._handle_ai_command(command_text, lowered, chat_id, actor, auto_reply)
|
return self._handle_ai_command(
|
||||||
|
command_text,
|
||||||
|
lowered,
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
principal=principal,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _permission_denied(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
chat_id: str | None,
|
||||||
|
actor: str,
|
||||||
|
auto_reply: bool,
|
||||||
|
principal: FeishuPrincipal | None,
|
||||||
|
reason: str,
|
||||||
|
content: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
self.feishu.audit.log(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=FeishuUserAuditAction.PERMISSION_DENIED,
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
target_id=principal.user_code if principal else None,
|
||||||
|
risk_level=AuditRiskLevel.MEDIUM,
|
||||||
|
response_payload={"result": "denied", "reason": reason},
|
||||||
|
status="denied",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
response = (
|
||||||
|
send_text_if_configured(self.feishu, chat_id, content, actor)
|
||||||
|
if auto_reply
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
return command_result(
|
||||||
|
FeishuCommandName.PERMISSION_DENIED,
|
||||||
|
FeishuReplyType.TEXT,
|
||||||
|
PERMISSION_DENIED_TITLE,
|
||||||
|
content,
|
||||||
|
response,
|
||||||
|
)
|
||||||
|
|
||||||
def _handle_report_command(
|
def _handle_report_command(
|
||||||
self,
|
self,
|
||||||
@@ -170,6 +346,7 @@ class FeishuCommandService:
|
|||||||
chat_id: str | None,
|
chat_id: str | None,
|
||||||
actor: str,
|
actor: str,
|
||||||
auto_reply: bool,
|
auto_reply: bool,
|
||||||
|
principal: FeishuPrincipal | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
prompt = command_text
|
prompt = command_text
|
||||||
for prefix in AI_COMMAND_PREFIXES:
|
for prefix in AI_COMMAND_PREFIXES:
|
||||||
@@ -178,12 +355,23 @@ class FeishuCommandService:
|
|||||||
break
|
break
|
||||||
if not prompt:
|
if not prompt:
|
||||||
prompt = DEFAULT_AI_PROMPT
|
prompt = DEFAULT_AI_PROMPT
|
||||||
ai_result = AIService(self.db).ask(
|
if get_settings().feishu_user_features_enabled and principal is not None:
|
||||||
prompt,
|
chat_type, chat_key = _principal_chat_context(principal)
|
||||||
context={},
|
ai_result = AIService(self.db).ask_personalized(
|
||||||
actor=actor,
|
principal.owner_id,
|
||||||
source=AuditSource.FEISHU,
|
chat_type,
|
||||||
)
|
chat_key,
|
||||||
|
prompt,
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
ai_result = AIService(self.db).ask(
|
||||||
|
prompt,
|
||||||
|
context={},
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
)
|
||||||
content = ai_result[AIResponseKey.ANSWER]
|
content = ai_result[AIResponseKey.ANSWER]
|
||||||
is_explicit_ai = any(
|
is_explicit_ai = any(
|
||||||
command_text.startswith(prefix) or lowered.startswith(prefix)
|
command_text.startswith(prefix) or lowered.startswith(prefix)
|
||||||
@@ -199,3 +387,76 @@ class FeishuCommandService:
|
|||||||
content,
|
content,
|
||||||
response,
|
response,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_mentions(value: Any, tenant_key: str) -> tuple[FeishuMention, ...]:
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return ()
|
||||||
|
mentions: list[FeishuMention] = []
|
||||||
|
for item in value:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
mention_id = item.get(FeishuPayloadKey.ID) or {}
|
||||||
|
if not isinstance(mention_id, dict):
|
||||||
|
mention_id = {}
|
||||||
|
mentions.append(
|
||||||
|
FeishuMention(
|
||||||
|
key=_optional_text(item.get(FeishuPayloadKey.KEY)),
|
||||||
|
name=_optional_text(item.get(FeishuPayloadKey.NAME)),
|
||||||
|
tenant_key=(
|
||||||
|
_optional_text(item.get(FeishuPayloadKey.TENANT_KEY))
|
||||||
|
or tenant_key
|
||||||
|
or None
|
||||||
|
),
|
||||||
|
open_id=_optional_text(
|
||||||
|
mention_id.get(FeishuPayloadKey.OPEN_ID)
|
||||||
|
or item.get(FeishuPayloadKey.OPEN_ID)
|
||||||
|
),
|
||||||
|
union_id=_optional_text(
|
||||||
|
mention_id.get(FeishuPayloadKey.UNION_ID)
|
||||||
|
or item.get(FeishuPayloadKey.UNION_ID)
|
||||||
|
),
|
||||||
|
user_id=_optional_text(
|
||||||
|
mention_id.get(FeishuPayloadKey.USER_ID)
|
||||||
|
or item.get(FeishuPayloadKey.USER_ID)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return tuple(mentions)
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_text(value: Any) -> str | None:
|
||||||
|
text = str(value or "").strip()
|
||||||
|
return text or None
|
||||||
|
|
||||||
|
|
||||||
|
def _required_capability(command_text: str) -> FeishuCapability | None:
|
||||||
|
if is_admin_command(command_text):
|
||||||
|
return FeishuCapability.USER_ADMINISTRATION
|
||||||
|
if (
|
||||||
|
command_text.startswith(COMPANY_RULE_COMMAND_PREFIXES)
|
||||||
|
or is_company_rule_command(command_text)
|
||||||
|
):
|
||||||
|
return FeishuCapability.COMPANY_RULES
|
||||||
|
if command_text.startswith(FINANCE_COMMAND_PREFIXES):
|
||||||
|
return FeishuCapability.COMPANY_REPORTS
|
||||||
|
if any(
|
||||||
|
keyword in command_text
|
||||||
|
for keyword in (
|
||||||
|
*DAILY_REPORT_KEYWORDS,
|
||||||
|
*PROJECT_WEEKLY_KEYWORDS,
|
||||||
|
*ATTENDANCE_KEYWORDS,
|
||||||
|
*RISK_KEYWORDS,
|
||||||
|
)
|
||||||
|
):
|
||||||
|
return FeishuCapability.COMPANY_REPORTS
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _principal_chat_context(principal: FeishuPrincipal) -> tuple[str, str]:
|
||||||
|
chat_type = principal.chat_type or "p2p"
|
||||||
|
if chat_type in {"group", "group_chat"}:
|
||||||
|
if not principal.chat_id:
|
||||||
|
raise ValueError("Verified group chat is missing chat_id")
|
||||||
|
return chat_type, principal.chat_id
|
||||||
|
return chat_type, principal.chat_id or principal.open_id
|
||||||
|
|||||||
@@ -9,13 +9,20 @@ def send_text_if_configured(
|
|||||||
chat_id: str | None,
|
chat_id: str | None,
|
||||||
text: str,
|
text: str,
|
||||||
actor: str,
|
actor: str,
|
||||||
|
*,
|
||||||
|
record_audit: bool = True,
|
||||||
) -> dict[str, Any] | None:
|
) -> dict[str, Any] | None:
|
||||||
"""Send a text reply only when Feishu credentials are configured."""
|
"""Send a text reply only when Feishu credentials are configured."""
|
||||||
|
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
if not (settings.feishu_app_id and settings.feishu_app_secret):
|
if not (settings.feishu_app_id and settings.feishu_app_secret):
|
||||||
return None
|
return None
|
||||||
return feishu.send_text(text, receive_id=chat_id, actor=actor)
|
return feishu.send_text(
|
||||||
|
text,
|
||||||
|
receive_id=chat_id,
|
||||||
|
actor=actor,
|
||||||
|
record_audit=record_audit,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def send_card_if_configured(
|
def send_card_if_configured(
|
||||||
|
|||||||
@@ -1,12 +1,20 @@
|
|||||||
|
from dataclasses import replace
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
from sqlalchemy.exc import IntegrityError
|
from sqlalchemy.exc import IntegrityError
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.application.feishu.commands import FeishuCommandService
|
from app.application.feishu.commands import FeishuCommandService
|
||||||
|
from app.core.config import get_settings
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.modules.audit.constants import AuditAction, AuditSource
|
from app.modules.audit.constants import AuditAction, AuditSource
|
||||||
from app.modules.audit.schemas import AuditLogCreate
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.feishu.app_tickets import (
|
||||||
|
APP_TICKET_EVENT_TYPE,
|
||||||
|
APP_TICKET_PAYLOAD_KEY,
|
||||||
|
FeishuAppTicketService,
|
||||||
|
)
|
||||||
from app.modules.feishu.constants import (
|
from app.modules.feishu.constants import (
|
||||||
FeishuCommandKey,
|
FeishuCommandKey,
|
||||||
FeishuEventReceiptKey,
|
FeishuEventReceiptKey,
|
||||||
@@ -16,6 +24,8 @@ from app.modules.feishu.constants import (
|
|||||||
)
|
)
|
||||||
from app.modules.feishu.models import FeishuEventReceipt
|
from app.modules.feishu.models import FeishuEventReceipt
|
||||||
from app.modules.feishu.service import FeishuService
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.principal import FeishuMention, FeishuPrincipal
|
||||||
|
from app.modules.feishu_users.services import FeishuIdentityService
|
||||||
|
|
||||||
FEISHU_EVENT_ACTIONS = {
|
FEISHU_EVENT_ACTIONS = {
|
||||||
FeishuEventSource.WEBHOOK: AuditAction.FEISHU_WEBHOOK_EVENT,
|
FeishuEventSource.WEBHOOK: AuditAction.FEISHU_WEBHOOK_EVENT,
|
||||||
@@ -38,10 +48,33 @@ class FeishuEventService:
|
|||||||
auto_reply: bool = True,
|
auto_reply: bool = True,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
self.feishu.verify_event(payload)
|
self.feishu.verify_event(payload)
|
||||||
|
return self._handle_verified_event(payload, source, auto_reply)
|
||||||
|
|
||||||
|
def _handle_verified_event(
|
||||||
|
self,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
source: str | FeishuEventSource,
|
||||||
|
auto_reply: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Handle an event after an HTTP verifier or the Feishu SDK accepted it."""
|
||||||
|
|
||||||
challenge = payload.get(FeishuPayloadKey.CHALLENGE)
|
challenge = payload.get(FeishuPayloadKey.CHALLENGE)
|
||||||
if challenge:
|
if challenge:
|
||||||
return {FeishuResponseKey.CHALLENGE: challenge}
|
return {FeishuResponseKey.CHALLENGE: challenge}
|
||||||
source_value = _normalize_source(source)
|
source_value = _normalize_source(source)
|
||||||
|
if _event_type(payload) == APP_TICKET_EVENT_TYPE:
|
||||||
|
return self._handle_app_ticket_event(payload, source_value)
|
||||||
|
user_features_enabled = get_settings().feishu_user_features_enabled
|
||||||
|
command = (
|
||||||
|
self.commands.extract_event_command(payload)
|
||||||
|
if user_features_enabled
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
principal = (
|
||||||
|
self._resolve_principal(payload, command)
|
||||||
|
if user_features_enabled and command
|
||||||
|
else None
|
||||||
|
)
|
||||||
event_identity = _event_identity(payload, source)
|
event_identity = _event_identity(payload, source)
|
||||||
if event_identity and not self._register_event(event_identity):
|
if event_identity and not self._register_event(event_identity):
|
||||||
return {
|
return {
|
||||||
@@ -51,7 +84,7 @@ class FeishuEventService:
|
|||||||
}
|
}
|
||||||
self.feishu.audit.log(
|
self.feishu.audit.log(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=ActorValue.FEISHU,
|
actor=principal.user_code if principal else ActorValue.FEISHU,
|
||||||
source=AuditSource.FEISHU,
|
source=AuditSource.FEISHU,
|
||||||
action=FEISHU_EVENT_ACTIONS[source_value],
|
action=FEISHU_EVENT_ACTIONS[source_value],
|
||||||
target_type=source_value,
|
target_type=source_value,
|
||||||
@@ -60,18 +93,32 @@ class FeishuEventService:
|
|||||||
if event_identity
|
if event_identity
|
||||||
else None
|
else None
|
||||||
),
|
),
|
||||||
request_payload=_audit_event_metadata(payload),
|
request_payload=_audit_event_metadata(
|
||||||
|
payload,
|
||||||
|
include_open_id=not user_features_enabled,
|
||||||
|
include_identity_context=user_features_enabled,
|
||||||
|
),
|
||||||
response_payload={FeishuResponseKey.ACCEPTED: True},
|
response_payload={FeishuResponseKey.ACCEPTED: True},
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
command = self.commands.extract_event_command(payload)
|
if command is None:
|
||||||
|
command = self.commands.extract_event_command(payload)
|
||||||
if not command:
|
if not command:
|
||||||
return {FeishuResponseKey.OK: True, FeishuResponseKey.HANDLED: False}
|
return {FeishuResponseKey.OK: True, FeishuResponseKey.HANDLED: False}
|
||||||
result = self.commands.handle_text(
|
result = self.commands.handle_text(
|
||||||
command[FeishuCommandKey.TEXT],
|
command[FeishuCommandKey.TEXT],
|
||||||
chat_id=command[FeishuCommandKey.CHAT_ID],
|
chat_id=command[FeishuCommandKey.CHAT_ID],
|
||||||
actor=command[FeishuCommandKey.ACTOR],
|
actor=(
|
||||||
|
principal.user_code
|
||||||
|
if principal
|
||||||
|
else (
|
||||||
|
ActorValue.FEISHU
|
||||||
|
if user_features_enabled
|
||||||
|
else command[FeishuCommandKey.ACTOR]
|
||||||
|
)
|
||||||
|
),
|
||||||
auto_reply=auto_reply,
|
auto_reply=auto_reply,
|
||||||
|
principal=principal,
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
FeishuResponseKey.OK: True,
|
FeishuResponseKey.OK: True,
|
||||||
@@ -79,6 +126,97 @@ class FeishuEventService:
|
|||||||
FeishuResponseKey.RESULT: result,
|
FeishuResponseKey.RESULT: result,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def _handle_app_ticket_event(
|
||||||
|
self,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
source: FeishuEventSource,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
settings = get_settings()
|
||||||
|
configured_app_id = str(settings.feishu_app_id or "").strip()
|
||||||
|
if not configured_app_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
detail="FEISHU_APP_ID is required for app ticket events",
|
||||||
|
)
|
||||||
|
|
||||||
|
app_id, ticket = _app_ticket_fields(payload)
|
||||||
|
if not app_id or not ticket:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Invalid Feishu app ticket event",
|
||||||
|
)
|
||||||
|
if app_id != configured_app_id:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Feishu app ticket app_id does not match configured application",
|
||||||
|
)
|
||||||
|
|
||||||
|
event_identity = _event_identity(payload, source)
|
||||||
|
if event_identity is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Feishu app ticket event identity is required",
|
||||||
|
)
|
||||||
|
if not self._register_event(event_identity):
|
||||||
|
return {
|
||||||
|
FeishuResponseKey.OK: True,
|
||||||
|
FeishuResponseKey.HANDLED: False,
|
||||||
|
FeishuResponseKey.DUPLICATE: True,
|
||||||
|
}
|
||||||
|
|
||||||
|
FeishuAppTicketService(self.db).store_verified(app_id, ticket)
|
||||||
|
self.feishu.audit.log(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=ActorValue.FEISHU,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=FEISHU_EVENT_ACTIONS[source],
|
||||||
|
target_type=source,
|
||||||
|
target_id=str(event_identity[FeishuEventReceiptKey.EVENT_KEY]),
|
||||||
|
request_payload=_audit_event_metadata(payload),
|
||||||
|
response_payload={FeishuResponseKey.ACCEPTED: True},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
FeishuResponseKey.OK: True,
|
||||||
|
FeishuResponseKey.HANDLED: True,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _resolve_principal(
|
||||||
|
self,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
command: dict[str, Any],
|
||||||
|
) -> FeishuPrincipal | None:
|
||||||
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
|
event = payload.get(FeishuPayloadKey.EVENT) or {}
|
||||||
|
sender = event.get(FeishuPayloadKey.SENDER) or {}
|
||||||
|
sender_id = sender.get(FeishuPayloadKey.SENDER_ID) or {}
|
||||||
|
tenant_key = str(header.get(FeishuPayloadKey.TENANT_KEY) or "").strip()
|
||||||
|
open_id = str(sender_id.get(FeishuPayloadKey.OPEN_ID) or "").strip()
|
||||||
|
if not tenant_key or not open_id:
|
||||||
|
return None
|
||||||
|
principal = FeishuIdentityService(self.db).resolve_or_register(
|
||||||
|
tenant_key=tenant_key,
|
||||||
|
open_id=open_id,
|
||||||
|
union_id=sender_id.get(FeishuPayloadKey.UNION_ID),
|
||||||
|
user_id=sender_id.get(FeishuPayloadKey.USER_ID),
|
||||||
|
)
|
||||||
|
mentions_value = command.get(FeishuCommandKey.MENTIONS)
|
||||||
|
mentions = (
|
||||||
|
tuple(
|
||||||
|
mention
|
||||||
|
for mention in mentions_value
|
||||||
|
if isinstance(mention, FeishuMention)
|
||||||
|
)
|
||||||
|
if isinstance(mentions_value, (list, tuple))
|
||||||
|
else ()
|
||||||
|
)
|
||||||
|
return replace(
|
||||||
|
principal,
|
||||||
|
chat_id=command.get(FeishuCommandKey.CHAT_ID),
|
||||||
|
chat_type=command.get(FeishuCommandKey.CHAT_TYPE),
|
||||||
|
mentions=mentions,
|
||||||
|
)
|
||||||
|
|
||||||
def _register_event(self, event_identity: dict[str, str | None]) -> bool:
|
def _register_event(self, event_identity: dict[str, str | None]) -> bool:
|
||||||
receipt = FeishuEventReceipt(
|
receipt = FeishuEventReceipt(
|
||||||
event_key=str(event_identity[FeishuEventReceiptKey.EVENT_KEY]),
|
event_key=str(event_identity[FeishuEventReceiptKey.EVENT_KEY]),
|
||||||
@@ -86,16 +224,21 @@ class FeishuEventService:
|
|||||||
event_id=event_identity.get(FeishuEventReceiptKey.EVENT_ID),
|
event_id=event_identity.get(FeishuEventReceiptKey.EVENT_ID),
|
||||||
message_id=event_identity.get(FeishuEventReceiptKey.MESSAGE_ID),
|
message_id=event_identity.get(FeishuEventReceiptKey.MESSAGE_ID),
|
||||||
)
|
)
|
||||||
self.db.add(receipt)
|
|
||||||
try:
|
try:
|
||||||
self.db.flush()
|
with self.db.begin_nested():
|
||||||
|
self.db.add(receipt)
|
||||||
|
self.db.flush()
|
||||||
except IntegrityError:
|
except IntegrityError:
|
||||||
self.db.rollback()
|
|
||||||
return False
|
return False
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
|
||||||
def _audit_event_metadata(payload: dict[str, Any]) -> dict[str, Any]:
|
def _audit_event_metadata(
|
||||||
|
payload: dict[str, Any],
|
||||||
|
*,
|
||||||
|
include_open_id: bool = True,
|
||||||
|
include_identity_context: bool = False,
|
||||||
|
) -> dict[str, Any]:
|
||||||
"""Keep webhook audit evidence without storing message content or tokens."""
|
"""Keep webhook audit evidence without storing message content or tokens."""
|
||||||
|
|
||||||
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
@@ -103,15 +246,23 @@ def _audit_event_metadata(payload: dict[str, Any]) -> dict[str, Any]:
|
|||||||
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
||||||
sender = event.get(FeishuPayloadKey.SENDER) or {}
|
sender = event.get(FeishuPayloadKey.SENDER) or {}
|
||||||
sender_id = sender.get(FeishuPayloadKey.SENDER_ID) or {}
|
sender_id = sender.get(FeishuPayloadKey.SENDER_ID) or {}
|
||||||
return {
|
metadata = {
|
||||||
"schema": payload.get("schema"),
|
"schema": payload.get("schema"),
|
||||||
FeishuPayloadKey.EVENT_ID: header.get(FeishuPayloadKey.EVENT_ID),
|
FeishuPayloadKey.EVENT_ID: header.get(FeishuPayloadKey.EVENT_ID),
|
||||||
FeishuPayloadKey.EVENT_TYPE: header.get(FeishuPayloadKey.EVENT_TYPE),
|
FeishuPayloadKey.EVENT_TYPE: _event_type(payload),
|
||||||
FeishuPayloadKey.MESSAGE_ID: message.get(FeishuPayloadKey.MESSAGE_ID),
|
FeishuPayloadKey.MESSAGE_ID: message.get(FeishuPayloadKey.MESSAGE_ID),
|
||||||
FeishuCommandKey.CHAT_ID: message.get(FeishuCommandKey.CHAT_ID),
|
FeishuCommandKey.CHAT_ID: message.get(FeishuCommandKey.CHAT_ID),
|
||||||
FeishuPayloadKey.MESSAGE_TYPE: message.get(FeishuPayloadKey.MESSAGE_TYPE),
|
FeishuPayloadKey.MESSAGE_TYPE: message.get(FeishuPayloadKey.MESSAGE_TYPE),
|
||||||
FeishuPayloadKey.OPEN_ID: sender_id.get(FeishuPayloadKey.OPEN_ID),
|
|
||||||
}
|
}
|
||||||
|
app_id, _ = _app_ticket_fields(payload)
|
||||||
|
if app_id:
|
||||||
|
metadata[FeishuPayloadKey.APP_ID] = app_id
|
||||||
|
if include_identity_context:
|
||||||
|
metadata[FeishuPayloadKey.TENANT_KEY] = header.get(FeishuPayloadKey.TENANT_KEY)
|
||||||
|
metadata[FeishuCommandKey.CHAT_TYPE] = message.get(FeishuPayloadKey.CHAT_TYPE)
|
||||||
|
if include_open_id:
|
||||||
|
metadata[FeishuPayloadKey.OPEN_ID] = sender_id.get(FeishuPayloadKey.OPEN_ID)
|
||||||
|
return metadata
|
||||||
|
|
||||||
|
|
||||||
def _normalize_source(source: str | FeishuEventSource) -> FeishuEventSource:
|
def _normalize_source(source: str | FeishuEventSource) -> FeishuEventSource:
|
||||||
@@ -126,15 +277,21 @@ def _event_identity(
|
|||||||
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
event = payload.get(FeishuPayloadKey.EVENT) or {}
|
event = payload.get(FeishuPayloadKey.EVENT) or {}
|
||||||
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
message = event.get(FeishuPayloadKey.MESSAGE) or {}
|
||||||
event_id = header.get(FeishuPayloadKey.EVENT_ID)
|
event_id = (
|
||||||
|
header.get(FeishuPayloadKey.EVENT_ID)
|
||||||
|
or payload.get(FeishuPayloadKey.EVENT_ID)
|
||||||
|
or payload.get(FeishuPayloadKey.UUID)
|
||||||
|
or event.get(FeishuPayloadKey.UUID)
|
||||||
|
)
|
||||||
message_id = message.get(FeishuPayloadKey.MESSAGE_ID)
|
message_id = message.get(FeishuPayloadKey.MESSAGE_ID)
|
||||||
stable_id = event_id or message_id
|
stable_id = event_id or message_id
|
||||||
if not stable_id:
|
if not stable_id:
|
||||||
return None
|
return None
|
||||||
event_type = header.get(FeishuPayloadKey.EVENT_TYPE)
|
event_type = _event_type(payload)
|
||||||
|
app_id, _ = _app_ticket_fields(payload)
|
||||||
|
tenant_key = header.get(FeishuPayloadKey.TENANT_KEY) or app_id or "unknown-tenant"
|
||||||
event_key = ":".join(
|
event_key = ":".join(
|
||||||
str(part)
|
str(part) for part in (tenant_key, event_type or FeishuPayloadKey.EVENT, stable_id)
|
||||||
for part in (source_value, event_type or FeishuPayloadKey.EVENT, stable_id)
|
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
FeishuEventReceiptKey.EVENT_KEY: event_key,
|
FeishuEventReceiptKey.EVENT_KEY: event_key,
|
||||||
@@ -142,3 +299,41 @@ def _event_identity(
|
|||||||
FeishuEventReceiptKey.EVENT_ID: str(event_id) if event_id else None,
|
FeishuEventReceiptKey.EVENT_ID: str(event_id) if event_id else None,
|
||||||
FeishuEventReceiptKey.MESSAGE_ID: str(message_id) if message_id else None,
|
FeishuEventReceiptKey.MESSAGE_ID: str(message_id) if message_id else None,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _event_type(payload: dict[str, Any]) -> str:
|
||||||
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
|
return str(
|
||||||
|
header.get(FeishuPayloadKey.EVENT_TYPE)
|
||||||
|
or payload.get(FeishuPayloadKey.EVENT_TYPE)
|
||||||
|
or payload.get("type")
|
||||||
|
or ""
|
||||||
|
).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _app_ticket_fields(payload: dict[str, Any]) -> tuple[str, str]:
|
||||||
|
header = payload.get(FeishuPayloadKey.HEADER)
|
||||||
|
event = payload.get(FeishuPayloadKey.EVENT)
|
||||||
|
data = payload.get(FeishuPayloadKey.DATA)
|
||||||
|
candidates = [
|
||||||
|
value
|
||||||
|
for value in (event, data, header, payload)
|
||||||
|
if isinstance(value, dict)
|
||||||
|
]
|
||||||
|
app_id = next(
|
||||||
|
(
|
||||||
|
str(candidate.get(FeishuPayloadKey.APP_ID) or "").strip()
|
||||||
|
for candidate in candidates
|
||||||
|
if candidate.get(FeishuPayloadKey.APP_ID)
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
)
|
||||||
|
ticket = next(
|
||||||
|
(
|
||||||
|
str(candidate.get(APP_TICKET_PAYLOAD_KEY) or "").strip()
|
||||||
|
for candidate in candidates
|
||||||
|
if candidate.get(APP_TICKET_PAYLOAD_KEY)
|
||||||
|
),
|
||||||
|
"",
|
||||||
|
)
|
||||||
|
return app_id, ticket
|
||||||
|
|||||||
@@ -1,9 +1,29 @@
|
|||||||
|
from app.application.feishu.handlers.admin import (
|
||||||
|
handle_admin_command,
|
||||||
|
is_admin_command,
|
||||||
|
)
|
||||||
from app.application.feishu.handlers.finance import handle_finance_command
|
from app.application.feishu.handlers.finance import handle_finance_command
|
||||||
from app.application.feishu.handlers.market import handle_market_command
|
from app.application.feishu.handlers.market import handle_market_command
|
||||||
from app.application.feishu.handlers.rules import handle_rule_command
|
from app.application.feishu.handlers.personalization import (
|
||||||
|
handle_personalization_command,
|
||||||
|
)
|
||||||
|
from app.application.feishu.handlers.personal_data import (
|
||||||
|
handle_personal_data_command,
|
||||||
|
)
|
||||||
|
from app.application.feishu.handlers.rules import (
|
||||||
|
handle_rule_command,
|
||||||
|
is_company_rule_command,
|
||||||
|
)
|
||||||
|
from app.application.feishu.handlers.subscriptions import handle_subscription_command
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
|
"handle_admin_command",
|
||||||
"handle_finance_command",
|
"handle_finance_command",
|
||||||
"handle_market_command",
|
"handle_market_command",
|
||||||
|
"handle_personalization_command",
|
||||||
|
"handle_personal_data_command",
|
||||||
"handle_rule_command",
|
"handle_rule_command",
|
||||||
|
"handle_subscription_command",
|
||||||
|
"is_admin_command",
|
||||||
|
"is_company_rule_command",
|
||||||
]
|
]
|
||||||
|
|||||||
192
app/application/feishu/handlers/admin.py
Normal file
192
app/application/feishu/handlers/admin.py
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.application.feishu.delivery import send_text_if_configured
|
||||||
|
from app.application.feishu.results import command_result
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FEISHU_USER_TARGET_TYPE,
|
||||||
|
FeishuUserAuditAction,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.principal import FeishuMention, FeishuPrincipal
|
||||||
|
from app.modules.feishu_users.services import (
|
||||||
|
FeishuIdentityService,
|
||||||
|
FeishuUserManagementService,
|
||||||
|
)
|
||||||
|
|
||||||
|
ADMIN_COMMAND_TITLE = "飞书用户管理"
|
||||||
|
ADMIN_COMMAND_HELP = (
|
||||||
|
"用户管理命令必须使用一个真实的飞书 @用户:\n"
|
||||||
|
"设为管理员 @用户\n"
|
||||||
|
"设为普通用户 @用户\n"
|
||||||
|
"停用用户 @用户\n"
|
||||||
|
"启用用户 @用户"
|
||||||
|
)
|
||||||
|
|
||||||
|
_ADMIN_COMMANDS: dict[str, tuple[FeishuCommandName, dict[str, str], str]] = {
|
||||||
|
"设为管理员": (
|
||||||
|
FeishuCommandName.USER_SET_ADMIN,
|
||||||
|
{"role": FeishuUserRole.ADMIN},
|
||||||
|
"已设为管理员",
|
||||||
|
),
|
||||||
|
"设为普通用户": (
|
||||||
|
FeishuCommandName.USER_SET_USER,
|
||||||
|
{"role": FeishuUserRole.USER},
|
||||||
|
"已设为普通用户",
|
||||||
|
),
|
||||||
|
"停用用户": (
|
||||||
|
FeishuCommandName.USER_DISABLE,
|
||||||
|
{"status": FeishuUserStatus.DISABLED},
|
||||||
|
"已停用",
|
||||||
|
),
|
||||||
|
"启用用户": (
|
||||||
|
FeishuCommandName.USER_ENABLE,
|
||||||
|
{"status": FeishuUserStatus.ACTIVE},
|
||||||
|
"已启用",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def is_admin_command(command_text: str) -> bool:
|
||||||
|
return any(command_text.startswith(prefix) for prefix in _ADMIN_COMMANDS)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_admin_command(
|
||||||
|
db: Session,
|
||||||
|
feishu: FeishuService,
|
||||||
|
*,
|
||||||
|
raw_text: str,
|
||||||
|
command_text: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""Manage a user selected only from verified structured mention metadata."""
|
||||||
|
|
||||||
|
matched = next(
|
||||||
|
(
|
||||||
|
(prefix, definition)
|
||||||
|
for prefix, definition in _ADMIN_COMMANDS.items()
|
||||||
|
if command_text.startswith(prefix)
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
if matched is None:
|
||||||
|
return None
|
||||||
|
command_prefix, definition = matched
|
||||||
|
command, changes, success_text = definition
|
||||||
|
target = _target_mention(raw_text, command_prefix, principal.mentions)
|
||||||
|
if target is None or not target.open_id:
|
||||||
|
_audit_denied(db, principal, "missing_or_ambiguous_structured_mention")
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
command,
|
||||||
|
ADMIN_COMMAND_HELP,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
target_tenant = target.tenant_key or principal.tenant_key
|
||||||
|
if target_tenant != principal.tenant_key:
|
||||||
|
_audit_denied(db, principal, "cross_tenant_target")
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
command,
|
||||||
|
"只能管理当前租户内通过飞书 @ 提及的用户。",
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
|
||||||
|
target_principal = FeishuIdentityService(db).resolve_or_register(
|
||||||
|
tenant_key=target_tenant,
|
||||||
|
open_id=target.open_id,
|
||||||
|
union_id=target.union_id,
|
||||||
|
user_id=target.user_id,
|
||||||
|
actor=principal.user_code,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
updated = FeishuUserManagementService(db).update_user(
|
||||||
|
target_principal.user_code,
|
||||||
|
changes=changes,
|
||||||
|
actor=principal.user_code,
|
||||||
|
)
|
||||||
|
display_name = target.name or updated.code
|
||||||
|
content = f"{display_name} {success_text}。"
|
||||||
|
except HTTPException as exc:
|
||||||
|
if exc.status_code == 409:
|
||||||
|
content = "操作已拒绝:不能停用或降级最后一个有效管理员。"
|
||||||
|
else:
|
||||||
|
content = "用户状态未修改,请确认目标用户后重试。"
|
||||||
|
return _result(feishu, command, content, principal, auto_reply)
|
||||||
|
|
||||||
|
|
||||||
|
def _target_mention(
|
||||||
|
raw_text: str,
|
||||||
|
command_text: str,
|
||||||
|
mentions: tuple[FeishuMention, ...],
|
||||||
|
) -> FeishuMention | None:
|
||||||
|
command_index = raw_text.find(command_text)
|
||||||
|
if command_index >= 0:
|
||||||
|
command_tail = raw_text[command_index + len(command_text) :]
|
||||||
|
candidates = [
|
||||||
|
mention
|
||||||
|
for mention in mentions
|
||||||
|
if mention.key and mention.key in command_tail
|
||||||
|
]
|
||||||
|
if len(candidates) == 1:
|
||||||
|
return candidates[0]
|
||||||
|
return None
|
||||||
|
if len(mentions) == 1:
|
||||||
|
return mentions[0]
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_denied(
|
||||||
|
db: Session,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
reason: str,
|
||||||
|
) -> None:
|
||||||
|
AuditService(db).record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=principal.user_code,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=FeishuUserAuditAction.UPDATE_DENIED,
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
risk_level=AuditRiskLevel.HIGH,
|
||||||
|
response_payload={"result": "denied", "reason": reason},
|
||||||
|
status="denied",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _result(
|
||||||
|
feishu: FeishuService,
|
||||||
|
command: FeishuCommandName,
|
||||||
|
content: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
response = (
|
||||||
|
send_text_if_configured(
|
||||||
|
feishu,
|
||||||
|
principal.chat_id,
|
||||||
|
content,
|
||||||
|
principal.user_code,
|
||||||
|
)
|
||||||
|
if auto_reply
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
return command_result(
|
||||||
|
command,
|
||||||
|
FeishuReplyType.TEXT,
|
||||||
|
ADMIN_COMMAND_TITLE,
|
||||||
|
content,
|
||||||
|
response,
|
||||||
|
)
|
||||||
@@ -9,6 +9,7 @@ from app.application.feishu.results import command_result
|
|||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
from app.modules.feishu.service import FeishuService
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
from app.modules.market.chart import render_market_chart
|
from app.modules.market.chart import render_market_chart
|
||||||
from app.modules.market.service import MarketService
|
from app.modules.market.service import MarketService
|
||||||
|
|
||||||
@@ -38,6 +39,7 @@ def handle_market_command(
|
|||||||
chat_id: str | None,
|
chat_id: str | None,
|
||||||
actor: str,
|
actor: str,
|
||||||
auto_reply: bool,
|
auto_reply: bool,
|
||||||
|
principal: FeishuPrincipal | None = None,
|
||||||
) -> dict[str, Any] | None:
|
) -> dict[str, Any] | None:
|
||||||
"""Handle market analysis and watchlist commands."""
|
"""Handle market analysis and watchlist commands."""
|
||||||
|
|
||||||
@@ -54,6 +56,31 @@ def handle_market_command(
|
|||||||
and not comparison
|
and not comparison
|
||||||
):
|
):
|
||||||
return None
|
return None
|
||||||
|
service = MarketService(db)
|
||||||
|
owner_id = principal.owner_id if principal is not None else None
|
||||||
|
if add:
|
||||||
|
item = service.add_watchlist(actor, add.group(1), owner_id=owner_id)
|
||||||
|
return _text_result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.WATCHLIST_ADD,
|
||||||
|
"自选股",
|
||||||
|
f"已加入自选:{item['symbol']}",
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if text == "查看自选":
|
||||||
|
items = service.watchlist(actor, owner_id=owner_id)
|
||||||
|
content = "自选股:" + ("、".join(item["symbol"] for item in items) or "暂无")
|
||||||
|
return _text_result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.WATCHLIST_LIST,
|
||||||
|
"自选股",
|
||||||
|
content,
|
||||||
|
chat_id,
|
||||||
|
actor,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
if not get_settings().market_analysis_enabled:
|
if not get_settings().market_analysis_enabled:
|
||||||
return _text_result(
|
return _text_result(
|
||||||
feishu,
|
feishu,
|
||||||
@@ -65,40 +92,6 @@ def handle_market_command(
|
|||||||
auto_reply,
|
auto_reply,
|
||||||
)
|
)
|
||||||
|
|
||||||
service = MarketService(db)
|
|
||||||
if add:
|
|
||||||
if get_settings().read_only_mode:
|
|
||||||
return _text_result(
|
|
||||||
feishu,
|
|
||||||
FeishuCommandName.WATCHLIST_ADD,
|
|
||||||
"自选股",
|
|
||||||
"当前为只读模式,不能修改自选股。请由管理员启用操作后重试。",
|
|
||||||
chat_id,
|
|
||||||
actor,
|
|
||||||
auto_reply,
|
|
||||||
)
|
|
||||||
item = service.add_watchlist(actor, add.group(1))
|
|
||||||
return _text_result(
|
|
||||||
feishu,
|
|
||||||
FeishuCommandName.WATCHLIST_ADD,
|
|
||||||
"自选股",
|
|
||||||
f"已加入自选:{item['symbol']}",
|
|
||||||
chat_id,
|
|
||||||
actor,
|
|
||||||
auto_reply,
|
|
||||||
)
|
|
||||||
if text == "查看自选":
|
|
||||||
items = service.watchlist(actor)
|
|
||||||
content = "自选股:" + ("、".join(item["symbol"] for item in items) or "暂无")
|
|
||||||
return _text_result(
|
|
||||||
feishu,
|
|
||||||
FeishuCommandName.WATCHLIST_LIST,
|
|
||||||
"自选股",
|
|
||||||
content,
|
|
||||||
chat_id,
|
|
||||||
actor,
|
|
||||||
auto_reply,
|
|
||||||
)
|
|
||||||
if text == "最新公告":
|
if text == "最新公告":
|
||||||
items = service.announcements(limit=10)["items"]
|
items = service.announcements(limit=10)["items"]
|
||||||
content = (
|
content = (
|
||||||
|
|||||||
267
app/application/feishu/handlers/personal_data.py
Normal file
267
app/application/feishu/handlers/personal_data.py
Normal file
@@ -0,0 +1,267 @@
|
|||||||
|
import re
|
||||||
|
from collections import Counter
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.application.feishu.delivery import send_text_if_configured
|
||||||
|
from app.application.feishu.personal_data import FeishuPersonalDataService
|
||||||
|
from app.application.feishu.results import command_result
|
||||||
|
from app.modules.ai_memory.constants import AIMemoryKind
|
||||||
|
from app.modules.ai_memory.models import AIMemoryEntry
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.business.models import MarketWatchlist
|
||||||
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
from app.modules.personalization.constants import PreferenceCategory
|
||||||
|
from app.modules.personalization.models import AIConversation
|
||||||
|
from app.modules.personalization.services import PreferenceService
|
||||||
|
from app.modules.subscriptions.models import PushSubscription
|
||||||
|
|
||||||
|
PERSONAL_DATA_TITLE = "我的数据"
|
||||||
|
_CONFIRM_PATTERN = re.compile(r"^确认忘记我\s+([A-Fa-f0-9]{8})$")
|
||||||
|
_COMMAND_PREFIXES = ("我的数据", "忘记我", "确认忘记我")
|
||||||
|
|
||||||
|
|
||||||
|
def handle_personal_data_command(
|
||||||
|
db: Session,
|
||||||
|
feishu: FeishuService,
|
||||||
|
command_text: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""Handle private data summaries and two-step account erasure."""
|
||||||
|
|
||||||
|
text = command_text.strip()
|
||||||
|
if not text.startswith(_COMMAND_PREFIXES):
|
||||||
|
return None
|
||||||
|
principal.require_active()
|
||||||
|
if principal.chat_type in {"group", "group_chat"}:
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_SUMMARY,
|
||||||
|
"为保护个人信息,请私聊机器人使用“我的数据”或“忘记我”。",
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if text == "我的数据":
|
||||||
|
content = _summary(db, principal)
|
||||||
|
_audit_summary(db, principal)
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_SUMMARY,
|
||||||
|
content,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if text == "忘记我":
|
||||||
|
confirmation = FeishuPersonalDataService(db).request_confirmation(principal)
|
||||||
|
_audit_erasure_request(db, principal)
|
||||||
|
content = (
|
||||||
|
"该操作会永久删除你的个人规则、记忆、偏好、兴趣、"
|
||||||
|
"会话、订阅和飞书身份映射。\n"
|
||||||
|
f"确认码:{confirmation.confirmation_code}\n"
|
||||||
|
f"有效期至:{confirmation.expires_at.strftime('%Y-%m-%d %H:%M:%S')} UTC\n"
|
||||||
|
f"确认命令:确认忘记我 {confirmation.confirmation_code}"
|
||||||
|
)
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_ERASURE_REQUEST,
|
||||||
|
content,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
|
||||||
|
match = _CONFIRM_PATTERN.fullmatch(text)
|
||||||
|
if match is None:
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_ERASURE_CONFIRM,
|
||||||
|
"格式不正确。请先发送“忘记我”获取一次性确认码。",
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
erased = FeishuPersonalDataService(db).confirm(principal, match.group(1))
|
||||||
|
except HTTPException as exc:
|
||||||
|
db.rollback()
|
||||||
|
content = (
|
||||||
|
"不能删除最后一个有效管理员,请先设置另一名管理员。"
|
||||||
|
if exc.status_code == 409
|
||||||
|
else "确认码无效或已过期,请重新发送“忘记我”。"
|
||||||
|
)
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_ERASURE_CONFIRM,
|
||||||
|
content,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.PERSONAL_DATA_ERASURE_CONFIRM,
|
||||||
|
"你的个人数据和飞书身份映射已删除。再次联系时会创建新的普通用户身份。",
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
audit_actor=erased.anonymous_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _summary(db: Session, principal: FeishuPrincipal) -> str:
|
||||||
|
owner_id = principal.owner_id
|
||||||
|
preferences = PreferenceService(db).list_preferences(owner_id)
|
||||||
|
interest_categories = {
|
||||||
|
PreferenceCategory.TOPIC,
|
||||||
|
PreferenceCategory.INTEREST,
|
||||||
|
}
|
||||||
|
profile_preferences = [
|
||||||
|
item for item in preferences if item["category"] not in interest_categories
|
||||||
|
]
|
||||||
|
interest_preferences = [
|
||||||
|
item for item in preferences if item["category"] in interest_categories
|
||||||
|
]
|
||||||
|
watchlist = list(
|
||||||
|
db.execute(
|
||||||
|
select(MarketWatchlist.symbol)
|
||||||
|
.where(
|
||||||
|
MarketWatchlist.owner_id == owner_id,
|
||||||
|
MarketWatchlist.enabled.is_(True),
|
||||||
|
)
|
||||||
|
.order_by(MarketWatchlist.symbol.asc())
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
subscription_statuses = Counter(
|
||||||
|
db.execute(
|
||||||
|
select(PushSubscription.status).where(
|
||||||
|
PushSubscription.owner_id == owner_id
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
personal_rule_count = _memory_count(
|
||||||
|
db,
|
||||||
|
owner_id,
|
||||||
|
AIMemoryKind.PERSONAL_RULE,
|
||||||
|
)
|
||||||
|
memory_count = _memory_count(db, owner_id, AIMemoryKind.MEMORY)
|
||||||
|
conversation_count = int(
|
||||||
|
db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(AIConversation)
|
||||||
|
.where(AIConversation.owner_id == owner_id)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
preference_text = "、".join(
|
||||||
|
f"{item['category']}={item['value']}" for item in profile_preferences[:10]
|
||||||
|
)
|
||||||
|
interest_values = [
|
||||||
|
*(str(item["value"]) for item in interest_preferences),
|
||||||
|
*watchlist,
|
||||||
|
]
|
||||||
|
interest_text = "、".join(interest_values[:10])
|
||||||
|
subscriptions_text = "、".join(
|
||||||
|
f"{status} {count}" for status, count in sorted(subscription_statuses.items())
|
||||||
|
)
|
||||||
|
return "\n".join(
|
||||||
|
[
|
||||||
|
"我的个人数据摘要:",
|
||||||
|
f"个人规则:{personal_rule_count} 条",
|
||||||
|
f"偏好:{preference_text or '暂无'}",
|
||||||
|
f"兴趣与自选:{interest_text or '暂无'}",
|
||||||
|
f"个人记忆:{memory_count} 条",
|
||||||
|
f"会话:{conversation_count} 个",
|
||||||
|
f"订阅:{subscriptions_text or '暂无'}",
|
||||||
|
"可发送“我的偏好”“查看规则”“我的订阅”查看明细。",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _memory_count(
|
||||||
|
db: Session,
|
||||||
|
owner_id: int,
|
||||||
|
kind: str,
|
||||||
|
) -> int:
|
||||||
|
return int(
|
||||||
|
db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(AIMemoryEntry)
|
||||||
|
.where(
|
||||||
|
AIMemoryEntry.owner_id == owner_id,
|
||||||
|
AIMemoryEntry.kind == kind,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_summary(db: Session, principal: FeishuPrincipal) -> None:
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action="personalization.data.summary",
|
||||||
|
risk_level=AuditRiskLevel.LOW,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_erasure_request(db: Session, principal: FeishuPrincipal) -> None:
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action="personalization.erasure.request",
|
||||||
|
risk_level=AuditRiskLevel.HIGH,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit(
|
||||||
|
db: Session,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
risk_level: str,
|
||||||
|
) -> None:
|
||||||
|
AuditService(db).record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=principal.user_code,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=action,
|
||||||
|
target_type="feishu-user",
|
||||||
|
target_id=principal.user_code,
|
||||||
|
risk_level=risk_level,
|
||||||
|
response_payload={"result": "success"},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _result(
|
||||||
|
feishu: FeishuService,
|
||||||
|
command: FeishuCommandName,
|
||||||
|
content: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
*,
|
||||||
|
audit_actor: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
response = (
|
||||||
|
send_text_if_configured(
|
||||||
|
feishu,
|
||||||
|
principal.chat_id,
|
||||||
|
content,
|
||||||
|
audit_actor or principal.user_code,
|
||||||
|
record_audit=audit_actor is None,
|
||||||
|
)
|
||||||
|
if auto_reply
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
return command_result(
|
||||||
|
command,
|
||||||
|
FeishuReplyType.TEXT,
|
||||||
|
PERSONAL_DATA_TITLE,
|
||||||
|
content,
|
||||||
|
response,
|
||||||
|
)
|
||||||
300
app/application/feishu/handlers/personalization.py
Normal file
300
app/application/feishu/handlers/personalization.py
Normal file
@@ -0,0 +1,300 @@
|
|||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.application.feishu.delivery import send_text_if_configured
|
||||||
|
from app.application.feishu.results import command_result
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
from app.modules.personalization.constants import PreferenceCategory
|
||||||
|
from app.modules.personalization.services import ConversationService, PreferenceService
|
||||||
|
|
||||||
|
PERSONALIZATION_TITLE = "个人设置"
|
||||||
|
PREFERENCE_HELP = (
|
||||||
|
"偏好指令格式:\n"
|
||||||
|
"记住偏好 语言:中文\n"
|
||||||
|
"记住偏好 语气:简洁\n"
|
||||||
|
"记住偏好 详略:详细\n"
|
||||||
|
"关注主题:人工智能\n"
|
||||||
|
"我的偏好\n"
|
||||||
|
"删除偏好 <偏好编号>"
|
||||||
|
)
|
||||||
|
_CATEGORY_LABELS = {
|
||||||
|
"语言": PreferenceCategory.LANGUAGE,
|
||||||
|
"语气": PreferenceCategory.TONE,
|
||||||
|
"详略": PreferenceCategory.DETAIL,
|
||||||
|
"主题": PreferenceCategory.TOPIC,
|
||||||
|
"兴趣": PreferenceCategory.INTEREST,
|
||||||
|
}
|
||||||
|
_CATEGORY_NAMES = {
|
||||||
|
PreferenceCategory.LANGUAGE: "语言",
|
||||||
|
PreferenceCategory.TONE: "语气",
|
||||||
|
PreferenceCategory.DETAIL: "详略",
|
||||||
|
PreferenceCategory.TOPIC: "关注主题",
|
||||||
|
PreferenceCategory.INTEREST: "兴趣",
|
||||||
|
}
|
||||||
|
_SET_PATTERN = re.compile(
|
||||||
|
r"^记住偏好\s+(语言|语气|详略|主题|兴趣)\s*[::]\s*(.+)$"
|
||||||
|
)
|
||||||
|
_TOPIC_PATTERN = re.compile(r"^关注主题\s*[::]?\s*(.+)$")
|
||||||
|
_DELETE_PATTERN = re.compile(
|
||||||
|
r"^删除偏好\s+(PREF-[A-Za-z0-9-]+)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_LIST_COMMANDS = {"我的偏好", "查看偏好", "我的兴趣", "查看兴趣"}
|
||||||
|
_HELP_COMMANDS = {"帮助", "使用帮助", "命令帮助"}
|
||||||
|
_COMMAND_PREFIXES = (
|
||||||
|
"记住偏好",
|
||||||
|
"关注主题",
|
||||||
|
"我的偏好",
|
||||||
|
"查看偏好",
|
||||||
|
"我的兴趣",
|
||||||
|
"查看兴趣",
|
||||||
|
"删除偏好",
|
||||||
|
"重置对话",
|
||||||
|
*_HELP_COMMANDS,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_personalization_command(
|
||||||
|
db: Session,
|
||||||
|
feishu: FeishuService,
|
||||||
|
command_text: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""Handle owner-scoped preferences, interests, help, and conversation reset."""
|
||||||
|
|
||||||
|
text = command_text.strip()
|
||||||
|
if not text.startswith(_COMMAND_PREFIXES):
|
||||||
|
return None
|
||||||
|
principal.require_active()
|
||||||
|
if text in _HELP_COMMANDS:
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.HELP,
|
||||||
|
_help_content(principal),
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
if text == "重置对话":
|
||||||
|
chat_type, chat_key = _conversation_key(principal)
|
||||||
|
reset = ConversationService(db).reset(
|
||||||
|
principal.owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action="personalization.conversation.reset",
|
||||||
|
target_type="ai-conversation",
|
||||||
|
response={"reset": reset},
|
||||||
|
)
|
||||||
|
content = "当前会话已重置。" if reset else "当前会话没有可清除的历史。"
|
||||||
|
return _result(
|
||||||
|
feishu,
|
||||||
|
FeishuCommandName.CONVERSATION_RESET,
|
||||||
|
content,
|
||||||
|
principal,
|
||||||
|
auto_reply,
|
||||||
|
)
|
||||||
|
|
||||||
|
service = PreferenceService(db)
|
||||||
|
set_match = _SET_PATTERN.fullmatch(text)
|
||||||
|
topic_match = _TOPIC_PATTERN.fullmatch(text)
|
||||||
|
delete_match = _DELETE_PATTERN.fullmatch(text)
|
||||||
|
try:
|
||||||
|
if set_match:
|
||||||
|
category = _CATEGORY_LABELS[set_match.group(1)]
|
||||||
|
record = service.upsert(
|
||||||
|
principal.owner_id,
|
||||||
|
category,
|
||||||
|
set_match.group(2),
|
||||||
|
)
|
||||||
|
_audit_preference(db, principal, "upsert", record)
|
||||||
|
content = (
|
||||||
|
"偏好已保存。\n"
|
||||||
|
f"编号:{record['code']}\n"
|
||||||
|
f"类别:{_category_name(record['category'])}\n"
|
||||||
|
f"内容:{record['value']}"
|
||||||
|
)
|
||||||
|
command = FeishuCommandName.PREFERENCE_SET
|
||||||
|
elif topic_match:
|
||||||
|
record = service.upsert(
|
||||||
|
principal.owner_id,
|
||||||
|
PreferenceCategory.TOPIC,
|
||||||
|
topic_match.group(1),
|
||||||
|
)
|
||||||
|
_audit_preference(db, principal, "upsert", record)
|
||||||
|
content = (
|
||||||
|
"关注主题已保存。\n"
|
||||||
|
f"编号:{record['code']}\n"
|
||||||
|
f"主题:{record['value']}"
|
||||||
|
)
|
||||||
|
command = FeishuCommandName.PREFERENCE_SET
|
||||||
|
elif text in _LIST_COMMANDS:
|
||||||
|
records = service.list_preferences(principal.owner_id)
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action="personalization.preference.list",
|
||||||
|
target_type="user-preference",
|
||||||
|
response={"count": len(records)},
|
||||||
|
)
|
||||||
|
content = _preference_list(records)
|
||||||
|
command = FeishuCommandName.PREFERENCE_LIST
|
||||||
|
elif delete_match:
|
||||||
|
code = delete_match.group(1)
|
||||||
|
service.delete(principal.owner_id, code)
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action="personalization.preference.delete",
|
||||||
|
target_type="user-preference",
|
||||||
|
target_id=code,
|
||||||
|
response={"deleted": True},
|
||||||
|
)
|
||||||
|
content = "偏好已删除。"
|
||||||
|
command = FeishuCommandName.PREFERENCE_DELETE
|
||||||
|
else:
|
||||||
|
content = PREFERENCE_HELP
|
||||||
|
command = FeishuCommandName.PREFERENCE_SET
|
||||||
|
except HTTPException as exc:
|
||||||
|
db.rollback()
|
||||||
|
if exc.status_code == 404:
|
||||||
|
content = "没有找到该偏好,请先发送“我的偏好”确认编号。"
|
||||||
|
elif "Sensitive preference" in str(exc.detail):
|
||||||
|
content = "该内容可能涉及敏感个人信息或密钥,已拒绝保存。"
|
||||||
|
else:
|
||||||
|
content = f"偏好指令未执行,请检查格式。\n\n{PREFERENCE_HELP}"
|
||||||
|
command = (
|
||||||
|
FeishuCommandName.PREFERENCE_DELETE
|
||||||
|
if delete_match
|
||||||
|
else FeishuCommandName.PREFERENCE_SET
|
||||||
|
)
|
||||||
|
return _result(feishu, command, content, principal, auto_reply)
|
||||||
|
|
||||||
|
|
||||||
|
def _preference_list(records: list[dict[str, Any]]) -> str:
|
||||||
|
if not records:
|
||||||
|
return "当前没有已保存的偏好或兴趣。\n\n" + PREFERENCE_HELP
|
||||||
|
lines = ["我的偏好与兴趣:"]
|
||||||
|
for record in records:
|
||||||
|
lines.append(
|
||||||
|
f"{record['code']}|{_category_name(record['category'])}"
|
||||||
|
f"|{record['value']}"
|
||||||
|
)
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _help_content(principal: FeishuPrincipal) -> str:
|
||||||
|
lines = [
|
||||||
|
"你可以使用:",
|
||||||
|
"问 <问题>",
|
||||||
|
"学习规则:<个人规则>;查看/修改/启用/停用/删除规则",
|
||||||
|
"记住偏好、关注主题、我的偏好、删除偏好",
|
||||||
|
"加入自选 <股票代码>;查看自选",
|
||||||
|
"订阅 <自然语言时间>:<提示词>;我的订阅、暂停、恢复、退订",
|
||||||
|
"设置时区、设置/关闭安静时段",
|
||||||
|
"重置对话、我的数据、忘记我",
|
||||||
|
]
|
||||||
|
if principal.is_admin:
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
"",
|
||||||
|
"管理员还可以使用:",
|
||||||
|
"日报、周报、财务、风险和考勤查询",
|
||||||
|
"学习/查看/修改/启停/删除公司规则",
|
||||||
|
"在当前群创建群订阅",
|
||||||
|
"通过真实 @用户 管理角色和状态",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _conversation_key(principal: FeishuPrincipal) -> tuple[str, str]:
|
||||||
|
chat_type = principal.chat_type or "p2p"
|
||||||
|
is_group = chat_type in {"group", "group_chat"}
|
||||||
|
chat_key = principal.chat_id if is_group else (principal.chat_id or principal.open_id)
|
||||||
|
if not chat_key:
|
||||||
|
raise HTTPException(status_code=422, detail="Missing Feishu chat identity")
|
||||||
|
return chat_type, chat_key
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_preference(
|
||||||
|
db: Session,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
action: str,
|
||||||
|
record: dict[str, Any],
|
||||||
|
) -> None:
|
||||||
|
_audit(
|
||||||
|
db,
|
||||||
|
principal,
|
||||||
|
action=f"personalization.preference.{action}",
|
||||||
|
target_type="user-preference",
|
||||||
|
target_id=str(record["code"]),
|
||||||
|
response={"category": record["category"]},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit(
|
||||||
|
db: Session,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
*,
|
||||||
|
action: str,
|
||||||
|
target_type: str,
|
||||||
|
target_id: str | None = None,
|
||||||
|
response: dict[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
AuditService(db).record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=principal.user_code,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=action,
|
||||||
|
target_type=target_type,
|
||||||
|
target_id=target_id,
|
||||||
|
risk_level=AuditRiskLevel.LOW,
|
||||||
|
response_payload=response,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _category_name(value: str) -> str:
|
||||||
|
try:
|
||||||
|
return _CATEGORY_NAMES[PreferenceCategory(value)]
|
||||||
|
except ValueError:
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _result(
|
||||||
|
feishu: FeishuService,
|
||||||
|
command: FeishuCommandName,
|
||||||
|
content: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
response = (
|
||||||
|
send_text_if_configured(
|
||||||
|
feishu,
|
||||||
|
principal.chat_id,
|
||||||
|
content,
|
||||||
|
principal.user_code,
|
||||||
|
)
|
||||||
|
if auto_reply
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
return command_result(
|
||||||
|
command,
|
||||||
|
FeishuReplyType.TEXT,
|
||||||
|
PERSONALIZATION_TITLE,
|
||||||
|
content,
|
||||||
|
response,
|
||||||
|
)
|
||||||
@@ -11,32 +11,60 @@ from app.modules.ai_memory.constants import AIMemoryStatus
|
|||||||
from app.modules.ai_memory.service import AIMemoryService
|
from app.modules.ai_memory.service import AIMemoryService
|
||||||
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
from app.modules.feishu.service import FeishuService
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.constants import FeishuCapability
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
|
||||||
RULE_TITLE = "AI 学习规则"
|
RULE_TITLE = "AI 学习规则"
|
||||||
RULE_CREATE_PATTERN = re.compile(r"^学习规则(?:\s+(\d{1,3}))?\s*[::]\s*(.*)$")
|
RULE_CREATE_PATTERN = re.compile(r"^学习规则(?:\s+(\d{1,3}))?\s*[::]\s*(.*)$")
|
||||||
MARKET_RULE_CREATE_PATTERN = re.compile(
|
MARKET_RULE_CREATE_PATTERN = re.compile(
|
||||||
r"^学习市场规则(?:\s+(\d{1,3}))?\s*[::]\s*(.*)$"
|
r"^学习市场规则(?:\s+(\d{1,3}))?\s*[::]\s*(.*)$"
|
||||||
)
|
)
|
||||||
|
RULE_UPDATE_PATTERN = re.compile(
|
||||||
|
r"^修改规则\s+(MEM-[A-Za-z0-9-]+)(?:\s+(\d{1,3}))?\s*[::]\s*(.*)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
RULE_DISABLE_PATTERN = re.compile(r"^停用规则\s+(MEM-[A-Za-z0-9-]+)$", re.IGNORECASE)
|
RULE_DISABLE_PATTERN = re.compile(r"^停用规则\s+(MEM-[A-Za-z0-9-]+)$", re.IGNORECASE)
|
||||||
RULE_ENABLE_PATTERN = re.compile(r"^启用规则\s+(MEM-[A-Za-z0-9-]+)$", re.IGNORECASE)
|
RULE_ENABLE_PATTERN = re.compile(r"^启用规则\s+(MEM-[A-Za-z0-9-]+)$", re.IGNORECASE)
|
||||||
|
RULE_DELETE_PATTERN = re.compile(r"^删除规则\s+(MEM-[A-Za-z0-9-]+)$", re.IGNORECASE)
|
||||||
RULE_LIST_COMMANDS = {"查看规则", "规则列表", "查看市场规则"}
|
RULE_LIST_COMMANDS = {"查看规则", "规则列表", "查看市场规则"}
|
||||||
RULE_COMMAND_PREFIXES = (
|
RULE_COMMAND_PREFIXES = (
|
||||||
|
"学习公司市场规则",
|
||||||
|
"学习公司规则",
|
||||||
|
"查看公司市场规则",
|
||||||
|
"查看公司规则",
|
||||||
|
"修改公司规则",
|
||||||
|
"启用公司规则",
|
||||||
|
"停用公司规则",
|
||||||
|
"删除公司规则",
|
||||||
"学习市场规则",
|
"学习市场规则",
|
||||||
"学习规则",
|
"学习规则",
|
||||||
"查看市场规则",
|
"查看市场规则",
|
||||||
"查看规则",
|
"查看规则",
|
||||||
"规则列表",
|
"规则列表",
|
||||||
|
"修改规则",
|
||||||
"停用规则",
|
"停用规则",
|
||||||
"启用规则",
|
"启用规则",
|
||||||
|
"删除规则",
|
||||||
)
|
)
|
||||||
RULE_COMMAND_HELP = (
|
RULE_COMMAND_HELP = (
|
||||||
"规则指令格式:\n"
|
"规则指令格式:\n"
|
||||||
"学习规则:<规则内容>\n"
|
|
||||||
"学习规则 80:<规则内容>\n"
|
"学习规则 80:<规则内容>\n"
|
||||||
"学习市场规则 80:<仅用于市场分析的规则内容>\n"
|
|
||||||
"查看规则\n"
|
"查看规则\n"
|
||||||
"停用规则 <规则编号>\n"
|
"修改规则 <编号> 80:<新内容>\n"
|
||||||
"启用规则 <规则编号>"
|
"启用规则 <编号>\n"
|
||||||
|
"停用规则 <编号>\n"
|
||||||
|
"删除规则 <编号>"
|
||||||
|
)
|
||||||
|
|
||||||
|
_COMPANY_REPLACEMENTS = (
|
||||||
|
("学习公司市场规则", "学习市场规则"),
|
||||||
|
("查看公司市场规则", "查看市场规则"),
|
||||||
|
("学习公司规则", "学习规则"),
|
||||||
|
("查看公司规则", "查看规则"),
|
||||||
|
("修改公司规则", "修改规则"),
|
||||||
|
("启用公司规则", "启用规则"),
|
||||||
|
("停用公司规则", "停用规则"),
|
||||||
|
("删除公司规则", "删除规则"),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -47,29 +75,40 @@ def handle_rule_command(
|
|||||||
chat_id: str | None,
|
chat_id: str | None,
|
||||||
actor: str,
|
actor: str,
|
||||||
auto_reply: bool,
|
auto_reply: bool,
|
||||||
|
principal: FeishuPrincipal | None = None,
|
||||||
) -> dict[str, Any] | None:
|
) -> dict[str, Any] | None:
|
||||||
"""Handle persistent AI rule commands."""
|
"""Handle company or owner-scoped persistent AI rule commands."""
|
||||||
|
|
||||||
if not command_text.startswith(RULE_COMMAND_PREFIXES):
|
if not command_text.startswith(RULE_COMMAND_PREFIXES):
|
||||||
return None
|
return None
|
||||||
command = _command_name(command_text)
|
normalized, company_rule = _normalize_company_command(command_text)
|
||||||
if command in {
|
owner_id: int | None = None
|
||||||
FeishuCommandName.RULE_CREATE,
|
if get_settings().feishu_user_features_enabled:
|
||||||
FeishuCommandName.RULE_DISABLE,
|
if principal is None:
|
||||||
FeishuCommandName.RULE_ENABLE,
|
return _result(
|
||||||
} and get_settings().read_only_mode:
|
feishu,
|
||||||
return _result(
|
FeishuCommandName.PERMISSION_DENIED,
|
||||||
feishu,
|
"个人规则只能由已验证的飞书账号管理。",
|
||||||
command,
|
chat_id,
|
||||||
"当前为只读模式,不能新增或修改学习规则。请由管理员启用操作后重试。",
|
actor,
|
||||||
chat_id,
|
auto_reply,
|
||||||
actor,
|
)
|
||||||
auto_reply,
|
if company_rule:
|
||||||
)
|
principal.require_capability(FeishuCapability.COMPANY_RULES)
|
||||||
|
else:
|
||||||
|
owner_id = principal.owner_id
|
||||||
|
|
||||||
|
command = _command_name(normalized)
|
||||||
content = RULE_COMMAND_HELP
|
content = RULE_COMMAND_HELP
|
||||||
try:
|
try:
|
||||||
command, content = _execute(db, command_text, command, actor)
|
command, content = _execute(
|
||||||
|
db,
|
||||||
|
normalized,
|
||||||
|
command,
|
||||||
|
actor,
|
||||||
|
owner_id=owner_id,
|
||||||
|
company_rule=company_rule or owner_id is None,
|
||||||
|
)
|
||||||
except HTTPException as exc:
|
except HTTPException as exc:
|
||||||
detail = str(exc.detail)
|
detail = str(exc.detail)
|
||||||
if "secret-like" in detail:
|
if "secret-like" in detail:
|
||||||
@@ -83,21 +122,65 @@ def handle_rule_command(
|
|||||||
return _result(feishu, command, content, chat_id, actor, auto_reply)
|
return _result(feishu, command, content, chat_id, actor, auto_reply)
|
||||||
|
|
||||||
|
|
||||||
|
def is_company_rule_command(command_text: str) -> bool:
|
||||||
|
return any(command_text.startswith(prefix) for prefix, _ in _COMPANY_REPLACEMENTS)
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_company_command(command_text: str) -> tuple[str, bool]:
|
||||||
|
for prefix, replacement in _COMPANY_REPLACEMENTS:
|
||||||
|
if command_text.startswith(prefix):
|
||||||
|
return replacement + command_text[len(prefix) :], True
|
||||||
|
return command_text, False
|
||||||
|
|
||||||
|
|
||||||
def _execute(
|
def _execute(
|
||||||
db: Session,
|
db: Session,
|
||||||
command_text: str,
|
command_text: str,
|
||||||
command: FeishuCommandName,
|
command: FeishuCommandName,
|
||||||
actor: str,
|
actor: str,
|
||||||
|
*,
|
||||||
|
owner_id: int | None,
|
||||||
|
company_rule: bool,
|
||||||
) -> tuple[FeishuCommandName, str]:
|
) -> tuple[FeishuCommandName, str]:
|
||||||
market_create_match = MARKET_RULE_CREATE_PATTERN.fullmatch(command_text)
|
market_create_match = MARKET_RULE_CREATE_PATTERN.fullmatch(command_text)
|
||||||
create_match = market_create_match or RULE_CREATE_PATTERN.fullmatch(command_text)
|
create_match = market_create_match or RULE_CREATE_PATTERN.fullmatch(command_text)
|
||||||
|
update_match = RULE_UPDATE_PATTERN.fullmatch(command_text)
|
||||||
disable_match = RULE_DISABLE_PATTERN.fullmatch(command_text)
|
disable_match = RULE_DISABLE_PATTERN.fullmatch(command_text)
|
||||||
enable_match = RULE_ENABLE_PATTERN.fullmatch(command_text)
|
enable_match = RULE_ENABLE_PATTERN.fullmatch(command_text)
|
||||||
|
delete_match = RULE_DELETE_PATTERN.fullmatch(command_text)
|
||||||
memory = AIMemoryService(db)
|
memory = AIMemoryService(db)
|
||||||
if create_match:
|
if create_match:
|
||||||
return command, _create_rule(memory, create_match, market_create_match is not None, actor)
|
return (
|
||||||
|
command,
|
||||||
|
_create_rule(
|
||||||
|
memory,
|
||||||
|
create_match,
|
||||||
|
market_create_match is not None,
|
||||||
|
actor,
|
||||||
|
owner_id,
|
||||||
|
company_rule,
|
||||||
|
),
|
||||||
|
)
|
||||||
if command_text in RULE_LIST_COMMANDS:
|
if command_text in RULE_LIST_COMMANDS:
|
||||||
return FeishuCommandName.RULE_LIST, _list_rules(memory, command_text)
|
return (
|
||||||
|
FeishuCommandName.RULE_LIST,
|
||||||
|
_list_rules(memory, command_text, owner_id, company_rule),
|
||||||
|
)
|
||||||
|
if update_match:
|
||||||
|
priority = int(update_match.group(2) or 50)
|
||||||
|
content = update_match.group(3).strip()
|
||||||
|
if not content:
|
||||||
|
return FeishuCommandName.RULE_UPDATE, "规则内容不能为空。"
|
||||||
|
rule = memory.update_rule(
|
||||||
|
code=update_match.group(1),
|
||||||
|
content=content,
|
||||||
|
priority=priority,
|
||||||
|
tags=None,
|
||||||
|
enabled=None,
|
||||||
|
actor=actor,
|
||||||
|
owner_id=owner_id,
|
||||||
|
)
|
||||||
|
return FeishuCommandName.RULE_UPDATE, _rule_state(rule, "已修改")
|
||||||
if disable_match or enable_match:
|
if disable_match or enable_match:
|
||||||
enabled = enable_match is not None
|
enabled = enable_match is not None
|
||||||
match = enable_match or disable_match
|
match = enable_match or disable_match
|
||||||
@@ -108,16 +191,16 @@ def _execute(
|
|||||||
tags=None,
|
tags=None,
|
||||||
enabled=enabled,
|
enabled=enabled,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
owner_id=owner_id,
|
||||||
)
|
)
|
||||||
state = "已启用" if enabled else "已停用"
|
state = "已启用" if enabled else "已停用"
|
||||||
return (
|
return (
|
||||||
FeishuCommandName.RULE_ENABLE if enabled else FeishuCommandName.RULE_DISABLE,
|
FeishuCommandName.RULE_ENABLE if enabled else FeishuCommandName.RULE_DISABLE,
|
||||||
f"规则{state}。\n"
|
_rule_state(rule, state),
|
||||||
f"编号:{rule['code']}\n"
|
|
||||||
f"优先级:{rule['importance']}\n"
|
|
||||||
f"范围:{rule['scope']} / {rule['subject']}\n"
|
|
||||||
f"状态:{state}",
|
|
||||||
)
|
)
|
||||||
|
if delete_match:
|
||||||
|
memory.delete_rule(delete_match.group(1), actor=actor, owner_id=owner_id)
|
||||||
|
return FeishuCommandName.RULE_DELETE, "规则已删除。"
|
||||||
return command, RULE_COMMAND_HELP
|
return command, RULE_COMMAND_HELP
|
||||||
|
|
||||||
|
|
||||||
@@ -126,6 +209,8 @@ def _create_rule(
|
|||||||
match: re.Match[str],
|
match: re.Match[str],
|
||||||
market_rule: bool,
|
market_rule: bool,
|
||||||
actor: str,
|
actor: str,
|
||||||
|
owner_id: int | None,
|
||||||
|
company_rule: bool,
|
||||||
) -> str:
|
) -> str:
|
||||||
priority = int(match.group(1) or 50)
|
priority = int(match.group(1) or 50)
|
||||||
content = match.group(2).strip()
|
content = match.group(2).strip()
|
||||||
@@ -136,29 +221,39 @@ def _create_rule(
|
|||||||
rule = memory.create_rule(
|
rule = memory.create_rule(
|
||||||
content=content,
|
content=content,
|
||||||
scope="market" if market_rule else "global",
|
scope="market" if market_rule else "global",
|
||||||
subject="market" if market_rule else "company",
|
subject=(
|
||||||
|
"market"
|
||||||
|
if market_rule
|
||||||
|
else ("company" if company_rule else "personal")
|
||||||
|
),
|
||||||
priority=priority,
|
priority=priority,
|
||||||
tags=["feishu", *(["market"] if market_rule else [])],
|
tags=[
|
||||||
|
"feishu",
|
||||||
|
"company" if company_rule else "personal",
|
||||||
|
*(["market"] if market_rule else []),
|
||||||
|
],
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
owner_id=owner_id,
|
||||||
)
|
)
|
||||||
return (
|
return _rule_state(rule, "已学习")
|
||||||
"规则已学习。\n"
|
|
||||||
f"编号:{rule['code']}\n"
|
|
||||||
f"优先级:{rule['importance']}\n"
|
|
||||||
f"范围:{rule['scope']} / {rule['subject']}\n"
|
|
||||||
"状态:已启用"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _list_rules(memory: AIMemoryService, command_text: str) -> str:
|
def _list_rules(
|
||||||
|
memory: AIMemoryService,
|
||||||
|
command_text: str,
|
||||||
|
owner_id: int | None,
|
||||||
|
company_rule: bool,
|
||||||
|
) -> str:
|
||||||
rules = memory.list_rules(
|
rules = memory.list_rules(
|
||||||
scope="market" if command_text == "查看市场规则" else None,
|
scope="market" if command_text == "查看市场规则" else None,
|
||||||
status_filter=AIMemoryStatus.ACTIVE,
|
status_filter=AIMemoryStatus.ACTIVE,
|
||||||
limit=20,
|
limit=20,
|
||||||
|
owner_id=owner_id,
|
||||||
)
|
)
|
||||||
if not rules:
|
if not rules:
|
||||||
return "当前没有已启用的学习规则。"
|
target = "公司" if company_rule else "个人"
|
||||||
lines = ["当前已启用的学习规则:"]
|
return f"当前没有已启用的{target}规则。"
|
||||||
|
lines = ["当前已启用的公司规则:" if company_rule else "当前已启用的个人规则:"]
|
||||||
for rule in rules:
|
for rule in rules:
|
||||||
rule_text = str(rule["content"])
|
rule_text = str(rule["content"])
|
||||||
if len(rule_text) > 80:
|
if len(rule_text) > 80:
|
||||||
@@ -170,11 +265,25 @@ def _list_rules(memory: AIMemoryService, command_text: str) -> str:
|
|||||||
return "\n\n".join(lines)
|
return "\n\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _rule_state(rule: dict[str, Any], state: str) -> str:
|
||||||
|
return (
|
||||||
|
f"规则{state}。\n"
|
||||||
|
f"编号:{rule['code']}\n"
|
||||||
|
f"优先级:{rule['importance']}\n"
|
||||||
|
f"范围:{rule['scope']} / {rule['subject']}\n"
|
||||||
|
f"状态:{state}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _command_name(command_text: str) -> FeishuCommandName:
|
def _command_name(command_text: str) -> FeishuCommandName:
|
||||||
if command_text.startswith("停用规则"):
|
if command_text.startswith("停用规则"):
|
||||||
return FeishuCommandName.RULE_DISABLE
|
return FeishuCommandName.RULE_DISABLE
|
||||||
if command_text.startswith("启用规则"):
|
if command_text.startswith("启用规则"):
|
||||||
return FeishuCommandName.RULE_ENABLE
|
return FeishuCommandName.RULE_ENABLE
|
||||||
|
if command_text.startswith("修改规则"):
|
||||||
|
return FeishuCommandName.RULE_UPDATE
|
||||||
|
if command_text.startswith("删除规则"):
|
||||||
|
return FeishuCommandName.RULE_DELETE
|
||||||
if command_text.startswith(("查看市场规则", "查看规则", "规则列表")):
|
if command_text.startswith(("查看市场规则", "查看规则", "规则列表")):
|
||||||
return FeishuCommandName.RULE_LIST
|
return FeishuCommandName.RULE_LIST
|
||||||
return FeishuCommandName.RULE_CREATE
|
return FeishuCommandName.RULE_CREATE
|
||||||
|
|||||||
321
app/application/feishu/handlers/subscriptions.py
Normal file
321
app/application/feishu/handlers/subscriptions.py
Normal file
@@ -0,0 +1,321 @@
|
|||||||
|
import re
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from fastapi import HTTPException
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.application.feishu.delivery import send_text_if_configured
|
||||||
|
from app.application.feishu.results import command_result
|
||||||
|
from app.modules.feishu.constants import FeishuCommandName, FeishuReplyType
|
||||||
|
from app.modules.feishu.service import FeishuService
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
from app.modules.subscriptions.constants import (
|
||||||
|
DAILY_DELIVERY_LIMIT_REACHED,
|
||||||
|
PushDeliveryStatus,
|
||||||
|
PushSubscriptionStatus,
|
||||||
|
SubscriptionScheduleType,
|
||||||
|
SubscriptionTargetType,
|
||||||
|
)
|
||||||
|
from app.modules.subscriptions.models import PushDelivery, PushSubscription
|
||||||
|
from app.modules.subscriptions.services import (
|
||||||
|
ScheduleParseError,
|
||||||
|
SubscriptionManagementService,
|
||||||
|
parse_schedule,
|
||||||
|
)
|
||||||
|
|
||||||
|
SUBSCRIPTION_TITLE = "订阅管理"
|
||||||
|
SUBSCRIPTION_HELP = (
|
||||||
|
"订阅指令格式:\n"
|
||||||
|
"订阅 每天 09:00:提示词\n"
|
||||||
|
"订阅 工作日 18:00:提示词\n"
|
||||||
|
"订阅 每周一 09:00:提示词\n"
|
||||||
|
"订阅 每月1号 09:00:提示词\n"
|
||||||
|
"订阅 每隔30分钟:提示词\n"
|
||||||
|
"我的订阅\n"
|
||||||
|
"暂停订阅 <订阅编号>\n"
|
||||||
|
"恢复订阅 <订阅编号>\n"
|
||||||
|
"退订 <订阅编号>\n"
|
||||||
|
"设置时区 Asia/Shanghai\n"
|
||||||
|
"设置安静时段 22:00-07:00\n"
|
||||||
|
"关闭安静时段"
|
||||||
|
)
|
||||||
|
_LIST_COMMANDS = {"我的订阅", "查看订阅"}
|
||||||
|
_PAUSE_PATTERN = re.compile(
|
||||||
|
r"^(?:暂停订阅|停用订阅)\s+(SUB-[A-Za-z0-9-]+)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_RESUME_PATTERN = re.compile(
|
||||||
|
r"^(?:恢复订阅|启用订阅)\s+(SUB-[A-Za-z0-9-]+)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_CANCEL_PATTERN = re.compile(
|
||||||
|
r"^(?:退订|取消订阅)\s+(SUB-[A-Za-z0-9-]+)$",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_TIMEZONE_PATTERN = re.compile(r"^设置时区\s+(\S+)$")
|
||||||
|
_QUIET_PATTERN = re.compile(
|
||||||
|
r"^设置安静时段\s+(\d{1,2}(?:[::]\d{1,2}))"
|
||||||
|
r"\s*(?:-|~|至|到)\s*(\d{1,2}(?:[::]\d{1,2}))$"
|
||||||
|
)
|
||||||
|
_CLOSE_QUIET_COMMAND = "关闭安静时段"
|
||||||
|
_COMMAND_PREFIXES = (
|
||||||
|
"订阅",
|
||||||
|
"我的订阅",
|
||||||
|
"查看订阅",
|
||||||
|
"暂停订阅",
|
||||||
|
"停用订阅",
|
||||||
|
"恢复订阅",
|
||||||
|
"启用订阅",
|
||||||
|
"退订",
|
||||||
|
"取消订阅",
|
||||||
|
"设置时区",
|
||||||
|
"设置安静时段",
|
||||||
|
"关闭安静时段",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_subscription_command(
|
||||||
|
db: Session,
|
||||||
|
feishu: FeishuService,
|
||||||
|
command_text: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any] | None:
|
||||||
|
"""Handle self-service subscriptions for a verified Feishu principal."""
|
||||||
|
|
||||||
|
text = command_text.strip()
|
||||||
|
if not text.startswith(_COMMAND_PREFIXES):
|
||||||
|
return None
|
||||||
|
service = SubscriptionManagementService(db)
|
||||||
|
command = _command_name(text)
|
||||||
|
try:
|
||||||
|
content = _execute(service, text, principal)
|
||||||
|
except HTTPException as exc:
|
||||||
|
db.rollback()
|
||||||
|
content = _error_content(exc)
|
||||||
|
return _result(feishu, command, content, principal, auto_reply)
|
||||||
|
|
||||||
|
|
||||||
|
def _execute(
|
||||||
|
service: SubscriptionManagementService,
|
||||||
|
command_text: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
) -> str:
|
||||||
|
if command_text.startswith("订阅"):
|
||||||
|
parts = _create_parts(command_text, principal.timezone)
|
||||||
|
if parts is None:
|
||||||
|
return f"无法识别订阅时间或提示词。\n\n{SUBSCRIPTION_HELP}"
|
||||||
|
schedule_expression, prompt = parts
|
||||||
|
if principal.chat_type in {"group", "group_chat"}:
|
||||||
|
subscription, schedule = service.create_group(
|
||||||
|
principal,
|
||||||
|
schedule_expression,
|
||||||
|
prompt,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
subscription, schedule = service.create_private(
|
||||||
|
principal,
|
||||||
|
schedule_expression,
|
||||||
|
prompt,
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
"订阅已启用。\n"
|
||||||
|
f"编号:{subscription.code}\n"
|
||||||
|
f"计划:{schedule.display}\n"
|
||||||
|
f"时区:{schedule.timezone}\n"
|
||||||
|
f"下次执行:{_format_next(schedule.next_run_at, schedule.timezone)}\n"
|
||||||
|
f"暂停命令:暂停订阅 {subscription.code}"
|
||||||
|
)
|
||||||
|
if command_text in _LIST_COMMANDS:
|
||||||
|
return _list_content(
|
||||||
|
service.list_for_owner(principal),
|
||||||
|
service.latest_deliveries_for_owner(principal),
|
||||||
|
)
|
||||||
|
|
||||||
|
pause_match = _PAUSE_PATTERN.fullmatch(command_text)
|
||||||
|
if pause_match:
|
||||||
|
record = service.pause(principal, pause_match.group(1))
|
||||||
|
return f"订阅已暂停。\n编号:{record.code}\n恢复命令:恢复订阅 {record.code}"
|
||||||
|
|
||||||
|
resume_match = _RESUME_PATTERN.fullmatch(command_text)
|
||||||
|
if resume_match:
|
||||||
|
record = service.resume(principal, resume_match.group(1))
|
||||||
|
return (
|
||||||
|
"订阅已恢复。\n"
|
||||||
|
f"编号:{record.code}\n"
|
||||||
|
f"下次执行:{_format_next(record.next_run_at, record.timezone)}\n"
|
||||||
|
f"暂停命令:暂停订阅 {record.code}"
|
||||||
|
)
|
||||||
|
|
||||||
|
cancel_match = _CANCEL_PATTERN.fullmatch(command_text)
|
||||||
|
if cancel_match:
|
||||||
|
record = service.cancel(principal, cancel_match.group(1))
|
||||||
|
return f"已退订。\n编号:{record.code}"
|
||||||
|
|
||||||
|
timezone_match = _TIMEZONE_PATTERN.fullmatch(command_text)
|
||||||
|
if timezone_match:
|
||||||
|
owner = service.set_timezone(principal, timezone_match.group(1))
|
||||||
|
return f"时区已设置为 {owner.timezone}。"
|
||||||
|
|
||||||
|
quiet_match = _QUIET_PATTERN.fullmatch(command_text)
|
||||||
|
if quiet_match:
|
||||||
|
owner = service.set_quiet_hours(
|
||||||
|
principal,
|
||||||
|
quiet_match.group(1),
|
||||||
|
quiet_match.group(2),
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
"安静时段已设置。\n"
|
||||||
|
f"{owner.quiet_hours_start.strftime('%H:%M')}"
|
||||||
|
f"-{owner.quiet_hours_end.strftime('%H:%M')}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if command_text == _CLOSE_QUIET_COMMAND:
|
||||||
|
service.clear_quiet_hours(principal)
|
||||||
|
return "安静时段已关闭。"
|
||||||
|
return SUBSCRIPTION_HELP
|
||||||
|
|
||||||
|
|
||||||
|
def _create_parts(command_text: str, timezone_name: str) -> tuple[str, str] | None:
|
||||||
|
payload = command_text.removeprefix("订阅").strip()
|
||||||
|
separator_indexes = [
|
||||||
|
index for index, character in enumerate(payload) if character in {":", ":"}
|
||||||
|
]
|
||||||
|
for index in reversed(separator_indexes):
|
||||||
|
schedule_expression = payload[:index].strip()
|
||||||
|
prompt = payload[index + 1 :].strip()
|
||||||
|
if not schedule_expression or not prompt:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
parse_schedule(schedule_expression, timezone_name)
|
||||||
|
except ScheduleParseError:
|
||||||
|
continue
|
||||||
|
return schedule_expression, prompt
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _list_content(
|
||||||
|
records: list[PushSubscription],
|
||||||
|
latest_deliveries: dict[int, PushDelivery],
|
||||||
|
) -> str:
|
||||||
|
if not records:
|
||||||
|
return "当前没有订阅。\n\n" + SUBSCRIPTION_HELP.splitlines()[0]
|
||||||
|
lines = ["我的订阅:"]
|
||||||
|
for record in records:
|
||||||
|
prompt = record.prompt if len(record.prompt) <= 40 else f"{record.prompt[:40]}…"
|
||||||
|
target = (
|
||||||
|
"私聊"
|
||||||
|
if record.target_type == SubscriptionTargetType.USER
|
||||||
|
else "当前群"
|
||||||
|
)
|
||||||
|
lines.append(
|
||||||
|
f"{record.code}|{_status_name(record.status)}|{target}\n"
|
||||||
|
f"{_schedule_name(record)}|下次 {_format_next(record.next_run_at, record.timezone)}\n"
|
||||||
|
f"{prompt}{_delivery_note(latest_deliveries.get(record.id))}"
|
||||||
|
)
|
||||||
|
return "\n\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _schedule_name(record: PushSubscription) -> str:
|
||||||
|
config = record.schedule_config
|
||||||
|
if record.schedule_type == SubscriptionScheduleType.ONCE:
|
||||||
|
return "单次"
|
||||||
|
if record.schedule_type == SubscriptionScheduleType.INTERVAL:
|
||||||
|
return f"每隔 {config['minutes']} 分钟"
|
||||||
|
clock = f"{int(config['hour']):02d}:{int(config['minute']):02d}"
|
||||||
|
if record.schedule_type == SubscriptionScheduleType.DAILY:
|
||||||
|
return f"每天 {clock}"
|
||||||
|
if record.schedule_type == SubscriptionScheduleType.WEEKDAY:
|
||||||
|
return f"工作日 {clock}"
|
||||||
|
if record.schedule_type == SubscriptionScheduleType.WEEKLY:
|
||||||
|
names = "一二三四五六日"
|
||||||
|
return f"每周{names[int(config['weekday'])]} {clock}"
|
||||||
|
return f"每月 {config['day']} 号 {clock}"
|
||||||
|
|
||||||
|
|
||||||
|
def _status_name(status_value: str) -> str:
|
||||||
|
return {
|
||||||
|
PushSubscriptionStatus.ACTIVE: "已启用",
|
||||||
|
PushSubscriptionStatus.PAUSED: "已暂停",
|
||||||
|
PushSubscriptionStatus.CANCELLED: "已退订",
|
||||||
|
PushSubscriptionStatus.COMPLETED: "已完成",
|
||||||
|
}.get(status_value, status_value)
|
||||||
|
|
||||||
|
|
||||||
|
def _delivery_note(delivery: PushDelivery | None) -> str:
|
||||||
|
if delivery is None:
|
||||||
|
return ""
|
||||||
|
if (
|
||||||
|
delivery.status == PushDeliveryStatus.SKIPPED
|
||||||
|
and delivery.last_error == DAILY_DELIVERY_LIMIT_REACHED
|
||||||
|
):
|
||||||
|
return "\n最近投递:因每日最多 96 条限制已跳过"
|
||||||
|
if delivery.status == PushDeliveryStatus.RETRY:
|
||||||
|
return "\n最近投递:发送失败,正在按 1/5/15 分钟重试"
|
||||||
|
if delivery.status == PushDeliveryStatus.FAILED:
|
||||||
|
return "\n最近投递:重试后仍失败,请联系管理员"
|
||||||
|
if delivery.status == PushDeliveryStatus.SKIPPED:
|
||||||
|
return "\n最近投递:因账号、订阅状态或安静时段限制已跳过"
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _format_next(value: datetime | None, timezone_name: str) -> str:
|
||||||
|
if value is None:
|
||||||
|
return "无"
|
||||||
|
aware = value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
|
return aware.astimezone(ZoneInfo(timezone_name)).strftime("%Y-%m-%d %H:%M")
|
||||||
|
|
||||||
|
|
||||||
|
def _command_name(command_text: str) -> FeishuCommandName:
|
||||||
|
if command_text in _LIST_COMMANDS:
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_LIST
|
||||||
|
if _PAUSE_PATTERN.fullmatch(command_text):
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_PAUSE
|
||||||
|
if _RESUME_PATTERN.fullmatch(command_text):
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_RESUME
|
||||||
|
if _CANCEL_PATTERN.fullmatch(command_text):
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_CANCEL
|
||||||
|
if _TIMEZONE_PATTERN.fullmatch(command_text):
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_TIMEZONE
|
||||||
|
if _QUIET_PATTERN.fullmatch(command_text) or command_text == _CLOSE_QUIET_COMMAND:
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_QUIET_HOURS
|
||||||
|
return FeishuCommandName.SUBSCRIPTION_CREATE
|
||||||
|
|
||||||
|
|
||||||
|
def _error_content(exc: HTTPException) -> str:
|
||||||
|
detail = str(exc.detail)
|
||||||
|
if exc.status_code == 404:
|
||||||
|
return "没有找到该订阅,请先发送“我的订阅”确认编号。"
|
||||||
|
if exc.status_code == 409 and "50" in detail:
|
||||||
|
return "已达到最多 50 个启用订阅,请先暂停或退订现有订阅。"
|
||||||
|
if exc.status_code == 403:
|
||||||
|
return "当前飞书账号无权执行该订阅操作。"
|
||||||
|
return f"订阅指令未执行:{detail}\n\n{SUBSCRIPTION_HELP}"
|
||||||
|
|
||||||
|
|
||||||
|
def _result(
|
||||||
|
feishu: FeishuService,
|
||||||
|
command: FeishuCommandName,
|
||||||
|
content: str,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
auto_reply: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
response = (
|
||||||
|
send_text_if_configured(
|
||||||
|
feishu,
|
||||||
|
principal.chat_id,
|
||||||
|
content,
|
||||||
|
principal.user_code,
|
||||||
|
)
|
||||||
|
if auto_reply
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
return command_result(
|
||||||
|
command,
|
||||||
|
FeishuReplyType.TEXT,
|
||||||
|
SUBSCRIPTION_TITLE,
|
||||||
|
content,
|
||||||
|
response,
|
||||||
|
)
|
||||||
324
app/application/feishu/personal_data.py
Normal file
324
app/application/feishu/personal_data.py
Normal file
@@ -0,0 +1,324 @@
|
|||||||
|
from collections.abc import Sequence
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import delete, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.config import get_settings
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource, AuditStatus
|
||||||
|
from app.modules.audit.models import AuditLog
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FEISHU_USER_NOT_FOUND,
|
||||||
|
LAST_ACTIVE_ADMIN_ERROR,
|
||||||
|
FeishuCapability,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
parse_admin_identities,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.bootstrap import admin_bootstrap_identity_hash
|
||||||
|
from app.modules.feishu_users.models import (
|
||||||
|
FeishuAdminBootstrapTombstone,
|
||||||
|
FeishuUser,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
from app.modules.personalization.schemas import ErasureConfirmation, ErasureResult
|
||||||
|
from app.modules.personalization.services.erasure import (
|
||||||
|
ErasureHook,
|
||||||
|
PersonalDataErasureService,
|
||||||
|
)
|
||||||
|
from app.modules.subscriptions.models import PushDelivery, PushSubscription
|
||||||
|
|
||||||
|
PERSONAL_DATA_ERASURE_ACTION = "feishu.user.personal_data_erased"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuPersonalDataService:
|
||||||
|
"""Coordinate Feishu identity erasure across personal-data domains."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
db: Session,
|
||||||
|
*,
|
||||||
|
extra_hooks: Sequence[ErasureHook] = (),
|
||||||
|
) -> None:
|
||||||
|
self.db = db
|
||||||
|
self.extra_hooks = tuple(extra_hooks)
|
||||||
|
self.erasure = PersonalDataErasureService(db)
|
||||||
|
|
||||||
|
def request_confirmation(
|
||||||
|
self,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
) -> ErasureConfirmation:
|
||||||
|
"""Issue a short-lived confirmation code for the authenticated user."""
|
||||||
|
|
||||||
|
principal.require_capability(FeishuCapability.PERSONAL_DATA)
|
||||||
|
user = self._find_principal_user(principal)
|
||||||
|
if user.status != FeishuUserStatus.ACTIVE:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Feishu user is disabled",
|
||||||
|
)
|
||||||
|
return self.erasure.request_confirmation(user.id)
|
||||||
|
|
||||||
|
def confirm(
|
||||||
|
self,
|
||||||
|
principal: FeishuPrincipal,
|
||||||
|
confirmation_code: str,
|
||||||
|
) -> ErasureResult:
|
||||||
|
"""Confirm and erase the authenticated user's personal data."""
|
||||||
|
|
||||||
|
principal.require_capability(FeishuCapability.PERSONAL_DATA)
|
||||||
|
user = self._lock_principal_user(principal)
|
||||||
|
if user.status != FeishuUserStatus.ACTIVE:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Feishu user is disabled",
|
||||||
|
)
|
||||||
|
return self._confirm_and_erase(
|
||||||
|
user,
|
||||||
|
confirmation_code,
|
||||||
|
audit_source=AuditSource.FEISHU,
|
||||||
|
)
|
||||||
|
|
||||||
|
def erase_by_user_code(self, code: str, *, actor: str) -> ErasureResult:
|
||||||
|
"""Erase a user selected by an authenticated internal service."""
|
||||||
|
|
||||||
|
if not actor.strip():
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Authenticated service actor is required",
|
||||||
|
)
|
||||||
|
user = self._lock_user(code=code)
|
||||||
|
confirmation = self.erasure.request_confirmation(user.id)
|
||||||
|
|
||||||
|
# request_confirmation commits by design. Lock and re-check the last-admin
|
||||||
|
# invariant in the deletion transaction before any personal row is removed.
|
||||||
|
user = self._lock_user(code=code)
|
||||||
|
return self._confirm_and_erase(
|
||||||
|
user,
|
||||||
|
confirmation.confirmation_code,
|
||||||
|
audit_source=AuditSource.API,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _find_principal_user(self, principal: FeishuPrincipal) -> FeishuUser:
|
||||||
|
user = self.db.execute(
|
||||||
|
select(FeishuUser).where(
|
||||||
|
FeishuUser.id == principal.owner_id,
|
||||||
|
FeishuUser.code == principal.user_code,
|
||||||
|
FeishuUser.tenant_key == principal.tenant_key,
|
||||||
|
FeishuUser.open_id == principal.open_id,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if user is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=FEISHU_USER_NOT_FOUND,
|
||||||
|
)
|
||||||
|
return user
|
||||||
|
|
||||||
|
def _lock_principal_user(self, principal: FeishuPrincipal) -> FeishuUser:
|
||||||
|
return self._lock_user(
|
||||||
|
owner_id=principal.owner_id,
|
||||||
|
code=principal.user_code,
|
||||||
|
tenant_key=principal.tenant_key,
|
||||||
|
open_id=principal.open_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _lock_user(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
owner_id: int | None = None,
|
||||||
|
code: str | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
|
open_id: str | None = None,
|
||||||
|
) -> FeishuUser:
|
||||||
|
active_admin_ids = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(FeishuUser.id)
|
||||||
|
.where(
|
||||||
|
FeishuUser.role == FeishuUserRole.ADMIN,
|
||||||
|
FeishuUser.status == FeishuUserStatus.ACTIVE,
|
||||||
|
)
|
||||||
|
.order_by(FeishuUser.id.asc())
|
||||||
|
.with_for_update()
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
filters = []
|
||||||
|
if owner_id is not None:
|
||||||
|
filters.append(FeishuUser.id == owner_id)
|
||||||
|
if code is not None:
|
||||||
|
filters.append(FeishuUser.code == code)
|
||||||
|
if tenant_key is not None:
|
||||||
|
filters.append(FeishuUser.tenant_key == tenant_key)
|
||||||
|
if open_id is not None:
|
||||||
|
filters.append(FeishuUser.open_id == open_id)
|
||||||
|
user = self.db.execute(
|
||||||
|
select(FeishuUser).where(*filters).with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if user is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=FEISHU_USER_NOT_FOUND,
|
||||||
|
)
|
||||||
|
if (
|
||||||
|
user.role == FeishuUserRole.ADMIN
|
||||||
|
and user.status == FeishuUserStatus.ACTIVE
|
||||||
|
and active_admin_ids == [user.id]
|
||||||
|
):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail=LAST_ACTIVE_ADMIN_ERROR,
|
||||||
|
)
|
||||||
|
return user
|
||||||
|
|
||||||
|
def _confirm_and_erase(
|
||||||
|
self,
|
||||||
|
user: FeishuUser,
|
||||||
|
confirmation_code: str,
|
||||||
|
*,
|
||||||
|
audit_source: str,
|
||||||
|
) -> ErasureResult:
|
||||||
|
identifiers = tuple(
|
||||||
|
sorted(
|
||||||
|
{
|
||||||
|
value
|
||||||
|
for value in (
|
||||||
|
user.code,
|
||||||
|
user.open_id,
|
||||||
|
user.union_id,
|
||||||
|
user.user_id,
|
||||||
|
)
|
||||||
|
if value
|
||||||
|
},
|
||||||
|
key=len,
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
configured_admins = parse_admin_identities(
|
||||||
|
getattr(get_settings(), "feishu_admin_identities", ())
|
||||||
|
)
|
||||||
|
bootstrap_identity_hash = (
|
||||||
|
admin_bootstrap_identity_hash(user.tenant_key, user.open_id)
|
||||||
|
if (user.tenant_key, user.open_id) in configured_admins
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
|
||||||
|
def delete_subscriptions(
|
||||||
|
db: Session,
|
||||||
|
owner_id: int,
|
||||||
|
_anonymous_id: str,
|
||||||
|
) -> dict[str, int]:
|
||||||
|
subscription_ids = select(PushSubscription.id).where(
|
||||||
|
PushSubscription.owner_id == owner_id
|
||||||
|
)
|
||||||
|
deliveries = _row_count(
|
||||||
|
db.execute(
|
||||||
|
delete(PushDelivery).where(
|
||||||
|
PushDelivery.subscription_id.in_(subscription_ids)
|
||||||
|
)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
subscriptions = _row_count(
|
||||||
|
db.execute(
|
||||||
|
delete(PushSubscription).where(
|
||||||
|
PushSubscription.owner_id == owner_id
|
||||||
|
)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"deliveries": deliveries,
|
||||||
|
"subscriptions": subscriptions,
|
||||||
|
}
|
||||||
|
|
||||||
|
def finalize_identity(
|
||||||
|
db: Session,
|
||||||
|
_owner_id: int,
|
||||||
|
anonymous_id: str,
|
||||||
|
) -> dict[str, int]:
|
||||||
|
# Flush the pending confirmation-request deletion before removing
|
||||||
|
# the identity that owns it.
|
||||||
|
db.flush()
|
||||||
|
anonymized_logs = _anonymize_audit_logs(
|
||||||
|
db,
|
||||||
|
identifiers=identifiers,
|
||||||
|
anonymous_id=anonymous_id,
|
||||||
|
)
|
||||||
|
if bootstrap_identity_hash is not None:
|
||||||
|
existing_tombstone = db.scalar(
|
||||||
|
select(FeishuAdminBootstrapTombstone.id).where(
|
||||||
|
FeishuAdminBootstrapTombstone.identity_hash
|
||||||
|
== bootstrap_identity_hash
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if existing_tombstone is None:
|
||||||
|
db.add(
|
||||||
|
FeishuAdminBootstrapTombstone(
|
||||||
|
identity_hash=bootstrap_identity_hash
|
||||||
|
)
|
||||||
|
)
|
||||||
|
db.delete(user)
|
||||||
|
db.flush()
|
||||||
|
db.add(
|
||||||
|
AuditLog(
|
||||||
|
actor=anonymous_id,
|
||||||
|
source=audit_source,
|
||||||
|
action=PERSONAL_DATA_ERASURE_ACTION,
|
||||||
|
target_type=None,
|
||||||
|
target_id=None,
|
||||||
|
risk_level=AuditRiskLevel.HIGH,
|
||||||
|
request_payload=None,
|
||||||
|
response_payload=None,
|
||||||
|
status=AuditStatus.SUCCESS,
|
||||||
|
request_id=None,
|
||||||
|
created_at=utc_now(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"audit_logs_anonymized": anonymized_logs,
|
||||||
|
"identity": 1,
|
||||||
|
}
|
||||||
|
|
||||||
|
return self.erasure.confirm_and_erase(
|
||||||
|
user.id,
|
||||||
|
confirmation_code,
|
||||||
|
before_hooks=(delete_subscriptions,),
|
||||||
|
extra_hooks=(*self.extra_hooks, finalize_identity),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _anonymize_audit_logs(
|
||||||
|
db: Session,
|
||||||
|
*,
|
||||||
|
identifiers: Sequence[str],
|
||||||
|
anonymous_id: str,
|
||||||
|
) -> int:
|
||||||
|
if not identifiers:
|
||||||
|
return 0
|
||||||
|
changed_count = 0
|
||||||
|
for record in db.execute(select(AuditLog)).scalars():
|
||||||
|
searchable_values = (
|
||||||
|
record.actor,
|
||||||
|
record.target_id,
|
||||||
|
record.request_payload,
|
||||||
|
record.response_payload,
|
||||||
|
)
|
||||||
|
if not any(
|
||||||
|
identifier in value
|
||||||
|
for value in searchable_values
|
||||||
|
if value is not None
|
||||||
|
for identifier in identifiers
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
record.actor = anonymous_id
|
||||||
|
record.target_type = None
|
||||||
|
record.target_id = None
|
||||||
|
record.request_payload = None
|
||||||
|
record.response_payload = None
|
||||||
|
record.request_id = None
|
||||||
|
changed_count += 1
|
||||||
|
db.flush()
|
||||||
|
return changed_count
|
||||||
|
|
||||||
|
|
||||||
|
def _row_count(value: int | None) -> int:
|
||||||
|
return max(0, int(value or 0))
|
||||||
@@ -5,7 +5,6 @@ from sqlalchemy import select
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.core.security import business_mutations_enabled
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.application.delivery import ReportDeliveryService
|
from app.application.delivery import ReportDeliveryService
|
||||||
@@ -66,12 +65,6 @@ class LifecyclePipelineService:
|
|||||||
force: bool = False,
|
force: bool = False,
|
||||||
actor: str = ActorValue.SCHEDULER,
|
actor: str = ActorValue.SCHEDULER,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
if not business_mutations_enabled():
|
|
||||||
return {
|
|
||||||
"period_key": self.period_key(report_type),
|
|
||||||
"deduplicated": False,
|
|
||||||
"status": "operations_disabled",
|
|
||||||
}
|
|
||||||
workflow, period_key, deduplicated = self.prepare(report_type, actor, force)
|
workflow, period_key, deduplicated = self.prepare(report_type, actor, force)
|
||||||
if deduplicated:
|
if deduplicated:
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.security import business_mutations_enabled
|
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.modules.feishu.service import FeishuService
|
from app.modules.feishu.service import FeishuService
|
||||||
from app.modules.market.chart import render_market_chart
|
from app.modules.market.chart import render_market_chart
|
||||||
@@ -51,12 +50,6 @@ class MarketPipelineService:
|
|||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
target = reference_date or date.today()
|
target = reference_date or date.today()
|
||||||
period_key = self.period_key(report_type, target)
|
period_key = self.period_key(report_type, target)
|
||||||
if not business_mutations_enabled():
|
|
||||||
return {
|
|
||||||
"period_key": period_key,
|
|
||||||
"status": "operations_disabled",
|
|
||||||
"deduplicated": False,
|
|
||||||
}
|
|
||||||
existing = self.find(period_key)
|
existing = self.find(period_key)
|
||||||
if existing is not None and existing.status == WorkflowStatus.COMPLETED and not force:
|
if existing is not None and existing.status == WorkflowStatus.COMPLETED and not force:
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -46,14 +46,23 @@ def create_scheduler(app: FastAPI | None = None) -> Any:
|
|||||||
enqueue_market_report,
|
enqueue_market_report,
|
||||||
enqueue_project_weekly_push,
|
enqueue_project_weekly_push,
|
||||||
enqueue_risk_progress_push,
|
enqueue_risk_progress_push,
|
||||||
|
enqueue_subscription_cycle,
|
||||||
enqueue_work_daily_push,
|
enqueue_work_daily_push,
|
||||||
enqueue_work_weekly_push,
|
enqueue_work_weekly_push,
|
||||||
)
|
)
|
||||||
from app.modules.observability.service import ObservabilityService
|
from app.modules.observability.service import ObservabilityService
|
||||||
|
from app.modules.personalization.services import ConversationService
|
||||||
from app.modules.reports.services import ReportService
|
from app.modules.reports.services import ReportService
|
||||||
|
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
scheduler = BackgroundScheduler(timezone="Asia/Shanghai")
|
scheduler = BackgroundScheduler(
|
||||||
|
timezone="Asia/Shanghai",
|
||||||
|
job_defaults={
|
||||||
|
"coalesce": True,
|
||||||
|
"max_instances": 1,
|
||||||
|
"misfire_grace_time": 300,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
def deliver_report(
|
def deliver_report(
|
||||||
state_key: str,
|
state_key: str,
|
||||||
@@ -189,7 +198,19 @@ def create_scheduler(app: FastAPI | None = None) -> Any:
|
|||||||
finally:
|
finally:
|
||||||
db.close()
|
db.close()
|
||||||
|
|
||||||
if settings.lifecycle_pipeline_enabled and not settings.read_only_mode:
|
def run_subscription_cycle() -> None:
|
||||||
|
dispatch = enqueue_subscription_cycle(actor=ActorValue.SCHEDULER)
|
||||||
|
_set_state(app, "last_subscription_cycle", dispatch)
|
||||||
|
|
||||||
|
def run_personalization_retention_cleanup() -> None:
|
||||||
|
db = SessionLocal()
|
||||||
|
try:
|
||||||
|
deleted = ConversationService(db).cleanup_expired_globally()
|
||||||
|
_set_state(app, "last_personalization_retention_cleanup", deleted)
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
if settings.lifecycle_pipeline_enabled:
|
||||||
scheduler.add_job(
|
scheduler.add_job(
|
||||||
run_daily_lifecycle,
|
run_daily_lifecycle,
|
||||||
trigger="cron",
|
trigger="cron",
|
||||||
@@ -266,7 +287,7 @@ def create_scheduler(app: FastAPI | None = None) -> Any:
|
|||||||
id="event_dispatch",
|
id="event_dispatch",
|
||||||
replace_existing=True,
|
replace_existing=True,
|
||||||
)
|
)
|
||||||
if settings.market_analysis_enabled and not settings.read_only_mode:
|
if settings.market_analysis_enabled:
|
||||||
scheduler.add_job(
|
scheduler.add_job(
|
||||||
run_market_premarket,
|
run_market_premarket,
|
||||||
trigger="cron",
|
trigger="cron",
|
||||||
@@ -301,9 +322,23 @@ def create_scheduler(app: FastAPI | None = None) -> Any:
|
|||||||
id="scheduler_heartbeat",
|
id="scheduler_heartbeat",
|
||||||
replace_existing=True,
|
replace_existing=True,
|
||||||
)
|
)
|
||||||
|
if settings.feishu_user_features_enabled:
|
||||||
|
scheduler.add_job(
|
||||||
|
run_subscription_cycle,
|
||||||
|
trigger="interval",
|
||||||
|
minutes=1,
|
||||||
|
id="subscription_delivery_cycle",
|
||||||
|
replace_existing=True,
|
||||||
|
)
|
||||||
|
scheduler.add_job(
|
||||||
|
run_personalization_retention_cleanup,
|
||||||
|
trigger="interval",
|
||||||
|
minutes=1,
|
||||||
|
id="personalization_retention_cleanup",
|
||||||
|
replace_existing=True,
|
||||||
|
)
|
||||||
if (
|
if (
|
||||||
not settings.lifecycle_pipeline_enabled
|
not settings.lifecycle_pipeline_enabled
|
||||||
and not settings.read_only_mode
|
|
||||||
and settings.legacy_sync_enabled
|
and settings.legacy_sync_enabled
|
||||||
and settings.legacy_project_query
|
and settings.legacy_project_query
|
||||||
):
|
):
|
||||||
@@ -317,7 +352,6 @@ def create_scheduler(app: FastAPI | None = None) -> Any:
|
|||||||
)
|
)
|
||||||
if (
|
if (
|
||||||
not settings.lifecycle_pipeline_enabled
|
not settings.lifecycle_pipeline_enabled
|
||||||
and not settings.read_only_mode
|
|
||||||
and settings.legacy_sync_enabled
|
and settings.legacy_sync_enabled
|
||||||
and settings.legacy_task_query
|
and settings.legacy_task_query
|
||||||
):
|
):
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ from app.tasks.constants import (
|
|||||||
TASK_RUN_LIFECYCLE,
|
TASK_RUN_LIFECYCLE,
|
||||||
TASK_RUN_MARKET_CLOSE,
|
TASK_RUN_MARKET_CLOSE,
|
||||||
TASK_RUN_MARKET_REPORT,
|
TASK_RUN_MARKET_REPORT,
|
||||||
|
TASK_RUN_SUBSCRIPTION_CYCLE,
|
||||||
)
|
)
|
||||||
from app.core.background.task_queue.dispatcher import dispatch_task
|
from app.core.background.task_queue.dispatcher import dispatch_task
|
||||||
from app.core.background.task_queue.events import enqueue_event_dispatch
|
from app.core.background.task_queue.events import enqueue_event_dispatch
|
||||||
@@ -30,6 +31,7 @@ from app.core.background.task_queue.reports import (
|
|||||||
enqueue_work_weekly_push,
|
enqueue_work_weekly_push,
|
||||||
)
|
)
|
||||||
from app.core.background.task_queue.risk import enqueue_risk_event_generation
|
from app.core.background.task_queue.risk import enqueue_risk_event_generation
|
||||||
|
from app.core.background.task_queue.subscriptions import enqueue_subscription_cycle
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
@@ -46,6 +48,7 @@ __all__ = [
|
|||||||
"TASK_RUN_LIFECYCLE",
|
"TASK_RUN_LIFECYCLE",
|
||||||
"TASK_RUN_MARKET_CLOSE",
|
"TASK_RUN_MARKET_CLOSE",
|
||||||
"TASK_RUN_MARKET_REPORT",
|
"TASK_RUN_MARKET_REPORT",
|
||||||
|
"TASK_RUN_SUBSCRIPTION_CYCLE",
|
||||||
"dispatch_task",
|
"dispatch_task",
|
||||||
"enqueue_attendance_summary_push",
|
"enqueue_attendance_summary_push",
|
||||||
"enqueue_daily_brief_push",
|
"enqueue_daily_brief_push",
|
||||||
@@ -58,6 +61,7 @@ __all__ = [
|
|||||||
"enqueue_project_weekly_push",
|
"enqueue_project_weekly_push",
|
||||||
"enqueue_risk_progress_push",
|
"enqueue_risk_progress_push",
|
||||||
"enqueue_risk_event_generation",
|
"enqueue_risk_event_generation",
|
||||||
|
"enqueue_subscription_cycle",
|
||||||
"enqueue_work_daily_push",
|
"enqueue_work_daily_push",
|
||||||
"enqueue_work_weekly_push",
|
"enqueue_work_weekly_push",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ from app.core.config import get_settings
|
|||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.database import SessionLocal
|
from app.core.database import SessionLocal
|
||||||
from app.application.pipelines import LifecyclePipelineService
|
from app.application.pipelines import LifecyclePipelineService
|
||||||
|
from app.modules.workflows.constants import WorkflowStatus, WorkflowType
|
||||||
|
|
||||||
|
|
||||||
def enqueue_lifecycle_report(
|
def enqueue_lifecycle_report(
|
||||||
@@ -28,16 +29,28 @@ def enqueue_lifecycle_report(
|
|||||||
if get_settings().task_queue_enabled:
|
if get_settings().task_queue_enabled:
|
||||||
from app.tasks import celery_app
|
from app.tasks import celery_app
|
||||||
|
|
||||||
result = celery_app.signature(
|
try:
|
||||||
TASK_RUN_LIFECYCLE,
|
result = celery_app.signature(
|
||||||
kwargs={
|
TASK_RUN_LIFECYCLE,
|
||||||
"report_type": report_type,
|
kwargs={
|
||||||
"receive_id": receive_id,
|
"report_type": report_type,
|
||||||
"receive_id_type": receive_id_type,
|
"receive_id": receive_id,
|
||||||
"force": force,
|
"receive_id_type": receive_id_type,
|
||||||
"actor": actor,
|
"force": force,
|
||||||
},
|
"actor": actor,
|
||||||
).apply_async()
|
},
|
||||||
|
).apply_async()
|
||||||
|
except Exception as exc:
|
||||||
|
service.workflows.start_or_update(
|
||||||
|
workflow_type=WorkflowType.LIFECYCLE_REPORT,
|
||||||
|
aggregate_type="report_period",
|
||||||
|
aggregate_id=period_key,
|
||||||
|
status_value=WorkflowStatus.FAILED,
|
||||||
|
action="enqueue_failed",
|
||||||
|
actor=actor,
|
||||||
|
payload={"error": str(exc)[:2000]},
|
||||||
|
)
|
||||||
|
raise
|
||||||
return {
|
return {
|
||||||
"workflow_code": workflow.code,
|
"workflow_code": workflow.code,
|
||||||
"period_key": period_key,
|
"period_key": period_key,
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
from app.tasks.constants import (
|
from app.tasks.constants import (
|
||||||
TASK_PUSH_ATTENDANCE_SUMMARY,
|
TASK_PUSH_ATTENDANCE_SUMMARY,
|
||||||
@@ -173,37 +174,44 @@ def _queue_celery_report_push(
|
|||||||
from app.modules.reports.services import ReportService
|
from app.modules.reports.services import ReportService
|
||||||
from app.tasks import celery_app
|
from app.tasks import celery_app
|
||||||
|
|
||||||
|
task_id = uuid4().hex
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
push_run = ReportService(db).create_push_run(
|
service = ReportService(db)
|
||||||
|
push_run = service.create_push_run(
|
||||||
report_type=report_type,
|
report_type=report_type,
|
||||||
title=title,
|
title=title,
|
||||||
receive_id=receive_id,
|
receive_id=receive_id,
|
||||||
receive_id_type=receive_id_type,
|
receive_id_type=receive_id_type,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
status=ReportPushStatus.QUEUED,
|
status=ReportPushStatus.QUEUED,
|
||||||
|
task_id=task_id,
|
||||||
)
|
)
|
||||||
async_result = celery_app.signature(
|
try:
|
||||||
task_name,
|
celery_app.signature(
|
||||||
kwargs={
|
task_name,
|
||||||
"receive_id": receive_id,
|
kwargs={
|
||||||
"receive_id_type": receive_id_type,
|
"receive_id": receive_id,
|
||||||
"actor": actor,
|
"receive_id_type": receive_id_type,
|
||||||
"push_run_code": push_run.code,
|
"actor": actor,
|
||||||
},
|
"push_run_code": push_run.code,
|
||||||
).apply_async()
|
},
|
||||||
ReportService(db).update_push_run(
|
).apply_async(task_id=task_id)
|
||||||
push_run.code,
|
except Exception as exc:
|
||||||
ReportPushStatus.QUEUED,
|
service.update_push_run(
|
||||||
task_id=async_result.id,
|
push_run.code,
|
||||||
)
|
ReportPushStatus.FAILED,
|
||||||
|
task_id=task_id,
|
||||||
|
error_message=str(exc),
|
||||||
|
)
|
||||||
|
raise
|
||||||
finally:
|
finally:
|
||||||
db.close()
|
db.close()
|
||||||
return {
|
return {
|
||||||
"queued": True,
|
"queued": True,
|
||||||
"mode": "celery",
|
"mode": "celery",
|
||||||
"task_name": task_name,
|
"task_name": task_name,
|
||||||
"task_id": async_result.id,
|
"task_id": task_id,
|
||||||
"push_run_code": push_run.code,
|
"push_run_code": push_run.code,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
14
app/core/background/task_queue/subscriptions.py
Normal file
14
app/core/background/task_queue/subscriptions.py
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
from app.application.delivery.subscriptions import run_subscription_cycle
|
||||||
|
from app.core.background.task_queue.dispatcher import dispatch_task
|
||||||
|
from app.core.constants import ActorValue
|
||||||
|
from app.tasks.constants import TASK_RUN_SUBSCRIPTION_CYCLE
|
||||||
|
|
||||||
|
|
||||||
|
def enqueue_subscription_cycle(
|
||||||
|
actor: str = ActorValue.SCHEDULER,
|
||||||
|
) -> dict:
|
||||||
|
return dispatch_task(
|
||||||
|
TASK_RUN_SUBSCRIPTION_CYCLE,
|
||||||
|
{"actor": actor},
|
||||||
|
lambda: run_subscription_cycle(actor=actor),
|
||||||
|
)
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
import json
|
import json
|
||||||
|
import os
|
||||||
from functools import lru_cache
|
from functools import lru_cache
|
||||||
from typing import Annotated, Any
|
from typing import Annotated, Any, Literal
|
||||||
|
|
||||||
from pydantic import Field, field_validator, model_validator
|
from pydantic import Field, field_validator, model_validator
|
||||||
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
|
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
|
||||||
@@ -12,11 +13,22 @@ from app.core.constants import (
|
|||||||
DEFAULT_OPENCLAW_ACTION_JSON,
|
DEFAULT_OPENCLAW_ACTION_JSON,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
_DOTENV_DISABLED = os.getenv("COMPANY_AI_DISABLE_DOTENV", "").strip().lower() in {
|
||||||
|
"1",
|
||||||
|
"true",
|
||||||
|
"yes",
|
||||||
|
"on",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
class Settings(BaseSettings):
|
class Settings(BaseSettings):
|
||||||
"""Runtime settings loaded from environment variables and `.env`."""
|
"""Runtime settings loaded from environment variables and `.env`."""
|
||||||
|
|
||||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
model_config = SettingsConfigDict(
|
||||||
|
env_file=None if _DOTENV_DISABLED else ".env",
|
||||||
|
env_file_encoding="utf-8",
|
||||||
|
extra="ignore",
|
||||||
|
)
|
||||||
|
|
||||||
app_name: str = "Company AI Management Platform"
|
app_name: str = "Company AI Management Platform"
|
||||||
app_env: str = "local"
|
app_env: str = "local"
|
||||||
@@ -45,9 +57,14 @@ class Settings(BaseSettings):
|
|||||||
feishu_base_url: str = "https://open.feishu.cn/open-apis"
|
feishu_base_url: str = "https://open.feishu.cn/open-apis"
|
||||||
feishu_app_id: str | None = None
|
feishu_app_id: str | None = None
|
||||||
feishu_app_secret: str | None = None
|
feishu_app_secret: str | None = None
|
||||||
|
feishu_app_type: Literal["self", "store"] = "self"
|
||||||
|
feishu_app_ticket: str | None = None
|
||||||
feishu_verification_token: str | None = None
|
feishu_verification_token: str | None = None
|
||||||
feishu_encrypt_key: str | None = None
|
feishu_encrypt_key: str | None = None
|
||||||
feishu_default_chat_id: str | None = None
|
feishu_default_chat_id: str | None = None
|
||||||
|
feishu_default_tenant_key: str | None = None
|
||||||
|
feishu_user_features_enabled: bool = False
|
||||||
|
feishu_admin_identities: Annotated[list[str], NoDecode] = Field(default_factory=list)
|
||||||
model_provider: str = DEFAULT_MODEL_PROVIDER
|
model_provider: str = DEFAULT_MODEL_PROVIDER
|
||||||
openclaw_base_url: str = "http://127.0.0.1:2070"
|
openclaw_base_url: str = "http://127.0.0.1:2070"
|
||||||
openclaw_http_url: str | None = None
|
openclaw_http_url: str | None = None
|
||||||
@@ -109,6 +126,7 @@ class Settings(BaseSettings):
|
|||||||
event_dispatch_lock_seconds: int = 300
|
event_dispatch_lock_seconds: int = 300
|
||||||
event_dispatch_cron_minute: str = "*/5"
|
event_dispatch_cron_minute: str = "*/5"
|
||||||
heartbeat_interval_seconds: int = 60
|
heartbeat_interval_seconds: int = 60
|
||||||
|
heartbeat_retention_seconds: int = Field(default=86400, ge=1)
|
||||||
ai_memory_enabled: bool = True
|
ai_memory_enabled: bool = True
|
||||||
ai_memory_auto_write_enabled: bool = True
|
ai_memory_auto_write_enabled: bool = True
|
||||||
ai_memory_recall_limit: int = 5
|
ai_memory_recall_limit: int = 5
|
||||||
@@ -135,6 +153,8 @@ class Settings(BaseSettings):
|
|||||||
ai_memory_forbidden_keys: list[str] = Field(
|
ai_memory_forbidden_keys: list[str] = Field(
|
||||||
default_factory=lambda: [
|
default_factory=lambda: [
|
||||||
"authorization",
|
"authorization",
|
||||||
|
"app_access_token",
|
||||||
|
"app_ticket",
|
||||||
"api_key",
|
"api_key",
|
||||||
"apikey",
|
"apikey",
|
||||||
"access_token",
|
"access_token",
|
||||||
@@ -147,6 +167,7 @@ class Settings(BaseSettings):
|
|||||||
"direct_llm_api_key",
|
"direct_llm_api_key",
|
||||||
"market_data_token",
|
"market_data_token",
|
||||||
"feishu_app_secret",
|
"feishu_app_secret",
|
||||||
|
"feishu_app_ticket",
|
||||||
"feishu_verification_token",
|
"feishu_verification_token",
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
@@ -166,11 +187,20 @@ class Settings(BaseSettings):
|
|||||||
return [str(item).strip() for item in data if str(item).strip()]
|
return [str(item).strip() for item in data if str(item).strip()]
|
||||||
return [item.strip() for item in text.split(",") if item.strip()]
|
return [item.strip() for item in text.split(",") if item.strip()]
|
||||||
|
|
||||||
|
@field_validator(
|
||||||
|
"feishu_app_type",
|
||||||
|
mode="before",
|
||||||
|
)
|
||||||
|
@classmethod
|
||||||
|
def normalize_feishu_app_type(cls, value: Any) -> str:
|
||||||
|
return str(value or "self").strip().lower()
|
||||||
|
|
||||||
@field_validator(
|
@field_validator(
|
||||||
"openclaw_allowed_tools",
|
"openclaw_allowed_tools",
|
||||||
"openclaw_allowed_actions",
|
"openclaw_allowed_actions",
|
||||||
"ai_memory_forbidden_keys",
|
"ai_memory_forbidden_keys",
|
||||||
"ai_memory_blocked_content_terms",
|
"ai_memory_blocked_content_terms",
|
||||||
|
"feishu_admin_identities",
|
||||||
mode="before",
|
mode="before",
|
||||||
)
|
)
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -240,19 +270,49 @@ class Settings(BaseSettings):
|
|||||||
def validate_production_safety(self) -> "Settings":
|
def validate_production_safety(self) -> "Settings":
|
||||||
if self.app_env.lower() not in {"prod", "production"}:
|
if self.app_env.lower() not in {"prod", "production"}:
|
||||||
return self
|
return self
|
||||||
|
def _enabled_keys(
|
||||||
|
legacy_key: str | None,
|
||||||
|
configured_keys: list[dict[str, Any]],
|
||||||
|
) -> set[str]:
|
||||||
|
values: set[str] = set()
|
||||||
|
if legacy_key and legacy_key.strip():
|
||||||
|
values.add(legacy_key)
|
||||||
|
for item in configured_keys:
|
||||||
|
enabled = item.get("enabled", True)
|
||||||
|
if not isinstance(enabled, bool):
|
||||||
|
enabled = str(enabled).strip().lower() not in {
|
||||||
|
"0", "false", "no", "off", "disabled"
|
||||||
|
}
|
||||||
|
key = item.get("key")
|
||||||
|
if enabled and key is not None and str(key).strip():
|
||||||
|
values.add(str(key))
|
||||||
|
return values
|
||||||
|
|
||||||
errors: list[str] = []
|
errors: list[str] = []
|
||||||
|
api_key_values = _enabled_keys(self.api_key, self.api_keys)
|
||||||
|
audit_key_values = _enabled_keys(self.audit_api_key, self.audit_api_keys)
|
||||||
if self.database_url.startswith("sqlite"):
|
if self.database_url.startswith("sqlite"):
|
||||||
errors.append("DATABASE_URL must use PostgreSQL in production")
|
errors.append("DATABASE_URL must use PostgreSQL in production")
|
||||||
if not self.api_key and not any(item.get("key") for item in self.api_keys):
|
if not api_key_values:
|
||||||
errors.append("API_KEY or API_KEYS is required in production")
|
errors.append("API_KEY or API_KEYS is required in production")
|
||||||
if not self.audit_api_key and not any(
|
if not audit_key_values:
|
||||||
item.get("key") for item in self.audit_api_keys
|
|
||||||
):
|
|
||||||
errors.append("AUDIT_API_KEY or AUDIT_API_KEYS is required in production")
|
errors.append("AUDIT_API_KEY or AUDIT_API_KEYS is required in production")
|
||||||
|
if api_key_values & audit_key_values:
|
||||||
|
errors.append(
|
||||||
|
"API_KEY/API_KEYS and AUDIT_API_KEY/AUDIT_API_KEYS cannot overlap in production"
|
||||||
|
)
|
||||||
if "*" in self.cors_origins:
|
if "*" in self.cors_origins:
|
||||||
errors.append("CORS_ORIGINS cannot contain '*' in production")
|
errors.append("CORS_ORIGINS cannot contain '*' in production")
|
||||||
if self.debug:
|
if self.debug:
|
||||||
errors.append("DEBUG must be false in production")
|
errors.append("DEBUG must be false in production")
|
||||||
|
if not self.mask_sensitive_responses:
|
||||||
|
errors.append("MASK_SENSITIVE_RESPONSES must be true in production")
|
||||||
|
if not self.read_only_mode:
|
||||||
|
errors.append("READ_ONLY_MODE must be true in production")
|
||||||
|
if self.feishu_user_features_enabled and not self.feishu_admin_identities:
|
||||||
|
errors.append(
|
||||||
|
"FEISHU_ADMIN_IDENTITIES is required when Feishu user features are enabled"
|
||||||
|
)
|
||||||
if errors:
|
if errors:
|
||||||
raise ValueError("; ".join(errors))
|
raise ValueError("; ".join(errors))
|
||||||
return self
|
return self
|
||||||
|
|||||||
@@ -6,26 +6,61 @@ from app.core.config import get_settings
|
|||||||
MASKED_VALUE = "[MASKED]"
|
MASKED_VALUE = "[MASKED]"
|
||||||
SENSITIVE_RESPONSE_KEYS = frozenset(
|
SENSITIVE_RESPONSE_KEYS = frozenset(
|
||||||
{
|
{
|
||||||
|
"access_token",
|
||||||
"account_number",
|
"account_number",
|
||||||
"api_key",
|
"api_key",
|
||||||
|
"apikey",
|
||||||
|
"app_access_token",
|
||||||
|
"app_ticket",
|
||||||
|
"audit_api_key",
|
||||||
|
"authorization",
|
||||||
"bank_account",
|
"bank_account",
|
||||||
"card_no",
|
"card_no",
|
||||||
|
"client_secret",
|
||||||
|
"cookie",
|
||||||
"direct_llm_api_key",
|
"direct_llm_api_key",
|
||||||
"email",
|
"email",
|
||||||
|
"encrypt_key",
|
||||||
"feishu_app_secret",
|
"feishu_app_secret",
|
||||||
|
"feishu_app_ticket",
|
||||||
|
"feishu_encrypt_key",
|
||||||
"feishu_verification_token",
|
"feishu_verification_token",
|
||||||
"hermes_api_key",
|
"hermes_api_key",
|
||||||
"id_card",
|
"id_card",
|
||||||
|
"market_data_token",
|
||||||
"mobile",
|
"mobile",
|
||||||
|
"openclaw_api_key",
|
||||||
"openclaw_gateway_token",
|
"openclaw_gateway_token",
|
||||||
"password",
|
"password",
|
||||||
"payment_account",
|
"payment_account",
|
||||||
"phone",
|
"phone",
|
||||||
|
"private_key",
|
||||||
|
"refresh_token",
|
||||||
"secret",
|
"secret",
|
||||||
|
"secret_key",
|
||||||
|
"set-cookie",
|
||||||
|
"set_cookie",
|
||||||
"tenant_access_token",
|
"tenant_access_token",
|
||||||
"token",
|
"token",
|
||||||
|
"x-api-key",
|
||||||
|
"x-audit-api-key",
|
||||||
|
"x_api_key",
|
||||||
|
"x_audit_api_key",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
NORMALIZED_SENSITIVE_RESPONSE_KEYS = frozenset(
|
||||||
|
"".join(character for character in key.casefold() if character.isalnum())
|
||||||
|
for key in SENSITIVE_RESPONSE_KEYS
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def is_sensitive_key(key: str) -> bool:
|
||||||
|
"""Return whether a key matches a built-in sensitive field name."""
|
||||||
|
|
||||||
|
normalized = "".join(
|
||||||
|
character for character in key.casefold() if character.isalnum()
|
||||||
|
)
|
||||||
|
return normalized in NORMALIZED_SENSITIVE_RESPONSE_KEYS
|
||||||
|
|
||||||
|
|
||||||
def mask_configured(value: Any, domain: str | None = None) -> Any:
|
def mask_configured(value: Any, domain: str | None = None) -> Any:
|
||||||
@@ -61,7 +96,7 @@ def mask_sensitive(
|
|||||||
|
|
||||||
def _should_mask(key: str, domain: str | None, configured_fields: set[str]) -> bool:
|
def _should_mask(key: str, domain: str | None, configured_fields: set[str]) -> bool:
|
||||||
field = key.lower()
|
field = key.lower()
|
||||||
if field in SENSITIVE_RESPONSE_KEYS or field in configured_fields:
|
if is_sensitive_key(key) or field in configured_fields:
|
||||||
return True
|
return True
|
||||||
if f"*.{field}" in configured_fields:
|
if f"*.{field}" in configured_fields:
|
||||||
return True
|
return True
|
||||||
|
|||||||
@@ -1,21 +1,7 @@
|
|||||||
from app.core.security.api_keys import ApiPrincipal, require_api_key, require_audit_api_key
|
from app.core.security.api_keys import ApiPrincipal, require_api_key, require_audit_api_key
|
||||||
from app.core.security.operation_guard import (
|
|
||||||
READ_ONLY_OPERATION_DISABLED,
|
|
||||||
require_operations_enabled,
|
|
||||||
)
|
|
||||||
from app.core.security.operation_policy import (
|
|
||||||
OperationsDisabledError,
|
|
||||||
business_mutations_enabled,
|
|
||||||
ensure_business_mutations_enabled,
|
|
||||||
)
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"ApiPrincipal",
|
"ApiPrincipal",
|
||||||
"OperationsDisabledError",
|
|
||||||
"READ_ONLY_OPERATION_DISABLED",
|
|
||||||
"business_mutations_enabled",
|
|
||||||
"ensure_business_mutations_enabled",
|
|
||||||
"require_api_key",
|
"require_api_key",
|
||||||
"require_audit_api_key",
|
"require_audit_api_key",
|
||||||
"require_operations_enabled",
|
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -1,16 +0,0 @@
|
|||||||
from fastapi import HTTPException, status
|
|
||||||
|
|
||||||
from app.core.security.operation_policy import business_mutations_enabled
|
|
||||||
|
|
||||||
|
|
||||||
READ_ONLY_OPERATION_DISABLED = "This service is read-only; data mutation operations are disabled"
|
|
||||||
|
|
||||||
|
|
||||||
def require_operations_enabled(detail: str = READ_ONLY_OPERATION_DISABLED) -> None:
|
|
||||||
"""Reject mutation-oriented endpoints when the product is running read-only."""
|
|
||||||
|
|
||||||
if not business_mutations_enabled():
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=status.HTTP_405_METHOD_NOT_ALLOWED,
|
|
||||||
detail=detail,
|
|
||||||
)
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
from app.core.config import get_settings
|
|
||||||
|
|
||||||
|
|
||||||
class OperationsDisabledError(RuntimeError):
|
|
||||||
"""Raised when a business mutation is attempted in read-only mode."""
|
|
||||||
|
|
||||||
|
|
||||||
def business_mutations_enabled() -> bool:
|
|
||||||
"""Return whether platform-owned business ledgers may be mutated."""
|
|
||||||
|
|
||||||
return not get_settings().read_only_mode
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_business_mutations_enabled() -> None:
|
|
||||||
"""Enforce read-only policy outside the HTTP transport layer."""
|
|
||||||
|
|
||||||
if not business_mutations_enabled():
|
|
||||||
raise OperationsDisabledError(
|
|
||||||
"Business mutations are disabled while READ_ONLY_MODE is enabled"
|
|
||||||
)
|
|
||||||
@@ -1,12 +1,13 @@
|
|||||||
|
import json
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
|
|
||||||
from app.modules.ai_agent.constants import (
|
from app.modules.ai_agent.constants import (
|
||||||
CHAT_USER_CONTENT_TEMPLATE,
|
|
||||||
COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
||||||
UNEXPECTED_HERMES_RESPONSE,
|
UNEXPECTED_HERMES_RESPONSE,
|
||||||
AIChatRole,
|
AIChatRole,
|
||||||
|
AIContextKey,
|
||||||
AIErrorKey,
|
AIErrorKey,
|
||||||
AIHttpPayloadKey,
|
AIHttpPayloadKey,
|
||||||
AIResponseKey,
|
AIResponseKey,
|
||||||
@@ -41,6 +42,7 @@ def _chat_completion_payload(response: Any, error_key: AIErrorKey) -> dict[str,
|
|||||||
|
|
||||||
|
|
||||||
def _chat_messages(prompt: str, context: dict[str, Any] | None = None) -> list[dict[str, str]]:
|
def _chat_messages(prompt: str, context: dict[str, Any] | None = None) -> list[dict[str, str]]:
|
||||||
|
request_context = context or {}
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
AIHttpPayloadKey.ROLE: AIChatRole.SYSTEM,
|
AIHttpPayloadKey.ROLE: AIChatRole.SYSTEM,
|
||||||
@@ -48,14 +50,57 @@ def _chat_messages(prompt: str, context: dict[str, Any] | None = None) -> list[d
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
AIHttpPayloadKey.ROLE: AIChatRole.USER,
|
AIHttpPayloadKey.ROLE: AIChatRole.USER,
|
||||||
AIHttpPayloadKey.CONTENT: CHAT_USER_CONTENT_TEMPLATE.format(
|
AIHttpPayloadKey.CONTENT: _ordered_context(prompt, request_context),
|
||||||
context=context or {},
|
|
||||||
task=prompt,
|
|
||||||
),
|
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def _ordered_context(prompt: str, context: dict[str, Any]) -> str:
|
||||||
|
"""Serialize trusted personalization layers in their required precedence."""
|
||||||
|
|
||||||
|
company_rules = context.get(AIContextKey.COMPANY_RULES)
|
||||||
|
if company_rules is None:
|
||||||
|
company_rules = context.get(AIContextKey.USER_RULES) or []
|
||||||
|
controlled_keys = {
|
||||||
|
AIContextKey.USER_RULES,
|
||||||
|
AIContextKey.COMPANY_RULES,
|
||||||
|
AIContextKey.PERSONAL_RULES,
|
||||||
|
AIContextKey.PREFERENCES,
|
||||||
|
AIContextKey.INTERESTS,
|
||||||
|
AIContextKey.LOCAL_MEMORY,
|
||||||
|
AIContextKey.CONVERSATION_HISTORY,
|
||||||
|
AIContextKey.PROVIDER_SESSION_ID,
|
||||||
|
AIContextKey.ALLOW_PROVIDER_MEMORY,
|
||||||
|
}
|
||||||
|
current_context = {
|
||||||
|
str(key): value for key, value in context.items() if key not in controlled_keys
|
||||||
|
}
|
||||||
|
sections = [
|
||||||
|
("公司规则", company_rules),
|
||||||
|
("个人规则", context.get(AIContextKey.PERSONAL_RULES) or []),
|
||||||
|
(
|
||||||
|
"当前请求",
|
||||||
|
{
|
||||||
|
"prompt": prompt,
|
||||||
|
"context": current_context,
|
||||||
|
},
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"个人偏好与兴趣",
|
||||||
|
{
|
||||||
|
"preferences": context.get(AIContextKey.PREFERENCES) or [],
|
||||||
|
"interests": context.get(AIContextKey.INTERESTS) or [],
|
||||||
|
},
|
||||||
|
),
|
||||||
|
("个人相关记忆", context.get(AIContextKey.LOCAL_MEMORY) or []),
|
||||||
|
("当前会话历史", context.get(AIContextKey.CONVERSATION_HISTORY) or []),
|
||||||
|
]
|
||||||
|
return "\n\n".join(
|
||||||
|
f"{title}:\n{json.dumps(value, ensure_ascii=False, default=str)}"
|
||||||
|
for title, value in sections
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _service_root(base_url: str, suffix: str) -> str:
|
def _service_root(base_url: str, suffix: str) -> str:
|
||||||
root = base_url.rstrip("/")
|
root = base_url.rstrip("/")
|
||||||
normalized_suffix = suffix.rstrip("/")
|
normalized_suffix = suffix.rstrip("/")
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ from app.modules.ai_agent.constants import (
|
|||||||
AUTHORIZATION_BEARER_TEMPLATE,
|
AUTHORIZATION_BEARER_TEMPLATE,
|
||||||
UNEXPECTED_HERMES_RESPONSE,
|
UNEXPECTED_HERMES_RESPONSE,
|
||||||
AIErrorKey,
|
AIErrorKey,
|
||||||
|
AIContextKey,
|
||||||
AIHttpHeader,
|
AIHttpHeader,
|
||||||
AIHttpPath,
|
AIHttpPath,
|
||||||
AIHttpPayloadKey,
|
AIHttpPayloadKey,
|
||||||
@@ -38,11 +39,16 @@ class HermesAdapter(AIAdapter):
|
|||||||
headers[AIHttpHeader.AUTHORIZATION] = AUTHORIZATION_BEARER_TEMPLATE.format(
|
headers[AIHttpHeader.AUTHORIZATION] = AUTHORIZATION_BEARER_TEMPLATE.format(
|
||||||
token=self.settings.hermes_api_key
|
token=self.settings.hermes_api_key
|
||||||
)
|
)
|
||||||
if self.settings.hermes_session_id:
|
request_context = context or {}
|
||||||
headers[AIHttpHeader.HERMES_SESSION_ID] = self.settings.hermes_session_id
|
session_id = (
|
||||||
|
request_context.get(AIContextKey.PROVIDER_SESSION_ID)
|
||||||
|
or self.settings.hermes_session_id
|
||||||
|
)
|
||||||
|
if session_id:
|
||||||
|
headers[AIHttpHeader.HERMES_SESSION_ID] = str(session_id)
|
||||||
payload = {
|
payload = {
|
||||||
AIHttpPayloadKey.MODEL: self.settings.hermes_model,
|
AIHttpPayloadKey.MODEL: self.settings.hermes_model,
|
||||||
AIHttpPayloadKey.MESSAGES: _chat_messages(prompt, context),
|
AIHttpPayloadKey.MESSAGES: _chat_messages(prompt, request_context),
|
||||||
AIHttpPayloadKey.STREAM: False,
|
AIHttpPayloadKey.STREAM: False,
|
||||||
}
|
}
|
||||||
with httpx.Client(timeout=300, trust_env=False) as client:
|
with httpx.Client(timeout=300, trust_env=False) as client:
|
||||||
|
|||||||
@@ -32,7 +32,14 @@ class OpenClawHermesAdapter(AIAdapter):
|
|||||||
|
|
||||||
def ask(self, prompt: str, context: dict[str, Any] | None = None) -> dict[str, Any]:
|
def ask(self, prompt: str, context: dict[str, Any] | None = None) -> dict[str, Any]:
|
||||||
base_context = context or {}
|
base_context = context or {}
|
||||||
recall = self._recall_memory(prompt, base_context)
|
allow_provider_memory = bool(
|
||||||
|
base_context.get(AIContextKey.ALLOW_PROVIDER_MEMORY, True)
|
||||||
|
)
|
||||||
|
recall = (
|
||||||
|
self._recall_memory(prompt, base_context)
|
||||||
|
if allow_provider_memory
|
||||||
|
else {AIResponseKey.ANSWER: "", AIResponseKey.RAW: {}}
|
||||||
|
)
|
||||||
openclaw = self._openclaw_context(base_context)
|
openclaw = self._openclaw_context(base_context)
|
||||||
hermes_context = {
|
hermes_context = {
|
||||||
**base_context,
|
**base_context,
|
||||||
@@ -41,10 +48,14 @@ class OpenClawHermesAdapter(AIAdapter):
|
|||||||
AIContextKey.OPENCLAW: openclaw,
|
AIContextKey.OPENCLAW: openclaw,
|
||||||
}
|
}
|
||||||
hermes_result = self.hermes.ask(prompt, hermes_context)
|
hermes_result = self.hermes.ask(prompt, hermes_context)
|
||||||
remember = self._remember_interaction(
|
remember = (
|
||||||
prompt,
|
self._remember_interaction(
|
||||||
base_context,
|
prompt,
|
||||||
hermes_result[AIResponseKey.ANSWER],
|
base_context,
|
||||||
|
hermes_result[AIResponseKey.ANSWER],
|
||||||
|
)
|
||||||
|
if allow_provider_memory
|
||||||
|
else {AIResponseKey.OK: False, AIResponseKey.RAW: {}}
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
AIResponseKey.ANSWER: hermes_result[AIResponseKey.ANSWER],
|
AIResponseKey.ANSWER: hermes_result[AIResponseKey.ANSWER],
|
||||||
|
|||||||
@@ -60,6 +60,14 @@ class AIContextKey(StrEnum):
|
|||||||
REQUEST_CONTEXT = "request_context"
|
REQUEST_CONTEXT = "request_context"
|
||||||
ASSISTANT_ANSWER = "assistant_answer"
|
ASSISTANT_ANSWER = "assistant_answer"
|
||||||
USER_RULES = "user_rules"
|
USER_RULES = "user_rules"
|
||||||
|
COMPANY_RULES = "company_rules"
|
||||||
|
PERSONAL_RULES = "personal_rules"
|
||||||
|
PREFERENCES = "preferences"
|
||||||
|
INTERESTS = "interests"
|
||||||
|
CONVERSATION_HISTORY = "conversation_history"
|
||||||
|
PROVIDER_SESSION_ID = "provider_session_id"
|
||||||
|
ALLOW_PROVIDER_MEMORY = "allow_provider_memory"
|
||||||
|
EXECUTION_MODE = "execution_mode"
|
||||||
|
|
||||||
|
|
||||||
class AIMemoryMode(StrEnum):
|
class AIMemoryMode(StrEnum):
|
||||||
@@ -67,6 +75,14 @@ class AIMemoryMode(StrEnum):
|
|||||||
WRITE = "memory_write"
|
WRITE = "memory_write"
|
||||||
|
|
||||||
|
|
||||||
|
class AIExecutionMode(StrEnum):
|
||||||
|
INTERNAL = "internal"
|
||||||
|
PERSONALIZED = "personalized"
|
||||||
|
PREFERENCE_EXTRACTION = "preference_extraction"
|
||||||
|
SCHEDULED_PRIVATE = "scheduled_private"
|
||||||
|
SCHEDULED_GROUP = "scheduled_group"
|
||||||
|
|
||||||
|
|
||||||
class AIHttpPath(StrEnum):
|
class AIHttpPath(StrEnum):
|
||||||
CHAT_COMPLETIONS = "/chat/completions"
|
CHAT_COMPLETIONS = "/chat/completions"
|
||||||
HEALTH = "/health"
|
HEALTH = "/health"
|
||||||
@@ -95,13 +111,6 @@ class AIHttpPayloadKey(StrEnum):
|
|||||||
MESSAGE = "message"
|
MESSAGE = "message"
|
||||||
|
|
||||||
|
|
||||||
class AIToolAuditKey(StrEnum):
|
|
||||||
TOOL = "tool"
|
|
||||||
ACTION = "action"
|
|
||||||
ARGS = "args"
|
|
||||||
SESSION_KEY = "session_key"
|
|
||||||
|
|
||||||
|
|
||||||
class AIChatRole(StrEnum):
|
class AIChatRole(StrEnum):
|
||||||
SYSTEM = "system"
|
SYSTEM = "system"
|
||||||
USER = "user"
|
USER = "user"
|
||||||
@@ -133,16 +142,25 @@ CHAT_USER_CONTENT_TEMPLATE = "Context:\n{context}\n\nTask:\n{task}"
|
|||||||
AUTHORIZATION_BEARER_TEMPLATE = "Bearer {token}"
|
AUTHORIZATION_BEARER_TEMPLATE = "Bearer {token}"
|
||||||
NOOP_PROVIDER_ANSWER = (
|
NOOP_PROVIDER_ANSWER = (
|
||||||
"AI provider is not configured yet. This is a deterministic placeholder. "
|
"AI provider is not configured yet. This is a deterministic placeholder. "
|
||||||
"Set MODEL_PROVIDER to openclaw_hermes, openclaw, hermes, or direct_llm "
|
"Set MODEL_PROVIDER to openclaw_hermes, hermes, or direct_llm after "
|
||||||
"after credentials are ready."
|
"credentials are ready."
|
||||||
|
)
|
||||||
|
AI_UNAVAILABLE_ANSWER = "AI 当前不可用,请稍后重试。"
|
||||||
|
PREFERENCE_EXTRACTION_INSTRUCTIONS = (
|
||||||
|
"Extract only durable user communication preferences from the supplied user text. "
|
||||||
|
"Allowed categories are language, tone, detail, topic, and interest. "
|
||||||
|
"Return strict JSON only in this shape: "
|
||||||
|
'{"preferences":[{"category":"language","value":"中文"}]}. '
|
||||||
|
"Return an empty preferences list when there is no durable preference. "
|
||||||
|
"Never return secrets, credentials, health, religion, politics, sexual orientation, "
|
||||||
|
"performance, compensation, or confidential financial information."
|
||||||
)
|
)
|
||||||
OPENCLAW_TOOL_COMPLETED_ANSWER = "OpenClaw tool invocation completed."
|
OPENCLAW_TOOL_COMPLETED_ANSWER = "OpenClaw tool invocation completed."
|
||||||
DIRECT_LLM_API_KEY_MISSING = "DIRECT_LLM_API_KEY is not configured"
|
DIRECT_LLM_API_KEY_MISSING = "DIRECT_LLM_API_KEY is not configured"
|
||||||
UNEXPECTED_HERMES_RESPONSE = "Unexpected chat completion response"
|
UNEXPECTED_HERMES_RESPONSE = "Unexpected chat completion response"
|
||||||
OPENCLAW_CHAT_PROVIDER_REQUIRED = (
|
OPENCLAW_CHAT_PROVIDER_REQUIRED = (
|
||||||
"OpenClaw Gateway is not configured as a chat provider. "
|
"OpenClaw Gateway is not exposed as a tool-execution provider. "
|
||||||
"Provide context.openclaw_tool for /tools/invoke, or use "
|
"Use MODEL_PROVIDER=hermes, openclaw_hermes, or direct_llm for AI answers."
|
||||||
"MODEL_PROVIDER=hermes/openclaw_hermes for AI answers."
|
|
||||||
)
|
)
|
||||||
OPENCLAW_TOOL_NOT_ALLOWED = "OpenClaw tool is not allowed"
|
OPENCLAW_TOOL_NOT_ALLOWED = "OpenClaw tool is not allowed"
|
||||||
OPENCLAW_ACTION_NOT_ALLOWED = "OpenClaw action is not allowed"
|
OPENCLAW_ACTION_NOT_ALLOWED = "OpenClaw action is not allowed"
|
||||||
|
|||||||
@@ -2,14 +2,13 @@ from fastapi import APIRouter, Depends
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.audit.constants import AuditSource
|
from app.modules.audit.constants import AuditSource
|
||||||
from app.modules.ai_agent.schemas import (
|
from app.modules.ai_agent.schemas import (
|
||||||
AIAskRequest,
|
AIAskRequest,
|
||||||
AIAskResponse,
|
AIAskResponse,
|
||||||
DraftPolicyRequest,
|
DraftPolicyRequest,
|
||||||
InvestmentResearchRequest,
|
InvestmentResearchRequest,
|
||||||
OpenClawToolInvokeRequest,
|
|
||||||
)
|
)
|
||||||
from app.modules.ai_agent.service import AIService
|
from app.modules.ai_agent.service import AIService
|
||||||
|
|
||||||
@@ -38,22 +37,6 @@ def provider_health(
|
|||||||
return AIService(db).provider_health(actor=principal.actor)
|
return AIService(db).provider_health(actor=principal.actor)
|
||||||
|
|
||||||
|
|
||||||
@router.post("/openclaw/tools/invoke")
|
|
||||||
def invoke_openclaw_tool(
|
|
||||||
payload: OpenClawToolInvokeRequest,
|
|
||||||
db: Session = Depends(get_db),
|
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
|
||||||
) -> dict:
|
|
||||||
require_operations_enabled()
|
|
||||||
return AIService(db).invoke_openclaw_tool(
|
|
||||||
tool=payload.tool,
|
|
||||||
action=payload.action,
|
|
||||||
args=payload.args,
|
|
||||||
session_key=payload.session_key,
|
|
||||||
actor=principal.actor,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/draft-policy", response_model=AIAskResponse)
|
@router.post("/draft-policy", response_model=AIAskResponse)
|
||||||
def draft_policy(
|
def draft_policy(
|
||||||
payload: DraftPolicyRequest,
|
payload: DraftPolicyRequest,
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from typing import Any
|
|||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.modules.ai_agent.constants import AIDefault, AIRiskPreference
|
from app.modules.ai_agent.constants import AIRiskPreference
|
||||||
from app.modules.audit.constants import AuditSource
|
from app.modules.audit.constants import AuditSource
|
||||||
|
|
||||||
|
|
||||||
@@ -15,17 +15,11 @@ class AIAskRequest(BaseModel):
|
|||||||
|
|
||||||
|
|
||||||
class AIAskResponse(BaseModel):
|
class AIAskResponse(BaseModel):
|
||||||
|
ok: bool = True
|
||||||
provider: str
|
provider: str
|
||||||
answer: str
|
answer: str
|
||||||
raw: dict[str, Any] = Field(default_factory=dict)
|
raw: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
error: str | None = None
|
||||||
|
|
||||||
class OpenClawToolInvokeRequest(BaseModel):
|
|
||||||
tool: str
|
|
||||||
action: str = AIDefault.ACTION_JSON
|
|
||||||
args: dict[str, Any] = Field(default_factory=dict)
|
|
||||||
session_key: str = AIDefault.SESSION_KEY_MAIN
|
|
||||||
actor: str = ActorValue.API
|
|
||||||
|
|
||||||
|
|
||||||
class DraftPolicyRequest(BaseModel):
|
class DraftPolicyRequest(BaseModel):
|
||||||
|
|||||||
@@ -1,20 +1,23 @@
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.modules.ai_agent.adapters import HermesAdapter, OpenClawAdapter, get_adapter
|
from app.modules.ai_agent.adapters import HermesAdapter, OpenClawAdapter, get_adapter
|
||||||
from app.modules.ai_agent.constants import (
|
from app.modules.ai_agent.constants import (
|
||||||
AIDefault,
|
AI_UNAVAILABLE_ANSWER,
|
||||||
AI_AUDIT_MAX_DEPTH,
|
AI_AUDIT_MAX_DEPTH,
|
||||||
AI_AUDIT_MAX_SEQUENCE_ITEMS,
|
AI_AUDIT_MAX_SEQUENCE_ITEMS,
|
||||||
AI_AUDIT_MAX_TEXT_LENGTH,
|
AI_AUDIT_MAX_TEXT_LENGTH,
|
||||||
AI_AUDIT_REDACTED_VALUE,
|
AI_AUDIT_REDACTED_VALUE,
|
||||||
AI_AUDIT_SENSITIVE_KEYS,
|
AI_AUDIT_SENSITIVE_KEYS,
|
||||||
AI_AUDIT_TRUNCATED_VALUE,
|
AI_AUDIT_TRUNCATED_VALUE,
|
||||||
AIToolAuditKey,
|
COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
||||||
|
PREFERENCE_EXTRACTION_INSTRUCTIONS,
|
||||||
AIContextKey,
|
AIContextKey,
|
||||||
|
AIExecutionMode,
|
||||||
AIProviderName,
|
AIProviderName,
|
||||||
AIRequestKey,
|
AIRequestKey,
|
||||||
AIResponseKey,
|
AIResponseKey,
|
||||||
@@ -30,6 +33,13 @@ from app.modules.audit.constants import (
|
|||||||
)
|
)
|
||||||
from app.modules.audit.schemas import AuditLogCreate
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
from app.modules.audit.service import AuditService
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.personalization.services import (
|
||||||
|
ConversationService,
|
||||||
|
PersonalizationContext,
|
||||||
|
PersonalizationContextService,
|
||||||
|
PreferenceService,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.services.preferences import contains_preference_signal
|
||||||
|
|
||||||
|
|
||||||
class AIService:
|
class AIService:
|
||||||
@@ -46,15 +56,30 @@ class AIService:
|
|||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
source: str = AuditSource.API,
|
source: str = AuditSource.API,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
request_context = dict(context or {})
|
||||||
|
_validate_external_context(request_context)
|
||||||
adapter = get_adapter()
|
adapter = get_adapter()
|
||||||
original_context = context or {}
|
if _is_unavailable_adapter(adapter):
|
||||||
adapter_context = dict(original_context)
|
response = _unavailable_response(adapter.provider_name)
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=source,
|
||||||
|
request_payload={
|
||||||
|
AIRequestKey.PROMPT: prompt,
|
||||||
|
AIRequestKey.CONTEXT: request_context,
|
||||||
|
},
|
||||||
|
response=response,
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
adapter_context = dict(request_context)
|
||||||
memory_service = AIMemoryService(self.db)
|
memory_service = AIMemoryService(self.db)
|
||||||
memory_scope = _memory_scope(original_context)
|
memory_scope = _memory_scope(request_context)
|
||||||
memory_subject = _memory_subject(original_context)
|
memory_subject = _memory_subject(request_context)
|
||||||
user_rules = memory_service.active_rules(
|
user_rules = memory_service.active_rules(
|
||||||
scope=memory_scope,
|
scope=memory_scope,
|
||||||
subject=memory_subject,
|
subject=memory_subject,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
if user_rules:
|
if user_rules:
|
||||||
adapter_context[AIContextKey.USER_RULES] = user_rules
|
adapter_context[AIContextKey.USER_RULES] = user_rules
|
||||||
@@ -63,6 +88,7 @@ class AIService:
|
|||||||
scope=memory_scope,
|
scope=memory_scope,
|
||||||
subject=memory_subject,
|
subject=memory_subject,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
if local_memory:
|
if local_memory:
|
||||||
adapter_context[AIContextKey.LOCAL_MEMORY] = local_memory
|
adapter_context[AIContextKey.LOCAL_MEMORY] = local_memory
|
||||||
@@ -73,9 +99,10 @@ class AIService:
|
|||||||
raw[AIResponseKey.LOCAL_MEMORY] = local_memory
|
raw[AIResponseKey.LOCAL_MEMORY] = local_memory
|
||||||
memory_record = memory_service.auto_write(
|
memory_record = memory_service.auto_write(
|
||||||
prompt=prompt,
|
prompt=prompt,
|
||||||
context=original_context,
|
context=request_context,
|
||||||
answer=answer,
|
answer=answer,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
if memory_record is not None:
|
if memory_record is not None:
|
||||||
raw[AIResponseKey.MEMORY_WRITE] = {
|
raw[AIResponseKey.MEMORY_WRITE] = {
|
||||||
@@ -83,10 +110,253 @@ class AIService:
|
|||||||
AIMemoryPayloadKey.STATUS: memory_record.status,
|
AIMemoryPayloadKey.STATUS: memory_record.status,
|
||||||
}
|
}
|
||||||
response = {
|
response = {
|
||||||
|
AIResponseKey.OK: True,
|
||||||
AIResponseKey.PROVIDER: adapter.provider_name,
|
AIResponseKey.PROVIDER: adapter.provider_name,
|
||||||
AIResponseKey.ANSWER: answer,
|
AIResponseKey.ANSWER: answer,
|
||||||
AIResponseKey.RAW: raw,
|
AIResponseKey.RAW: raw,
|
||||||
}
|
}
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=source,
|
||||||
|
request_payload={
|
||||||
|
AIRequestKey.PROMPT: prompt,
|
||||||
|
AIRequestKey.CONTEXT: request_context,
|
||||||
|
},
|
||||||
|
response=response,
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
def ask_personalized(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str,
|
||||||
|
chat_key: str,
|
||||||
|
prompt: str,
|
||||||
|
actor: str = ActorValue.FEISHU,
|
||||||
|
source: str = AuditSource.FEISHU,
|
||||||
|
scope: str = AIMemoryScope.USER,
|
||||||
|
subject: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Answer with one verified owner's isolated personalization context."""
|
||||||
|
|
||||||
|
_validate_owner_id(owner_id)
|
||||||
|
_validate_prompt(prompt)
|
||||||
|
session_id = ConversationService.provider_session_id(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
adapter = get_adapter()
|
||||||
|
if _is_unavailable_adapter(adapter):
|
||||||
|
response = _unavailable_response(adapter.provider_name)
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=source,
|
||||||
|
request_payload={
|
||||||
|
"owner_id": owner_id,
|
||||||
|
"chat_type": chat_type,
|
||||||
|
AIContextKey.EXECUTION_MODE: AIExecutionMode.PERSONALIZED,
|
||||||
|
},
|
||||||
|
response=response,
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
memory_subject = subject or f"owner:{owner_id}"
|
||||||
|
personalization = PersonalizationContextService(self.db).build(
|
||||||
|
owner_id=owner_id,
|
||||||
|
request=prompt,
|
||||||
|
system_constraints=COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
||||||
|
chat_type=chat_type,
|
||||||
|
chat_key=chat_key,
|
||||||
|
scope=scope,
|
||||||
|
subject=memory_subject,
|
||||||
|
actor=actor,
|
||||||
|
include_company_rules=True,
|
||||||
|
include_personal_context=True,
|
||||||
|
include_history=True,
|
||||||
|
)
|
||||||
|
adapter_context = _adapter_context(
|
||||||
|
personalization,
|
||||||
|
execution_mode=AIExecutionMode.PERSONALIZED,
|
||||||
|
allow_provider_memory=False,
|
||||||
|
provider_session_id=session_id,
|
||||||
|
)
|
||||||
|
result = adapter.ask(prompt, adapter_context)
|
||||||
|
answer = _required_answer(result)
|
||||||
|
raw = dict(result.get(AIResponseKey.RAW, {}))
|
||||||
|
|
||||||
|
ConversationService(self.db).record_turn(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
user_content=prompt,
|
||||||
|
assistant_content=answer,
|
||||||
|
provider_name=str(adapter.provider_name),
|
||||||
|
)
|
||||||
|
memory_record = AIMemoryService(self.db).auto_write(
|
||||||
|
prompt=prompt,
|
||||||
|
context={
|
||||||
|
AIMemoryPayloadKey.SCOPE: scope,
|
||||||
|
AIMemoryPayloadKey.SUBJECT: memory_subject,
|
||||||
|
},
|
||||||
|
answer=answer,
|
||||||
|
actor=actor,
|
||||||
|
owner_id=owner_id,
|
||||||
|
)
|
||||||
|
if memory_record is not None:
|
||||||
|
raw[AIResponseKey.MEMORY_WRITE] = {
|
||||||
|
AIMemoryPayloadKey.CODE: memory_record.code,
|
||||||
|
AIMemoryPayloadKey.STATUS: memory_record.status,
|
||||||
|
}
|
||||||
|
saved_preferences = self._extract_preferences(
|
||||||
|
adapter=adapter,
|
||||||
|
owner_id=owner_id,
|
||||||
|
user_text=prompt,
|
||||||
|
provider_session_id=session_id,
|
||||||
|
)
|
||||||
|
if saved_preferences:
|
||||||
|
raw["preferences_saved"] = [
|
||||||
|
{
|
||||||
|
"code": item["code"],
|
||||||
|
"category": item["category"],
|
||||||
|
}
|
||||||
|
for item in saved_preferences
|
||||||
|
]
|
||||||
|
|
||||||
|
response = {
|
||||||
|
AIResponseKey.OK: True,
|
||||||
|
AIResponseKey.PROVIDER: adapter.provider_name,
|
||||||
|
AIResponseKey.ANSWER: answer,
|
||||||
|
AIResponseKey.RAW: raw,
|
||||||
|
}
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=source,
|
||||||
|
request_payload={
|
||||||
|
"owner_id": owner_id,
|
||||||
|
"chat_type": chat_type,
|
||||||
|
AIContextKey.EXECUTION_MODE: AIExecutionMode.PERSONALIZED,
|
||||||
|
},
|
||||||
|
response={
|
||||||
|
AIResponseKey.OK: True,
|
||||||
|
AIResponseKey.PROVIDER: adapter.provider_name,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
def generate_scheduled(
|
||||||
|
self,
|
||||||
|
prompt: str,
|
||||||
|
owner_id: int | None,
|
||||||
|
group: bool,
|
||||||
|
actor: str = "subscription-system",
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Generate side-effect-free scheduled content within its target boundary."""
|
||||||
|
|
||||||
|
_validate_prompt(prompt)
|
||||||
|
if not group:
|
||||||
|
if owner_id is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Private scheduled generation requires an owner",
|
||||||
|
)
|
||||||
|
_validate_owner_id(owner_id)
|
||||||
|
|
||||||
|
adapter = get_adapter()
|
||||||
|
execution_mode = (
|
||||||
|
AIExecutionMode.SCHEDULED_GROUP
|
||||||
|
if group
|
||||||
|
else AIExecutionMode.SCHEDULED_PRIVATE
|
||||||
|
)
|
||||||
|
if _is_unavailable_adapter(adapter):
|
||||||
|
response = _unavailable_response(adapter.provider_name)
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
request_payload={AIContextKey.EXECUTION_MODE: execution_mode},
|
||||||
|
response=response,
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
context_service = PersonalizationContextService(self.db)
|
||||||
|
if group:
|
||||||
|
personalization = context_service.build_group_scheduled(
|
||||||
|
request=prompt,
|
||||||
|
system_constraints=COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
personalization = context_service.build_private_scheduled(
|
||||||
|
owner_id=owner_id,
|
||||||
|
request=prompt,
|
||||||
|
system_constraints=COMPANY_MANAGEMENT_SYSTEM_INSTRUCTIONS,
|
||||||
|
actor=actor,
|
||||||
|
scope=AIMemoryScope.USER,
|
||||||
|
subject=f"owner:{owner_id}",
|
||||||
|
)
|
||||||
|
adapter_context = _adapter_context(
|
||||||
|
personalization,
|
||||||
|
execution_mode=execution_mode,
|
||||||
|
allow_provider_memory=False,
|
||||||
|
provider_session_id=None,
|
||||||
|
)
|
||||||
|
result = adapter.ask(prompt, adapter_context)
|
||||||
|
response = {
|
||||||
|
AIResponseKey.OK: True,
|
||||||
|
AIResponseKey.PROVIDER: adapter.provider_name,
|
||||||
|
AIResponseKey.ANSWER: _required_answer(result),
|
||||||
|
AIResponseKey.RAW: dict(result.get(AIResponseKey.RAW, {})),
|
||||||
|
}
|
||||||
|
self._audit_ai_response(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
request_payload={AIContextKey.EXECUTION_MODE: execution_mode},
|
||||||
|
response={
|
||||||
|
AIResponseKey.OK: True,
|
||||||
|
AIResponseKey.PROVIDER: adapter.provider_name,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return response
|
||||||
|
|
||||||
|
def _extract_preferences(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
adapter: Any,
|
||||||
|
owner_id: int,
|
||||||
|
user_text: str,
|
||||||
|
provider_session_id: str,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
if not contains_preference_signal(user_text):
|
||||||
|
return []
|
||||||
|
extraction_context = {
|
||||||
|
"preference_source_text": user_text,
|
||||||
|
AIContextKey.PROVIDER_SESSION_ID: f"{provider_session_id}-preferences",
|
||||||
|
AIContextKey.ALLOW_PROVIDER_MEMORY: False,
|
||||||
|
AIContextKey.EXECUTION_MODE: AIExecutionMode.PREFERENCE_EXTRACTION,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
extraction = adapter.ask(
|
||||||
|
PREFERENCE_EXTRACTION_INSTRUCTIONS,
|
||||||
|
extraction_context,
|
||||||
|
)
|
||||||
|
structured_payload = extraction.get(AIResponseKey.ANSWER, "")
|
||||||
|
return PreferenceService(self.db).save_auto_extraction(
|
||||||
|
owner_id,
|
||||||
|
provider_name=str(adapter.provider_name),
|
||||||
|
user_text=user_text,
|
||||||
|
structured_payload=structured_payload,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
# Preference inference must never block or change the primary answer.
|
||||||
|
return []
|
||||||
|
|
||||||
|
def _audit_ai_response(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
actor: str,
|
||||||
|
source: str,
|
||||||
|
request_payload: dict[str, Any],
|
||||||
|
response: dict[str, Any],
|
||||||
|
) -> None:
|
||||||
self.audit.log(
|
self.audit.log(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
@@ -94,14 +364,10 @@ class AIService:
|
|||||||
action=AuditAction.AI_ASK,
|
action=AuditAction.AI_ASK,
|
||||||
target_type=AuditTargetType.AI,
|
target_type=AuditTargetType.AI,
|
||||||
risk_level=AuditRiskLevel.MEDIUM,
|
risk_level=AuditRiskLevel.MEDIUM,
|
||||||
request_payload=_audit_safe_payload({
|
request_payload=_audit_safe_payload(request_payload),
|
||||||
AIRequestKey.PROMPT: prompt,
|
|
||||||
AIRequestKey.CONTEXT: context or {},
|
|
||||||
}),
|
|
||||||
response_payload=_audit_safe_payload(response),
|
response_payload=_audit_safe_payload(response),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return response
|
|
||||||
|
|
||||||
def run_skill(
|
def run_skill(
|
||||||
self,
|
self,
|
||||||
@@ -139,35 +405,6 @@ class AIService:
|
|||||||
)
|
)
|
||||||
return response
|
return response
|
||||||
|
|
||||||
def invoke_openclaw_tool(
|
|
||||||
self,
|
|
||||||
tool: str,
|
|
||||||
action: str = AIDefault.ACTION_JSON,
|
|
||||||
args: dict[str, Any] | None = None,
|
|
||||||
session_key: str = AIDefault.SESSION_KEY_MAIN,
|
|
||||||
actor: str = ActorValue.API,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
result = OpenClawAdapter(get_settings()).invoke_tool(tool, action, args or {}, session_key)
|
|
||||||
response = {AIResponseKey.PROVIDER: AIProviderName.OPENCLAW, AIResponseKey.RESULT: result}
|
|
||||||
self.audit.log(
|
|
||||||
AuditLogCreate(
|
|
||||||
actor=actor,
|
|
||||||
source=AuditSource.OPENCLAW,
|
|
||||||
action=AuditAction.OPENCLAW_TOOLS_INVOKE,
|
|
||||||
target_type=AuditTargetType.OPENCLAW_TOOL,
|
|
||||||
target_id=tool,
|
|
||||||
risk_level=AuditRiskLevel.HIGH,
|
|
||||||
request_payload=_audit_safe_payload({
|
|
||||||
AIToolAuditKey.TOOL: tool,
|
|
||||||
AIToolAuditKey.ACTION: action,
|
|
||||||
AIToolAuditKey.ARGS: args or {},
|
|
||||||
AIToolAuditKey.SESSION_KEY: session_key,
|
|
||||||
}),
|
|
||||||
response_payload=_audit_safe_payload(result),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return response
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _health_result(check: Any) -> dict[str, Any]:
|
def _health_result(check: Any) -> dict[str, Any]:
|
||||||
try:
|
try:
|
||||||
@@ -247,3 +484,103 @@ def _memory_scope(context: dict[str, Any]) -> str:
|
|||||||
def _memory_subject(context: dict[str, Any]) -> str | None:
|
def _memory_subject(context: dict[str, Any]) -> str | None:
|
||||||
value = context.get(AIContextKey.MEMORY_SUBJECT) or context.get(AIMemoryPayloadKey.SUBJECT)
|
value = context.get(AIContextKey.MEMORY_SUBJECT) or context.get(AIMemoryPayloadKey.SUBJECT)
|
||||||
return str(value) if value else None
|
return str(value) if value else None
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_external_context(context: dict[str, Any]) -> None:
|
||||||
|
controlled_keys = {
|
||||||
|
AIContextKey.OPENCLAW_TOOL,
|
||||||
|
AIContextKey.OPENCLAW_ACTION,
|
||||||
|
AIContextKey.OPENCLAW_ARGS,
|
||||||
|
AIContextKey.OPENCLAW_SESSION_KEY,
|
||||||
|
AIContextKey.AGENT_PIPELINE,
|
||||||
|
AIContextKey.HERMES_MEMORY,
|
||||||
|
AIContextKey.LOCAL_MEMORY,
|
||||||
|
AIContextKey.OPENCLAW,
|
||||||
|
AIContextKey.MODE,
|
||||||
|
AIContextKey.USER_PROMPT,
|
||||||
|
AIContextKey.REQUEST_CONTEXT,
|
||||||
|
AIContextKey.ASSISTANT_ANSWER,
|
||||||
|
AIContextKey.USER_RULES,
|
||||||
|
AIContextKey.COMPANY_RULES,
|
||||||
|
AIContextKey.PERSONAL_RULES,
|
||||||
|
AIContextKey.PREFERENCES,
|
||||||
|
AIContextKey.INTERESTS,
|
||||||
|
AIContextKey.CONVERSATION_HISTORY,
|
||||||
|
AIContextKey.PROVIDER_SESSION_ID,
|
||||||
|
AIContextKey.ALLOW_PROVIDER_MEMORY,
|
||||||
|
AIContextKey.EXECUTION_MODE,
|
||||||
|
"system_constraints",
|
||||||
|
"current_request",
|
||||||
|
"personal_memory",
|
||||||
|
"owner_id",
|
||||||
|
"tenant_key",
|
||||||
|
"open_id",
|
||||||
|
}
|
||||||
|
supplied = {str(key) for key in context}
|
||||||
|
if supplied.intersection(str(key) for key in controlled_keys):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Controlled AI context fields are not allowed",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _adapter_context(
|
||||||
|
personalization: PersonalizationContext,
|
||||||
|
*,
|
||||||
|
execution_mode: AIExecutionMode,
|
||||||
|
allow_provider_memory: bool,
|
||||||
|
provider_session_id: str | None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
context: dict[str, Any] = {
|
||||||
|
AIContextKey.COMPANY_RULES: personalization.company_rules,
|
||||||
|
AIContextKey.PERSONAL_RULES: personalization.personal_rules,
|
||||||
|
AIContextKey.PREFERENCES: personalization.preferences,
|
||||||
|
AIContextKey.INTERESTS: personalization.interests,
|
||||||
|
AIContextKey.LOCAL_MEMORY: personalization.personal_memory,
|
||||||
|
AIContextKey.CONVERSATION_HISTORY: personalization.conversation_history,
|
||||||
|
}
|
||||||
|
if provider_session_id:
|
||||||
|
context[AIContextKey.PROVIDER_SESSION_ID] = provider_session_id
|
||||||
|
context[AIContextKey.ALLOW_PROVIDER_MEMORY] = allow_provider_memory
|
||||||
|
context[AIContextKey.EXECUTION_MODE] = execution_mode
|
||||||
|
return context
|
||||||
|
|
||||||
|
|
||||||
|
def _is_unavailable_adapter(adapter: Any) -> bool:
|
||||||
|
return str(getattr(adapter, "provider_name", "")).strip().lower() == AIProviderName.NOOP
|
||||||
|
|
||||||
|
|
||||||
|
def _unavailable_response(provider_name: Any) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
AIResponseKey.OK: False,
|
||||||
|
AIResponseKey.PROVIDER: str(provider_name or AIProviderName.NOOP),
|
||||||
|
AIResponseKey.ANSWER: AI_UNAVAILABLE_ANSWER,
|
||||||
|
AIResponseKey.RAW: {"reason": "provider_unavailable"},
|
||||||
|
AIResponseKey.ERROR: "AI provider unavailable",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _required_answer(result: dict[str, Any]) -> str:
|
||||||
|
answer = str(result.get(AIResponseKey.ANSWER) or "").strip()
|
||||||
|
if not answer:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||||
|
detail="AI provider returned an empty answer",
|
||||||
|
)
|
||||||
|
return answer
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_owner_id(owner_id: int) -> None:
|
||||||
|
if owner_id <= 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Valid AI owner is required",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_prompt(prompt: str) -> None:
|
||||||
|
if not str(prompt).strip():
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="AI prompt is required",
|
||||||
|
)
|
||||||
|
|||||||
@@ -19,6 +19,13 @@ class AIMemorySource(StrEnum):
|
|||||||
HERMES = "hermes"
|
HERMES = "hermes"
|
||||||
API = "api"
|
API = "api"
|
||||||
USER_RULE = "user_rule"
|
USER_RULE = "user_rule"
|
||||||
|
LEGACY_COMPANY = "legacy_company"
|
||||||
|
|
||||||
|
|
||||||
|
class AIMemoryKind(StrEnum):
|
||||||
|
COMPANY_RULE = "company_rule"
|
||||||
|
PERSONAL_RULE = "personal_rule"
|
||||||
|
MEMORY = "memory"
|
||||||
|
|
||||||
|
|
||||||
class AIMemoryResponseKey(StrEnum):
|
class AIMemoryResponseKey(StrEnum):
|
||||||
|
|||||||
@@ -1,19 +1,46 @@
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import JSON, DateTime, Integer, String, Text
|
from sqlalchemy import JSON, DateTime, ForeignKey, Integer, String, Text, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.modules.ai_memory.constants import AIMemoryScope, AIMemorySource, AIMemoryStatus
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
|
from app.modules.ai_memory.constants import (
|
||||||
|
AIMemoryKind,
|
||||||
|
AIMemoryScope,
|
||||||
|
AIMemorySource,
|
||||||
|
AIMemoryStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class AIMemoryEntry(Base):
|
class AIMemoryEntry(Base):
|
||||||
__tablename__ = "ai_memory_entries"
|
__tablename__ = "ai_memory_entries"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"owner_id",
|
||||||
|
"fingerprint",
|
||||||
|
name="uq_ai_memory_owner_fingerprint",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
code: Mapped[str] = mapped_column(String(64), unique=True, index=True)
|
code: Mapped[str] = mapped_column(String(64), unique=True, index=True)
|
||||||
|
fingerprint: Mapped[str | None] = mapped_column(
|
||||||
|
String(64), nullable=True, index=True
|
||||||
|
)
|
||||||
|
owner_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("feishu_users.id", ondelete="CASCADE"),
|
||||||
|
nullable=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner: Mapped[FeishuUser | None] = relationship()
|
||||||
|
kind: Mapped[str] = mapped_column(
|
||||||
|
String(32),
|
||||||
|
default=AIMemoryKind.MEMORY,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
scope: Mapped[str] = mapped_column(String(64), default=AIMemoryScope.GLOBAL, index=True)
|
scope: Mapped[str] = mapped_column(String(64), default=AIMemoryScope.GLOBAL, index=True)
|
||||||
subject: Mapped[str] = mapped_column(String(128), index=True)
|
subject: Mapped[str] = mapped_column(String(128), index=True)
|
||||||
content: Mapped[str] = mapped_column(Text)
|
content: Mapped[str] = mapped_column(Text)
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ from fastapi import APIRouter, Depends, Query
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.ai_memory.constants import AIMemoryResponseKey, AIMemoryStatus
|
from app.modules.ai_memory.constants import AIMemoryResponseKey, AIMemoryStatus
|
||||||
from app.modules.ai_memory.schemas import (
|
from app.modules.ai_memory.schemas import (
|
||||||
AIMemoryRecallRequest,
|
AIMemoryRecallRequest,
|
||||||
@@ -22,14 +22,15 @@ def list_memory(
|
|||||||
limit: int = Query(default=100, ge=1, le=500),
|
limit: int = Query(default=100, ge=1, le=500),
|
||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
return {
|
items = AIMemoryService(db).list_entries(
|
||||||
AIMemoryResponseKey.ITEMS: AIMemoryService(db).list_entries(
|
scope=scope,
|
||||||
scope=scope,
|
subject=subject,
|
||||||
subject=subject,
|
status_filter=status,
|
||||||
status_filter=status,
|
limit=limit,
|
||||||
limit=limit,
|
owner_id=None,
|
||||||
)
|
)
|
||||||
}
|
db.commit()
|
||||||
|
return {AIMemoryResponseKey.ITEMS: items}
|
||||||
|
|
||||||
|
|
||||||
@router.post("/memory/recall")
|
@router.post("/memory/recall")
|
||||||
@@ -44,6 +45,7 @@ def recall_memory(
|
|||||||
subject=payload.subject,
|
subject=payload.subject,
|
||||||
limit=payload.limit,
|
limit=payload.limit,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
return {AIMemoryResponseKey.ITEMS: items}
|
return {AIMemoryResponseKey.ITEMS: items}
|
||||||
|
|
||||||
@@ -62,6 +64,7 @@ def list_rules(
|
|||||||
subject=subject,
|
subject=subject,
|
||||||
status_filter=status,
|
status_filter=status,
|
||||||
limit=limit,
|
limit=limit,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -72,7 +75,6 @@ def create_rule(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return {
|
return {
|
||||||
AIMemoryResponseKey.DATA: AIMemoryService(db).create_rule(
|
AIMemoryResponseKey.DATA: AIMemoryService(db).create_rule(
|
||||||
content=payload.content,
|
content=payload.content,
|
||||||
@@ -81,6 +83,7 @@ def create_rule(
|
|||||||
priority=payload.priority,
|
priority=payload.priority,
|
||||||
tags=payload.tags,
|
tags=payload.tags,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -92,7 +95,6 @@ def update_rule(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return {
|
return {
|
||||||
AIMemoryResponseKey.DATA: AIMemoryService(db).update_rule(
|
AIMemoryResponseKey.DATA: AIMemoryService(db).update_rule(
|
||||||
code=code,
|
code=code,
|
||||||
@@ -101,5 +103,20 @@ def update_rule(
|
|||||||
tags=payload.tags,
|
tags=payload.tags,
|
||||||
enabled=payload.enabled,
|
enabled=payload.enabled,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
|
owner_id=None,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/rules/{code}")
|
||||||
|
def delete_rule(
|
||||||
|
code: str,
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
|
) -> dict:
|
||||||
|
AIMemoryService(db).delete_rule(
|
||||||
|
code=code,
|
||||||
|
actor=principal.actor,
|
||||||
|
owner_id=None,
|
||||||
|
)
|
||||||
|
return {AIMemoryResponseKey.DATA: {"code": code, "deleted": True}}
|
||||||
|
|||||||
@@ -14,6 +14,8 @@ class AIMemoryRecallRequest(BaseModel):
|
|||||||
|
|
||||||
class AIMemoryRead(BaseModel):
|
class AIMemoryRead(BaseModel):
|
||||||
code: str
|
code: str
|
||||||
|
owner_id: int | None
|
||||||
|
kind: str
|
||||||
scope: str
|
scope: str
|
||||||
subject: str
|
subject: str
|
||||||
content: str
|
content: str
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
|
from hashlib import sha256
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
from sqlalchemy import func, or_, select
|
from sqlalchemy import delete, func, or_, select, update
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
@@ -15,6 +17,7 @@ from app.modules.ai_memory.constants import (
|
|||||||
AI_MEMORY_MAX_CONTENT_LENGTH,
|
AI_MEMORY_MAX_CONTENT_LENGTH,
|
||||||
AI_MEMORY_MAX_SUMMARY_LENGTH,
|
AI_MEMORY_MAX_SUMMARY_LENGTH,
|
||||||
AI_MEMORY_MIN_AUTO_WRITE_LENGTH,
|
AI_MEMORY_MIN_AUTO_WRITE_LENGTH,
|
||||||
|
AIMemoryKind,
|
||||||
AIMemoryPayloadKey,
|
AIMemoryPayloadKey,
|
||||||
AIMemoryScope,
|
AIMemoryScope,
|
||||||
AIMemorySource,
|
AIMemorySource,
|
||||||
@@ -50,10 +53,15 @@ class AIMemoryService:
|
|||||||
subject: str | None = None,
|
subject: str | None = None,
|
||||||
status_filter: str = AIMemoryStatus.ACTIVE,
|
status_filter: str = AIMemoryStatus.ACTIVE,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
|
self._archive_expired()
|
||||||
stmt = (
|
stmt = (
|
||||||
select(AIMemoryEntry)
|
select(AIMemoryEntry)
|
||||||
.where(AIMemoryEntry.status == status_filter)
|
.where(
|
||||||
|
AIMemoryEntry.status == status_filter,
|
||||||
|
_owner_filter(owner_id),
|
||||||
|
)
|
||||||
.order_by(AIMemoryEntry.importance.desc(), AIMemoryEntry.id.desc())
|
.order_by(AIMemoryEntry.importance.desc(), AIMemoryEntry.id.desc())
|
||||||
.limit(bounded_limit(limit))
|
.limit(bounded_limit(limit))
|
||||||
)
|
)
|
||||||
@@ -70,16 +78,20 @@ class AIMemoryService:
|
|||||||
subject: str | None = None,
|
subject: str | None = None,
|
||||||
limit: int | None = None,
|
limit: int | None = None,
|
||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
if not settings.ai_memory_enabled:
|
if not settings.ai_memory_enabled:
|
||||||
return []
|
return []
|
||||||
limit_value = bounded_limit(limit or settings.ai_memory_recall_limit)
|
limit_value = bounded_limit(limit or settings.ai_memory_recall_limit)
|
||||||
|
self._archive_expired()
|
||||||
now = utc_now()
|
now = utc_now()
|
||||||
stmt = (
|
stmt = (
|
||||||
select(AIMemoryEntry)
|
select(AIMemoryEntry)
|
||||||
.where(
|
.where(
|
||||||
AIMemoryEntry.status == AIMemoryStatus.ACTIVE,
|
AIMemoryEntry.status == AIMemoryStatus.ACTIVE,
|
||||||
|
AIMemoryEntry.kind == AIMemoryKind.MEMORY,
|
||||||
|
_owner_filter(owner_id),
|
||||||
or_(AIMemoryEntry.expires_at.is_(None), AIMemoryEntry.expires_at > now),
|
or_(AIMemoryEntry.expires_at.is_(None), AIMemoryEntry.expires_at > now),
|
||||||
AIMemoryEntry.scope.in_({AIMemoryScope.GLOBAL, scope}),
|
AIMemoryEntry.scope.in_({AIMemoryScope.GLOBAL, scope}),
|
||||||
)
|
)
|
||||||
@@ -95,8 +107,6 @@ class AIMemoryService:
|
|||||||
)
|
)
|
||||||
candidates = list(self.db.execute(stmt).scalars())
|
candidates = list(self.db.execute(stmt).scalars())
|
||||||
items = [item for item in candidates if _matches_query(item, query)]
|
items = [item for item in candidates if _matches_query(item, query)]
|
||||||
if not items:
|
|
||||||
items = candidates[:limit_value]
|
|
||||||
items = items[:limit_value]
|
items = items[:limit_value]
|
||||||
for item in items:
|
for item in items:
|
||||||
item.last_used_at = now
|
item.last_used_at = now
|
||||||
@@ -126,11 +136,13 @@ class AIMemoryService:
|
|||||||
context: dict[str, Any],
|
context: dict[str, Any],
|
||||||
answer: str,
|
answer: str,
|
||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> AIMemoryEntry | None:
|
) -> AIMemoryEntry | None:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
if not settings.ai_memory_enabled or not settings.ai_memory_auto_write_enabled:
|
if not settings.ai_memory_enabled or not settings.ai_memory_auto_write_enabled:
|
||||||
return None
|
return None
|
||||||
content = _build_memory_content(prompt, context, answer)
|
content = _build_memory_content(prompt, context, answer)
|
||||||
|
self._archive_expired()
|
||||||
if len(content) < AI_MEMORY_MIN_AUTO_WRITE_LENGTH:
|
if len(content) < AI_MEMORY_MIN_AUTO_WRITE_LENGTH:
|
||||||
return None
|
return None
|
||||||
scope = str(context.get(AIMemoryPayloadKey.SCOPE) or AIMemoryText.DEFAULT_SCOPE)
|
scope = str(context.get(AIMemoryPayloadKey.SCOPE) or AIMemoryText.DEFAULT_SCOPE)
|
||||||
@@ -143,39 +155,60 @@ class AIMemoryService:
|
|||||||
},
|
},
|
||||||
settings.ai_memory_forbidden_keys,
|
settings.ai_memory_forbidden_keys,
|
||||||
):
|
):
|
||||||
|
safe_content = str(AIMemoryText.REJECTED_SECRET)
|
||||||
record = self._create_entry(
|
record = self._create_entry(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
subject=subject,
|
subject=subject,
|
||||||
content=str(AIMemoryText.REJECTED_SECRET),
|
content=safe_content,
|
||||||
summary=str(AIMemoryText.REJECTED_SECRET),
|
summary=safe_content,
|
||||||
tags=[str(AIMemoryText.AUTO_TAG)],
|
tags=[str(AIMemoryText.AUTO_TAG)],
|
||||||
source=AIMemorySource.AUTO,
|
source=AIMemorySource.AUTO,
|
||||||
importance=0,
|
importance=0,
|
||||||
status_value=AIMemoryStatus.REJECTED,
|
status_value=AIMemoryStatus.REJECTED,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
fingerprint=_memory_fingerprint(
|
||||||
|
owner_id,
|
||||||
|
scope,
|
||||||
|
subject,
|
||||||
|
content,
|
||||||
|
AIMemoryStatus.REJECTED,
|
||||||
|
),
|
||||||
|
owner_id=owner_id,
|
||||||
|
kind=AIMemoryKind.MEMORY,
|
||||||
expires_at=utc_now()
|
expires_at=utc_now()
|
||||||
+ timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
+ timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
||||||
)
|
)
|
||||||
return record
|
return record
|
||||||
if _contains_blocked_content(content, settings.ai_memory_blocked_content_terms):
|
if _contains_blocked_content(content, settings.ai_memory_blocked_content_terms):
|
||||||
|
safe_content = str(AIMemoryText.REJECTED_SENSITIVE_FACT)
|
||||||
return self._create_entry(
|
return self._create_entry(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
subject=subject,
|
subject=subject,
|
||||||
content=str(AIMemoryText.REJECTED_SENSITIVE_FACT),
|
content=safe_content,
|
||||||
summary=str(AIMemoryText.REJECTED_SENSITIVE_FACT),
|
summary=safe_content,
|
||||||
tags=[str(AIMemoryText.AUTO_TAG)],
|
tags=[str(AIMemoryText.AUTO_TAG)],
|
||||||
source=AIMemorySource.AUTO,
|
source=AIMemorySource.AUTO,
|
||||||
importance=0,
|
importance=0,
|
||||||
status_value=AIMemoryStatus.REJECTED,
|
status_value=AIMemoryStatus.REJECTED,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
|
fingerprint=_memory_fingerprint(
|
||||||
|
owner_id,
|
||||||
|
scope,
|
||||||
|
subject,
|
||||||
|
content,
|
||||||
|
AIMemoryStatus.REJECTED,
|
||||||
|
),
|
||||||
|
owner_id=owner_id,
|
||||||
|
kind=AIMemoryKind.MEMORY,
|
||||||
expires_at=utc_now()
|
expires_at=utc_now()
|
||||||
+ timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
+ timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
||||||
)
|
)
|
||||||
summary = _truncate(answer, AI_MEMORY_MAX_SUMMARY_LENGTH)
|
summary = _truncate(answer, AI_MEMORY_MAX_SUMMARY_LENGTH)
|
||||||
|
stored_content = _truncate(content, AI_MEMORY_MAX_CONTENT_LENGTH)
|
||||||
record = self._create_entry(
|
record = self._create_entry(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
subject=subject,
|
subject=subject,
|
||||||
content=_truncate(content, AI_MEMORY_MAX_CONTENT_LENGTH),
|
content=stored_content,
|
||||||
summary=summary,
|
summary=summary,
|
||||||
tags=[str(AIMemoryText.AUTO_TAG)],
|
tags=[str(AIMemoryText.AUTO_TAG)],
|
||||||
source=AIMemorySource.AUTO,
|
source=AIMemorySource.AUTO,
|
||||||
@@ -183,6 +216,15 @@ class AIMemoryService:
|
|||||||
status_value=AIMemoryStatus.ACTIVE,
|
status_value=AIMemoryStatus.ACTIVE,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
expires_at=utc_now() + timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
expires_at=utc_now() + timedelta(days=settings.ai_memory_auto_write_ttl_days),
|
||||||
|
fingerprint=_memory_fingerprint(
|
||||||
|
owner_id,
|
||||||
|
scope,
|
||||||
|
subject,
|
||||||
|
stored_content,
|
||||||
|
AIMemoryStatus.ACTIVE,
|
||||||
|
),
|
||||||
|
owner_id=owner_id,
|
||||||
|
kind=AIMemoryKind.MEMORY,
|
||||||
)
|
)
|
||||||
return record
|
return record
|
||||||
|
|
||||||
@@ -192,10 +234,19 @@ class AIMemoryService:
|
|||||||
subject: str | None = None,
|
subject: str | None = None,
|
||||||
status_filter: str | None = None,
|
status_filter: str | None = None,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
|
kind = (
|
||||||
|
AIMemoryKind.COMPANY_RULE
|
||||||
|
if owner_id is None
|
||||||
|
else AIMemoryKind.PERSONAL_RULE
|
||||||
|
)
|
||||||
stmt = (
|
stmt = (
|
||||||
select(AIMemoryEntry)
|
select(AIMemoryEntry)
|
||||||
.where(AIMemoryEntry.source == AIMemorySource.USER_RULE)
|
.where(
|
||||||
|
AIMemoryEntry.kind == kind,
|
||||||
|
_owner_filter(owner_id),
|
||||||
|
)
|
||||||
.order_by(AIMemoryEntry.importance.desc(), AIMemoryEntry.id.asc())
|
.order_by(AIMemoryEntry.importance.desc(), AIMemoryEntry.id.asc())
|
||||||
.limit(bounded_limit(limit))
|
.limit(bounded_limit(limit))
|
||||||
)
|
)
|
||||||
@@ -212,11 +263,19 @@ class AIMemoryService:
|
|||||||
scope: str = AIMemoryScope.GLOBAL,
|
scope: str = AIMemoryScope.GLOBAL,
|
||||||
subject: str | None = None,
|
subject: str | None = None,
|
||||||
limit: int = 50,
|
limit: int = 50,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
|
self._archive_expired()
|
||||||
|
kind = (
|
||||||
|
AIMemoryKind.COMPANY_RULE
|
||||||
|
if owner_id is None
|
||||||
|
else AIMemoryKind.PERSONAL_RULE
|
||||||
|
)
|
||||||
stmt = (
|
stmt = (
|
||||||
select(AIMemoryEntry)
|
select(AIMemoryEntry)
|
||||||
.where(
|
.where(
|
||||||
AIMemoryEntry.source == AIMemorySource.USER_RULE,
|
AIMemoryEntry.kind == kind,
|
||||||
|
_owner_filter(owner_id),
|
||||||
AIMemoryEntry.status == AIMemoryStatus.ACTIVE,
|
AIMemoryEntry.status == AIMemoryStatus.ACTIVE,
|
||||||
AIMemoryEntry.scope.in_({AIMemoryScope.GLOBAL, scope}),
|
AIMemoryEntry.scope.in_({AIMemoryScope.GLOBAL, scope}),
|
||||||
)
|
)
|
||||||
@@ -249,6 +308,7 @@ class AIMemoryService:
|
|||||||
priority: int,
|
priority: int,
|
||||||
tags: list[str] | None,
|
tags: list[str] | None,
|
||||||
actor: str,
|
actor: str,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
self._validate_rule(content, priority)
|
self._validate_rule(content, priority)
|
||||||
record = self._create_entry(
|
record = self._create_entry(
|
||||||
@@ -262,6 +322,12 @@ class AIMemoryService:
|
|||||||
status_value=AIMemoryStatus.ACTIVE,
|
status_value=AIMemoryStatus.ACTIVE,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
audit_action=AuditAction.AI_RULE_CREATE,
|
audit_action=AuditAction.AI_RULE_CREATE,
|
||||||
|
owner_id=owner_id,
|
||||||
|
kind=(
|
||||||
|
AIMemoryKind.COMPANY_RULE
|
||||||
|
if owner_id is None
|
||||||
|
else AIMemoryKind.PERSONAL_RULE
|
||||||
|
),
|
||||||
)
|
)
|
||||||
return serialize_model(record)
|
return serialize_model(record)
|
||||||
|
|
||||||
@@ -273,11 +339,18 @@ class AIMemoryService:
|
|||||||
tags: list[str] | None,
|
tags: list[str] | None,
|
||||||
enabled: bool | None,
|
enabled: bool | None,
|
||||||
actor: str,
|
actor: str,
|
||||||
|
owner_id: int | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
kind = (
|
||||||
|
AIMemoryKind.COMPANY_RULE
|
||||||
|
if owner_id is None
|
||||||
|
else AIMemoryKind.PERSONAL_RULE
|
||||||
|
)
|
||||||
record = self.db.execute(
|
record = self.db.execute(
|
||||||
select(AIMemoryEntry).where(
|
select(AIMemoryEntry).where(
|
||||||
AIMemoryEntry.code == code,
|
AIMemoryEntry.code == code,
|
||||||
AIMemoryEntry.source == AIMemorySource.USER_RULE,
|
AIMemoryEntry.kind == kind,
|
||||||
|
_owner_filter(owner_id),
|
||||||
)
|
)
|
||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
if record is None:
|
if record is None:
|
||||||
@@ -312,6 +385,50 @@ class AIMemoryService:
|
|||||||
self.db.refresh(record)
|
self.db.refresh(record)
|
||||||
return serialize_model(record)
|
return serialize_model(record)
|
||||||
|
|
||||||
|
def delete_rule(
|
||||||
|
self,
|
||||||
|
code: str,
|
||||||
|
actor: str,
|
||||||
|
owner_id: int | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Delete a rule only within the requested company or personal owner scope."""
|
||||||
|
|
||||||
|
kind = (
|
||||||
|
AIMemoryKind.COMPANY_RULE
|
||||||
|
if owner_id is None
|
||||||
|
else AIMemoryKind.PERSONAL_RULE
|
||||||
|
)
|
||||||
|
record = self.db.execute(
|
||||||
|
select(AIMemoryEntry).where(
|
||||||
|
AIMemoryEntry.code == code,
|
||||||
|
AIMemoryEntry.kind == kind,
|
||||||
|
_owner_filter(owner_id),
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="AI rule not found")
|
||||||
|
self.db.delete(record)
|
||||||
|
self.audit.record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.AI_MEMORY,
|
||||||
|
action=AuditAction.AI_RULE_UPDATE,
|
||||||
|
target_type=AuditTargetType.AI_MEMORY,
|
||||||
|
target_id=code,
|
||||||
|
risk_level=AuditRiskLevel.MEDIUM,
|
||||||
|
response_payload={"deleted": True},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
|
||||||
|
def delete_owner_entries(self, owner_id: int) -> int:
|
||||||
|
"""Stage deletion of all personal rules and memories for an owner."""
|
||||||
|
|
||||||
|
result = self.db.execute(
|
||||||
|
delete(AIMemoryEntry).where(AIMemoryEntry.owner_id == owner_id)
|
||||||
|
)
|
||||||
|
return max(0, int(result.rowcount or 0))
|
||||||
|
|
||||||
def _validate_rule(self, content: str, priority: int) -> None:
|
def _validate_rule(self, content: str, priority: int) -> None:
|
||||||
if not AI_USER_RULE_MIN_PRIORITY <= priority <= AI_USER_RULE_MAX_PRIORITY:
|
if not AI_USER_RULE_MIN_PRIORITY <= priority <= AI_USER_RULE_MAX_PRIORITY:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
@@ -325,11 +442,46 @@ class AIMemoryService:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def count_by_status(self) -> dict[str, int]:
|
def count_by_status(self) -> dict[str, int]:
|
||||||
|
self._archive_expired()
|
||||||
rows = self.db.execute(
|
rows = self.db.execute(
|
||||||
select(AIMemoryEntry.status, func.count()).group_by(AIMemoryEntry.status)
|
select(AIMemoryEntry.status, func.count()).group_by(AIMemoryEntry.status)
|
||||||
).all()
|
).all()
|
||||||
return {str(status_value): int(count) for status_value, count in rows}
|
return {str(status_value): int(count) for status_value, count in rows}
|
||||||
|
|
||||||
|
def _archive_expired(self) -> int:
|
||||||
|
now = utc_now()
|
||||||
|
result = self.db.execute(
|
||||||
|
update(AIMemoryEntry)
|
||||||
|
.where(
|
||||||
|
AIMemoryEntry.status.in_(
|
||||||
|
{
|
||||||
|
AIMemoryStatus.ACTIVE,
|
||||||
|
AIMemoryStatus.REJECTED,
|
||||||
|
}
|
||||||
|
),
|
||||||
|
AIMemoryEntry.expires_at.is_not(None),
|
||||||
|
AIMemoryEntry.expires_at <= now,
|
||||||
|
)
|
||||||
|
.values(
|
||||||
|
status=AIMemoryStatus.ARCHIVED,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
archived = max(0, int(result.rowcount or 0))
|
||||||
|
return archived
|
||||||
|
|
||||||
|
def _find_by_fingerprint(
|
||||||
|
self,
|
||||||
|
owner_id: int | None,
|
||||||
|
fingerprint: str,
|
||||||
|
) -> AIMemoryEntry | None:
|
||||||
|
return self.db.execute(
|
||||||
|
select(AIMemoryEntry).where(
|
||||||
|
AIMemoryEntry.fingerprint == fingerprint,
|
||||||
|
_owner_filter(owner_id),
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
def _create_entry(
|
def _create_entry(
|
||||||
self,
|
self,
|
||||||
scope: str,
|
scope: str,
|
||||||
@@ -343,12 +495,23 @@ class AIMemoryService:
|
|||||||
actor: str,
|
actor: str,
|
||||||
audit_action: str = AuditAction.AI_MEMORY_WRITE,
|
audit_action: str = AuditAction.AI_MEMORY_WRITE,
|
||||||
expires_at: datetime | None = None,
|
expires_at: datetime | None = None,
|
||||||
|
fingerprint: str | None = None,
|
||||||
|
owner_id: int | None = None,
|
||||||
|
kind: str = AIMemoryKind.MEMORY,
|
||||||
) -> AIMemoryEntry:
|
) -> AIMemoryEntry:
|
||||||
|
if fingerprint:
|
||||||
|
existing = self._find_by_fingerprint(owner_id, fingerprint)
|
||||||
|
if existing is not None:
|
||||||
|
return self._reuse_entry(existing, status_value, expires_at)
|
||||||
|
|
||||||
record = AIMemoryEntry(
|
record = AIMemoryEntry(
|
||||||
code=(
|
code=(
|
||||||
f"{AI_MEMORY_CODE_PREFIX}-{utc_now():%Y%m%d%H%M%S%f}-"
|
f"{AI_MEMORY_CODE_PREFIX}-{utc_now():%Y%m%d%H%M%S%f}-"
|
||||||
f"{uuid4().hex[:8]}"
|
f"{uuid4().hex[:8]}"
|
||||||
),
|
),
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
owner_id=owner_id,
|
||||||
|
kind=kind,
|
||||||
scope=scope,
|
scope=scope,
|
||||||
subject=subject,
|
subject=subject,
|
||||||
content=content,
|
content=content,
|
||||||
@@ -360,8 +523,20 @@ class AIMemoryService:
|
|||||||
actor=actor,
|
actor=actor,
|
||||||
expires_at=expires_at,
|
expires_at=expires_at,
|
||||||
)
|
)
|
||||||
self.db.add(record)
|
if fingerprint:
|
||||||
self.db.flush()
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError:
|
||||||
|
existing = self._find_by_fingerprint(owner_id, fingerprint)
|
||||||
|
if existing is None:
|
||||||
|
raise
|
||||||
|
return self._reuse_entry(existing, status_value, expires_at)
|
||||||
|
else:
|
||||||
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
|
||||||
self.audit.record(
|
self.audit.record(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
@@ -397,6 +572,21 @@ class AIMemoryService:
|
|||||||
self.db.refresh(record)
|
self.db.refresh(record)
|
||||||
return record
|
return record
|
||||||
|
|
||||||
|
def _reuse_entry(
|
||||||
|
self,
|
||||||
|
record: AIMemoryEntry,
|
||||||
|
status_value: str,
|
||||||
|
expires_at: datetime | None,
|
||||||
|
) -> AIMemoryEntry:
|
||||||
|
if record.status != AIMemoryStatus.ARCHIVED:
|
||||||
|
return record
|
||||||
|
record.status = status_value
|
||||||
|
record.expires_at = expires_at
|
||||||
|
record.updated_at = utc_now()
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
def _build_memory_content(prompt: str, context: dict[str, Any], answer: str) -> str:
|
def _build_memory_content(prompt: str, context: dict[str, Any], answer: str) -> str:
|
||||||
context_text = ", ".join(
|
context_text = ", ".join(
|
||||||
@@ -427,6 +617,24 @@ def _contains_blocked_content(value: str, blocked_terms: list[str]) -> bool:
|
|||||||
return any(term.lower() in lowered for term in blocked_terms if term.strip())
|
return any(term.lower() in lowered for term in blocked_terms if term.strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _memory_fingerprint(
|
||||||
|
owner_id: int | None,
|
||||||
|
scope: str,
|
||||||
|
subject: str,
|
||||||
|
content: str,
|
||||||
|
status_value: str,
|
||||||
|
) -> str:
|
||||||
|
owner_key = "company" if owner_id is None else f"owner:{owner_id}"
|
||||||
|
value = "\0".join((owner_key, scope, subject, status_value, content))
|
||||||
|
return sha256(value.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _owner_filter(owner_id: int | None):
|
||||||
|
if owner_id is None:
|
||||||
|
return AIMemoryEntry.owner_id.is_(None)
|
||||||
|
return AIMemoryEntry.owner_id == owner_id
|
||||||
|
|
||||||
|
|
||||||
def _matches_query(entry: AIMemoryEntry, query: str) -> bool:
|
def _matches_query(entry: AIMemoryEntry, query: str) -> bool:
|
||||||
query_text = query.lower().strip()
|
query_text = query.lower().strip()
|
||||||
if not query_text:
|
if not query_text:
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from enum import StrEnum
|
|||||||
class AuditAction(StrEnum):
|
class AuditAction(StrEnum):
|
||||||
AI_ASK = "ai.ask"
|
AI_ASK = "ai.ask"
|
||||||
AI_PROVIDER_HEALTH = "ai.provider_health"
|
AI_PROVIDER_HEALTH = "ai.provider_health"
|
||||||
OPENCLAW_TOOLS_INVOKE = "openclaw.tools.invoke"
|
|
||||||
GENERATE_EVENTS = "generate_events"
|
GENERATE_EVENTS = "generate_events"
|
||||||
FEISHU_WEBHOOK_EVENT = "webhook_event"
|
FEISHU_WEBHOOK_EVENT = "webhook_event"
|
||||||
FEISHU_LONG_CONNECTION_EVENT = "long_connection_event"
|
FEISHU_LONG_CONNECTION_EVENT = "long_connection_event"
|
||||||
@@ -34,7 +33,6 @@ class AuditRiskLevel(StrEnum):
|
|||||||
|
|
||||||
class AuditSource(StrEnum):
|
class AuditSource(StrEnum):
|
||||||
API = "api"
|
API = "api"
|
||||||
OPENCLAW = "openclaw"
|
|
||||||
RISK = "risk"
|
RISK = "risk"
|
||||||
FEISHU = "feishu"
|
FEISHU = "feishu"
|
||||||
LEGACY_MYSQL = "legacy_mysql"
|
LEGACY_MYSQL = "legacy_mysql"
|
||||||
@@ -47,7 +45,6 @@ class AuditSource(StrEnum):
|
|||||||
|
|
||||||
class AuditTargetType(StrEnum):
|
class AuditTargetType(StrEnum):
|
||||||
AI = "ai"
|
AI = "ai"
|
||||||
OPENCLAW_TOOL = "openclaw_tool"
|
|
||||||
RISK_EVENTS = "risk-events"
|
RISK_EVENTS = "risk-events"
|
||||||
WORK_REPORTS = "work-reports"
|
WORK_REPORTS = "work-reports"
|
||||||
ENTERPRISE_ANALYTICS = "enterprise-analytics"
|
ENTERPRISE_ANALYTICS = "enterprise-analytics"
|
||||||
@@ -62,21 +59,3 @@ class AuditStatus(StrEnum):
|
|||||||
|
|
||||||
|
|
||||||
AUDIT_REDACTED_VALUE = "[REDACTED]"
|
AUDIT_REDACTED_VALUE = "[REDACTED]"
|
||||||
AUDIT_SENSITIVE_KEYS = frozenset(
|
|
||||||
{
|
|
||||||
"authorization",
|
|
||||||
"api_key",
|
|
||||||
"apikey",
|
|
||||||
"access_token",
|
|
||||||
"tenant_access_token",
|
|
||||||
"token",
|
|
||||||
"secret",
|
|
||||||
"password",
|
|
||||||
"openclaw_gateway_token",
|
|
||||||
"hermes_api_key",
|
|
||||||
"direct_llm_api_key",
|
|
||||||
"market_data_token",
|
|
||||||
"feishu_app_secret",
|
|
||||||
"feishu_verification_token",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|||||||
@@ -4,9 +4,10 @@ from typing import Any
|
|||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.http.masking import is_sensitive_key
|
||||||
from app.core.http.pagination import bounded_limit
|
from app.core.http.pagination import bounded_limit
|
||||||
from app.core.http.request_context import get_request_id
|
from app.core.http.request_context import get_request_id
|
||||||
from app.modules.audit.constants import AUDIT_REDACTED_VALUE, AUDIT_SENSITIVE_KEYS
|
from app.modules.audit.constants import AUDIT_REDACTED_VALUE
|
||||||
from app.modules.audit.models import AuditLog
|
from app.modules.audit.models import AuditLog
|
||||||
from app.modules.audit.schemas import AuditLogCreate
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
|
||||||
@@ -16,7 +17,7 @@ def _redact(value: Any) -> Any:
|
|||||||
safe: dict[str, Any] = {}
|
safe: dict[str, Any] = {}
|
||||||
for key, item in value.items():
|
for key, item in value.items():
|
||||||
key_text = str(key)
|
key_text = str(key)
|
||||||
if key_text.lower() in AUDIT_SENSITIVE_KEYS:
|
if is_sensitive_key(key_text):
|
||||||
safe[key_text] = AUDIT_REDACTED_VALUE
|
safe[key_text] = AUDIT_REDACTED_VALUE
|
||||||
else:
|
else:
|
||||||
safe[key_text] = _redact(item)
|
safe[key_text] = _redact(item)
|
||||||
@@ -34,6 +35,12 @@ def _dump(value: Any | None) -> str | None:
|
|||||||
if value is None:
|
if value is None:
|
||||||
return None
|
return None
|
||||||
if isinstance(value, str):
|
if isinstance(value, str):
|
||||||
|
try:
|
||||||
|
parsed = json.loads(value)
|
||||||
|
except (json.JSONDecodeError, TypeError):
|
||||||
|
return value
|
||||||
|
if isinstance(parsed, (dict, list)):
|
||||||
|
return json.dumps(_redact(parsed), ensure_ascii=False, default=str)
|
||||||
return value
|
return value
|
||||||
return json.dumps(_redact(value), ensure_ascii=False, default=str)
|
return json.dumps(_redact(value), ensure_ascii=False, default=str)
|
||||||
|
|
||||||
|
|||||||
@@ -1,10 +1,20 @@
|
|||||||
from datetime import date, datetime
|
from datetime import date, datetime
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from sqlalchemy import Boolean, Date, DateTime, Integer, Numeric, String, UniqueConstraint
|
from sqlalchemy import (
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
Boolean,
|
||||||
|
Date,
|
||||||
|
DateTime,
|
||||||
|
ForeignKey,
|
||||||
|
Integer,
|
||||||
|
Numeric,
|
||||||
|
String,
|
||||||
|
UniqueConstraint,
|
||||||
|
)
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
from app.modules.business.models.common import TimestampMixin
|
from app.modules.business.models.common import TimestampMixin
|
||||||
|
|
||||||
|
|
||||||
@@ -80,8 +90,16 @@ class MarketAnnouncement(Base, TimestampMixin):
|
|||||||
|
|
||||||
class MarketWatchlist(Base, TimestampMixin):
|
class MarketWatchlist(Base, TimestampMixin):
|
||||||
__tablename__ = "market_watchlists"
|
__tablename__ = "market_watchlists"
|
||||||
__table_args__ = (UniqueConstraint("actor", "symbol", name="uq_market_watchlist_actor_symbol"),)
|
__table_args__ = (
|
||||||
|
UniqueConstraint("owner_id", "symbol", name="uq_market_watchlist_owner_symbol"),
|
||||||
|
)
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True)
|
||||||
|
owner_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("feishu_users.id", ondelete="CASCADE"),
|
||||||
|
nullable=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner: Mapped[FeishuUser | None] = relationship()
|
||||||
actor: Mapped[str] = mapped_column(String(128), index=True)
|
actor: Mapped[str] = mapped_column(String(128), index=True)
|
||||||
symbol: Mapped[str] = mapped_column(String(32), index=True)
|
symbol: Mapped[str] = mapped_column(String(32), index=True)
|
||||||
enabled: Mapped[bool] = mapped_column(Boolean, default=True, index=True)
|
enabled: Mapped[bool] = mapped_column(Boolean, default=True, index=True)
|
||||||
|
|||||||
@@ -15,4 +15,6 @@ def dashboard_summary(
|
|||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
_ = principal
|
_ = principal
|
||||||
return mask_configured(DashboardService(db).summary())
|
result = mask_configured(DashboardService(db).summary())
|
||||||
|
db.commit()
|
||||||
|
return result
|
||||||
|
|||||||
@@ -4,8 +4,7 @@ from sqlalchemy import func, select
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.modules.ai_memory.constants import AIMemoryStatus
|
from app.modules.ai_memory.constants import AIMemoryStatus
|
||||||
from app.modules.ai_memory.models import AIMemoryEntry
|
from app.modules.ai_memory.service import AIMemoryService
|
||||||
from app.modules.audit.models import AuditLog
|
|
||||||
from app.modules.business.constants import DONE_STATUSES, PROJECT_CLOSED_STATUSES, StatusValue
|
from app.modules.business.constants import DONE_STATUSES, PROJECT_CLOSED_STATUSES, StatusValue
|
||||||
from app.modules.business.models import (
|
from app.modules.business.models import (
|
||||||
LegacySyncRun,
|
LegacySyncRun,
|
||||||
@@ -53,7 +52,8 @@ class DashboardService:
|
|||||||
WorkflowInstance,
|
WorkflowInstance,
|
||||||
WorkflowInstance.status == WorkflowStatus.FAILED,
|
WorkflowInstance.status == WorkflowStatus.FAILED,
|
||||||
)
|
)
|
||||||
active_ai_memory = self._count(AIMemoryEntry, AIMemoryEntry.status == AIMemoryStatus.ACTIVE)
|
memory_counts = AIMemoryService(self.db).count_by_status()
|
||||||
|
active_ai_memory = memory_counts.get(AIMemoryStatus.ACTIVE, 0)
|
||||||
heartbeat_summary = ObservabilityService(self.db).heartbeat_summary()
|
heartbeat_summary = ObservabilityService(self.db).heartbeat_summary()
|
||||||
latest_reports = self.db.execute(
|
latest_reports = self.db.execute(
|
||||||
select(WorkReport).order_by(WorkReport.id.desc()).limit(5)
|
select(WorkReport).order_by(WorkReport.id.desc()).limit(5)
|
||||||
@@ -64,9 +64,6 @@ class DashboardService:
|
|||||||
latest_sync_runs = self.db.execute(
|
latest_sync_runs = self.db.execute(
|
||||||
select(LegacySyncRun).order_by(LegacySyncRun.id.desc()).limit(10)
|
select(LegacySyncRun).order_by(LegacySyncRun.id.desc()).limit(10)
|
||||||
).scalars()
|
).scalars()
|
||||||
latest_audit_logs = self.db.execute(
|
|
||||||
select(AuditLog).order_by(AuditLog.id.desc()).limit(10)
|
|
||||||
).scalars()
|
|
||||||
risk_summary = self.risks.summary()
|
risk_summary = self.risks.summary()
|
||||||
return {
|
return {
|
||||||
"metrics": {
|
"metrics": {
|
||||||
@@ -94,7 +91,6 @@ class DashboardService:
|
|||||||
"latest_reports": [serialize_model(item) for item in latest_reports],
|
"latest_reports": [serialize_model(item) for item in latest_reports],
|
||||||
"latest_push_runs": [serialize_model(item) for item in latest_push_runs],
|
"latest_push_runs": [serialize_model(item) for item in latest_push_runs],
|
||||||
"latest_sync_runs": [serialize_model(item) for item in latest_sync_runs],
|
"latest_sync_runs": [serialize_model(item) for item in latest_sync_runs],
|
||||||
"latest_audit_logs": [serialize_model(item) for item in latest_audit_logs],
|
|
||||||
}
|
}
|
||||||
|
|
||||||
def _count(self, model: type, *conditions: Any) -> int:
|
def _count(self, model: type, *conditions: Any) -> int:
|
||||||
|
|||||||
@@ -1,7 +1,9 @@
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
from sqlalchemy import func, select
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
@@ -35,7 +37,10 @@ class EventQueryMixin:
|
|||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
now = utc_now()
|
now = utc_now()
|
||||||
record = DomainEvent(
|
record = DomainEvent(
|
||||||
event_id=f"{EVENT_CODE_PREFIX}-{now:%Y%m%d%H%M%S%f}",
|
event_id=(
|
||||||
|
f"{EVENT_CODE_PREFIX}-{now:%Y%m%d%H%M%S%f}-"
|
||||||
|
f"{uuid4().hex[:8]}"
|
||||||
|
),
|
||||||
event_type=event_type,
|
event_type=event_type,
|
||||||
source=source,
|
source=source,
|
||||||
aggregate_type=aggregate_type,
|
aggregate_type=aggregate_type,
|
||||||
@@ -46,8 +51,20 @@ class EventQueryMixin:
|
|||||||
next_attempt_at=now,
|
next_attempt_at=now,
|
||||||
max_attempts=settings.event_dispatch_max_attempts,
|
max_attempts=settings.event_dispatch_max_attempts,
|
||||||
)
|
)
|
||||||
self.db.add(record)
|
if not idempotency_key:
|
||||||
self.db.flush()
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
return record
|
||||||
|
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError:
|
||||||
|
existing = self._find_idempotent_event(idempotency_key)
|
||||||
|
if existing is None:
|
||||||
|
raise
|
||||||
|
return existing
|
||||||
return record
|
return record
|
||||||
|
|
||||||
def emit(
|
def emit(
|
||||||
|
|||||||
63
app/modules/feishu/app_tickets.py
Normal file
63
app/modules/feishu/app_tickets.py
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.feishu.models import FeishuAppTicket
|
||||||
|
|
||||||
|
APP_TICKET_EVENT_TYPE = "app_ticket"
|
||||||
|
APP_TICKET_PAYLOAD_KEY = "app_ticket"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuAppTicketService:
|
||||||
|
"""Persist the latest ticket received through a verified Feishu event."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
|
||||||
|
def get_ticket(self, app_id: str) -> str | None:
|
||||||
|
app_id_value = str(app_id).strip()
|
||||||
|
if not app_id_value:
|
||||||
|
return None
|
||||||
|
return self.db.scalar(
|
||||||
|
select(FeishuAppTicket.app_ticket).where(
|
||||||
|
FeishuAppTicket.app_id == app_id_value
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def store_verified(self, app_id: str, ticket: str) -> FeishuAppTicket:
|
||||||
|
app_id_value = str(app_id).strip()
|
||||||
|
ticket_value = str(ticket).strip()
|
||||||
|
if not app_id_value or not ticket_value:
|
||||||
|
raise ValueError("Verified Feishu app ticket fields are required")
|
||||||
|
|
||||||
|
now = utc_now()
|
||||||
|
record = self.db.execute(
|
||||||
|
select(FeishuAppTicket)
|
||||||
|
.where(FeishuAppTicket.app_id == app_id_value)
|
||||||
|
.with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
record = FeishuAppTicket(
|
||||||
|
app_id=app_id_value,
|
||||||
|
app_ticket=ticket_value,
|
||||||
|
received_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError:
|
||||||
|
record = self.db.execute(
|
||||||
|
select(FeishuAppTicket)
|
||||||
|
.where(FeishuAppTicket.app_id == app_id_value)
|
||||||
|
.with_for_update()
|
||||||
|
).scalar_one()
|
||||||
|
|
||||||
|
record.app_ticket = ticket_value
|
||||||
|
record.received_at = now
|
||||||
|
record.updated_at = now
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return record
|
||||||
@@ -1,67 +1,185 @@
|
|||||||
import json
|
import json
|
||||||
import time
|
import time
|
||||||
|
from threading import RLock
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.constants import BEARER_TOKEN_TEMPLATE, HttpHeader
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
|
from app.core.constants import BEARER_TOKEN_TEMPLATE, HttpHeader
|
||||||
from app.modules.feishu.constants import (
|
from app.modules.feishu.constants import (
|
||||||
|
FEISHU_APP_TICKET_MISSING,
|
||||||
|
FEISHU_APP_TOKEN_PATH,
|
||||||
FEISHU_AUTH_MISSING,
|
FEISHU_AUTH_MISSING,
|
||||||
FEISHU_DEFAULT_TOKEN_EXPIRE_SECONDS,
|
FEISHU_DEFAULT_TOKEN_EXPIRE_SECONDS,
|
||||||
FEISHU_MESSAGE_PATH,
|
|
||||||
FEISHU_IMAGE_PATH,
|
FEISHU_IMAGE_PATH,
|
||||||
|
FEISHU_MESSAGE_PATH,
|
||||||
FEISHU_RECEIVE_ID_MISSING,
|
FEISHU_RECEIVE_ID_MISSING,
|
||||||
|
FEISHU_STORE_TENANT_TOKEN_PATH,
|
||||||
FEISHU_SUCCESS_CODE,
|
FEISHU_SUCCESS_CODE,
|
||||||
|
FEISHU_TENANT_KEY_MISSING,
|
||||||
FEISHU_TENANT_TOKEN_PATH,
|
FEISHU_TENANT_TOKEN_PATH,
|
||||||
FEISHU_TOKEN_EXPIRE_SAFETY_SECONDS,
|
FEISHU_TOKEN_EXPIRE_SAFETY_SECONDS,
|
||||||
|
FeishuAppType,
|
||||||
FeishuMessageType,
|
FeishuMessageType,
|
||||||
FeishuPayloadKey,
|
FeishuPayloadKey,
|
||||||
FeishuReceiveIdType,
|
FeishuReceiveIdType,
|
||||||
)
|
)
|
||||||
|
from app.modules.feishu.errors import FeishuAPIError
|
||||||
|
|
||||||
|
_RETRYABLE_PROVIDER_CODES = frozenset(
|
||||||
|
{
|
||||||
|
99991400,
|
||||||
|
99991401,
|
||||||
|
99991402,
|
||||||
|
99991403,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
_RETRYABLE_PROVIDER_TERMS = (
|
||||||
|
"rate limit",
|
||||||
|
"too many request",
|
||||||
|
"temporar",
|
||||||
|
"timeout",
|
||||||
|
"busy",
|
||||||
|
"限流",
|
||||||
|
"频率",
|
||||||
|
"超时",
|
||||||
|
"繁忙",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class FeishuClient:
|
class FeishuClient:
|
||||||
"""Small Feishu Open Platform client for tenant token and message APIs."""
|
"""Small Feishu Open Platform client with tenant-isolated token caches."""
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self, db: Session | None = None) -> None:
|
||||||
self.settings = get_settings()
|
self.settings = get_settings()
|
||||||
|
self.db = db
|
||||||
self._tenant_access_token: str | None = None
|
self._tenant_access_token: str | None = None
|
||||||
self._token_expires_at: float = 0
|
self._token_expires_at: float = 0
|
||||||
|
self._app_access_tokens: dict[str, tuple[str, float]] = {}
|
||||||
|
self._store_tenant_access_tokens: dict[tuple[str, str], tuple[str, float]] = {}
|
||||||
|
self._token_lock = RLock()
|
||||||
|
|
||||||
def _is_configured(self) -> bool:
|
def _is_configured(self) -> bool:
|
||||||
return bool(self.settings.feishu_app_id and self.settings.feishu_app_secret)
|
return bool(self.settings.feishu_app_id and self.settings.feishu_app_secret)
|
||||||
|
|
||||||
def _get_tenant_access_token(self) -> str:
|
def _get_tenant_access_token(self, tenant_key: str | None = None) -> str:
|
||||||
if not self._is_configured():
|
if not self._is_configured():
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
detail=FEISHU_AUTH_MISSING,
|
detail=FEISHU_AUTH_MISSING,
|
||||||
)
|
)
|
||||||
if self._tenant_access_token and time.time() < self._token_expires_at:
|
if self.settings.feishu_app_type == FeishuAppType.STORE:
|
||||||
return self._tenant_access_token
|
return self._get_store_tenant_access_token(tenant_key)
|
||||||
|
return self._get_self_tenant_access_token()
|
||||||
|
|
||||||
url = f"{self.settings.feishu_base_url}{FEISHU_TENANT_TOKEN_PATH}"
|
def _get_self_tenant_access_token(self) -> str:
|
||||||
payload = {
|
with self._token_lock:
|
||||||
FeishuPayloadKey.APP_ID: self.settings.feishu_app_id,
|
if (
|
||||||
FeishuPayloadKey.APP_SECRET: self.settings.feishu_app_secret,
|
self._tenant_access_token
|
||||||
}
|
and time.time() < self._token_expires_at
|
||||||
with httpx.Client(timeout=20) as client:
|
):
|
||||||
response = client.post(url, json=payload)
|
return self._tenant_access_token
|
||||||
response.raise_for_status()
|
data = self._post(
|
||||||
data = response.json()
|
f"{self.settings.feishu_base_url}{FEISHU_TENANT_TOKEN_PATH}",
|
||||||
if data.get(FeishuPayloadKey.CODE) != FEISHU_SUCCESS_CODE:
|
operation="Feishu self tenant token request",
|
||||||
raise HTTPException(
|
json={
|
||||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
FeishuPayloadKey.APP_ID: self.settings.feishu_app_id,
|
||||||
detail={FeishuPayloadKey.FEISHU_ERROR: data},
|
FeishuPayloadKey.APP_SECRET: self.settings.feishu_app_secret,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
self._tenant_access_token = data[FeishuPayloadKey.TENANT_ACCESS_TOKEN]
|
token = self._required_token(
|
||||||
expire_seconds = int(
|
data,
|
||||||
data.get(FeishuPayloadKey.EXPIRE, FEISHU_DEFAULT_TOKEN_EXPIRE_SECONDS)
|
FeishuPayloadKey.TENANT_ACCESS_TOKEN,
|
||||||
)
|
"Feishu self tenant token response",
|
||||||
self._token_expires_at = time.time() + expire_seconds - FEISHU_TOKEN_EXPIRE_SAFETY_SECONDS
|
)
|
||||||
return self._tenant_access_token
|
self._tenant_access_token = token
|
||||||
|
self._token_expires_at = self._expires_at(data)
|
||||||
|
return token
|
||||||
|
|
||||||
|
def _get_store_app_access_token(self) -> str:
|
||||||
|
app_id = str(self.settings.feishu_app_id)
|
||||||
|
with self._token_lock:
|
||||||
|
cached = self._get_cached(self._app_access_tokens, app_id)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
app_ticket = self._get_app_ticket(app_id)
|
||||||
|
data = self._post(
|
||||||
|
f"{self.settings.feishu_base_url}{FEISHU_APP_TOKEN_PATH}",
|
||||||
|
operation="Feishu store app token request",
|
||||||
|
json={
|
||||||
|
FeishuPayloadKey.APP_ID: app_id,
|
||||||
|
FeishuPayloadKey.APP_SECRET: self.settings.feishu_app_secret,
|
||||||
|
FeishuPayloadKey.APP_TICKET: app_ticket,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
token = self._required_token(
|
||||||
|
data,
|
||||||
|
FeishuPayloadKey.APP_ACCESS_TOKEN,
|
||||||
|
"Feishu store app token response",
|
||||||
|
)
|
||||||
|
self._app_access_tokens[app_id] = (token, self._expires_at(data))
|
||||||
|
return token
|
||||||
|
|
||||||
|
def _get_store_tenant_access_token(self, tenant_key: str | None) -> str:
|
||||||
|
normalized_tenant_key = str(tenant_key or "").strip()
|
||||||
|
if not normalized_tenant_key:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail=FEISHU_TENANT_KEY_MISSING,
|
||||||
|
)
|
||||||
|
app_id = str(self.settings.feishu_app_id)
|
||||||
|
cache_key = (app_id, normalized_tenant_key)
|
||||||
|
with self._token_lock:
|
||||||
|
cached = self._get_cached(
|
||||||
|
self._store_tenant_access_tokens,
|
||||||
|
cache_key,
|
||||||
|
)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
app_access_token = self._get_store_app_access_token()
|
||||||
|
data = self._post(
|
||||||
|
f"{self.settings.feishu_base_url}{FEISHU_STORE_TENANT_TOKEN_PATH}",
|
||||||
|
operation="Feishu store tenant token request",
|
||||||
|
json={
|
||||||
|
FeishuPayloadKey.APP_ACCESS_TOKEN: app_access_token,
|
||||||
|
FeishuPayloadKey.TENANT_KEY: normalized_tenant_key,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
token = self._required_token(
|
||||||
|
data,
|
||||||
|
FeishuPayloadKey.TENANT_ACCESS_TOKEN,
|
||||||
|
"Feishu store tenant token response",
|
||||||
|
)
|
||||||
|
self._store_tenant_access_tokens[cache_key] = (
|
||||||
|
token,
|
||||||
|
self._expires_at(data),
|
||||||
|
)
|
||||||
|
return token
|
||||||
|
|
||||||
|
def _get_app_ticket(self, app_id: str) -> str:
|
||||||
|
ticket: Any = None
|
||||||
|
if self.db is not None:
|
||||||
|
from app.modules.feishu.app_tickets import FeishuAppTicketService
|
||||||
|
|
||||||
|
ticket = FeishuAppTicketService(self.db).get_ticket(app_id)
|
||||||
|
if ticket is not None and not isinstance(ticket, str):
|
||||||
|
ticket = getattr(ticket, "app_ticket", None) or getattr(
|
||||||
|
ticket,
|
||||||
|
"ticket",
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
normalized = str(ticket or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
normalized = str(self.settings.feishu_app_ticket or "").strip()
|
||||||
|
if not normalized:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
detail=FEISHU_APP_TICKET_MISSING,
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
def send_message(
|
def send_message(
|
||||||
self,
|
self,
|
||||||
@@ -69,77 +187,207 @@ class FeishuClient:
|
|||||||
receive_id_type: str,
|
receive_id_type: str,
|
||||||
msg_type: str,
|
msg_type: str,
|
||||||
content: dict[str, Any],
|
content: dict[str, Any],
|
||||||
|
uuid: str | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
token = self._get_tenant_access_token()
|
token = self._get_tenant_access_token(tenant_key)
|
||||||
url = f"{self.settings.feishu_base_url}{FEISHU_MESSAGE_PATH}"
|
|
||||||
headers = {HttpHeader.AUTHORIZATION: BEARER_TOKEN_TEMPLATE.format(token=token)}
|
headers = {HttpHeader.AUTHORIZATION: BEARER_TOKEN_TEMPLATE.format(token=token)}
|
||||||
params = {FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type}
|
|
||||||
payload = {
|
payload = {
|
||||||
FeishuPayloadKey.RECEIVE_ID: receive_id,
|
FeishuPayloadKey.RECEIVE_ID: receive_id,
|
||||||
FeishuPayloadKey.MESSAGE_TYPE: msg_type,
|
FeishuPayloadKey.MESSAGE_TYPE: msg_type,
|
||||||
FeishuPayloadKey.CONTENT: json.dumps(content, ensure_ascii=False),
|
FeishuPayloadKey.CONTENT: json.dumps(content, ensure_ascii=False),
|
||||||
}
|
}
|
||||||
with httpx.Client(timeout=20) as client:
|
if uuid:
|
||||||
response = client.post(url, headers=headers, params=params, json=payload)
|
payload[FeishuPayloadKey.UUID] = uuid
|
||||||
response.raise_for_status()
|
return self._post(
|
||||||
data = response.json()
|
f"{self.settings.feishu_base_url}{FEISHU_MESSAGE_PATH}",
|
||||||
return data
|
operation="Feishu message request",
|
||||||
|
headers=headers,
|
||||||
|
params={FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type},
|
||||||
|
json=payload,
|
||||||
|
)
|
||||||
|
|
||||||
def send_text(
|
def send_text(
|
||||||
self,
|
self,
|
||||||
text: str,
|
text: str,
|
||||||
receive_id: str | None = None,
|
receive_id: str | None = None,
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
||||||
) -> dict:
|
uuid: str | None = None,
|
||||||
chat_id = receive_id or self.settings.feishu_default_chat_id
|
tenant_key: str | None = None,
|
||||||
if not chat_id:
|
) -> dict[str, Any]:
|
||||||
|
target_id = receive_id or self.settings.feishu_default_chat_id
|
||||||
|
if not target_id:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST,
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
detail=FEISHU_RECEIVE_ID_MISSING,
|
detail=FEISHU_RECEIVE_ID_MISSING,
|
||||||
)
|
)
|
||||||
|
resolved_tenant_key = tenant_key
|
||||||
|
if receive_id is None and not resolved_tenant_key:
|
||||||
|
resolved_tenant_key = self.settings.feishu_default_tenant_key
|
||||||
return self.send_message(
|
return self.send_message(
|
||||||
chat_id,
|
target_id,
|
||||||
receive_id_type,
|
receive_id_type,
|
||||||
FeishuMessageType.TEXT,
|
FeishuMessageType.TEXT,
|
||||||
{FeishuPayloadKey.TEXT: text},
|
{FeishuPayloadKey.TEXT: text},
|
||||||
|
uuid,
|
||||||
|
resolved_tenant_key,
|
||||||
)
|
)
|
||||||
|
|
||||||
def upload_image(
|
def upload_image(
|
||||||
self,
|
self,
|
||||||
image: bytes,
|
image: bytes,
|
||||||
filename: str = "lifecycle-report.png",
|
filename: str = "lifecycle-report.png",
|
||||||
|
tenant_key: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
token = self._get_tenant_access_token()
|
resolved_tenant_key = tenant_key or self.settings.feishu_default_tenant_key
|
||||||
url = f"{self.settings.feishu_base_url}{FEISHU_IMAGE_PATH}"
|
token = self._get_tenant_access_token(resolved_tenant_key)
|
||||||
headers = {HttpHeader.AUTHORIZATION: BEARER_TOKEN_TEMPLATE.format(token=token)}
|
headers = {HttpHeader.AUTHORIZATION: BEARER_TOKEN_TEMPLATE.format(token=token)}
|
||||||
with httpx.Client(timeout=30) as client:
|
return self._post(
|
||||||
response = client.post(
|
f"{self.settings.feishu_base_url}{FEISHU_IMAGE_PATH}",
|
||||||
url,
|
operation="Feishu image upload request",
|
||||||
headers=headers,
|
timeout=30,
|
||||||
data={FeishuPayloadKey.IMAGE_TYPE: FeishuPayloadKey.MESSAGE},
|
headers=headers,
|
||||||
files={
|
data={FeishuPayloadKey.IMAGE_TYPE: FeishuPayloadKey.MESSAGE},
|
||||||
FeishuPayloadKey.IMAGE: (filename, image, "image/png"),
|
files={
|
||||||
},
|
FeishuPayloadKey.IMAGE: (filename, image, "image/png"),
|
||||||
)
|
},
|
||||||
response.raise_for_status()
|
)
|
||||||
data = response.json()
|
|
||||||
if data.get(FeishuPayloadKey.CODE) != FEISHU_SUCCESS_CODE:
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
|
||||||
detail={FeishuPayloadKey.FEISHU_ERROR: data},
|
|
||||||
)
|
|
||||||
return data
|
|
||||||
|
|
||||||
def send_card(
|
def send_card(
|
||||||
self,
|
self,
|
||||||
card: dict[str, Any],
|
card: dict[str, Any],
|
||||||
receive_id: str | None = None,
|
receive_id: str | None = None,
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
||||||
) -> dict:
|
uuid: str | None = None,
|
||||||
chat_id = receive_id or self.settings.feishu_default_chat_id
|
tenant_key: str | None = None,
|
||||||
if not chat_id:
|
) -> dict[str, Any]:
|
||||||
|
target_id = receive_id or self.settings.feishu_default_chat_id
|
||||||
|
if not target_id:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST,
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
detail=FEISHU_RECEIVE_ID_MISSING,
|
detail=FEISHU_RECEIVE_ID_MISSING,
|
||||||
)
|
)
|
||||||
return self.send_message(chat_id, receive_id_type, FeishuMessageType.INTERACTIVE, card)
|
resolved_tenant_key = tenant_key
|
||||||
|
if receive_id is None and not resolved_tenant_key:
|
||||||
|
resolved_tenant_key = self.settings.feishu_default_tenant_key
|
||||||
|
return self.send_message(
|
||||||
|
target_id,
|
||||||
|
receive_id_type,
|
||||||
|
FeishuMessageType.INTERACTIVE,
|
||||||
|
card,
|
||||||
|
uuid,
|
||||||
|
resolved_tenant_key,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _post(
|
||||||
|
self,
|
||||||
|
url: str,
|
||||||
|
*,
|
||||||
|
operation: str,
|
||||||
|
timeout: int = 20,
|
||||||
|
**kwargs: Any,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
with httpx.Client(timeout=timeout) as client:
|
||||||
|
response = client.post(url, **kwargs)
|
||||||
|
except httpx.HTTPError:
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} failed",
|
||||||
|
retryable=True,
|
||||||
|
) from None
|
||||||
|
if not status.HTTP_200_OK <= response.status_code < status.HTTP_300_MULTIPLE_CHOICES:
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} returned an HTTP error",
|
||||||
|
retryable=_is_retryable_http_status(response.status_code),
|
||||||
|
http_status=response.status_code,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
data = response.json()
|
||||||
|
except ValueError:
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} response was not valid JSON",
|
||||||
|
retryable=True,
|
||||||
|
http_status=response.status_code,
|
||||||
|
) from None
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} response was not a JSON object",
|
||||||
|
retryable=True,
|
||||||
|
http_status=response.status_code,
|
||||||
|
)
|
||||||
|
provider_code = data.get(FeishuPayloadKey.CODE)
|
||||||
|
if provider_code != FEISHU_SUCCESS_CODE:
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} returned a non-zero business code",
|
||||||
|
retryable=_is_retryable_business_error(data),
|
||||||
|
http_status=response.status_code,
|
||||||
|
provider_code=provider_code,
|
||||||
|
provider_response={
|
||||||
|
FeishuPayloadKey.CODE: provider_code,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return data
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _required_token(
|
||||||
|
data: dict[str, Any],
|
||||||
|
key: FeishuPayloadKey,
|
||||||
|
operation: str,
|
||||||
|
) -> str:
|
||||||
|
token = data.get(key)
|
||||||
|
if not isinstance(token, str) or not token.strip():
|
||||||
|
raise FeishuAPIError(
|
||||||
|
f"{operation} did not include the required credential",
|
||||||
|
retryable=True,
|
||||||
|
provider_code=data.get(FeishuPayloadKey.CODE),
|
||||||
|
provider_response={
|
||||||
|
FeishuPayloadKey.CODE: data.get(FeishuPayloadKey.CODE),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return token
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _expires_at(data: dict[str, Any]) -> float:
|
||||||
|
try:
|
||||||
|
expire_seconds = int(
|
||||||
|
data.get(
|
||||||
|
FeishuPayloadKey.EXPIRE,
|
||||||
|
FEISHU_DEFAULT_TOKEN_EXPIRE_SECONDS,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
expire_seconds = FEISHU_DEFAULT_TOKEN_EXPIRE_SECONDS
|
||||||
|
usable_seconds = max(
|
||||||
|
1,
|
||||||
|
expire_seconds - FEISHU_TOKEN_EXPIRE_SAFETY_SECONDS,
|
||||||
|
)
|
||||||
|
return time.time() + usable_seconds
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _get_cached(
|
||||||
|
cache: dict[Any, tuple[str, float]],
|
||||||
|
key: Any,
|
||||||
|
) -> str | None:
|
||||||
|
entry = cache.get(key)
|
||||||
|
if entry is None:
|
||||||
|
return None
|
||||||
|
token, expires_at = entry
|
||||||
|
if time.time() < expires_at:
|
||||||
|
return token
|
||||||
|
cache.pop(key, None)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _is_retryable_http_status(status_code: int) -> bool:
|
||||||
|
return (
|
||||||
|
status_code == status.HTTP_429_TOO_MANY_REQUESTS
|
||||||
|
or status_code >= status.HTTP_500_INTERNAL_SERVER_ERROR
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_retryable_business_error(data: dict[str, Any]) -> bool:
|
||||||
|
code = data.get(FeishuPayloadKey.CODE)
|
||||||
|
if code in _RETRYABLE_PROVIDER_CODES:
|
||||||
|
return True
|
||||||
|
message = str(data.get("msg") or "").casefold()
|
||||||
|
return any(term in message for term in _RETRYABLE_PROVIDER_TERMS)
|
||||||
|
|||||||
@@ -3,6 +3,12 @@ from enum import StrEnum
|
|||||||
|
|
||||||
class FeishuReceiveIdType(StrEnum):
|
class FeishuReceiveIdType(StrEnum):
|
||||||
CHAT_ID = "chat_id"
|
CHAT_ID = "chat_id"
|
||||||
|
OPEN_ID = "open_id"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuAppType(StrEnum):
|
||||||
|
SELF = "self"
|
||||||
|
STORE = "store"
|
||||||
|
|
||||||
|
|
||||||
class FeishuMessageType(StrEnum):
|
class FeishuMessageType(StrEnum):
|
||||||
@@ -16,24 +22,30 @@ class FeishuEventSource(StrEnum):
|
|||||||
|
|
||||||
|
|
||||||
class FeishuPayloadKey(StrEnum):
|
class FeishuPayloadKey(StrEnum):
|
||||||
|
APP_ACCESS_TOKEN = "app_access_token"
|
||||||
APP_ID = "app_id"
|
APP_ID = "app_id"
|
||||||
APP_SECRET = "app_secret"
|
APP_SECRET = "app_secret"
|
||||||
|
APP_TICKET = "app_ticket"
|
||||||
CARD = "card"
|
CARD = "card"
|
||||||
CHALLENGE = "challenge"
|
CHALLENGE = "challenge"
|
||||||
|
CHAT_TYPE = "chat_type"
|
||||||
CODE = "code"
|
CODE = "code"
|
||||||
CONFIG = "config"
|
CONFIG = "config"
|
||||||
CONTENT = "content"
|
CONTENT = "content"
|
||||||
DIV = "div"
|
DIV = "div"
|
||||||
DATA = "data"
|
DATA = "data"
|
||||||
ELEMENTS = "elements"
|
ELEMENTS = "elements"
|
||||||
|
ENCRYPT = "encrypt"
|
||||||
EXPIRE = "expire"
|
EXPIRE = "expire"
|
||||||
FEISHU_ERROR = "feishu_error"
|
FEISHU_ERROR = "feishu_error"
|
||||||
HEADER = "header"
|
HEADER = "header"
|
||||||
IMAGE = "image"
|
IMAGE = "image"
|
||||||
IMAGE_KEY = "image_key"
|
IMAGE_KEY = "image_key"
|
||||||
IMAGE_TYPE = "image_type"
|
IMAGE_TYPE = "image_type"
|
||||||
|
ID = "id"
|
||||||
IMG = "img"
|
IMG = "img"
|
||||||
IMG_KEY = "img_key"
|
IMG_KEY = "img_key"
|
||||||
|
KEY = "key"
|
||||||
ALT = "alt"
|
ALT = "alt"
|
||||||
EVENT = "event"
|
EVENT = "event"
|
||||||
EVENT_ID = "event_id"
|
EVENT_ID = "event_id"
|
||||||
@@ -42,6 +54,8 @@ class FeishuPayloadKey(StrEnum):
|
|||||||
MESSAGE = "message"
|
MESSAGE = "message"
|
||||||
MESSAGE_ID = "message_id"
|
MESSAGE_ID = "message_id"
|
||||||
MESSAGE_TYPE = "msg_type"
|
MESSAGE_TYPE = "msg_type"
|
||||||
|
MENTIONS = "mentions"
|
||||||
|
NAME = "name"
|
||||||
OPEN_ID = "open_id"
|
OPEN_ID = "open_id"
|
||||||
PLAIN_TEXT = "plain_text"
|
PLAIN_TEXT = "plain_text"
|
||||||
RECEIVE_ID = "receive_id"
|
RECEIVE_ID = "receive_id"
|
||||||
@@ -50,17 +64,23 @@ class FeishuPayloadKey(StrEnum):
|
|||||||
SENDER_ID = "sender_id"
|
SENDER_ID = "sender_id"
|
||||||
TAG = "tag"
|
TAG = "tag"
|
||||||
TENANT_ACCESS_TOKEN = "tenant_access_token"
|
TENANT_ACCESS_TOKEN = "tenant_access_token"
|
||||||
|
TENANT_KEY = "tenant_key"
|
||||||
TEXT = "text"
|
TEXT = "text"
|
||||||
TITLE = "title"
|
TITLE = "title"
|
||||||
TOKEN = "token"
|
TOKEN = "token"
|
||||||
|
UNION_ID = "union_id"
|
||||||
USER_ID = "user_id"
|
USER_ID = "user_id"
|
||||||
|
UUID = "uuid"
|
||||||
WIDE_SCREEN_MODE = "wide_screen_mode"
|
WIDE_SCREEN_MODE = "wide_screen_mode"
|
||||||
|
|
||||||
|
|
||||||
class FeishuCommandKey(StrEnum):
|
class FeishuCommandKey(StrEnum):
|
||||||
TEXT = "text"
|
TEXT = "text"
|
||||||
CHAT_ID = "chat_id"
|
CHAT_ID = "chat_id"
|
||||||
|
CHAT_TYPE = "chat_type"
|
||||||
ACTOR = "actor"
|
ACTOR = "actor"
|
||||||
|
MENTIONS = "mentions"
|
||||||
|
PRINCIPAL = "principal"
|
||||||
|
|
||||||
|
|
||||||
class FeishuResponseKey(StrEnum):
|
class FeishuResponseKey(StrEnum):
|
||||||
@@ -85,10 +105,32 @@ class FeishuCommandResultKey(StrEnum):
|
|||||||
|
|
||||||
|
|
||||||
class FeishuCommandName(StrEnum):
|
class FeishuCommandName(StrEnum):
|
||||||
|
PERMISSION_DENIED = "permission_denied"
|
||||||
|
HELP = "help"
|
||||||
|
USER_SET_ADMIN = "user_set_admin"
|
||||||
|
USER_SET_USER = "user_set_user"
|
||||||
|
USER_DISABLE = "user_disable"
|
||||||
|
USER_ENABLE = "user_enable"
|
||||||
|
PREFERENCE_SET = "preference_set"
|
||||||
|
PREFERENCE_LIST = "preference_list"
|
||||||
|
PREFERENCE_DELETE = "preference_delete"
|
||||||
|
CONVERSATION_RESET = "conversation_reset"
|
||||||
|
PERSONAL_DATA_SUMMARY = "personal_data_summary"
|
||||||
|
PERSONAL_DATA_ERASURE_REQUEST = "personal_data_erasure_request"
|
||||||
|
PERSONAL_DATA_ERASURE_CONFIRM = "personal_data_erasure_confirm"
|
||||||
|
SUBSCRIPTION_CREATE = "subscription_create"
|
||||||
|
SUBSCRIPTION_LIST = "subscription_list"
|
||||||
|
SUBSCRIPTION_PAUSE = "subscription_pause"
|
||||||
|
SUBSCRIPTION_RESUME = "subscription_resume"
|
||||||
|
SUBSCRIPTION_CANCEL = "subscription_cancel"
|
||||||
|
SUBSCRIPTION_TIMEZONE = "subscription_timezone"
|
||||||
|
SUBSCRIPTION_QUIET_HOURS = "subscription_quiet_hours"
|
||||||
RULE_CREATE = "rule_create"
|
RULE_CREATE = "rule_create"
|
||||||
RULE_LIST = "rule_list"
|
RULE_LIST = "rule_list"
|
||||||
RULE_DISABLE = "rule_disable"
|
RULE_DISABLE = "rule_disable"
|
||||||
RULE_ENABLE = "rule_enable"
|
RULE_ENABLE = "rule_enable"
|
||||||
|
RULE_UPDATE = "rule_update"
|
||||||
|
RULE_DELETE = "rule_delete"
|
||||||
FINANCE_NEEDS = "finance_needs"
|
FINANCE_NEEDS = "finance_needs"
|
||||||
PROJECT_FINANCE = "project_finance"
|
PROJECT_FINANCE = "project_finance"
|
||||||
MARKET_OVERVIEW = "market_overview"
|
MARKET_OVERVIEW = "market_overview"
|
||||||
@@ -123,11 +165,15 @@ class FeishuCardKey(StrEnum):
|
|||||||
VALUE = "value"
|
VALUE = "value"
|
||||||
|
|
||||||
|
|
||||||
|
FEISHU_APP_TOKEN_PATH = "/auth/v3/app_access_token"
|
||||||
|
FEISHU_STORE_TENANT_TOKEN_PATH = "/auth/v3/tenant_access_token"
|
||||||
FEISHU_TENANT_TOKEN_PATH = "/auth/v3/tenant_access_token/internal"
|
FEISHU_TENANT_TOKEN_PATH = "/auth/v3/tenant_access_token/internal"
|
||||||
FEISHU_MESSAGE_PATH = "/im/v1/messages"
|
FEISHU_MESSAGE_PATH = "/im/v1/messages"
|
||||||
FEISHU_IMAGE_PATH = "/im/v1/images"
|
FEISHU_IMAGE_PATH = "/im/v1/images"
|
||||||
FEISHU_DEFAULT_OPEN_API_DOMAIN = "https://open.feishu.cn"
|
FEISHU_DEFAULT_OPEN_API_DOMAIN = "https://open.feishu.cn"
|
||||||
FEISHU_AUTH_MISSING = "Feishu app credentials are not configured"
|
FEISHU_AUTH_MISSING = "Feishu app credentials are not configured"
|
||||||
|
FEISHU_APP_TICKET_MISSING = "Feishu store app ticket is not available"
|
||||||
|
FEISHU_TENANT_KEY_MISSING = "tenant_key is required for Feishu store apps"
|
||||||
FEISHU_VERIFICATION_TOKEN_REQUIRED = "FEISHU_VERIFICATION_TOKEN is required"
|
FEISHU_VERIFICATION_TOKEN_REQUIRED = "FEISHU_VERIFICATION_TOKEN is required"
|
||||||
FEISHU_INVALID_TOKEN = "Invalid Feishu token"
|
FEISHU_INVALID_TOKEN = "Invalid Feishu token"
|
||||||
FEISHU_RECEIVE_ID_MISSING = "receive_id or FEISHU_DEFAULT_CHAT_ID is required"
|
FEISHU_RECEIVE_ID_MISSING = "receive_id or FEISHU_DEFAULT_CHAT_ID is required"
|
||||||
|
|||||||
29
app/modules/feishu/errors.py
Normal file
29
app/modules/feishu/errors.py
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuAPIError(HTTPException):
|
||||||
|
"""Normalized outbound Feishu failure with retry classification."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
detail: str,
|
||||||
|
*,
|
||||||
|
retryable: bool,
|
||||||
|
http_status: int | None = None,
|
||||||
|
provider_code: int | str | None = None,
|
||||||
|
provider_response: dict[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
super().__init__(
|
||||||
|
status_code=(
|
||||||
|
status.HTTP_503_SERVICE_UNAVAILABLE
|
||||||
|
if retryable
|
||||||
|
else status.HTTP_502_BAD_GATEWAY
|
||||||
|
),
|
||||||
|
detail=detail,
|
||||||
|
)
|
||||||
|
self.retryable = retryable
|
||||||
|
self.http_status = http_status
|
||||||
|
self.provider_code = provider_code
|
||||||
|
self.provider_response = provider_response or {}
|
||||||
131
app/modules/feishu/event_verification.py
Normal file
131
app/modules/feishu/event_verification.py
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
import base64
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
from hashlib import sha256
|
||||||
|
from secrets import compare_digest
|
||||||
|
from typing import Any, Mapping
|
||||||
|
|
||||||
|
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||||
|
from cryptography.hazmat.primitives.padding import PKCS7
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
|
||||||
|
from app.core.config import get_settings
|
||||||
|
from app.modules.feishu.constants import FeishuPayloadKey
|
||||||
|
|
||||||
|
_SIGNATURE_MAX_AGE_SECONDS = 300
|
||||||
|
_SIGNATURE_HEADER = "x-lark-signature"
|
||||||
|
_TIMESTAMP_HEADER = "x-lark-request-timestamp"
|
||||||
|
_NONCE_HEADER = "x-lark-request-nonce"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuWebhookVerifier:
|
||||||
|
"""Verify, decrypt, and normalize an HTTP webhook before business handling."""
|
||||||
|
|
||||||
|
def verify(
|
||||||
|
self,
|
||||||
|
raw_body: bytes,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
settings = get_settings()
|
||||||
|
if settings.feishu_encrypt_key:
|
||||||
|
self._verify_signature(raw_body, headers, settings.feishu_encrypt_key)
|
||||||
|
payload = self._load_json(raw_body)
|
||||||
|
if FeishuPayloadKey.ENCRYPT in payload:
|
||||||
|
if not settings.feishu_encrypt_key:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
detail="FEISHU_ENCRYPT_KEY is required for encrypted webhooks",
|
||||||
|
)
|
||||||
|
payload = self._decrypt(
|
||||||
|
str(payload[FeishuPayloadKey.ENCRYPT]),
|
||||||
|
settings.feishu_encrypt_key,
|
||||||
|
)
|
||||||
|
self._verify_token(payload, settings.feishu_verification_token)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _load_json(raw_body: bytes) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
payload = json.loads(raw_body)
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Invalid Feishu webhook JSON",
|
||||||
|
) from exc
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail="Invalid Feishu webhook payload",
|
||||||
|
)
|
||||||
|
return payload
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _verify_signature(
|
||||||
|
raw_body: bytes,
|
||||||
|
headers: Mapping[str, str],
|
||||||
|
encrypt_key: str,
|
||||||
|
) -> None:
|
||||||
|
normalized = {str(key).lower(): str(value) for key, value in headers.items()}
|
||||||
|
timestamp = normalized.get(_TIMESTAMP_HEADER)
|
||||||
|
nonce = normalized.get(_NONCE_HEADER)
|
||||||
|
signature = normalized.get(_SIGNATURE_HEADER)
|
||||||
|
if not timestamp or not nonce or not signature:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Missing Feishu webhook signature headers",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
request_time = int(timestamp)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid Feishu webhook timestamp",
|
||||||
|
) from exc
|
||||||
|
if abs(int(time.time()) - request_time) > _SIGNATURE_MAX_AGE_SECONDS:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Expired Feishu webhook signature",
|
||||||
|
)
|
||||||
|
signed = (
|
||||||
|
timestamp.encode("utf-8")
|
||||||
|
+ nonce.encode("utf-8")
|
||||||
|
+ encrypt_key.encode("utf-8")
|
||||||
|
+ raw_body
|
||||||
|
)
|
||||||
|
expected = sha256(signed).hexdigest()
|
||||||
|
if not compare_digest(signature, expected):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid Feishu webhook signature",
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _decrypt(encrypted: str, encrypt_key: str) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
key = sha256(encrypt_key.encode("utf-8")).digest()
|
||||||
|
encrypted_bytes = base64.b64decode(encrypted, validate=True)
|
||||||
|
decryptor = Cipher(algorithms.AES(key), modes.CBC(key[:16])).decryptor()
|
||||||
|
padded = decryptor.update(encrypted_bytes) + decryptor.finalize()
|
||||||
|
unpadder = PKCS7(algorithms.AES.block_size).unpadder()
|
||||||
|
cleartext = unpadder.update(padded) + unpadder.finalize()
|
||||||
|
except (ValueError, TypeError) as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid encrypted Feishu webhook",
|
||||||
|
) from exc
|
||||||
|
return FeishuWebhookVerifier._load_json(cleartext)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _verify_token(payload: dict[str, Any], expected: str | None) -> None:
|
||||||
|
if not expected:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
detail="FEISHU_VERIFICATION_TOKEN is required",
|
||||||
|
)
|
||||||
|
header = payload.get(FeishuPayloadKey.HEADER) or {}
|
||||||
|
token = payload.get(FeishuPayloadKey.TOKEN) or header.get(FeishuPayloadKey.TOKEN)
|
||||||
|
if not token or not compare_digest(str(token), expected):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||||
|
detail="Invalid Feishu token",
|
||||||
|
)
|
||||||
@@ -31,10 +31,18 @@ def _sdk_event_to_payload(event: Any) -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
def _handle_message_event(event: Any) -> None:
|
def _handle_message_event(event: Any) -> None:
|
||||||
|
_handle_verified_sdk_event(event)
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_app_ticket_event(event: Any) -> None:
|
||||||
|
_handle_verified_sdk_event(event)
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_verified_sdk_event(event: Any) -> None:
|
||||||
payload = _sdk_event_to_payload(event)
|
payload = _sdk_event_to_payload(event)
|
||||||
db = SessionLocal()
|
db = SessionLocal()
|
||||||
try:
|
try:
|
||||||
result = FeishuEventService(db).handle_event(
|
result = FeishuEventService(db)._handle_verified_event(
|
||||||
payload,
|
payload,
|
||||||
source=FeishuEventSource.LONG_CONNECTION,
|
source=FeishuEventSource.LONG_CONNECTION,
|
||||||
auto_reply=True,
|
auto_reply=True,
|
||||||
@@ -62,6 +70,7 @@ def run_long_connection() -> None:
|
|||||||
settings.feishu_verification_token or "",
|
settings.feishu_verification_token or "",
|
||||||
)
|
)
|
||||||
.register_p2_im_message_receive_v1(_handle_message_event)
|
.register_p2_im_message_receive_v1(_handle_message_event)
|
||||||
|
.register_p1_customized_event("app_ticket", _handle_app_ticket_event)
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
client = lark.ws.Client(
|
client = lark.ws.Client(
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Integer, String
|
from sqlalchemy import DateTime, Integer, String, Text
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
@@ -16,3 +16,17 @@ class FeishuEventReceipt(Base):
|
|||||||
event_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
event_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
||||||
message_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
message_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
||||||
received_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
received_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuAppTicket(Base):
|
||||||
|
__tablename__ = "feishu_app_tickets"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
app_id: Mapped[str] = mapped_column(String(128), unique=True, index=True)
|
||||||
|
app_ticket: Mapped[str] = mapped_column(Text)
|
||||||
|
received_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime,
|
||||||
|
default=utc_now,
|
||||||
|
onupdate=utc_now,
|
||||||
|
)
|
||||||
|
|||||||
@@ -1,12 +1,11 @@
|
|||||||
from typing import Any
|
from fastapi import APIRouter, Depends, Request
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.security import ApiPrincipal, require_api_key
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.application.feishu import FeishuCommandService, FeishuEventService
|
from app.application.feishu import FeishuCommandService, FeishuEventService
|
||||||
from app.modules.feishu.constants import FeishuEventSource, FeishuPayloadKey, FeishuResponseKey
|
from app.modules.feishu.constants import FeishuEventSource, FeishuPayloadKey, FeishuResponseKey
|
||||||
|
from app.modules.feishu.event_verification import FeishuWebhookVerifier
|
||||||
from app.modules.feishu.schemas import (
|
from app.modules.feishu.schemas import (
|
||||||
FeishuCardMessage,
|
FeishuCardMessage,
|
||||||
FeishuCommandRequest,
|
FeishuCommandRequest,
|
||||||
@@ -20,10 +19,11 @@ router = APIRouter()
|
|||||||
|
|
||||||
|
|
||||||
@router.post("/webhook")
|
@router.post("/webhook")
|
||||||
def feishu_webhook(payload: dict[str, Any], db: Session = Depends(get_db)) -> dict:
|
async def feishu_webhook(request: Request, db: Session = Depends(get_db)) -> dict:
|
||||||
"""Handle Feishu webhook challenge and text command events."""
|
"""Handle Feishu webhook challenge and text command events."""
|
||||||
|
|
||||||
return FeishuEventService(db).handle_event(
|
payload = FeishuWebhookVerifier().verify(await request.body(), request.headers)
|
||||||
|
return FeishuEventService(db)._handle_verified_event(
|
||||||
payload,
|
payload,
|
||||||
source=FeishuEventSource.WEBHOOK,
|
source=FeishuEventSource.WEBHOOK,
|
||||||
auto_reply=True,
|
auto_reply=True,
|
||||||
@@ -41,6 +41,7 @@ def send_text(
|
|||||||
receive_id=payload.receive_id,
|
receive_id=payload.receive_id,
|
||||||
receive_id_type=payload.receive_id_type,
|
receive_id_type=payload.receive_id_type,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
|
tenant_key=payload.tenant_key,
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
FeishuResponseKey.OK: result.get(FeishuPayloadKey.CODE) == 0,
|
FeishuResponseKey.OK: result.get(FeishuPayloadKey.CODE) == 0,
|
||||||
@@ -59,6 +60,7 @@ def send_card(
|
|||||||
receive_id=payload.receive_id,
|
receive_id=payload.receive_id,
|
||||||
receive_id_type=payload.receive_id_type,
|
receive_id_type=payload.receive_id_type,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
|
tenant_key=payload.tenant_key,
|
||||||
)
|
)
|
||||||
return {
|
return {
|
||||||
FeishuResponseKey.OK: result.get(FeishuPayloadKey.CODE) == 0,
|
FeishuResponseKey.OK: result.get(FeishuPayloadKey.CODE) == 0,
|
||||||
@@ -82,4 +84,5 @@ def preview_command(
|
|||||||
chat_id=payload.chat_id,
|
chat_id=payload.chat_id,
|
||||||
actor=principal.actor,
|
actor=principal.actor,
|
||||||
auto_reply=payload.auto_reply,
|
auto_reply=payload.auto_reply,
|
||||||
|
tenant_key=payload.tenant_key,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -12,12 +12,14 @@ class FeishuTextMessage(BaseModel):
|
|||||||
description="chat_id or open_id depending on type.",
|
description="chat_id or open_id depending on type.",
|
||||||
)
|
)
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID
|
||||||
|
tenant_key: str | None = Field(default=None, max_length=128)
|
||||||
text: str
|
text: str
|
||||||
|
|
||||||
|
|
||||||
class FeishuCardMessage(BaseModel):
|
class FeishuCardMessage(BaseModel):
|
||||||
receive_id: str | None = None
|
receive_id: str | None = None
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID
|
||||||
|
tenant_key: str | None = Field(default=None, max_length=128)
|
||||||
card: dict[str, Any]
|
card: dict[str, Any]
|
||||||
|
|
||||||
|
|
||||||
@@ -38,6 +40,7 @@ class FeishuSendResult(BaseModel):
|
|||||||
class FeishuCommandRequest(BaseModel):
|
class FeishuCommandRequest(BaseModel):
|
||||||
text: str
|
text: str
|
||||||
chat_id: str | None = None
|
chat_id: str | None = None
|
||||||
|
tenant_key: str | None = Field(default=None, max_length=128)
|
||||||
actor: str = ActorValue.API
|
actor: str = ActorValue.API
|
||||||
auto_reply: bool = False
|
auto_reply: bool = False
|
||||||
|
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
from hashlib import sha256
|
||||||
from secrets import compare_digest
|
from secrets import compare_digest
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -22,10 +23,18 @@ from app.modules.feishu.constants import (
|
|||||||
class FeishuService:
|
class FeishuService:
|
||||||
"""Send Feishu messages and record audit entries for outbound actions."""
|
"""Send Feishu messages and record audit entries for outbound actions."""
|
||||||
|
|
||||||
def __init__(self, db: Session):
|
def __init__(self, db: Session, tenant_key: str | None = None):
|
||||||
self.db = db
|
self.db = db
|
||||||
self.audit = AuditService(db)
|
self.audit = AuditService(db)
|
||||||
self.client = FeishuClient()
|
self.client = FeishuClient(db)
|
||||||
|
self.tenant_key = _optional_text(tenant_key) or _optional_text(
|
||||||
|
get_settings().feishu_default_tenant_key
|
||||||
|
)
|
||||||
|
|
||||||
|
def set_tenant_key(self, tenant_key: str | None) -> None:
|
||||||
|
"""Set the default tenant used by subsequent outbound operations."""
|
||||||
|
|
||||||
|
self.tenant_key = _optional_text(tenant_key)
|
||||||
|
|
||||||
def verify_event(self, payload: dict[str, Any]) -> None:
|
def verify_event(self, payload: dict[str, Any]) -> None:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
@@ -49,21 +58,32 @@ class FeishuService:
|
|||||||
receive_id: str | None = None,
|
receive_id: str | None = None,
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
||||||
actor: str = ActorValue.SYSTEM,
|
actor: str = ActorValue.SYSTEM,
|
||||||
|
uuid: str | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
|
record_audit: bool = True,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
result = self.client.send_text(text, receive_id, receive_id_type)
|
result = self.client.send_text(
|
||||||
self.audit.log(
|
text,
|
||||||
AuditLogCreate(
|
receive_id,
|
||||||
actor=actor,
|
receive_id_type,
|
||||||
source=AuditSource.FEISHU,
|
uuid,
|
||||||
action=AuditAction.FEISHU_SEND_TEXT,
|
tenant_key=self._resolve_tenant_key(tenant_key),
|
||||||
request_payload={
|
|
||||||
FeishuPayloadKey.RECEIVE_ID: receive_id,
|
|
||||||
FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type,
|
|
||||||
FeishuPayloadKey.TEXT: text,
|
|
||||||
},
|
|
||||||
response_payload=result,
|
|
||||||
)
|
|
||||||
)
|
)
|
||||||
|
if record_audit:
|
||||||
|
self.audit.log(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=AuditAction.FEISHU_SEND_TEXT,
|
||||||
|
request_payload={
|
||||||
|
"receive_target_hash": _target_fingerprint(receive_id),
|
||||||
|
FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type,
|
||||||
|
"content_length": len(text),
|
||||||
|
FeishuPayloadKey.UUID: uuid,
|
||||||
|
},
|
||||||
|
response_payload=result,
|
||||||
|
)
|
||||||
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def send_card(
|
def send_card(
|
||||||
@@ -72,17 +92,26 @@ class FeishuService:
|
|||||||
receive_id: str | None = None,
|
receive_id: str | None = None,
|
||||||
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
receive_id_type: str = FeishuReceiveIdType.CHAT_ID,
|
||||||
actor: str = ActorValue.SYSTEM,
|
actor: str = ActorValue.SYSTEM,
|
||||||
|
uuid: str | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
result = self.client.send_card(card, receive_id, receive_id_type)
|
result = self.client.send_card(
|
||||||
|
card,
|
||||||
|
receive_id,
|
||||||
|
receive_id_type,
|
||||||
|
uuid,
|
||||||
|
tenant_key=self._resolve_tenant_key(tenant_key),
|
||||||
|
)
|
||||||
self.audit.log(
|
self.audit.log(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
source=AuditSource.FEISHU,
|
source=AuditSource.FEISHU,
|
||||||
action=AuditAction.FEISHU_SEND_CARD,
|
action=AuditAction.FEISHU_SEND_CARD,
|
||||||
request_payload={
|
request_payload={
|
||||||
FeishuPayloadKey.RECEIVE_ID: receive_id,
|
"receive_target_hash": _target_fingerprint(receive_id),
|
||||||
FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type,
|
FeishuPayloadKey.RECEIVE_ID_TYPE: receive_id_type,
|
||||||
FeishuPayloadKey.CARD: card,
|
"card_element_count": len(card.get(FeishuPayloadKey.ELEMENTS) or []),
|
||||||
|
FeishuPayloadKey.UUID: uuid,
|
||||||
},
|
},
|
||||||
response_payload=result,
|
response_payload=result,
|
||||||
)
|
)
|
||||||
@@ -93,8 +122,12 @@ class FeishuService:
|
|||||||
self,
|
self,
|
||||||
image: bytes,
|
image: bytes,
|
||||||
actor: str = ActorValue.SYSTEM,
|
actor: str = ActorValue.SYSTEM,
|
||||||
|
tenant_key: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
result = self.client.upload_image(image)
|
result = self.client.upload_image(
|
||||||
|
image,
|
||||||
|
tenant_key=self._resolve_tenant_key(tenant_key),
|
||||||
|
)
|
||||||
self.audit.log(
|
self.audit.log(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
@@ -106,6 +139,9 @@ class FeishuService:
|
|||||||
)
|
)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
|
def _resolve_tenant_key(self, tenant_key: str | None) -> str | None:
|
||||||
|
return _optional_text(tenant_key) or self.tenant_key
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def build_basic_card(
|
def build_basic_card(
|
||||||
title: str,
|
title: str,
|
||||||
@@ -144,3 +180,14 @@ class FeishuService:
|
|||||||
},
|
},
|
||||||
FeishuPayloadKey.ELEMENTS: elements,
|
FeishuPayloadKey.ELEMENTS: elements,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _target_fingerprint(receive_id: str | None) -> str | None:
|
||||||
|
if not receive_id:
|
||||||
|
return None
|
||||||
|
return sha256(receive_id.encode("utf-8")).hexdigest()[:16]
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_text(value: str | None) -> str | None:
|
||||||
|
text = str(value or "").strip()
|
||||||
|
return text or None
|
||||||
|
|||||||
22
app/modules/feishu_users/__init__.py
Normal file
22
app/modules/feishu_users/__init__.py
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FeishuCapability,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
|
from app.modules.feishu_users.principal import FeishuMention, FeishuPrincipal
|
||||||
|
from app.modules.feishu_users.services import (
|
||||||
|
FeishuIdentityService,
|
||||||
|
FeishuUserManagementService,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"FeishuCapability",
|
||||||
|
"FeishuIdentityService",
|
||||||
|
"FeishuMention",
|
||||||
|
"FeishuPrincipal",
|
||||||
|
"FeishuUser",
|
||||||
|
"FeishuUserManagementService",
|
||||||
|
"FeishuUserRole",
|
||||||
|
"FeishuUserStatus",
|
||||||
|
]
|
||||||
21
app/modules/feishu_users/bootstrap.py
Normal file
21
app/modules/feishu_users/bootstrap.py
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
from hashlib import sha256
|
||||||
|
|
||||||
|
_ADMIN_BOOTSTRAP_TOMBSTONE_DOMAIN = (
|
||||||
|
b"company-ai-platform:feishu-admin-bootstrap-tombstone:v1\0"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def admin_bootstrap_identity_hash(tenant_key: str, open_id: str) -> str:
|
||||||
|
"""Return a domain-separated digest used only to prevent admin re-grants."""
|
||||||
|
|
||||||
|
tenant_bytes = tenant_key.encode("utf-8")
|
||||||
|
open_id_bytes = open_id.encode("utf-8")
|
||||||
|
identity = b"".join(
|
||||||
|
(
|
||||||
|
len(tenant_bytes).to_bytes(4, "big"),
|
||||||
|
tenant_bytes,
|
||||||
|
len(open_id_bytes).to_bytes(4, "big"),
|
||||||
|
open_id_bytes,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return sha256(_ADMIN_BOOTSTRAP_TOMBSTONE_DOMAIN + identity).hexdigest()
|
||||||
86
app/modules/feishu_users/constants.py
Normal file
86
app/modules/feishu_users/constants.py
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
from enum import StrEnum
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserRole(StrEnum):
|
||||||
|
USER = "user"
|
||||||
|
ADMIN = "admin"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserStatus(StrEnum):
|
||||||
|
ACTIVE = "active"
|
||||||
|
DISABLED = "disabled"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuCapability(StrEnum):
|
||||||
|
PERSONAL_AI = "personal_ai"
|
||||||
|
PERSONAL_DATA = "personal_data"
|
||||||
|
PRIVATE_SUBSCRIPTION = "private_subscription"
|
||||||
|
PERSONAL_MARKET = "personal_market"
|
||||||
|
COMPANY_REPORTS = "company_reports"
|
||||||
|
COMPANY_RULES = "company_rules"
|
||||||
|
USER_ADMINISTRATION = "user_administration"
|
||||||
|
GROUP_SUBSCRIPTION = "group_subscription"
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserAuditAction(StrEnum):
|
||||||
|
REGISTER = "feishu.user.register"
|
||||||
|
AUTHENTICATE = "feishu.user.authenticate"
|
||||||
|
PERMISSION_DENIED = "feishu.permission.denied"
|
||||||
|
LIST = "feishu.user.list"
|
||||||
|
READ = "feishu.user.read"
|
||||||
|
UPDATE = "feishu.user.update"
|
||||||
|
UPDATE_DENIED = "feishu.user.update_denied"
|
||||||
|
|
||||||
|
|
||||||
|
FEISHU_USER_CODE_PREFIX = "FSU"
|
||||||
|
FEISHU_USER_TARGET_TYPE = "feishu-user"
|
||||||
|
DEFAULT_FEISHU_USER_TIMEZONE = "Asia/Shanghai"
|
||||||
|
LAST_ACTIVE_ADMIN_ERROR = "The last active Feishu administrator cannot be changed"
|
||||||
|
FEISHU_USER_NOT_FOUND = "Feishu user not found"
|
||||||
|
INVALID_FEISHU_IDENTITY = "tenant_key and open_id are required"
|
||||||
|
INVALID_FEISHU_TIMEZONE = "Invalid IANA timezone"
|
||||||
|
INVALID_QUIET_HOURS = "quiet_hours_start and quiet_hours_end must both be set or cleared"
|
||||||
|
INVALID_ADMIN_IDENTITY = (
|
||||||
|
"FEISHU_ADMIN_IDENTITIES entries must use tenant_key:open_id format"
|
||||||
|
)
|
||||||
|
|
||||||
|
_USER_CAPABILITIES = frozenset(
|
||||||
|
{
|
||||||
|
FeishuCapability.PERSONAL_AI,
|
||||||
|
FeishuCapability.PERSONAL_DATA,
|
||||||
|
FeishuCapability.PRIVATE_SUBSCRIPTION,
|
||||||
|
FeishuCapability.PERSONAL_MARKET,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
_ADMIN_CAPABILITIES = frozenset(FeishuCapability)
|
||||||
|
|
||||||
|
|
||||||
|
def capabilities_for_role(role: str | FeishuUserRole) -> frozenset[FeishuCapability]:
|
||||||
|
"""Return the fixed capability set for a Feishu user role."""
|
||||||
|
|
||||||
|
if FeishuUserRole(role) == FeishuUserRole.ADMIN:
|
||||||
|
return _ADMIN_CAPABILITIES
|
||||||
|
return _USER_CAPABILITIES
|
||||||
|
|
||||||
|
|
||||||
|
def parse_admin_identities(
|
||||||
|
value: str | Iterable[str] | None,
|
||||||
|
) -> frozenset[tuple[str, str]]:
|
||||||
|
"""Parse exact tenant/open-id pairs used only for initial administrator creation."""
|
||||||
|
|
||||||
|
if value is None:
|
||||||
|
return frozenset()
|
||||||
|
entries = value.split(",") if isinstance(value, str) else value
|
||||||
|
identities: set[tuple[str, str]] = set()
|
||||||
|
for entry in entries:
|
||||||
|
text = str(entry).strip()
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
tenant_key, separator, open_id = text.partition(":")
|
||||||
|
tenant_key = tenant_key.strip()
|
||||||
|
open_id = open_id.strip()
|
||||||
|
if not separator or not tenant_key or not open_id:
|
||||||
|
raise ValueError(INVALID_ADMIN_IDENTITY)
|
||||||
|
identities.add((tenant_key, open_id))
|
||||||
|
return frozenset(identities)
|
||||||
63
app/modules/feishu_users/models.py
Normal file
63
app/modules/feishu_users/models.py
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
from datetime import datetime, time
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, Integer, String, Time, UniqueConstraint
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from app.core.database import Base
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
DEFAULT_FEISHU_USER_TIMEZONE,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUser(Base):
|
||||||
|
__tablename__ = "feishu_users"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"tenant_key",
|
||||||
|
"open_id",
|
||||||
|
name="uq_feishu_user_tenant_open_id",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
code: Mapped[str] = mapped_column(String(64), unique=True, index=True)
|
||||||
|
tenant_key: Mapped[str] = mapped_column(String(128), index=True)
|
||||||
|
open_id: Mapped[str] = mapped_column(String(128), index=True)
|
||||||
|
union_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
||||||
|
user_id: Mapped[str | None] = mapped_column(String(128), nullable=True, index=True)
|
||||||
|
role: Mapped[str] = mapped_column(
|
||||||
|
String(32),
|
||||||
|
default=FeishuUserRole.USER,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
status: Mapped[str] = mapped_column(
|
||||||
|
String(32),
|
||||||
|
default=FeishuUserStatus.ACTIVE,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
timezone: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
default=DEFAULT_FEISHU_USER_TIMEZONE,
|
||||||
|
)
|
||||||
|
quiet_hours_start: Mapped[time | None] = mapped_column(Time, nullable=True)
|
||||||
|
quiet_hours_end: Mapped[time | None] = mapped_column(Time, nullable=True)
|
||||||
|
last_active_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime,
|
||||||
|
default=utc_now,
|
||||||
|
onupdate=utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuAdminBootstrapTombstone(Base):
|
||||||
|
"""Irreversible marker preventing a deleted initial admin from re-bootstrap."""
|
||||||
|
|
||||||
|
__tablename__ = "feishu_admin_bootstrap_tombstones"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
identity_hash: Mapped[str] = mapped_column(String(64), unique=True, index=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
102
app/modules/feishu_users/principal.py
Normal file
102
app/modules/feishu_users/principal.py
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import time
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FeishuCapability,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
capabilities_for_role,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class FeishuMention:
|
||||||
|
"""Structured mention identity supplied by a verified Feishu event."""
|
||||||
|
|
||||||
|
key: str | None = None
|
||||||
|
name: str | None = None
|
||||||
|
tenant_key: str | None = None
|
||||||
|
open_id: str | None = None
|
||||||
|
union_id: str | None = None
|
||||||
|
user_id: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class FeishuPrincipal:
|
||||||
|
"""Authenticated Feishu user plus the current chat context."""
|
||||||
|
|
||||||
|
owner_id: int
|
||||||
|
user_code: str
|
||||||
|
tenant_key: str
|
||||||
|
open_id: str
|
||||||
|
union_id: str | None
|
||||||
|
feishu_user_id: str | None
|
||||||
|
role: str
|
||||||
|
status: str
|
||||||
|
timezone: str
|
||||||
|
quiet_hours_start: time | None
|
||||||
|
quiet_hours_end: time | None
|
||||||
|
chat_id: str | None = None
|
||||||
|
chat_type: str | None = None
|
||||||
|
mentions: tuple[FeishuMention, ...] = ()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_active(self) -> bool:
|
||||||
|
return self.status == FeishuUserStatus.ACTIVE
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_admin(self) -> bool:
|
||||||
|
return self.role == FeishuUserRole.ADMIN
|
||||||
|
|
||||||
|
def has_capability(self, capability: str | FeishuCapability) -> bool:
|
||||||
|
if not self.is_active:
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
required = FeishuCapability(capability)
|
||||||
|
return required in capabilities_for_role(self.role)
|
||||||
|
except ValueError:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def require_active(self) -> None:
|
||||||
|
if not self.is_active:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Feishu user is disabled",
|
||||||
|
)
|
||||||
|
|
||||||
|
def require_capability(self, capability: str | FeishuCapability) -> None:
|
||||||
|
self.require_active()
|
||||||
|
if not self.has_capability(capability):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_403_FORBIDDEN,
|
||||||
|
detail="Feishu user is not authorized for this capability",
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_user(
|
||||||
|
cls,
|
||||||
|
user: FeishuUser,
|
||||||
|
*,
|
||||||
|
chat_id: str | None = None,
|
||||||
|
chat_type: str | None = None,
|
||||||
|
mentions: tuple[FeishuMention, ...] = (),
|
||||||
|
) -> "FeishuPrincipal":
|
||||||
|
return cls(
|
||||||
|
owner_id=user.id,
|
||||||
|
user_code=user.code,
|
||||||
|
tenant_key=user.tenant_key,
|
||||||
|
open_id=user.open_id,
|
||||||
|
union_id=user.union_id,
|
||||||
|
feishu_user_id=user.user_id,
|
||||||
|
role=user.role,
|
||||||
|
status=user.status,
|
||||||
|
timezone=user.timezone,
|
||||||
|
quiet_hours_start=user.quiet_hours_start,
|
||||||
|
quiet_hours_end=user.quiet_hours_end,
|
||||||
|
chat_id=chat_id,
|
||||||
|
chat_type=chat_type,
|
||||||
|
mentions=mentions,
|
||||||
|
)
|
||||||
80
app/modules/feishu_users/routes.py
Normal file
80
app/modules/feishu_users/routes.py
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.application.feishu.personal_data import FeishuPersonalDataService
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
|
from app.modules.feishu_users.constants import FeishuUserRole, FeishuUserStatus
|
||||||
|
from app.modules.feishu_users.schemas import (
|
||||||
|
FeishuUserListRead,
|
||||||
|
FeishuUserRead,
|
||||||
|
FeishuUserUpdate,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.services import FeishuUserManagementService
|
||||||
|
from app.modules.personalization.schemas import ErasureResult
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/users", dependencies=[Depends(require_api_key)])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=FeishuUserListRead)
|
||||||
|
def list_users(
|
||||||
|
role: FeishuUserRole | None = None,
|
||||||
|
status_filter: FeishuUserStatus | None = Query(default=None, alias="status"),
|
||||||
|
tenant_key: str | None = None,
|
||||||
|
limit: int = Query(default=100, ge=1, le=500),
|
||||||
|
offset: int = Query(default=0, ge=0),
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
|
) -> dict:
|
||||||
|
items, total = FeishuUserManagementService(db).list_users(
|
||||||
|
role=role,
|
||||||
|
status_filter=status_filter,
|
||||||
|
tenant_key=tenant_key,
|
||||||
|
limit=limit,
|
||||||
|
offset=offset,
|
||||||
|
actor=principal.actor,
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"items": items,
|
||||||
|
"total": total,
|
||||||
|
"limit": limit,
|
||||||
|
"offset": offset,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{code}", response_model=FeishuUserRead)
|
||||||
|
def get_user(
|
||||||
|
code: str,
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
|
) -> FeishuUserRead:
|
||||||
|
return FeishuUserManagementService(db).get_user(
|
||||||
|
code,
|
||||||
|
actor=principal.actor,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/{code}", response_model=FeishuUserRead)
|
||||||
|
def update_user(
|
||||||
|
code: str,
|
||||||
|
payload: FeishuUserUpdate,
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
|
) -> FeishuUserRead:
|
||||||
|
return FeishuUserManagementService(db).update_user(
|
||||||
|
code,
|
||||||
|
changes=payload.model_dump(exclude_unset=True),
|
||||||
|
actor=principal.actor,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/{code}/personal-data", response_model=ErasureResult)
|
||||||
|
def erase_user_personal_data(
|
||||||
|
code: str,
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
|
) -> ErasureResult:
|
||||||
|
return FeishuPersonalDataService(db).erase_by_user_code(
|
||||||
|
code,
|
||||||
|
actor=principal.actor,
|
||||||
|
)
|
||||||
40
app/modules/feishu_users/schemas.py
Normal file
40
app/modules/feishu_users/schemas.py
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
from datetime import datetime, time
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from app.modules.feishu_users.constants import FeishuUserRole, FeishuUserStatus
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserRead(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
code: str
|
||||||
|
tenant_key: str
|
||||||
|
open_id: str
|
||||||
|
union_id: str | None
|
||||||
|
user_id: str | None
|
||||||
|
role: str
|
||||||
|
status: str
|
||||||
|
timezone: str
|
||||||
|
quiet_hours_start: time | None
|
||||||
|
quiet_hours_end: time | None
|
||||||
|
last_active_at: datetime
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserListRead(BaseModel):
|
||||||
|
items: list[FeishuUserRead]
|
||||||
|
total: int
|
||||||
|
limit: int
|
||||||
|
offset: int
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserUpdate(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
role: FeishuUserRole | None = None
|
||||||
|
status: FeishuUserStatus | None = None
|
||||||
|
timezone: str | None = Field(default=None, min_length=1, max_length=64)
|
||||||
|
quiet_hours_start: time | None = None
|
||||||
|
quiet_hours_end: time | None = None
|
||||||
9
app/modules/feishu_users/services/__init__.py
Normal file
9
app/modules/feishu_users/services/__init__.py
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
from app.modules.feishu_users.services.identity import FeishuIdentityService
|
||||||
|
from app.modules.feishu_users.services.management import (
|
||||||
|
FeishuUserManagementService,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"FeishuIdentityService",
|
||||||
|
"FeishuUserManagementService",
|
||||||
|
]
|
||||||
161
app/modules/feishu_users/services/identity.py
Normal file
161
app/modules/feishu_users/services/identity.py
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
from collections.abc import Iterable
|
||||||
|
from typing import Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import inspect, select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.config import get_settings
|
||||||
|
from app.core.constants import ActorValue
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FEISHU_USER_CODE_PREFIX,
|
||||||
|
FEISHU_USER_TARGET_TYPE,
|
||||||
|
INVALID_FEISHU_IDENTITY,
|
||||||
|
FeishuUserAuditAction,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
parse_admin_identities,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.bootstrap import admin_bootstrap_identity_hash
|
||||||
|
from app.modules.feishu_users.models import (
|
||||||
|
FeishuAdminBootstrapTombstone,
|
||||||
|
FeishuUser,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.principal import FeishuPrincipal
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuIdentityService:
|
||||||
|
"""Resolve or create identities only after the caller verifies the Feishu event."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
db: Session,
|
||||||
|
admin_identities: str | Iterable[str] | None = None,
|
||||||
|
):
|
||||||
|
self.db = db
|
||||||
|
self.audit = AuditService(db)
|
||||||
|
configured = (
|
||||||
|
admin_identities
|
||||||
|
if admin_identities is not None
|
||||||
|
else getattr(get_settings(), "feishu_admin_identities", ())
|
||||||
|
)
|
||||||
|
self.admin_identities = parse_admin_identities(configured)
|
||||||
|
|
||||||
|
def resolve_or_register(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
tenant_key: str,
|
||||||
|
open_id: str,
|
||||||
|
union_id: str | None = None,
|
||||||
|
user_id: str | None = None,
|
||||||
|
actor: str = ActorValue.FEISHU,
|
||||||
|
) -> FeishuPrincipal:
|
||||||
|
"""Return a principal for a previously verified Feishu sender."""
|
||||||
|
|
||||||
|
tenant_key = _required_identity_part(tenant_key)
|
||||||
|
open_id = _required_identity_part(open_id)
|
||||||
|
union_id = _optional_identity_part(union_id)
|
||||||
|
user_id = _optional_identity_part(user_id)
|
||||||
|
record = self.get_by_identity(tenant_key=tenant_key, open_id=open_id)
|
||||||
|
created = False
|
||||||
|
if record is None:
|
||||||
|
candidate = FeishuUser(
|
||||||
|
code=f"{FEISHU_USER_CODE_PREFIX}-{uuid4().hex[:20].upper()}",
|
||||||
|
tenant_key=tenant_key,
|
||||||
|
open_id=open_id,
|
||||||
|
union_id=union_id,
|
||||||
|
user_id=user_id,
|
||||||
|
role=self._initial_role(tenant_key, open_id),
|
||||||
|
status=FeishuUserStatus.ACTIVE,
|
||||||
|
last_active_at=utc_now(),
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(candidate)
|
||||||
|
self.db.flush()
|
||||||
|
record = candidate
|
||||||
|
created = True
|
||||||
|
except IntegrityError:
|
||||||
|
record = self.get_by_identity(tenant_key=tenant_key, open_id=open_id)
|
||||||
|
if record is None:
|
||||||
|
raise
|
||||||
|
|
||||||
|
record.last_active_at = utc_now()
|
||||||
|
if union_id:
|
||||||
|
record.union_id = union_id
|
||||||
|
if user_id:
|
||||||
|
record.user_id = user_id
|
||||||
|
self.audit.record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.FEISHU,
|
||||||
|
action=(
|
||||||
|
FeishuUserAuditAction.REGISTER
|
||||||
|
if created
|
||||||
|
else FeishuUserAuditAction.AUTHENTICATE
|
||||||
|
),
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
target_id=record.code,
|
||||||
|
risk_level=AuditRiskLevel.LOW,
|
||||||
|
response_payload={
|
||||||
|
"created": created,
|
||||||
|
"role": record.role,
|
||||||
|
"status": record.status,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
if created and inspect(self.db.get_bind()).has_table("market_watchlists"):
|
||||||
|
# Import locally so the identity domain does not create a module cycle.
|
||||||
|
from app.modules.market.service import MarketService
|
||||||
|
|
||||||
|
MarketService(self.db).claim_legacy_watchlist(record.id, record.open_id)
|
||||||
|
return FeishuPrincipal.from_user(record)
|
||||||
|
|
||||||
|
def get_by_identity(self, *, tenant_key: str, open_id: str) -> FeishuUser | None:
|
||||||
|
return self.db.execute(
|
||||||
|
select(FeishuUser).where(
|
||||||
|
FeishuUser.tenant_key == tenant_key,
|
||||||
|
FeishuUser.open_id == open_id,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
def get_by_code(self, code: str) -> FeishuUser | None:
|
||||||
|
return self.db.execute(
|
||||||
|
select(FeishuUser).where(FeishuUser.code == code)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
def _initial_role(self, tenant_key: str, open_id: str) -> str:
|
||||||
|
if (tenant_key, open_id) not in self.admin_identities:
|
||||||
|
return FeishuUserRole.USER
|
||||||
|
identity_hash = admin_bootstrap_identity_hash(tenant_key, open_id)
|
||||||
|
was_erased = self.db.scalar(
|
||||||
|
select(FeishuAdminBootstrapTombstone.id)
|
||||||
|
.where(
|
||||||
|
FeishuAdminBootstrapTombstone.identity_hash == identity_hash
|
||||||
|
)
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
return FeishuUserRole.USER if was_erased is not None else FeishuUserRole.ADMIN
|
||||||
|
|
||||||
|
|
||||||
|
def _required_identity_part(value: Any) -> str:
|
||||||
|
text = str(value or "").strip()
|
||||||
|
if not text:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail=INVALID_FEISHU_IDENTITY,
|
||||||
|
)
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_identity_part(value: Any) -> str | None:
|
||||||
|
text = str(value or "").strip()
|
||||||
|
return text or None
|
||||||
290
app/modules/feishu_users/services/management.py
Normal file
290
app/modules/feishu_users/services/management.py
Normal file
@@ -0,0 +1,290 @@
|
|||||||
|
from datetime import time
|
||||||
|
from typing import Any
|
||||||
|
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.constants import ActorValue
|
||||||
|
from app.modules.audit.constants import AuditRiskLevel, AuditSource
|
||||||
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
|
from app.modules.audit.service import AuditService
|
||||||
|
from app.modules.feishu_users.constants import (
|
||||||
|
FEISHU_USER_NOT_FOUND,
|
||||||
|
FEISHU_USER_TARGET_TYPE,
|
||||||
|
INVALID_FEISHU_TIMEZONE,
|
||||||
|
INVALID_QUIET_HOURS,
|
||||||
|
LAST_ACTIVE_ADMIN_ERROR,
|
||||||
|
FeishuUserAuditAction,
|
||||||
|
FeishuUserRole,
|
||||||
|
FeishuUserStatus,
|
||||||
|
)
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
|
|
||||||
|
_UPDATABLE_FIELDS = frozenset(
|
||||||
|
{
|
||||||
|
"role",
|
||||||
|
"status",
|
||||||
|
"timezone",
|
||||||
|
"quiet_hours_start",
|
||||||
|
"quiet_hours_end",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FeishuUserManagementService:
|
||||||
|
"""Manage Feishu users while preserving an active administrator."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
self.audit = AuditService(db)
|
||||||
|
|
||||||
|
def list_users(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
role: str | None = None,
|
||||||
|
status_filter: str | None = None,
|
||||||
|
tenant_key: str | None = None,
|
||||||
|
limit: int = 100,
|
||||||
|
offset: int = 0,
|
||||||
|
actor: str | None = None,
|
||||||
|
) -> tuple[list[FeishuUser], int]:
|
||||||
|
filters = []
|
||||||
|
if role:
|
||||||
|
filters.append(FeishuUser.role == FeishuUserRole(role))
|
||||||
|
if status_filter:
|
||||||
|
filters.append(FeishuUser.status == FeishuUserStatus(status_filter))
|
||||||
|
if tenant_key:
|
||||||
|
filters.append(FeishuUser.tenant_key == tenant_key)
|
||||||
|
total = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count()).select_from(FeishuUser).where(*filters)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
items = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(FeishuUser)
|
||||||
|
.where(*filters)
|
||||||
|
.order_by(FeishuUser.created_at.desc(), FeishuUser.id.desc())
|
||||||
|
.offset(offset)
|
||||||
|
.limit(limit)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
if actor:
|
||||||
|
self._audit_read(
|
||||||
|
actor=actor,
|
||||||
|
action=FeishuUserAuditAction.LIST,
|
||||||
|
response_payload={"count": len(items), "total": total},
|
||||||
|
)
|
||||||
|
return items, total
|
||||||
|
|
||||||
|
def get_user(self, code: str, *, actor: str | None = None) -> FeishuUser:
|
||||||
|
record = self._find_user(code)
|
||||||
|
if actor:
|
||||||
|
self._audit_read(
|
||||||
|
actor=actor,
|
||||||
|
action=FeishuUserAuditAction.READ,
|
||||||
|
target_id=record.code,
|
||||||
|
)
|
||||||
|
return record
|
||||||
|
|
||||||
|
def update_user(
|
||||||
|
self,
|
||||||
|
code: str,
|
||||||
|
*,
|
||||||
|
changes: dict[str, Any],
|
||||||
|
actor: str = ActorValue.API,
|
||||||
|
) -> FeishuUser:
|
||||||
|
unexpected = set(changes) - _UPDATABLE_FIELDS
|
||||||
|
if unexpected:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail=f"Unsupported Feishu user fields: {', '.join(sorted(unexpected))}",
|
||||||
|
)
|
||||||
|
active_admin_ids = (
|
||||||
|
self._active_admin_ids()
|
||||||
|
if {"role", "status"} & changes.keys()
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
record = self.db.execute(
|
||||||
|
select(FeishuUser)
|
||||||
|
.where(FeishuUser.code == code)
|
||||||
|
.with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=FEISHU_USER_NOT_FOUND,
|
||||||
|
)
|
||||||
|
if not changes:
|
||||||
|
return record
|
||||||
|
|
||||||
|
normalized = self._normalized_changes(record, changes)
|
||||||
|
proposed_role = normalized.get("role", record.role)
|
||||||
|
proposed_status = normalized.get("status", record.status)
|
||||||
|
removes_active_admin = (
|
||||||
|
record.role == FeishuUserRole.ADMIN
|
||||||
|
and record.status == FeishuUserStatus.ACTIVE
|
||||||
|
and (
|
||||||
|
proposed_role != FeishuUserRole.ADMIN
|
||||||
|
or proposed_status != FeishuUserStatus.ACTIVE
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if removes_active_admin and active_admin_ids == [record.id]:
|
||||||
|
self.audit.record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.API,
|
||||||
|
action=FeishuUserAuditAction.UPDATE_DENIED,
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
target_id=record.code,
|
||||||
|
risk_level=AuditRiskLevel.HIGH,
|
||||||
|
request_payload={
|
||||||
|
"role": proposed_role,
|
||||||
|
"status": proposed_status,
|
||||||
|
},
|
||||||
|
response_payload={
|
||||||
|
"result": "denied",
|
||||||
|
"reason": "last_active_admin",
|
||||||
|
},
|
||||||
|
status="denied",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail=LAST_ACTIVE_ADMIN_ERROR,
|
||||||
|
)
|
||||||
|
|
||||||
|
before = _auditable_state(record)
|
||||||
|
for field, value in normalized.items():
|
||||||
|
setattr(record, field, value)
|
||||||
|
after = _auditable_state(record)
|
||||||
|
self.audit.record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.API,
|
||||||
|
action=FeishuUserAuditAction.UPDATE,
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
target_id=record.code,
|
||||||
|
risk_level=AuditRiskLevel.HIGH,
|
||||||
|
request_payload={"before": before, "after": after},
|
||||||
|
response_payload={"updated_fields": sorted(normalized)},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return record
|
||||||
|
|
||||||
|
def _find_user(self, code: str) -> FeishuUser:
|
||||||
|
record = self.db.execute(
|
||||||
|
select(FeishuUser).where(FeishuUser.code == code)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail=FEISHU_USER_NOT_FOUND,
|
||||||
|
)
|
||||||
|
return record
|
||||||
|
|
||||||
|
def _normalized_changes(
|
||||||
|
self,
|
||||||
|
record: FeishuUser,
|
||||||
|
changes: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized = dict(changes)
|
||||||
|
if "role" in normalized:
|
||||||
|
if normalized["role"] is None:
|
||||||
|
raise _unprocessable("role cannot be null")
|
||||||
|
normalized["role"] = FeishuUserRole(normalized["role"])
|
||||||
|
if "status" in normalized:
|
||||||
|
if normalized["status"] is None:
|
||||||
|
raise _unprocessable("status cannot be null")
|
||||||
|
normalized["status"] = FeishuUserStatus(normalized["status"])
|
||||||
|
if "timezone" in normalized:
|
||||||
|
timezone = str(normalized["timezone"] or "").strip()
|
||||||
|
if not timezone:
|
||||||
|
raise _unprocessable(INVALID_FEISHU_TIMEZONE)
|
||||||
|
try:
|
||||||
|
ZoneInfo(timezone)
|
||||||
|
except ZoneInfoNotFoundError as exc:
|
||||||
|
raise _unprocessable(INVALID_FEISHU_TIMEZONE) from exc
|
||||||
|
normalized["timezone"] = timezone
|
||||||
|
for field in ("quiet_hours_start", "quiet_hours_end"):
|
||||||
|
if field in normalized:
|
||||||
|
normalized[field] = _optional_time(normalized[field])
|
||||||
|
quiet_start = normalized.get("quiet_hours_start", record.quiet_hours_start)
|
||||||
|
quiet_end = normalized.get("quiet_hours_end", record.quiet_hours_end)
|
||||||
|
if (quiet_start is None) != (quiet_end is None):
|
||||||
|
raise _unprocessable(INVALID_QUIET_HOURS)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _active_admin_ids(self) -> list[int]:
|
||||||
|
return list(
|
||||||
|
self.db.execute(
|
||||||
|
select(FeishuUser.id)
|
||||||
|
.where(
|
||||||
|
FeishuUser.role == FeishuUserRole.ADMIN,
|
||||||
|
FeishuUser.status == FeishuUserStatus.ACTIVE,
|
||||||
|
)
|
||||||
|
.order_by(FeishuUser.id.asc())
|
||||||
|
.with_for_update()
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
|
||||||
|
def _audit_read(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
actor: str,
|
||||||
|
action: str,
|
||||||
|
target_id: str | None = None,
|
||||||
|
response_payload: dict[str, Any] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.audit.record(
|
||||||
|
AuditLogCreate(
|
||||||
|
actor=actor,
|
||||||
|
source=AuditSource.API,
|
||||||
|
action=action,
|
||||||
|
target_type=FEISHU_USER_TARGET_TYPE,
|
||||||
|
target_id=target_id,
|
||||||
|
risk_level=AuditRiskLevel.LOW,
|
||||||
|
response_payload=response_payload,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _auditable_state(record: FeishuUser) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"role": record.role,
|
||||||
|
"status": record.status,
|
||||||
|
"timezone": record.timezone,
|
||||||
|
"quiet_hours_start": (
|
||||||
|
record.quiet_hours_start.isoformat()
|
||||||
|
if record.quiet_hours_start is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
"quiet_hours_end": (
|
||||||
|
record.quiet_hours_end.isoformat()
|
||||||
|
if record.quiet_hours_end is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_time(value: Any) -> time | None:
|
||||||
|
if value is None or isinstance(value, time):
|
||||||
|
return value
|
||||||
|
try:
|
||||||
|
return time.fromisoformat(str(value))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise _unprocessable("Invalid quiet-hours time") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _unprocessable(detail: str) -> HTTPException:
|
||||||
|
return HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail=detail,
|
||||||
|
)
|
||||||
@@ -75,6 +75,8 @@ class LegacyQueryError(StrEnum):
|
|||||||
PROJECT_QUERY_NOT_CONFIGURED = "LEGACY_PROJECT_QUERY is not configured. Configure it first."
|
PROJECT_QUERY_NOT_CONFIGURED = "LEGACY_PROJECT_QUERY is not configured. Configure it first."
|
||||||
TASK_QUERY_NOT_CONFIGURED = "LEGACY_TASK_QUERY is not configured. Configure it first."
|
TASK_QUERY_NOT_CONFIGURED = "LEGACY_TASK_QUERY is not configured. Configure it first."
|
||||||
ONLY_SELECT_ALLOWED = "Only SELECT statements are allowed"
|
ONLY_SELECT_ALLOWED = "Only SELECT statements are allowed"
|
||||||
|
SQL_COMMENTS_NOT_ALLOWED = "SQL comments are not allowed in readonly queries"
|
||||||
|
SINGLE_STATEMENT_REQUIRED = "Only one SQL statement is allowed"
|
||||||
FORBIDDEN_SQL_TOKEN = "Forbidden SQL token in readonly query"
|
FORBIDDEN_SQL_TOKEN = "Forbidden SQL token in readonly query"
|
||||||
INVALID_LIMIT = "Invalid readonly query limit"
|
INVALID_LIMIT = "Invalid readonly query limit"
|
||||||
APP_DB_UNAVAILABLE = "Application database session is not available"
|
APP_DB_UNAVAILABLE = "Application database session is not available"
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from sqlalchemy.orm import Session
|
|||||||
from app.core.background.task_queue import enqueue_legacy_project_sync, enqueue_legacy_task_sync
|
from app.core.background.task_queue import enqueue_legacy_project_sync, enqueue_legacy_task_sync
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.http.masking import mask_configured
|
from app.core.http.masking import mask_configured
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.legacy_mysql.schemas import (
|
from app.modules.legacy_mysql.schemas import (
|
||||||
LegacyProjectSyncRequest,
|
LegacyProjectSyncRequest,
|
||||||
LegacyProjectSyncResult,
|
LegacyProjectSyncResult,
|
||||||
@@ -60,7 +60,6 @@ def sync_projects(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
result = LegacyMySQLService(db).sync_projects(
|
result = LegacyMySQLService(db).sync_projects(
|
||||||
source_query=payload.source_query,
|
source_query=payload.source_query,
|
||||||
source_query_name=payload.source_query_name,
|
source_query_name=payload.source_query_name,
|
||||||
@@ -77,7 +76,6 @@ def enqueue_sync_projects(
|
|||||||
payload: LegacyProjectSyncRequest,
|
payload: LegacyProjectSyncRequest,
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return enqueue_legacy_project_sync(
|
return enqueue_legacy_project_sync(
|
||||||
source_query=payload.source_query,
|
source_query=payload.source_query,
|
||||||
source_query_name=payload.source_query_name,
|
source_query_name=payload.source_query_name,
|
||||||
@@ -94,7 +92,6 @@ def sync_tasks(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
result = LegacyMySQLService(db).sync_tasks(
|
result = LegacyMySQLService(db).sync_tasks(
|
||||||
source_query=payload.source_query,
|
source_query=payload.source_query,
|
||||||
source_query_name=payload.source_query_name,
|
source_query_name=payload.source_query_name,
|
||||||
@@ -111,7 +108,6 @@ def enqueue_sync_tasks(
|
|||||||
payload: LegacyTaskSyncRequest,
|
payload: LegacyTaskSyncRequest,
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return enqueue_legacy_task_sync(
|
return enqueue_legacy_task_sync(
|
||||||
source_query=payload.source_query,
|
source_query=payload.source_query,
|
||||||
source_query_name=payload.source_query_name,
|
source_query_name=payload.source_query_name,
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ from sqlalchemy.engine import RowMapping
|
|||||||
from app.modules.legacy_mysql.constants import LEGACY_SQL_TRAILING_TERMINATOR, LegacyQueryName
|
from app.modules.legacy_mysql.constants import LEGACY_SQL_TRAILING_TERMINATOR, LegacyQueryName
|
||||||
|
|
||||||
FORBIDDEN_SQL_TOKENS = {
|
FORBIDDEN_SQL_TOKENS = {
|
||||||
|
"benchmark",
|
||||||
|
"call",
|
||||||
"insert",
|
"insert",
|
||||||
"update",
|
"update",
|
||||||
"delete",
|
"delete",
|
||||||
@@ -14,9 +16,21 @@ FORBIDDEN_SQL_TOKENS = {
|
|||||||
"alter",
|
"alter",
|
||||||
"truncate",
|
"truncate",
|
||||||
"create",
|
"create",
|
||||||
|
"do",
|
||||||
|
"dumpfile",
|
||||||
|
"execute",
|
||||||
"replace",
|
"replace",
|
||||||
"grant",
|
"grant",
|
||||||
|
"get_lock",
|
||||||
|
"handler",
|
||||||
|
"into",
|
||||||
|
"load_file",
|
||||||
|
"lock",
|
||||||
|
"outfile",
|
||||||
|
"release_lock",
|
||||||
"revoke",
|
"revoke",
|
||||||
|
"set",
|
||||||
|
"sleep",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from fastapi import HTTPException, status
|
|||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.security import ensure_business_mutations_enabled
|
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.modules.audit.constants import AuditAction, AuditRiskLevel, AuditSource, AuditStatus
|
from app.modules.audit.constants import AuditAction, AuditRiskLevel, AuditSource, AuditStatus
|
||||||
from app.modules.audit.schemas import AuditLogCreate
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
@@ -44,7 +43,6 @@ class LegacyProjectSyncMixin:
|
|||||||
dry_run: bool = True,
|
dry_run: bool = True,
|
||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
ensure_business_mutations_enabled()
|
|
||||||
if self.db is None:
|
if self.db is None:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import re
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from fastapi import HTTPException, status
|
from fastapi import HTTPException, status
|
||||||
@@ -23,6 +24,13 @@ from app.modules.legacy_mysql.constants import (
|
|||||||
|
|
||||||
from app.modules.legacy_mysql.services.common import FORBIDDEN_SQL_TOKENS, _normalize_sql, _query_name_text, _row_to_dict
|
from app.modules.legacy_mysql.services.common import FORBIDDEN_SQL_TOKENS, _normalize_sql, _query_name_text, _row_to_dict
|
||||||
|
|
||||||
|
_SQL_COMMENT_MARKERS = ("--", "#", "/*", "*/")
|
||||||
|
_SQL_QUOTED_CONTENT_PATTERN = re.compile(
|
||||||
|
r"""'(?:''|\\.|[^'])*'|"(?:""|\\.|[^"])*"|`(?:``|[^`])*`""",
|
||||||
|
flags=re.DOTALL,
|
||||||
|
)
|
||||||
|
_SQL_WORD_PATTERN = re.compile(r"[a-z_]+")
|
||||||
|
|
||||||
|
|
||||||
class LegacyQueryMixin:
|
class LegacyQueryMixin:
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@@ -36,13 +44,27 @@ class LegacyQueryMixin:
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _ensure_readonly(sql: str) -> None:
|
def _ensure_readonly(sql: str) -> None:
|
||||||
stripped = sql.strip().lower()
|
stripped = sql.strip()
|
||||||
if not stripped.startswith(LEGACY_SELECT_PREFIX):
|
scrubbed = _SQL_QUOTED_CONTENT_PATTERN.sub(" ", stripped)
|
||||||
|
if any(marker in scrubbed for marker in _SQL_COMMENT_MARKERS):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail=LegacyQueryError.SQL_COMMENTS_NOT_ALLOWED,
|
||||||
|
)
|
||||||
|
statement = scrubbed.rstrip()
|
||||||
|
if statement.endswith(LEGACY_SQL_TRAILING_TERMINATOR):
|
||||||
|
statement = statement[:-1].rstrip()
|
||||||
|
if LEGACY_SQL_TRAILING_TERMINATOR in statement:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
|
detail=LegacyQueryError.SINGLE_STATEMENT_REQUIRED,
|
||||||
|
)
|
||||||
|
if not re.match(rf"^{LEGACY_SELECT_PREFIX}\b", statement, flags=re.IGNORECASE):
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST,
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
detail=LegacyQueryError.ONLY_SELECT_ALLOWED,
|
detail=LegacyQueryError.ONLY_SELECT_ALLOWED,
|
||||||
)
|
)
|
||||||
tokens = {token.strip(" ;,\n\t") for token in stripped.replace("(", " ").split()}
|
tokens = set(_SQL_WORD_PATTERN.findall(statement.lower()))
|
||||||
if tokens & FORBIDDEN_SQL_TOKENS:
|
if tokens & FORBIDDEN_SQL_TOKENS:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST,
|
status_code=status.HTTP_400_BAD_REQUEST,
|
||||||
@@ -117,9 +139,10 @@ class LegacyQueryMixin:
|
|||||||
engine = self._ensure_engine()
|
engine = self._ensure_engine()
|
||||||
params = dict(params or {})
|
params = dict(params or {})
|
||||||
try:
|
try:
|
||||||
params[LegacyResponseKey.LIMIT] = bounded_limit(
|
limit_value = bounded_limit(
|
||||||
params.get(LegacyResponseKey.LIMIT, limit)
|
params.get(LegacyResponseKey.LIMIT, limit)
|
||||||
)
|
)
|
||||||
|
params[LegacyResponseKey.LIMIT] = limit_value
|
||||||
except (TypeError, ValueError) as exc:
|
except (TypeError, ValueError) as exc:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
@@ -130,7 +153,10 @@ class LegacyQueryMixin:
|
|||||||
limited_sql = f"{sql.rstrip(LEGACY_SQL_TRAILING_TERMINATOR)}{LEGACY_LIMIT_CLAUSE}"
|
limited_sql = f"{sql.rstrip(LEGACY_SQL_TRAILING_TERMINATOR)}{LEGACY_LIMIT_CLAUSE}"
|
||||||
with engine.connect() as conn:
|
with engine.connect() as conn:
|
||||||
result = conn.execute(text(limited_sql), params)
|
result = conn.execute(text(limited_sql), params)
|
||||||
rows = [_row_to_dict(row) for row in result.mappings().all()]
|
rows = [
|
||||||
|
_row_to_dict(row)
|
||||||
|
for row in result.mappings().fetchmany(limit_value)
|
||||||
|
]
|
||||||
columns = list(rows[0].keys()) if rows else []
|
columns = list(rows[0].keys()) if rows else []
|
||||||
return {
|
return {
|
||||||
LegacyResponseKey.COLUMNS: columns,
|
LegacyResponseKey.COLUMNS: columns,
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ from fastapi import HTTPException, status
|
|||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.security import ensure_business_mutations_enabled
|
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.modules.audit.constants import AuditAction, AuditRiskLevel, AuditSource, AuditStatus
|
from app.modules.audit.constants import AuditAction, AuditRiskLevel, AuditSource, AuditStatus
|
||||||
from app.modules.audit.schemas import AuditLogCreate
|
from app.modules.audit.schemas import AuditLogCreate
|
||||||
@@ -44,7 +43,6 @@ class LegacyTaskSyncMixin:
|
|||||||
dry_run: bool = True,
|
dry_run: bool = True,
|
||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
ensure_business_mutations_enabled()
|
|
||||||
if self.db is None:
|
if self.db is None:
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.background.task_queue.market import enqueue_market_report
|
from app.core.background.task_queue.market import enqueue_market_report
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.market.service import MarketService
|
from app.modules.market.service import MarketService
|
||||||
|
|
||||||
router = APIRouter(dependencies=[Depends(require_api_key)])
|
router = APIRouter(dependencies=[Depends(require_api_key)])
|
||||||
@@ -89,13 +89,11 @@ def announcements(
|
|||||||
|
|
||||||
@router.post("/sync/daily")
|
@router.post("/sync/daily")
|
||||||
def sync_daily(trade_date: date, db: Session = Depends(get_db)) -> dict:
|
def sync_daily(trade_date: date, db: Session = Depends(get_db)) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return MarketService(db).sync_daily(trade_date)
|
return MarketService(db).sync_daily(trade_date)
|
||||||
|
|
||||||
|
|
||||||
@router.post("/sync/macro")
|
@router.post("/sync/macro")
|
||||||
def sync_macro(reference_date: date | None = None, db: Session = Depends(get_db)) -> dict:
|
def sync_macro(reference_date: date | None = None, db: Session = Depends(get_db)) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return MarketService(db).sync_macro(reference_date)
|
return MarketService(db).sync_macro(reference_date)
|
||||||
|
|
||||||
|
|
||||||
@@ -103,13 +101,11 @@ def sync_macro(reference_date: date | None = None, db: Session = Depends(get_db)
|
|||||||
def sync_announcements(
|
def sync_announcements(
|
||||||
start_date: date, end_date: date, db: Session = Depends(get_db)
|
start_date: date, end_date: date, db: Session = Depends(get_db)
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return {"processed": MarketService(db).sync_announcements(start_date, end_date)}
|
return {"processed": MarketService(db).sync_announcements(start_date, end_date)}
|
||||||
|
|
||||||
|
|
||||||
@router.post("/reports/enqueue")
|
@router.post("/reports/enqueue")
|
||||||
def enqueue_report(payload: MarketReportRequest) -> dict:
|
def enqueue_report(payload: MarketReportRequest) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return enqueue_market_report(payload.report_type, payload.reference_date, payload.force)
|
return enqueue_market_report(payload.report_type, payload.reference_date, payload.force)
|
||||||
|
|
||||||
|
|
||||||
@@ -119,7 +115,6 @@ def add_watchlist(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return MarketService(db).add_watchlist(principal.actor, payload.symbol)
|
return MarketService(db).add_watchlist(principal.actor, payload.symbol)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -780,15 +780,31 @@ class MarketService:
|
|||||||
report["content"] = "\n".join(lines)
|
report["content"] = "\n".join(lines)
|
||||||
return report
|
return report
|
||||||
|
|
||||||
def add_watchlist(self, actor: str, symbol: str) -> dict[str, Any]:
|
def add_watchlist(
|
||||||
|
self,
|
||||||
|
actor: str,
|
||||||
|
symbol: str,
|
||||||
|
owner_id: int | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
code = normalize_symbol(symbol)
|
code = normalize_symbol(symbol)
|
||||||
|
owner_clause = (
|
||||||
|
MarketWatchlist.owner_id.is_(None)
|
||||||
|
if owner_id is None
|
||||||
|
else MarketWatchlist.owner_id == owner_id
|
||||||
|
)
|
||||||
record = self.db.execute(
|
record = self.db.execute(
|
||||||
select(MarketWatchlist).where(
|
select(MarketWatchlist).where(
|
||||||
MarketWatchlist.actor == actor, MarketWatchlist.symbol == code
|
owner_clause,
|
||||||
|
MarketWatchlist.symbol == code,
|
||||||
|
*(
|
||||||
|
(MarketWatchlist.actor == actor,)
|
||||||
|
if owner_id is None
|
||||||
|
else ()
|
||||||
|
),
|
||||||
)
|
)
|
||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
if record is None:
|
if record is None:
|
||||||
record = MarketWatchlist(actor=actor, symbol=code)
|
record = MarketWatchlist(owner_id=owner_id, actor=actor, symbol=code)
|
||||||
self.db.add(record)
|
self.db.add(record)
|
||||||
else:
|
else:
|
||||||
record.enabled = True
|
record.enabled = True
|
||||||
@@ -804,16 +820,77 @@ class MarketService:
|
|||||||
response_payload={"enabled": True},
|
response_payload={"enabled": True},
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return {"actor": actor, "symbol": code, "enabled": True}
|
return {
|
||||||
|
"actor": actor,
|
||||||
|
"owner_id": owner_id,
|
||||||
|
"symbol": code,
|
||||||
|
"enabled": True,
|
||||||
|
}
|
||||||
|
|
||||||
def watchlist(self, actor: str) -> list[dict[str, Any]]:
|
def watchlist(
|
||||||
|
self,
|
||||||
|
actor: str,
|
||||||
|
owner_id: int | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
owner_clause = (
|
||||||
|
MarketWatchlist.owner_id.is_(None)
|
||||||
|
if owner_id is None
|
||||||
|
else MarketWatchlist.owner_id == owner_id
|
||||||
|
)
|
||||||
records = self.db.execute(
|
records = self.db.execute(
|
||||||
select(MarketWatchlist).where(
|
select(MarketWatchlist).where(
|
||||||
MarketWatchlist.actor == actor, MarketWatchlist.enabled.is_(True)
|
owner_clause,
|
||||||
|
MarketWatchlist.enabled.is_(True),
|
||||||
|
*(
|
||||||
|
(MarketWatchlist.actor == actor,)
|
||||||
|
if owner_id is None
|
||||||
|
else ()
|
||||||
|
),
|
||||||
)
|
)
|
||||||
).scalars()
|
).scalars()
|
||||||
return [{"symbol": r.symbol} for r in records]
|
return [{"symbol": r.symbol} for r in records]
|
||||||
|
|
||||||
|
def claim_legacy_watchlist(self, owner_id: int, open_id: str) -> int:
|
||||||
|
"""Claim still-unowned rows created by the verified legacy Feishu actor."""
|
||||||
|
|
||||||
|
legacy_records = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(MarketWatchlist).where(
|
||||||
|
MarketWatchlist.owner_id.is_(None),
|
||||||
|
MarketWatchlist.actor == open_id,
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
claimed = 0
|
||||||
|
for legacy in legacy_records:
|
||||||
|
existing = self.db.execute(
|
||||||
|
select(MarketWatchlist).where(
|
||||||
|
MarketWatchlist.owner_id == owner_id,
|
||||||
|
MarketWatchlist.symbol == legacy.symbol,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if existing is not None:
|
||||||
|
existing.enabled = existing.enabled or legacy.enabled
|
||||||
|
self.db.delete(legacy)
|
||||||
|
continue
|
||||||
|
legacy.owner_id = owner_id
|
||||||
|
claimed += 1
|
||||||
|
self.db.commit()
|
||||||
|
return claimed
|
||||||
|
|
||||||
|
def delete_owner_watchlist(self, owner_id: int) -> int:
|
||||||
|
"""Stage deletion of all personal watchlist rows for an owner."""
|
||||||
|
|
||||||
|
records = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(MarketWatchlist).where(MarketWatchlist.owner_id == owner_id)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
for record in records:
|
||||||
|
self.db.delete(record)
|
||||||
|
self.db.flush()
|
||||||
|
return len(records)
|
||||||
|
|
||||||
def _ai(self, skill: AISkillId, report: dict[str, Any], actor: str) -> dict[str, Any]:
|
def _ai(self, skill: AISkillId, report: dict[str, Any], actor: str) -> dict[str, Any]:
|
||||||
try:
|
try:
|
||||||
result = AIService(self.db).run_skill(
|
result = AIService(self.db).run_skill(
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
from sqlalchemy import DateTime, Integer, String
|
from sqlalchemy import DateTime, Integer, String, UniqueConstraint
|
||||||
from sqlalchemy.orm import Mapped, mapped_column
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
from app.core.database import Base
|
from app.core.database import Base
|
||||||
@@ -9,6 +9,9 @@ from app.core.utils.time import utc_now
|
|||||||
|
|
||||||
class SystemHeartbeat(Base):
|
class SystemHeartbeat(Base):
|
||||||
__tablename__ = "system_heartbeats"
|
__tablename__ = "system_heartbeats"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("component", "instance_id", name="uq_system_heartbeat_component_instance"),
|
||||||
|
)
|
||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
component: Mapped[str] = mapped_column(String(128), index=True)
|
component: Mapped[str] = mapped_column(String(128), index=True)
|
||||||
|
|||||||
@@ -32,4 +32,6 @@ def ready(
|
|||||||
def metrics(
|
def metrics(
|
||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
return ObservabilityService(db).metrics()
|
result = ObservabilityService(db).metrics()
|
||||||
|
db.commit()
|
||||||
|
return result
|
||||||
|
|||||||
@@ -1,7 +1,10 @@
|
|||||||
|
from collections.abc import Callable
|
||||||
from datetime import timedelta
|
from datetime import timedelta
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import select, text
|
from sqlalchemy import and_, func, or_, select, text
|
||||||
|
from sqlalchemy.dialects.postgresql import insert as postgresql_insert
|
||||||
|
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.core.config import get_settings
|
from app.core.config import get_settings
|
||||||
@@ -18,6 +21,10 @@ from app.modules.audit.schemas import AuditLogCreate
|
|||||||
from app.modules.audit.service import AuditService
|
from app.modules.audit.service import AuditService
|
||||||
from app.modules.events.constants import EventStatus
|
from app.modules.events.constants import EventStatus
|
||||||
from app.modules.events.services import EventService
|
from app.modules.events.services import EventService
|
||||||
|
from app.modules.feishu.app_tickets import FeishuAppTicketService
|
||||||
|
from app.modules.feishu.constants import FeishuAppType
|
||||||
|
from app.modules.feishu_users.constants import FeishuUserStatus
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
from app.modules.observability.constants import (
|
from app.modules.observability.constants import (
|
||||||
HeartbeatStatus,
|
HeartbeatStatus,
|
||||||
ObservabilityKey,
|
ObservabilityKey,
|
||||||
@@ -27,6 +34,11 @@ from app.modules.observability.constants import (
|
|||||||
from app.modules.observability.models import SystemHeartbeat
|
from app.modules.observability.models import SystemHeartbeat
|
||||||
from app.modules.workflows.constants import WorkflowStatus
|
from app.modules.workflows.constants import WorkflowStatus
|
||||||
from app.modules.workflows.service import WorkflowService
|
from app.modules.workflows.service import WorkflowService
|
||||||
|
from app.modules.subscriptions.constants import (
|
||||||
|
PushDeliveryStatus,
|
||||||
|
PushSubscriptionStatus,
|
||||||
|
)
|
||||||
|
from app.modules.subscriptions.models import PushDelivery, PushSubscription
|
||||||
|
|
||||||
|
|
||||||
class ObservabilityService:
|
class ObservabilityService:
|
||||||
@@ -40,31 +52,42 @@ class ObservabilityService:
|
|||||||
|
|
||||||
def ready(self) -> dict[str, Any]:
|
def ready(self) -> dict[str, Any]:
|
||||||
checks = {
|
checks = {
|
||||||
ObservabilityKey.DATABASE: self._database_check(),
|
ObservabilityKey.DATABASE: self._safe_call(self._database_check),
|
||||||
ObservabilityKey.REDIS: self._redis_check(),
|
ObservabilityKey.REDIS: self._safe_call(self._redis_check),
|
||||||
ObservabilityKey.EVENTS: self._events_check(),
|
ObservabilityKey.EVENTS: self._safe_call(self._events_check),
|
||||||
ObservabilityKey.WORKFLOWS: self._workflows_check(),
|
ObservabilityKey.WORKFLOWS: self._safe_call(self._workflows_check),
|
||||||
ObservabilityKey.HEARTBEATS: self._heartbeats_check(),
|
ObservabilityKey.HEARTBEATS: self._safe_call(self._heartbeats_check),
|
||||||
}
|
"feishu_subscriptions": self._safe_call(
|
||||||
degraded = any(
|
self._feishu_subscriptions_check
|
||||||
item[ObservabilityKey.STATUS]
|
|
||||||
in {ObservabilityStatus.DEGRADED, ObservabilityStatus.ERROR}
|
|
||||||
for item in checks.values()
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
ObservabilityKey.STATUS: (
|
|
||||||
ObservabilityStatus.DEGRADED if degraded else ObservabilityStatus.OK
|
|
||||||
),
|
),
|
||||||
|
}
|
||||||
|
statuses = {item[ObservabilityKey.STATUS] for item in checks.values()}
|
||||||
|
if statuses & {ObservabilityStatus.ERROR, ObservabilityStatus.DEGRADED}:
|
||||||
|
overall_status = ObservabilityStatus.DEGRADED
|
||||||
|
else:
|
||||||
|
overall_status = ObservabilityStatus.OK
|
||||||
|
return {
|
||||||
|
ObservabilityKey.STATUS: overall_status,
|
||||||
ObservabilityKey.CHECKS: checks,
|
ObservabilityKey.CHECKS: checks,
|
||||||
}
|
}
|
||||||
|
|
||||||
def metrics(self) -> dict[str, Any]:
|
def metrics(self) -> dict[str, Any]:
|
||||||
return {
|
return {
|
||||||
ObservabilityKey.METRICS: {
|
ObservabilityKey.METRICS: {
|
||||||
ObservabilityKey.EVENTS: EventService(self.db).count_by_status(),
|
ObservabilityKey.EVENTS: self._safe_call(
|
||||||
ObservabilityKey.WORKFLOWS: WorkflowService(self.db).count_by_status(),
|
lambda: EventService(self.db).count_by_status()
|
||||||
ObservabilityKey.AI_MEMORY: AIMemoryService(self.db).count_by_status(),
|
),
|
||||||
ObservabilityKey.HEARTBEATS: self.heartbeat_summary(),
|
ObservabilityKey.WORKFLOWS: self._safe_call(
|
||||||
|
lambda: WorkflowService(self.db).count_by_status()
|
||||||
|
),
|
||||||
|
ObservabilityKey.AI_MEMORY: self._safe_call(
|
||||||
|
lambda: AIMemoryService(self.db).count_by_status()
|
||||||
|
),
|
||||||
|
ObservabilityKey.HEARTBEATS: self._safe_call(
|
||||||
|
self.heartbeat_summary
|
||||||
|
),
|
||||||
|
"feishu_users": self._safe_call(self._feishu_user_metrics),
|
||||||
|
"subscriptions": self._safe_call(self._subscription_metrics),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -76,24 +99,7 @@ class ObservabilityService:
|
|||||||
actor: str = ActorValue.SYSTEM,
|
actor: str = ActorValue.SYSTEM,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
now = utc_now()
|
now = utc_now()
|
||||||
record = self.db.execute(
|
record = self._upsert_heartbeat(component, instance_id, status_value, now)
|
||||||
select(SystemHeartbeat).where(
|
|
||||||
SystemHeartbeat.component == component,
|
|
||||||
SystemHeartbeat.instance_id == instance_id,
|
|
||||||
)
|
|
||||||
).scalar_one_or_none()
|
|
||||||
if record is None:
|
|
||||||
record = SystemHeartbeat(
|
|
||||||
component=component,
|
|
||||||
instance_id=instance_id,
|
|
||||||
status=status_value,
|
|
||||||
last_seen_at=now,
|
|
||||||
)
|
|
||||||
self.db.add(record)
|
|
||||||
else:
|
|
||||||
record.status = status_value
|
|
||||||
record.last_seen_at = now
|
|
||||||
record.updated_at = now
|
|
||||||
AuditService(self.db).record(
|
AuditService(self.db).record(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
@@ -115,7 +121,13 @@ class ObservabilityService:
|
|||||||
return self._serialize_heartbeat(record)
|
return self._serialize_heartbeat(record)
|
||||||
|
|
||||||
def heartbeat_summary(self) -> dict[str, Any]:
|
def heartbeat_summary(self) -> dict[str, Any]:
|
||||||
records = list(self.db.execute(select(SystemHeartbeat)).scalars())
|
records = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(SystemHeartbeat)
|
||||||
|
.where(SystemHeartbeat.last_seen_at >= self._heartbeat_retention_threshold())
|
||||||
|
.order_by(SystemHeartbeat.component.asc(), SystemHeartbeat.instance_id.asc())
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
threshold = self._heartbeat_stale_threshold()
|
threshold = self._heartbeat_stale_threshold()
|
||||||
stale = [item for item in records if item.last_seen_at < threshold]
|
stale = [item for item in records if item.last_seen_at < threshold]
|
||||||
active = len(records) - len(stale)
|
active = len(records) - len(stale)
|
||||||
@@ -132,31 +144,75 @@ class ObservabilityService:
|
|||||||
],
|
],
|
||||||
}
|
}
|
||||||
|
|
||||||
def _database_check(self) -> dict[str, Any]:
|
def _safe_call(self, operation: Callable[[], Any]) -> Any:
|
||||||
try:
|
try:
|
||||||
self.db.execute(text("select 1")).scalar()
|
return operation()
|
||||||
except Exception as exc:
|
except Exception:
|
||||||
|
self.db.rollback()
|
||||||
return {
|
return {
|
||||||
ObservabilityKey.STATUS: ObservabilityStatus.ERROR,
|
ObservabilityKey.STATUS: ObservabilityStatus.ERROR,
|
||||||
ObservabilityMetricKey.ERROR: str(exc),
|
ObservabilityMetricKey.ERROR: "unavailable",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def _database_check(self) -> dict[str, Any]:
|
||||||
|
self.db.execute(text("select 1")).scalar()
|
||||||
return {ObservabilityKey.STATUS: ObservabilityStatus.OK}
|
return {ObservabilityKey.STATUS: ObservabilityStatus.OK}
|
||||||
|
|
||||||
def _redis_check(self) -> dict[str, Any]:
|
def _redis_check(self) -> dict[str, Any]:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
if not settings.task_queue_enabled:
|
if not settings.task_queue_enabled:
|
||||||
return {ObservabilityKey.STATUS: ObservabilityStatus.SKIPPED}
|
return {ObservabilityKey.STATUS: ObservabilityStatus.SKIPPED}
|
||||||
try:
|
from redis import Redis
|
||||||
from redis import Redis
|
|
||||||
|
|
||||||
Redis.from_url(settings.redis_url, socket_connect_timeout=1).ping()
|
client = Redis.from_url(
|
||||||
except Exception as exc:
|
settings.redis_url,
|
||||||
return {
|
socket_connect_timeout=1,
|
||||||
ObservabilityKey.STATUS: ObservabilityStatus.DEGRADED,
|
socket_timeout=1,
|
||||||
ObservabilityMetricKey.ERROR: str(exc),
|
)
|
||||||
}
|
try:
|
||||||
|
client.ping()
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
return {ObservabilityKey.STATUS: ObservabilityStatus.OK}
|
return {ObservabilityKey.STATUS: ObservabilityStatus.OK}
|
||||||
|
|
||||||
|
def _upsert_heartbeat(
|
||||||
|
self,
|
||||||
|
component: str,
|
||||||
|
instance_id: str,
|
||||||
|
status_value: str,
|
||||||
|
now: Any,
|
||||||
|
) -> SystemHeartbeat:
|
||||||
|
dialect_name = self.db.get_bind().dialect.name
|
||||||
|
insert_factory = {
|
||||||
|
"postgresql": postgresql_insert,
|
||||||
|
"sqlite": sqlite_insert,
|
||||||
|
}.get(dialect_name)
|
||||||
|
if insert_factory is None:
|
||||||
|
raise RuntimeError(f"Unsupported heartbeat database dialect: {dialect_name}")
|
||||||
|
statement = insert_factory(SystemHeartbeat).values(
|
||||||
|
component=component,
|
||||||
|
instance_id=instance_id,
|
||||||
|
status=status_value,
|
||||||
|
last_seen_at=now,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
statement = statement.on_conflict_do_update(
|
||||||
|
index_elements=["component", "instance_id"],
|
||||||
|
set_={
|
||||||
|
"status": status_value,
|
||||||
|
"last_seen_at": now,
|
||||||
|
"updated_at": now,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self.db.execute(statement)
|
||||||
|
return self.db.execute(
|
||||||
|
select(SystemHeartbeat).where(
|
||||||
|
SystemHeartbeat.component == component,
|
||||||
|
SystemHeartbeat.instance_id == instance_id,
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
|
||||||
def _events_check(self) -> dict[str, Any]:
|
def _events_check(self) -> dict[str, Any]:
|
||||||
counts = EventService(self.db).count_by_status()
|
counts = EventService(self.db).count_by_status()
|
||||||
failed = counts.get(EventStatus.FAILED, 0)
|
failed = counts.get(EventStatus.FAILED, 0)
|
||||||
@@ -196,6 +252,134 @@ class ObservabilityService:
|
|||||||
],
|
],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def _feishu_subscriptions_check(self) -> dict[str, Any]:
|
||||||
|
active = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(PushSubscription)
|
||||||
|
.where(PushSubscription.status == PushSubscriptionStatus.ACTIVE)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
processable_delivery_filter = or_(
|
||||||
|
and_(
|
||||||
|
PushDelivery.status.in_(
|
||||||
|
[
|
||||||
|
PushDeliveryStatus.PENDING,
|
||||||
|
PushDeliveryStatus.RETRY,
|
||||||
|
]
|
||||||
|
),
|
||||||
|
PushDelivery.next_attempt_at.is_not(None),
|
||||||
|
),
|
||||||
|
and_(
|
||||||
|
PushDelivery.status == PushDeliveryStatus.PROCESSING,
|
||||||
|
PushDelivery.locked_until.is_not(None),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
processable_deliveries = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(PushDelivery)
|
||||||
|
.where(processable_delivery_filter)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
if not active and not processable_deliveries:
|
||||||
|
return {
|
||||||
|
ObservabilityKey.STATUS: ObservabilityStatus.SKIPPED,
|
||||||
|
"active": 0,
|
||||||
|
"processable_deliveries": 0,
|
||||||
|
}
|
||||||
|
settings = get_settings()
|
||||||
|
app_id = str(settings.feishu_app_id or "").strip()
|
||||||
|
credentials_configured = bool(app_id and settings.feishu_app_secret)
|
||||||
|
active_tenant_count = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count(func.distinct(FeishuUser.tenant_key)))
|
||||||
|
.select_from(PushSubscription)
|
||||||
|
.join(FeishuUser, FeishuUser.id == PushSubscription.owner_id)
|
||||||
|
.outerjoin(
|
||||||
|
PushDelivery,
|
||||||
|
PushDelivery.subscription_id == PushSubscription.id,
|
||||||
|
)
|
||||||
|
.where(
|
||||||
|
or_(
|
||||||
|
PushSubscription.status == PushSubscriptionStatus.ACTIVE,
|
||||||
|
processable_delivery_filter,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
ticket_configured = False
|
||||||
|
default_tenant_configured = bool(
|
||||||
|
str(settings.feishu_default_tenant_key or "").strip()
|
||||||
|
)
|
||||||
|
reasons: list[str] = []
|
||||||
|
if not credentials_configured:
|
||||||
|
reasons.append("credentials_missing")
|
||||||
|
if settings.feishu_app_type == FeishuAppType.STORE:
|
||||||
|
database_ticket = (
|
||||||
|
FeishuAppTicketService(self.db).get_ticket(app_id)
|
||||||
|
if app_id
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
ticket_configured = bool(
|
||||||
|
str(database_ticket or settings.feishu_app_ticket or "").strip()
|
||||||
|
)
|
||||||
|
if not ticket_configured:
|
||||||
|
reasons.append("app_ticket_missing")
|
||||||
|
if not default_tenant_configured:
|
||||||
|
reasons.append("default_tenant_missing")
|
||||||
|
elif active_tenant_count > 1:
|
||||||
|
reasons.append("self_app_multiple_tenants")
|
||||||
|
return {
|
||||||
|
ObservabilityKey.STATUS: (
|
||||||
|
ObservabilityStatus.OK
|
||||||
|
if not reasons
|
||||||
|
else ObservabilityStatus.DEGRADED
|
||||||
|
),
|
||||||
|
"active": active,
|
||||||
|
"processable_deliveries": processable_deliveries,
|
||||||
|
"app_type": settings.feishu_app_type,
|
||||||
|
"credentials_configured": credentials_configured,
|
||||||
|
"ticket_configured": ticket_configured,
|
||||||
|
"default_tenant_configured": default_tenant_configured,
|
||||||
|
"active_tenant_count": active_tenant_count,
|
||||||
|
"reasons": reasons,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _feishu_user_metrics(self) -> dict[str, int]:
|
||||||
|
active = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(FeishuUser)
|
||||||
|
.where(FeishuUser.status == FeishuUserStatus.ACTIVE)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
return {"active": active}
|
||||||
|
|
||||||
|
def _subscription_metrics(self) -> dict[str, int]:
|
||||||
|
active = int(
|
||||||
|
self.db.scalar(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(PushSubscription)
|
||||||
|
.where(PushSubscription.status == PushSubscriptionStatus.ACTIVE)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
delivery_rows = self.db.execute(
|
||||||
|
select(PushDelivery.status, func.count()).group_by(PushDelivery.status)
|
||||||
|
).all()
|
||||||
|
deliveries = {str(status_value): int(count) for status_value, count in delivery_rows}
|
||||||
|
return {
|
||||||
|
"active": active,
|
||||||
|
"pending": deliveries.get(PushDeliveryStatus.PENDING, 0)
|
||||||
|
+ deliveries.get(PushDeliveryStatus.RETRY, 0),
|
||||||
|
"failed": deliveries.get(PushDeliveryStatus.FAILED, 0),
|
||||||
|
}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _serialize_heartbeat(record: SystemHeartbeat) -> dict[str, Any]:
|
def _serialize_heartbeat(record: SystemHeartbeat) -> dict[str, Any]:
|
||||||
return {
|
return {
|
||||||
@@ -209,3 +393,12 @@ class ObservabilityService:
|
|||||||
def _heartbeat_stale_threshold() -> Any:
|
def _heartbeat_stale_threshold() -> Any:
|
||||||
settings = get_settings()
|
settings = get_settings()
|
||||||
return utc_now() - timedelta(seconds=settings.heartbeat_interval_seconds * 3)
|
return utc_now() - timedelta(seconds=settings.heartbeat_interval_seconds * 3)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _heartbeat_retention_threshold() -> Any:
|
||||||
|
settings = get_settings()
|
||||||
|
retention_seconds = max(
|
||||||
|
settings.heartbeat_retention_seconds,
|
||||||
|
settings.heartbeat_interval_seconds * 3,
|
||||||
|
)
|
||||||
|
return utc_now() - timedelta(seconds=retention_seconds)
|
||||||
|
|||||||
23
app/modules/personalization/__init__.py
Normal file
23
app/modules/personalization/__init__.py
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
from app.modules.personalization.models import (
|
||||||
|
AIConversation,
|
||||||
|
AIConversationMessage,
|
||||||
|
PersonalDataErasureRequest,
|
||||||
|
UserPreference,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.services import (
|
||||||
|
ConversationService,
|
||||||
|
PersonalDataErasureService,
|
||||||
|
PersonalizationContextService,
|
||||||
|
PreferenceService,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AIConversation",
|
||||||
|
"AIConversationMessage",
|
||||||
|
"ConversationService",
|
||||||
|
"PersonalDataErasureRequest",
|
||||||
|
"PersonalDataErasureService",
|
||||||
|
"PersonalizationContextService",
|
||||||
|
"PreferenceService",
|
||||||
|
"UserPreference",
|
||||||
|
]
|
||||||
101
app/modules/personalization/constants.py
Normal file
101
app/modules/personalization/constants.py
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
from enum import StrEnum
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceCategory(StrEnum):
|
||||||
|
LANGUAGE = "language"
|
||||||
|
TONE = "tone"
|
||||||
|
DETAIL = "detail"
|
||||||
|
TOPIC = "topic"
|
||||||
|
INTEREST = "interest"
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceSource(StrEnum):
|
||||||
|
EXPLICIT = "explicit"
|
||||||
|
AUTO = "auto"
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationRole(StrEnum):
|
||||||
|
USER = "user"
|
||||||
|
ASSISTANT = "assistant"
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationChatType(StrEnum):
|
||||||
|
PRIVATE = "private"
|
||||||
|
GROUP = "group"
|
||||||
|
|
||||||
|
|
||||||
|
class PersonalizationContextKey(StrEnum):
|
||||||
|
SYSTEM_CONSTRAINTS = "system_constraints"
|
||||||
|
COMPANY_RULES = "company_rules"
|
||||||
|
PERSONAL_RULES = "personal_rules"
|
||||||
|
CURRENT_REQUEST = "current_request"
|
||||||
|
PREFERENCES = "preferences"
|
||||||
|
INTERESTS = "interests"
|
||||||
|
PERSONAL_MEMORY = "personal_memory"
|
||||||
|
CONVERSATION_HISTORY = "conversation_history"
|
||||||
|
|
||||||
|
|
||||||
|
PREFERENCE_CODE_PREFIX = "PREF"
|
||||||
|
CONVERSATION_CODE_PREFIX = "CONV"
|
||||||
|
PREFERENCE_MAX_VALUE_LENGTH = 1000
|
||||||
|
CONVERSATION_MAX_CONTENT_LENGTH = 20_000
|
||||||
|
CONVERSATION_RETENTION_DAYS = 30
|
||||||
|
CONVERSATION_MAX_TURNS = 20
|
||||||
|
CONVERSATION_MAX_MESSAGES = CONVERSATION_MAX_TURNS * 2
|
||||||
|
ERASURE_CONFIRMATION_TTL_MINUTES = 10
|
||||||
|
|
||||||
|
UNAVAILABLE_AI_PROVIDERS = frozenset({"", "noop"})
|
||||||
|
PREFERENCE_SIGNAL_TERMS = (
|
||||||
|
"以后",
|
||||||
|
"记住",
|
||||||
|
"偏好",
|
||||||
|
"喜欢",
|
||||||
|
"希望",
|
||||||
|
"请用",
|
||||||
|
"请保持",
|
||||||
|
"关注",
|
||||||
|
"感兴趣",
|
||||||
|
"prefer",
|
||||||
|
"preference",
|
||||||
|
"i like",
|
||||||
|
"interested in",
|
||||||
|
)
|
||||||
|
|
||||||
|
# These terms identify categories that must never become an inferred personal profile.
|
||||||
|
# General topics such as public market news remain allowed; the financial terms below are
|
||||||
|
# intentionally limited to private account, compensation, and confidential-company facts.
|
||||||
|
SENSITIVE_PREFERENCE_TERMS = (
|
||||||
|
"api key",
|
||||||
|
"api_key",
|
||||||
|
"access token",
|
||||||
|
"access_token",
|
||||||
|
"password",
|
||||||
|
"secret",
|
||||||
|
"token",
|
||||||
|
"密码",
|
||||||
|
"密钥",
|
||||||
|
"令牌",
|
||||||
|
"健康",
|
||||||
|
"病史",
|
||||||
|
"疾病",
|
||||||
|
"诊断",
|
||||||
|
"医疗记录",
|
||||||
|
"宗教",
|
||||||
|
"信仰",
|
||||||
|
"政治立场",
|
||||||
|
"党派",
|
||||||
|
"选举倾向",
|
||||||
|
"性取向",
|
||||||
|
"同性恋",
|
||||||
|
"异性恋",
|
||||||
|
"绩效",
|
||||||
|
"考核结果",
|
||||||
|
"银行账号",
|
||||||
|
"银行卡",
|
||||||
|
"工资",
|
||||||
|
"薪资",
|
||||||
|
"个人收入",
|
||||||
|
"财务秘密",
|
||||||
|
"未公开财务",
|
||||||
|
"保密预算",
|
||||||
|
)
|
||||||
131
app/modules/personalization/models.py
Normal file
131
app/modules/personalization/models.py
Normal file
@@ -0,0 +1,131 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import DateTime, ForeignKey, Integer, String, Text, UniqueConstraint
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
|
||||||
|
from app.core.database import Base
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.feishu_users.models import FeishuUser
|
||||||
|
from app.modules.personalization.constants import (
|
||||||
|
CONVERSATION_CODE_PREFIX,
|
||||||
|
PREFERENCE_CODE_PREFIX,
|
||||||
|
PreferenceSource,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _public_code(prefix: str) -> str:
|
||||||
|
return f"{prefix}-{uuid4().hex}"
|
||||||
|
|
||||||
|
|
||||||
|
class UserPreference(Base):
|
||||||
|
__tablename__ = "user_preferences"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"owner_id",
|
||||||
|
"category",
|
||||||
|
"normalized_value",
|
||||||
|
name="uq_user_preference_owner_category_value",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
code: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
default=lambda: _public_code(PREFERENCE_CODE_PREFIX),
|
||||||
|
unique=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("feishu_users.id", ondelete="CASCADE"),
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner: Mapped[FeishuUser] = relationship()
|
||||||
|
category: Mapped[str] = mapped_column(String(32), index=True)
|
||||||
|
value: Mapped[str] = mapped_column(Text)
|
||||||
|
normalized_value: Mapped[str] = mapped_column(String(1000))
|
||||||
|
source: Mapped[str] = mapped_column(
|
||||||
|
String(32),
|
||||||
|
default=PreferenceSource.EXPLICIT,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime,
|
||||||
|
default=utc_now,
|
||||||
|
onupdate=utc_now,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AIConversation(Base):
|
||||||
|
__tablename__ = "ai_conversations"
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"owner_id",
|
||||||
|
"chat_type",
|
||||||
|
"chat_key",
|
||||||
|
name="uq_ai_conversation_owner_chat",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
code: Mapped[str] = mapped_column(
|
||||||
|
String(64),
|
||||||
|
default=lambda: _public_code(CONVERSATION_CODE_PREFIX),
|
||||||
|
unique=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("feishu_users.id", ondelete="CASCADE"),
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner: Mapped[FeishuUser] = relationship()
|
||||||
|
chat_type: Mapped[str] = mapped_column(String(32), index=True)
|
||||||
|
chat_key: Mapped[str] = mapped_column(String(256), index=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime,
|
||||||
|
default=utc_now,
|
||||||
|
onupdate=utc_now,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
messages: Mapped[list["AIConversationMessage"]] = relationship(
|
||||||
|
back_populates="conversation",
|
||||||
|
cascade="all, delete-orphan",
|
||||||
|
passive_deletes=True,
|
||||||
|
order_by="AIConversationMessage.id",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class AIConversationMessage(Base):
|
||||||
|
__tablename__ = "ai_conversation_messages"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
conversation_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("ai_conversations.id", ondelete="CASCADE"),
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
role: Mapped[str] = mapped_column(String(32), index=True)
|
||||||
|
content: Mapped[str] = mapped_column(Text)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
conversation: Mapped[AIConversation] = relationship(back_populates="messages")
|
||||||
|
|
||||||
|
|
||||||
|
class PersonalDataErasureRequest(Base):
|
||||||
|
__tablename__ = "personal_data_erasure_requests"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
owner_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("feishu_users.id", ondelete="CASCADE"),
|
||||||
|
unique=True,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
owner: Mapped[FeishuUser] = relationship()
|
||||||
|
token_hash: Mapped[str] = mapped_column(String(64))
|
||||||
|
expires_at: Mapped[datetime] = mapped_column(DateTime, index=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=utc_now, index=True)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
|
DateTime,
|
||||||
|
default=utc_now,
|
||||||
|
onupdate=utc_now,
|
||||||
|
)
|
||||||
65
app/modules/personalization/schemas.py
Normal file
65
app/modules/personalization/schemas.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
from app.modules.personalization.constants import PreferenceCategory, PreferenceSource
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceCreate(BaseModel):
|
||||||
|
category: PreferenceCategory
|
||||||
|
value: str = Field(..., min_length=1, max_length=1000)
|
||||||
|
source: PreferenceSource = PreferenceSource.EXPLICIT
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceUpdate(BaseModel):
|
||||||
|
category: PreferenceCategory | None = None
|
||||||
|
value: str | None = Field(default=None, min_length=1, max_length=1000)
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceRead(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
code: str
|
||||||
|
category: str
|
||||||
|
value: str
|
||||||
|
source: str
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class ExtractedPreference(BaseModel):
|
||||||
|
category: PreferenceCategory
|
||||||
|
value: str = Field(..., min_length=1, max_length=1000)
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceExtractionPayload(BaseModel):
|
||||||
|
preferences: list[ExtractedPreference] = Field(default_factory=list, max_length=20)
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationMessageRead(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
role: str
|
||||||
|
content: str
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationRead(BaseModel):
|
||||||
|
model_config = ConfigDict(from_attributes=True)
|
||||||
|
|
||||||
|
code: str
|
||||||
|
chat_type: str
|
||||||
|
chat_key: str
|
||||||
|
messages: list[ConversationMessageRead] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class ErasureConfirmation(BaseModel):
|
||||||
|
confirmation_code: str
|
||||||
|
expires_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class ErasureResult(BaseModel):
|
||||||
|
anonymous_id: str
|
||||||
|
deleted: dict[str, int] = Field(default_factory=dict)
|
||||||
|
extra: dict[str, Any] = Field(default_factory=dict)
|
||||||
15
app/modules/personalization/services/__init__.py
Normal file
15
app/modules/personalization/services/__init__.py
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
from app.modules.personalization.services.context import (
|
||||||
|
PersonalizationContext,
|
||||||
|
PersonalizationContextService,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.services.conversations import ConversationService
|
||||||
|
from app.modules.personalization.services.erasure import PersonalDataErasureService
|
||||||
|
from app.modules.personalization.services.preferences import PreferenceService
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ConversationService",
|
||||||
|
"PersonalDataErasureService",
|
||||||
|
"PersonalizationContext",
|
||||||
|
"PersonalizationContextService",
|
||||||
|
"PreferenceService",
|
||||||
|
]
|
||||||
191
app/modules/personalization/services/context.py
Normal file
191
app/modules/personalization/services/context.py
Normal file
@@ -0,0 +1,191 @@
|
|||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.constants import ActorValue
|
||||||
|
from app.modules.ai_memory.constants import AIMemoryScope
|
||||||
|
from app.modules.ai_memory.service import AIMemoryService
|
||||||
|
from app.modules.business.models import MarketWatchlist
|
||||||
|
from app.modules.personalization.constants import (
|
||||||
|
PersonalizationContextKey,
|
||||||
|
PreferenceCategory,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.services.conversations import ConversationService
|
||||||
|
from app.modules.personalization.services.preferences import PreferenceService
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PersonalizationContext:
|
||||||
|
"""Ordered, provider-neutral context sections for one AI request."""
|
||||||
|
|
||||||
|
system_constraints: str
|
||||||
|
company_rules: list[dict[str, Any]]
|
||||||
|
personal_rules: list[dict[str, Any]]
|
||||||
|
current_request: str
|
||||||
|
preferences: list[dict[str, Any]]
|
||||||
|
interests: list[dict[str, Any]]
|
||||||
|
personal_memory: list[dict[str, Any]]
|
||||||
|
conversation_history: list[dict[str, Any]]
|
||||||
|
provider_session_id: str | None = field(default=None)
|
||||||
|
|
||||||
|
def as_ordered_dict(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
PersonalizationContextKey.SYSTEM_CONSTRAINTS: self.system_constraints,
|
||||||
|
PersonalizationContextKey.COMPANY_RULES: self.company_rules,
|
||||||
|
PersonalizationContextKey.PERSONAL_RULES: self.personal_rules,
|
||||||
|
PersonalizationContextKey.CURRENT_REQUEST: self.current_request,
|
||||||
|
PersonalizationContextKey.PREFERENCES: self.preferences,
|
||||||
|
PersonalizationContextKey.INTERESTS: self.interests,
|
||||||
|
PersonalizationContextKey.PERSONAL_MEMORY: self.personal_memory,
|
||||||
|
PersonalizationContextKey.CONVERSATION_HISTORY: self.conversation_history,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class PersonalizationContextService:
|
||||||
|
"""Load only the explicitly requested company and owner-scoped context layers."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
self.memory = AIMemoryService(db)
|
||||||
|
self.preferences = PreferenceService(db)
|
||||||
|
self.conversations = ConversationService(db)
|
||||||
|
|
||||||
|
def build(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
owner_id: int | None,
|
||||||
|
request: str,
|
||||||
|
system_constraints: str,
|
||||||
|
chat_type: str | None = None,
|
||||||
|
chat_key: str | None = None,
|
||||||
|
scope: str = AIMemoryScope.GLOBAL,
|
||||||
|
subject: str | None = None,
|
||||||
|
actor: str = ActorValue.SYSTEM,
|
||||||
|
include_company_rules: bool = True,
|
||||||
|
include_personal_context: bool = True,
|
||||||
|
include_history: bool = True,
|
||||||
|
) -> PersonalizationContext:
|
||||||
|
company_rules = (
|
||||||
|
self.memory.active_rules(
|
||||||
|
scope=scope,
|
||||||
|
subject=subject,
|
||||||
|
owner_id=None,
|
||||||
|
)
|
||||||
|
if include_company_rules
|
||||||
|
else []
|
||||||
|
)
|
||||||
|
personal_rules: list[dict[str, Any]] = []
|
||||||
|
preference_items: list[dict[str, Any]] = []
|
||||||
|
interests: list[dict[str, Any]] = []
|
||||||
|
personal_memory: list[dict[str, Any]] = []
|
||||||
|
history: list[dict[str, Any]] = []
|
||||||
|
provider_session_id: str | None = None
|
||||||
|
|
||||||
|
if owner_id is not None and include_personal_context:
|
||||||
|
personal_rules = self.memory.active_rules(
|
||||||
|
scope=scope,
|
||||||
|
subject=subject,
|
||||||
|
owner_id=owner_id,
|
||||||
|
)
|
||||||
|
all_preferences = self.preferences.list_preferences(owner_id)
|
||||||
|
interest_categories = {
|
||||||
|
PreferenceCategory.TOPIC,
|
||||||
|
PreferenceCategory.INTEREST,
|
||||||
|
}
|
||||||
|
for preference in all_preferences:
|
||||||
|
if preference["category"] in interest_categories:
|
||||||
|
interests.append(preference)
|
||||||
|
else:
|
||||||
|
preference_items.append(preference)
|
||||||
|
interests.extend(self._watchlist_interests(owner_id))
|
||||||
|
personal_memory = self.memory.recall(
|
||||||
|
query=request,
|
||||||
|
scope=scope,
|
||||||
|
subject=subject,
|
||||||
|
actor=actor,
|
||||||
|
owner_id=owner_id,
|
||||||
|
)
|
||||||
|
if include_history and chat_type and chat_key:
|
||||||
|
history = self.conversations.history(owner_id, chat_type, chat_key)
|
||||||
|
provider_session_id = self.conversations.provider_session_id(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
|
||||||
|
return PersonalizationContext(
|
||||||
|
system_constraints=system_constraints,
|
||||||
|
company_rules=company_rules,
|
||||||
|
personal_rules=personal_rules,
|
||||||
|
current_request=request,
|
||||||
|
preferences=preference_items,
|
||||||
|
interests=interests,
|
||||||
|
personal_memory=personal_memory,
|
||||||
|
conversation_history=history,
|
||||||
|
provider_session_id=provider_session_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
def build_private_scheduled(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
owner_id: int,
|
||||||
|
request: str,
|
||||||
|
system_constraints: str,
|
||||||
|
scope: str = AIMemoryScope.GLOBAL,
|
||||||
|
subject: str | None = None,
|
||||||
|
actor: str = ActorValue.SYSTEM,
|
||||||
|
) -> PersonalizationContext:
|
||||||
|
"""Build private scheduled context without company data or conversation history."""
|
||||||
|
|
||||||
|
return self.build(
|
||||||
|
owner_id=owner_id,
|
||||||
|
request=request,
|
||||||
|
system_constraints=system_constraints,
|
||||||
|
scope=scope,
|
||||||
|
subject=subject,
|
||||||
|
actor=actor,
|
||||||
|
include_company_rules=False,
|
||||||
|
include_personal_context=True,
|
||||||
|
include_history=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def build_group_scheduled(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
request: str,
|
||||||
|
system_constraints: str,
|
||||||
|
scope: str = AIMemoryScope.GLOBAL,
|
||||||
|
subject: str | None = None,
|
||||||
|
) -> PersonalizationContext:
|
||||||
|
"""Build group scheduled context without any creator profile."""
|
||||||
|
|
||||||
|
return self.build(
|
||||||
|
owner_id=None,
|
||||||
|
request=request,
|
||||||
|
system_constraints=system_constraints,
|
||||||
|
scope=scope,
|
||||||
|
subject=subject,
|
||||||
|
include_company_rules=True,
|
||||||
|
include_personal_context=False,
|
||||||
|
include_history=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _watchlist_interests(self, owner_id: int) -> list[dict[str, Any]]:
|
||||||
|
symbols = self.db.execute(
|
||||||
|
select(MarketWatchlist.symbol)
|
||||||
|
.where(
|
||||||
|
MarketWatchlist.owner_id == owner_id,
|
||||||
|
MarketWatchlist.enabled.is_(True),
|
||||||
|
)
|
||||||
|
.order_by(MarketWatchlist.symbol.asc())
|
||||||
|
).scalars()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"category": "watchlist",
|
||||||
|
"value": symbol,
|
||||||
|
"source": "market_watchlist",
|
||||||
|
}
|
||||||
|
for symbol in symbols
|
||||||
|
]
|
||||||
323
app/modules/personalization/services/conversations.py
Normal file
323
app/modules/personalization/services/conversations.py
Normal file
@@ -0,0 +1,323 @@
|
|||||||
|
from datetime import timedelta
|
||||||
|
from hashlib import sha256
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import delete, exists, select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.personalization.constants import (
|
||||||
|
CONVERSATION_MAX_CONTENT_LENGTH,
|
||||||
|
CONVERSATION_MAX_MESSAGES,
|
||||||
|
CONVERSATION_RETENTION_DAYS,
|
||||||
|
UNAVAILABLE_AI_PROVIDERS,
|
||||||
|
ConversationChatType,
|
||||||
|
ConversationRole,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.models import AIConversation, AIConversationMessage
|
||||||
|
|
||||||
|
|
||||||
|
class ConversationService:
|
||||||
|
"""Persist isolated Feishu conversations with bounded history."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
|
||||||
|
def history(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str | ConversationChatType,
|
||||||
|
chat_key: str,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
owner_id, chat_type_value, chat_key_value = _conversation_identity(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
self.cleanup_expired(owner_id=owner_id)
|
||||||
|
conversation = self._find(owner_id, chat_type_value, chat_key_value)
|
||||||
|
if conversation is None:
|
||||||
|
self.db.commit()
|
||||||
|
return []
|
||||||
|
messages = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(AIConversationMessage)
|
||||||
|
.where(AIConversationMessage.conversation_id == conversation.id)
|
||||||
|
.order_by(
|
||||||
|
AIConversationMessage.created_at.desc(),
|
||||||
|
AIConversationMessage.id.desc(),
|
||||||
|
)
|
||||||
|
.limit(CONVERSATION_MAX_MESSAGES)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
messages.reverse()
|
||||||
|
return [_serialize_message(message) for message in messages]
|
||||||
|
|
||||||
|
def record_turn(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str | ConversationChatType,
|
||||||
|
chat_key: str,
|
||||||
|
*,
|
||||||
|
user_content: str,
|
||||||
|
assistant_content: str,
|
||||||
|
provider_name: str,
|
||||||
|
ai_available: bool = True,
|
||||||
|
) -> bool:
|
||||||
|
"""Record one complete turn only after a real AI answer succeeds."""
|
||||||
|
|
||||||
|
if not ai_available or provider_name.strip().lower() in UNAVAILABLE_AI_PROVIDERS:
|
||||||
|
return False
|
||||||
|
owner_id, chat_type_value, chat_key_value = _conversation_identity(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
user_text = _message_content(user_content)
|
||||||
|
assistant_text = _message_content(assistant_content)
|
||||||
|
self.cleanup_expired(owner_id=owner_id)
|
||||||
|
conversation = self._get_or_create(
|
||||||
|
owner_id,
|
||||||
|
chat_type_value,
|
||||||
|
chat_key_value,
|
||||||
|
)
|
||||||
|
now = utc_now()
|
||||||
|
self.db.add_all(
|
||||||
|
[
|
||||||
|
AIConversationMessage(
|
||||||
|
conversation_id=conversation.id,
|
||||||
|
role=ConversationRole.USER,
|
||||||
|
content=user_text,
|
||||||
|
created_at=now,
|
||||||
|
),
|
||||||
|
AIConversationMessage(
|
||||||
|
conversation_id=conversation.id,
|
||||||
|
role=ConversationRole.ASSISTANT,
|
||||||
|
content=assistant_text,
|
||||||
|
created_at=now,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
conversation.updated_at = now
|
||||||
|
self.db.flush()
|
||||||
|
self._trim(conversation.id)
|
||||||
|
self.db.commit()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def reset(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str | ConversationChatType,
|
||||||
|
chat_key: str,
|
||||||
|
) -> bool:
|
||||||
|
owner_id, chat_type_value, chat_key_value = _conversation_identity(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
conversation = self._find(owner_id, chat_type_value, chat_key_value)
|
||||||
|
if conversation is None:
|
||||||
|
return False
|
||||||
|
self.db.execute(
|
||||||
|
delete(AIConversationMessage).where(
|
||||||
|
AIConversationMessage.conversation_id == conversation.id
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.db.delete(conversation)
|
||||||
|
self.db.commit()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def cleanup_expired(self, owner_id: int | None = None) -> int:
|
||||||
|
"""Stage retention cleanup for one owner or all owners.
|
||||||
|
|
||||||
|
The surrounding operation owns the transaction. Interactive history and
|
||||||
|
write paths use this method before completing their own commit.
|
||||||
|
"""
|
||||||
|
|
||||||
|
return self._cleanup_expired(owner_id)["conversation_messages"]
|
||||||
|
|
||||||
|
def cleanup_expired_globally(self) -> dict[str, int]:
|
||||||
|
"""Delete expired messages for every owner and commit the maintenance run."""
|
||||||
|
|
||||||
|
deleted = self._cleanup_expired(owner_id=None)
|
||||||
|
self.db.commit()
|
||||||
|
return deleted
|
||||||
|
|
||||||
|
def _cleanup_expired(self, owner_id: int | None) -> dict[str, int]:
|
||||||
|
cutoff = utc_now() - timedelta(days=CONVERSATION_RETENTION_DAYS)
|
||||||
|
conversation_ids = select(AIConversation.id)
|
||||||
|
if owner_id is not None:
|
||||||
|
conversation_ids = conversation_ids.where(AIConversation.owner_id == owner_id)
|
||||||
|
message_result = self.db.execute(
|
||||||
|
delete(AIConversationMessage).where(
|
||||||
|
AIConversationMessage.conversation_id.in_(conversation_ids),
|
||||||
|
AIConversationMessage.created_at < cutoff,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
empty_conversations = delete(AIConversation).where(
|
||||||
|
~exists(
|
||||||
|
select(AIConversationMessage.id).where(
|
||||||
|
AIConversationMessage.conversation_id == AIConversation.id
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if owner_id is not None:
|
||||||
|
empty_conversations = empty_conversations.where(
|
||||||
|
AIConversation.owner_id == owner_id
|
||||||
|
)
|
||||||
|
conversation_result = self.db.execute(empty_conversations)
|
||||||
|
return {
|
||||||
|
"conversation_messages": max(0, int(message_result.rowcount or 0)),
|
||||||
|
"conversations": max(0, int(conversation_result.rowcount or 0)),
|
||||||
|
}
|
||||||
|
|
||||||
|
def delete_owner_conversations(self, owner_id: int) -> dict[str, int]:
|
||||||
|
conversation_ids = select(AIConversation.id).where(
|
||||||
|
AIConversation.owner_id == owner_id
|
||||||
|
)
|
||||||
|
message_result = self.db.execute(
|
||||||
|
delete(AIConversationMessage).where(
|
||||||
|
AIConversationMessage.conversation_id.in_(conversation_ids)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
conversation_result = self.db.execute(
|
||||||
|
delete(AIConversation).where(AIConversation.owner_id == owner_id)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"conversation_messages": max(0, int(message_result.rowcount or 0)),
|
||||||
|
"conversations": max(0, int(conversation_result.rowcount or 0)),
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def provider_session_id(
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str | ConversationChatType,
|
||||||
|
chat_key: str,
|
||||||
|
) -> str:
|
||||||
|
owner_id, chat_type_value, chat_key_value = _conversation_identity(
|
||||||
|
owner_id,
|
||||||
|
chat_type,
|
||||||
|
chat_key,
|
||||||
|
)
|
||||||
|
digest = sha256(
|
||||||
|
f"{owner_id}\0{chat_type_value}\0{chat_key_value}".encode("utf-8")
|
||||||
|
).hexdigest()
|
||||||
|
return f"feishu-{digest}"
|
||||||
|
|
||||||
|
def _find(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str,
|
||||||
|
chat_key: str,
|
||||||
|
) -> AIConversation | None:
|
||||||
|
return self.db.execute(
|
||||||
|
select(AIConversation).where(
|
||||||
|
AIConversation.owner_id == owner_id,
|
||||||
|
AIConversation.chat_type == chat_type,
|
||||||
|
AIConversation.chat_key == chat_key,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
def _get_or_create(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str,
|
||||||
|
chat_key: str,
|
||||||
|
) -> AIConversation:
|
||||||
|
existing = self._find(owner_id, chat_type, chat_key)
|
||||||
|
if existing is not None:
|
||||||
|
return existing
|
||||||
|
conversation = AIConversation(
|
||||||
|
owner_id=owner_id,
|
||||||
|
chat_type=chat_type,
|
||||||
|
chat_key=chat_key,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(conversation)
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError:
|
||||||
|
conversation = self._find(owner_id, chat_type, chat_key)
|
||||||
|
if conversation is None:
|
||||||
|
raise
|
||||||
|
return conversation
|
||||||
|
|
||||||
|
def _trim(self, conversation_id: int) -> int:
|
||||||
|
stale_ids = list(
|
||||||
|
self.db.execute(
|
||||||
|
select(AIConversationMessage.id)
|
||||||
|
.where(AIConversationMessage.conversation_id == conversation_id)
|
||||||
|
.order_by(
|
||||||
|
AIConversationMessage.created_at.desc(),
|
||||||
|
AIConversationMessage.id.desc(),
|
||||||
|
)
|
||||||
|
.offset(CONVERSATION_MAX_MESSAGES)
|
||||||
|
).scalars()
|
||||||
|
)
|
||||||
|
if not stale_ids:
|
||||||
|
return 0
|
||||||
|
result = self.db.execute(
|
||||||
|
delete(AIConversationMessage).where(
|
||||||
|
AIConversationMessage.id.in_(stale_ids)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return max(0, int(result.rowcount or 0))
|
||||||
|
|
||||||
|
|
||||||
|
def _conversation_identity(
|
||||||
|
owner_id: int,
|
||||||
|
chat_type: str | ConversationChatType,
|
||||||
|
chat_key: str,
|
||||||
|
) -> tuple[int, str, str]:
|
||||||
|
if owner_id <= 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Valid conversation owner is required",
|
||||||
|
)
|
||||||
|
raw_chat_type = str(chat_type).strip().lower()
|
||||||
|
aliases = {
|
||||||
|
"p2p": ConversationChatType.PRIVATE,
|
||||||
|
"private": ConversationChatType.PRIVATE,
|
||||||
|
"group": ConversationChatType.GROUP,
|
||||||
|
"group_chat": ConversationChatType.GROUP,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
chat_type_value = str(aliases[raw_chat_type])
|
||||||
|
except KeyError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Unsupported conversation chat type",
|
||||||
|
) from exc
|
||||||
|
chat_key_value = str(chat_key).strip()
|
||||||
|
if not chat_key_value or len(chat_key_value) > 256:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Valid conversation chat key is required",
|
||||||
|
)
|
||||||
|
return owner_id, chat_type_value, chat_key_value
|
||||||
|
|
||||||
|
|
||||||
|
def _message_content(value: str) -> str:
|
||||||
|
content = str(value).strip()
|
||||||
|
if not content:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Conversation message is required",
|
||||||
|
)
|
||||||
|
if len(content) > CONVERSATION_MAX_CONTENT_LENGTH:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Conversation message is too long",
|
||||||
|
)
|
||||||
|
return content
|
||||||
|
|
||||||
|
|
||||||
|
def _serialize_message(message: AIConversationMessage) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"role": message.role,
|
||||||
|
"content": message.content,
|
||||||
|
"created_at": message.created_at.isoformat(),
|
||||||
|
}
|
||||||
205
app/modules/personalization/services/erasure.py
Normal file
205
app/modules/personalization/services/erasure.py
Normal file
@@ -0,0 +1,205 @@
|
|||||||
|
from collections.abc import Callable, Mapping, Sequence
|
||||||
|
from datetime import timedelta
|
||||||
|
from hashlib import sha256
|
||||||
|
from hmac import compare_digest
|
||||||
|
from secrets import token_hex
|
||||||
|
from typing import Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from sqlalchemy import delete, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.core.utils.time import utc_now
|
||||||
|
from app.modules.ai_memory.models import AIMemoryEntry
|
||||||
|
from app.modules.business.models import MarketWatchlist
|
||||||
|
from app.modules.personalization.constants import ERASURE_CONFIRMATION_TTL_MINUTES
|
||||||
|
from app.modules.personalization.models import (
|
||||||
|
AIConversation,
|
||||||
|
AIConversationMessage,
|
||||||
|
PersonalDataErasureRequest,
|
||||||
|
UserPreference,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.schemas import ErasureConfirmation, ErasureResult
|
||||||
|
|
||||||
|
ErasureHook = Callable[
|
||||||
|
[Session, int, str],
|
||||||
|
int | Mapping[str, int] | None,
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class PersonalDataErasureService:
|
||||||
|
"""Issue one-time confirmations and erase owner data in one transaction."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
|
||||||
|
def request_confirmation(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
*,
|
||||||
|
ttl_minutes: int = ERASURE_CONFIRMATION_TTL_MINUTES,
|
||||||
|
) -> ErasureConfirmation:
|
||||||
|
_validate_owner(owner_id)
|
||||||
|
if ttl_minutes <= 0 or ttl_minutes > 60:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Invalid erasure confirmation lifetime",
|
||||||
|
)
|
||||||
|
confirmation_code = token_hex(4).upper()
|
||||||
|
now = utc_now()
|
||||||
|
expires_at = now + timedelta(minutes=ttl_minutes)
|
||||||
|
record = self.db.execute(
|
||||||
|
select(PersonalDataErasureRequest).where(
|
||||||
|
PersonalDataErasureRequest.owner_id == owner_id
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
record = PersonalDataErasureRequest(
|
||||||
|
owner_id=owner_id,
|
||||||
|
token_hash=_token_hash(owner_id, confirmation_code),
|
||||||
|
expires_at=expires_at,
|
||||||
|
)
|
||||||
|
self.db.add(record)
|
||||||
|
else:
|
||||||
|
record.token_hash = _token_hash(owner_id, confirmation_code)
|
||||||
|
record.expires_at = expires_at
|
||||||
|
record.updated_at = now
|
||||||
|
self.db.commit()
|
||||||
|
return ErasureConfirmation(
|
||||||
|
confirmation_code=confirmation_code,
|
||||||
|
expires_at=expires_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
def confirm_and_erase(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
confirmation_code: str,
|
||||||
|
*,
|
||||||
|
before_hooks: Sequence[ErasureHook] = (),
|
||||||
|
extra_hooks: Sequence[ErasureHook] = (),
|
||||||
|
) -> ErasureResult:
|
||||||
|
"""Erase core personal tables and run integration hooks before one commit."""
|
||||||
|
|
||||||
|
_validate_owner(owner_id)
|
||||||
|
request = self.db.execute(
|
||||||
|
select(PersonalDataErasureRequest)
|
||||||
|
.where(PersonalDataErasureRequest.owner_id == owner_id)
|
||||||
|
.with_for_update()
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if request is None or request.expires_at <= utc_now():
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Erasure confirmation is invalid or expired",
|
||||||
|
)
|
||||||
|
supplied_hash = _token_hash(owner_id, confirmation_code.strip().upper())
|
||||||
|
if not compare_digest(supplied_hash, request.token_hash):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Erasure confirmation is invalid or expired",
|
||||||
|
)
|
||||||
|
|
||||||
|
anonymous_id = f"anonymous-{uuid4().hex}"
|
||||||
|
deleted: dict[str, int] = {}
|
||||||
|
extra: dict[str, Any] = {}
|
||||||
|
try:
|
||||||
|
for index, hook in enumerate(before_hooks):
|
||||||
|
_merge_hook_result(
|
||||||
|
hook(self.db, owner_id, anonymous_id),
|
||||||
|
index=index,
|
||||||
|
prefix="before",
|
||||||
|
deleted=deleted,
|
||||||
|
extra=extra,
|
||||||
|
)
|
||||||
|
conversation_ids = select(AIConversation.id).where(
|
||||||
|
AIConversation.owner_id == owner_id
|
||||||
|
)
|
||||||
|
deleted["conversation_messages"] = _row_count(
|
||||||
|
self.db.execute(
|
||||||
|
delete(AIConversationMessage).where(
|
||||||
|
AIConversationMessage.conversation_id.in_(conversation_ids)
|
||||||
|
)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
deleted["conversations"] = _row_count(
|
||||||
|
self.db.execute(
|
||||||
|
delete(AIConversation).where(AIConversation.owner_id == owner_id)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
deleted["preferences"] = _row_count(
|
||||||
|
self.db.execute(
|
||||||
|
delete(UserPreference).where(UserPreference.owner_id == owner_id)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
deleted["ai_memory"] = _row_count(
|
||||||
|
self.db.execute(
|
||||||
|
delete(AIMemoryEntry).where(AIMemoryEntry.owner_id == owner_id)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
deleted["watchlist"] = _row_count(
|
||||||
|
self.db.execute(
|
||||||
|
delete(MarketWatchlist).where(MarketWatchlist.owner_id == owner_id)
|
||||||
|
).rowcount
|
||||||
|
)
|
||||||
|
self.db.delete(request)
|
||||||
|
|
||||||
|
for index, hook in enumerate(extra_hooks):
|
||||||
|
_merge_hook_result(
|
||||||
|
hook(self.db, owner_id, anonymous_id),
|
||||||
|
index=index,
|
||||||
|
prefix="extra",
|
||||||
|
deleted=deleted,
|
||||||
|
extra=extra,
|
||||||
|
)
|
||||||
|
self.db.commit()
|
||||||
|
except Exception:
|
||||||
|
self.db.rollback()
|
||||||
|
raise
|
||||||
|
return ErasureResult(
|
||||||
|
anonymous_id=anonymous_id,
|
||||||
|
deleted=deleted,
|
||||||
|
extra=extra,
|
||||||
|
)
|
||||||
|
|
||||||
|
def purge_expired_confirmations(self) -> int:
|
||||||
|
result = self.db.execute(
|
||||||
|
delete(PersonalDataErasureRequest).where(
|
||||||
|
PersonalDataErasureRequest.expires_at <= utc_now()
|
||||||
|
)
|
||||||
|
)
|
||||||
|
count = _row_count(result.rowcount)
|
||||||
|
self.db.commit()
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def _token_hash(owner_id: int, confirmation_code: str) -> str:
|
||||||
|
return sha256(f"{owner_id}\0{confirmation_code}".encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_owner(owner_id: int) -> None:
|
||||||
|
if owner_id <= 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Valid erasure owner is required",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _row_count(value: int | None) -> int:
|
||||||
|
return max(0, int(value or 0))
|
||||||
|
|
||||||
|
|
||||||
|
def _merge_hook_result(
|
||||||
|
hook_result: int | Mapping[str, int] | None,
|
||||||
|
*,
|
||||||
|
index: int,
|
||||||
|
prefix: str,
|
||||||
|
deleted: dict[str, int],
|
||||||
|
extra: dict[str, Any],
|
||||||
|
) -> None:
|
||||||
|
if isinstance(hook_result, Mapping):
|
||||||
|
for key, value in hook_result.items():
|
||||||
|
deleted[str(key)] = int(value)
|
||||||
|
elif isinstance(hook_result, int):
|
||||||
|
deleted[f"{prefix}_{index}"] = hook_result
|
||||||
|
elif hook_result is not None:
|
||||||
|
extra[f"{prefix}_{index}"] = hook_result
|
||||||
315
app/modules/personalization/services/preferences.py
Normal file
315
app/modules/personalization/services/preferences.py
Normal file
@@ -0,0 +1,315 @@
|
|||||||
|
import json
|
||||||
|
import re
|
||||||
|
import unicodedata
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
from pydantic import ValidationError
|
||||||
|
from sqlalchemy import delete, select
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.modules.personalization.constants import (
|
||||||
|
PREFERENCE_MAX_VALUE_LENGTH,
|
||||||
|
PREFERENCE_SIGNAL_TERMS,
|
||||||
|
SENSITIVE_PREFERENCE_TERMS,
|
||||||
|
UNAVAILABLE_AI_PROVIDERS,
|
||||||
|
PreferenceCategory,
|
||||||
|
PreferenceSource,
|
||||||
|
)
|
||||||
|
from app.modules.personalization.models import UserPreference
|
||||||
|
from app.modules.personalization.schemas import PreferenceExtractionPayload
|
||||||
|
|
||||||
|
|
||||||
|
class PreferenceService:
|
||||||
|
"""Manage explicit and safely extracted preferences inside one owner boundary."""
|
||||||
|
|
||||||
|
def __init__(self, db: Session):
|
||||||
|
self.db = db
|
||||||
|
|
||||||
|
def list_preferences(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
category: str | PreferenceCategory | None = None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
_validate_owner(owner_id)
|
||||||
|
stmt = (
|
||||||
|
select(UserPreference)
|
||||||
|
.where(UserPreference.owner_id == owner_id)
|
||||||
|
.order_by(UserPreference.category.asc(), UserPreference.id.asc())
|
||||||
|
)
|
||||||
|
if category is not None:
|
||||||
|
stmt = stmt.where(UserPreference.category == _category_value(category))
|
||||||
|
return [_serialize(item) for item in self.db.execute(stmt).scalars()]
|
||||||
|
|
||||||
|
def upsert(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
category: str | PreferenceCategory,
|
||||||
|
value: str,
|
||||||
|
source: str | PreferenceSource = PreferenceSource.EXPLICIT,
|
||||||
|
*,
|
||||||
|
commit: bool = True,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Create one owner-scoped preference or reuse its normalized equivalent."""
|
||||||
|
|
||||||
|
_validate_owner(owner_id)
|
||||||
|
category_value, clean_value, normalized = validate_preference(category, value)
|
||||||
|
source_value = _source_value(source)
|
||||||
|
existing = self.db.execute(
|
||||||
|
select(UserPreference).where(
|
||||||
|
UserPreference.owner_id == owner_id,
|
||||||
|
UserPreference.category == category_value,
|
||||||
|
UserPreference.normalized_value == normalized,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if existing is not None:
|
||||||
|
existing.value = clean_value
|
||||||
|
if source_value == PreferenceSource.EXPLICIT:
|
||||||
|
existing.source = source_value
|
||||||
|
if commit:
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(existing)
|
||||||
|
return _serialize(existing)
|
||||||
|
|
||||||
|
record = UserPreference(
|
||||||
|
owner_id=owner_id,
|
||||||
|
category=category_value,
|
||||||
|
value=clean_value,
|
||||||
|
normalized_value=normalized,
|
||||||
|
source=source_value,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
self.db.add(record)
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError:
|
||||||
|
record = self.db.execute(
|
||||||
|
select(UserPreference).where(
|
||||||
|
UserPreference.owner_id == owner_id,
|
||||||
|
UserPreference.category == category_value,
|
||||||
|
UserPreference.normalized_value == normalized,
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
if commit:
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return _serialize(record)
|
||||||
|
|
||||||
|
def update(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
code: str,
|
||||||
|
*,
|
||||||
|
category: str | PreferenceCategory | None = None,
|
||||||
|
value: str | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
record = self._owned_record(owner_id, code)
|
||||||
|
next_category = category if category is not None else record.category
|
||||||
|
next_value = value if value is not None else record.value
|
||||||
|
category_value, clean_value, normalized = validate_preference(
|
||||||
|
next_category,
|
||||||
|
next_value,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with self.db.begin_nested():
|
||||||
|
record.category = category_value
|
||||||
|
record.value = clean_value
|
||||||
|
record.normalized_value = normalized
|
||||||
|
self.db.flush()
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_409_CONFLICT,
|
||||||
|
detail="Preference already exists",
|
||||||
|
) from exc
|
||||||
|
self.db.commit()
|
||||||
|
self.db.refresh(record)
|
||||||
|
return _serialize(record)
|
||||||
|
|
||||||
|
def delete(self, owner_id: int, code: str) -> None:
|
||||||
|
record = self._owned_record(owner_id, code)
|
||||||
|
self.db.delete(record)
|
||||||
|
self.db.commit()
|
||||||
|
|
||||||
|
def delete_matching(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
*,
|
||||||
|
category: str | PreferenceCategory,
|
||||||
|
value: str,
|
||||||
|
) -> bool:
|
||||||
|
category_value, _, normalized = validate_preference(category, value)
|
||||||
|
record = self.db.execute(
|
||||||
|
select(UserPreference).where(
|
||||||
|
UserPreference.owner_id == owner_id,
|
||||||
|
UserPreference.category == category_value,
|
||||||
|
UserPreference.normalized_value == normalized,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
return False
|
||||||
|
self.db.delete(record)
|
||||||
|
self.db.commit()
|
||||||
|
return True
|
||||||
|
|
||||||
|
def save_auto_extraction(
|
||||||
|
self,
|
||||||
|
owner_id: int,
|
||||||
|
*,
|
||||||
|
provider_name: str,
|
||||||
|
user_text: str,
|
||||||
|
structured_payload: str | dict[str, Any] | list[Any],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
"""Persist only allowlisted, non-sensitive output from a real AI provider."""
|
||||||
|
|
||||||
|
if provider_name.strip().lower() in UNAVAILABLE_AI_PROVIDERS:
|
||||||
|
return []
|
||||||
|
if not contains_preference_signal(user_text):
|
||||||
|
return []
|
||||||
|
candidates = _parse_extraction_payload(structured_payload)
|
||||||
|
saved: list[dict[str, Any]] = []
|
||||||
|
for candidate in candidates:
|
||||||
|
try:
|
||||||
|
saved.append(
|
||||||
|
self.upsert(
|
||||||
|
owner_id=owner_id,
|
||||||
|
category=candidate.category,
|
||||||
|
value=candidate.value,
|
||||||
|
source=PreferenceSource.AUTO,
|
||||||
|
commit=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
except HTTPException:
|
||||||
|
# Automatic extraction is intentionally silent. Invalid or sensitive
|
||||||
|
# candidates are discarded without creating a rejected profile row.
|
||||||
|
continue
|
||||||
|
if saved:
|
||||||
|
self.db.commit()
|
||||||
|
return saved
|
||||||
|
|
||||||
|
def delete_owner_preferences(self, owner_id: int) -> int:
|
||||||
|
"""Stage deletion of every preference for an owner."""
|
||||||
|
|
||||||
|
result = self.db.execute(
|
||||||
|
delete(UserPreference).where(UserPreference.owner_id == owner_id)
|
||||||
|
)
|
||||||
|
return max(0, int(result.rowcount or 0))
|
||||||
|
|
||||||
|
def _owned_record(self, owner_id: int, code: str) -> UserPreference:
|
||||||
|
_validate_owner(owner_id)
|
||||||
|
record = self.db.execute(
|
||||||
|
select(UserPreference).where(
|
||||||
|
UserPreference.owner_id == owner_id,
|
||||||
|
UserPreference.code == code,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if record is None:
|
||||||
|
# Deliberately indistinguishable from a nonexistent code.
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_404_NOT_FOUND,
|
||||||
|
detail="Preference not found",
|
||||||
|
)
|
||||||
|
return record
|
||||||
|
|
||||||
|
|
||||||
|
def contains_preference_signal(value: str) -> bool:
|
||||||
|
text = unicodedata.normalize("NFKC", value).casefold()
|
||||||
|
return any(term in text for term in PREFERENCE_SIGNAL_TERMS)
|
||||||
|
|
||||||
|
|
||||||
|
def validate_preference(
|
||||||
|
category: str | PreferenceCategory,
|
||||||
|
value: str,
|
||||||
|
) -> tuple[str, str, str]:
|
||||||
|
category_value = _category_value(category)
|
||||||
|
clean_value = _clean_value(value)
|
||||||
|
if is_sensitive_preference(clean_value):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Sensitive preference content is not allowed",
|
||||||
|
)
|
||||||
|
return category_value, clean_value, normalize_preference_value(clean_value)
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_preference_value(value: str) -> str:
|
||||||
|
normalized = unicodedata.normalize("NFKC", value)
|
||||||
|
normalized = re.sub(r"\s+", " ", normalized).strip()
|
||||||
|
return normalized.casefold()
|
||||||
|
|
||||||
|
|
||||||
|
def is_sensitive_preference(value: str) -> bool:
|
||||||
|
normalized = unicodedata.normalize("NFKC", value).casefold()
|
||||||
|
return any(term.casefold() in normalized for term in SENSITIVE_PREFERENCE_TERMS)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_extraction_payload(
|
||||||
|
payload: str | dict[str, Any] | list[Any],
|
||||||
|
) -> list[Any]:
|
||||||
|
parsed: Any = payload
|
||||||
|
if isinstance(payload, str):
|
||||||
|
try:
|
||||||
|
parsed = json.loads(payload)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return []
|
||||||
|
if isinstance(parsed, list):
|
||||||
|
parsed = {"preferences": parsed}
|
||||||
|
elif isinstance(parsed, dict) and "preferences" not in parsed:
|
||||||
|
parsed = {"preferences": [parsed]}
|
||||||
|
try:
|
||||||
|
return PreferenceExtractionPayload.model_validate(parsed).preferences
|
||||||
|
except ValidationError:
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def _category_value(category: str | PreferenceCategory) -> str:
|
||||||
|
try:
|
||||||
|
return str(PreferenceCategory(category))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Unsupported preference category",
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _source_value(source: str | PreferenceSource) -> str:
|
||||||
|
try:
|
||||||
|
return str(PreferenceSource(source))
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Unsupported preference source",
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _clean_value(value: str) -> str:
|
||||||
|
clean_value = unicodedata.normalize("NFKC", str(value)).strip()
|
||||||
|
if not clean_value:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Preference value is required",
|
||||||
|
)
|
||||||
|
if len(clean_value) > PREFERENCE_MAX_VALUE_LENGTH:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Preference value is too long",
|
||||||
|
)
|
||||||
|
return clean_value
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_owner(owner_id: int) -> None:
|
||||||
|
if owner_id <= 0:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="Valid preference owner is required",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _serialize(record: UserPreference) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"code": record.code,
|
||||||
|
"category": record.category,
|
||||||
|
"value": record.value,
|
||||||
|
"source": record.source,
|
||||||
|
"created_at": record.created_at.isoformat(),
|
||||||
|
"updated_at": record.updated_at.isoformat(),
|
||||||
|
}
|
||||||
@@ -14,7 +14,7 @@ from app.core.background.task_queue import (
|
|||||||
enqueue_work_weekly_push,
|
enqueue_work_weekly_push,
|
||||||
)
|
)
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.reports.constants import ReportPushKey
|
from app.modules.reports.constants import ReportPushKey
|
||||||
from app.modules.reports.schemas import (
|
from app.modules.reports.schemas import (
|
||||||
LifecycleRunRequest,
|
LifecycleRunRequest,
|
||||||
@@ -134,7 +134,6 @@ def enqueue_lifecycle(
|
|||||||
payload: LifecycleRunRequest,
|
payload: LifecycleRunRequest,
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return enqueue_lifecycle_report(
|
return enqueue_lifecycle_report(
|
||||||
report_type=payload.report_type,
|
report_type=payload.report_type,
|
||||||
receive_id=payload.receive_id,
|
receive_id=payload.receive_id,
|
||||||
@@ -167,8 +166,6 @@ def generate_work_report(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
if payload.persist:
|
|
||||||
require_operations_enabled()
|
|
||||||
return ReportService(db).generate_work_report(
|
return ReportService(db).generate_work_report(
|
||||||
report_type=payload.report_type,
|
report_type=payload.report_type,
|
||||||
reporter=payload.reporter,
|
reporter=payload.reporter,
|
||||||
|
|||||||
@@ -1,6 +1,10 @@
|
|||||||
from datetime import date
|
from datetime import date
|
||||||
|
from hashlib import sha256
|
||||||
|
import json
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import HTTPException, status
|
||||||
|
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.modules.audit.constants import AuditAction, AuditSource, AuditTargetType
|
from app.modules.audit.constants import AuditAction, AuditSource, AuditTargetType
|
||||||
@@ -26,7 +30,7 @@ from app.modules.reports.constants import (
|
|||||||
ReportTitle,
|
ReportTitle,
|
||||||
)
|
)
|
||||||
|
|
||||||
from app.modules.reports.services.common import _json_safe, _money, _next_code, _rate
|
from app.modules.reports.services.common import _json_safe, _money, _rate
|
||||||
|
|
||||||
|
|
||||||
class ReportEnterpriseAnalyticsMixin:
|
class ReportEnterpriseAnalyticsMixin:
|
||||||
@@ -39,8 +43,15 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
actor: str = ActorValue.API,
|
actor: str = ActorValue.API,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Build V3 read-only finance, procurement, performance, and operations analytics."""
|
"""Build V3 read-only finance, procurement, performance, and operations analytics."""
|
||||||
|
if period_start and period_end and period_start > period_end:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail="period_start must be before or equal to period_end",
|
||||||
|
)
|
||||||
|
|
||||||
code = _next_code("ANALYTICS")
|
include_global_metrics = not any(
|
||||||
|
value is not None for value in (project_code, owner, period_start, period_end)
|
||||||
|
)
|
||||||
lifecycle = self.project_lifecycle_report(
|
lifecycle = self.project_lifecycle_report(
|
||||||
project_code=project_code,
|
project_code=project_code,
|
||||||
owner=owner,
|
owner=owner,
|
||||||
@@ -61,9 +72,17 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
MetricKey.BUDGET_TOTAL: projects[MetricKey.BUDGET_TOTAL],
|
MetricKey.BUDGET_TOTAL: projects[MetricKey.BUDGET_TOTAL],
|
||||||
MetricKey.ACTUAL_TOTAL: projects[MetricKey.ACTUAL_TOTAL],
|
MetricKey.ACTUAL_TOTAL: projects[MetricKey.ACTUAL_TOTAL],
|
||||||
MetricKey.BUDGET_USAGE_RATE: projects[MetricKey.BUDGET_USAGE_RATE],
|
MetricKey.BUDGET_USAGE_RATE: projects[MetricKey.BUDGET_USAGE_RATE],
|
||||||
MetricKey.CURRENT_BALANCE_TOTAL: funds[MetricKey.CURRENT_BALANCE_TOTAL],
|
MetricKey.CURRENT_BALANCE_TOTAL: (
|
||||||
MetricKey.NET_POSITION: funds[MetricKey.NET_POSITION],
|
funds[MetricKey.CURRENT_BALANCE_TOTAL]
|
||||||
MetricKey.RISK_ACCOUNTS: funds[MetricKey.RISK_ACCOUNTS],
|
if include_global_metrics
|
||||||
|
else 0
|
||||||
|
),
|
||||||
|
MetricKey.NET_POSITION: (
|
||||||
|
funds[MetricKey.NET_POSITION] if include_global_metrics else 0
|
||||||
|
),
|
||||||
|
MetricKey.RISK_ACCOUNTS: (
|
||||||
|
funds[MetricKey.RISK_ACCOUNTS] if include_global_metrics else 0
|
||||||
|
),
|
||||||
MetricKey.PAYMENT_EXPOSURE: (
|
MetricKey.PAYMENT_EXPOSURE: (
|
||||||
procurements[MetricKey.ACTUAL_TOTAL] + expenses[MetricKey.AMOUNT_TOTAL]
|
procurements[MetricKey.ACTUAL_TOTAL] + expenses[MetricKey.AMOUNT_TOTAL]
|
||||||
),
|
),
|
||||||
@@ -77,7 +96,7 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
MetricKey.ACTUAL_TOTAL: procurements[MetricKey.ACTUAL_TOTAL],
|
MetricKey.ACTUAL_TOTAL: procurements[MetricKey.ACTUAL_TOTAL],
|
||||||
MetricKey.DELIVERY_RISK: procurements[MetricKey.PENDING_DELIVERY],
|
MetricKey.DELIVERY_RISK: procurements[MetricKey.PENDING_DELIVERY],
|
||||||
}
|
}
|
||||||
performance = self._enterprise_performance_stats()
|
performance = self._enterprise_performance_stats(include_global_metrics)
|
||||||
operations = {
|
operations = {
|
||||||
MetricKey.READINESS_SCORE: health[MetricKey.SCORE],
|
MetricKey.READINESS_SCORE: health[MetricKey.SCORE],
|
||||||
MetricKey.LEVEL: health[MetricKey.LEVEL],
|
MetricKey.LEVEL: health[MetricKey.LEVEL],
|
||||||
@@ -97,9 +116,8 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
operations,
|
operations,
|
||||||
recommendations,
|
recommendations,
|
||||||
)
|
)
|
||||||
report = _json_safe(
|
snapshot = _json_safe(
|
||||||
{
|
{
|
||||||
EnterpriseAnalyticsKey.CODE: code,
|
|
||||||
EnterpriseAnalyticsKey.TITLE: ReportTitle.ENTERPRISE_ANALYTICS,
|
EnterpriseAnalyticsKey.TITLE: ReportTitle.ENTERPRISE_ANALYTICS,
|
||||||
EnterpriseAnalyticsKey.FILTERS: lifecycle[LifecycleResponseKey.FILTERS],
|
EnterpriseAnalyticsKey.FILTERS: lifecycle[LifecycleResponseKey.FILTERS],
|
||||||
EnterpriseAnalyticsKey.FINANCE: finance,
|
EnterpriseAnalyticsKey.FINANCE: finance,
|
||||||
@@ -111,6 +129,16 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
EnterpriseAnalyticsKey.CONTENT: "\n".join(lines),
|
EnterpriseAnalyticsKey.CONTENT: "\n".join(lines),
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
canonical_snapshot = json.dumps(
|
||||||
|
snapshot,
|
||||||
|
ensure_ascii=False,
|
||||||
|
sort_keys=True,
|
||||||
|
separators=(",", ":"),
|
||||||
|
default=str,
|
||||||
|
)
|
||||||
|
digest = sha256(canonical_snapshot.encode("utf-8")).hexdigest()[:24].upper()
|
||||||
|
code = f"ANALYTICS-{digest}"
|
||||||
|
report = {EnterpriseAnalyticsKey.CODE: code, **snapshot}
|
||||||
AuditService(self.db).record(
|
AuditService(self.db).record(
|
||||||
AuditLogCreate(
|
AuditLogCreate(
|
||||||
actor=actor,
|
actor=actor,
|
||||||
@@ -136,7 +164,18 @@ class ReportEnterpriseAnalyticsMixin:
|
|||||||
self.db.commit()
|
self.db.commit()
|
||||||
return report
|
return report
|
||||||
|
|
||||||
def _enterprise_performance_stats(self) -> dict[str, Any]:
|
def _enterprise_performance_stats(self, include_global: bool) -> dict[str, Any]:
|
||||||
|
if not include_global:
|
||||||
|
return {
|
||||||
|
MetricKey.TOTAL: 0,
|
||||||
|
MetricKey.CONFIRMED: 0,
|
||||||
|
MetricKey.CONFIRMED_RATE: 0.0,
|
||||||
|
MetricKey.AVERAGE_AUTO_SCORE: 0.0,
|
||||||
|
MetricKey.AVERAGE_CONFIRMED_SCORE: 0.0,
|
||||||
|
MetricKey.WEIGHT_TOTAL: 0,
|
||||||
|
MetricKey.BY_STATUS: {},
|
||||||
|
}
|
||||||
|
|
||||||
total = self._count(PerformanceMetric)
|
total = self._count(PerformanceMetric)
|
||||||
confirmed = self._count(PerformanceMetric, PerformanceMetric.confirmed_score.is_not(None))
|
confirmed = self._count(PerformanceMetric, PerformanceMetric.confirmed_score.is_not(None))
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -45,7 +45,9 @@ class ReportLifecycleReportMixin:
|
|||||||
task_conditions,
|
task_conditions,
|
||||||
risk_conditions,
|
risk_conditions,
|
||||||
)
|
)
|
||||||
include_global_risk = not (project_code or owner)
|
include_global_risk = not any(
|
||||||
|
value is not None for value in (project_code, owner, period_start, period_end)
|
||||||
|
)
|
||||||
health = self._lifecycle_health(
|
health = self._lifecycle_health(
|
||||||
project_stats,
|
project_stats,
|
||||||
task_stats,
|
task_stats,
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ class ReportPushRunMixin:
|
|||||||
receive_id_type: str,
|
receive_id_type: str,
|
||||||
actor: str,
|
actor: str,
|
||||||
status: str = ReportPushStatus.PENDING,
|
status: str = ReportPushStatus.PENDING,
|
||||||
|
task_id: str | None = None,
|
||||||
idempotency_key: str | None = None,
|
idempotency_key: str | None = None,
|
||||||
) -> ReportPushRun:
|
) -> ReportPushRun:
|
||||||
if idempotency_key:
|
if idempotency_key:
|
||||||
@@ -43,6 +44,7 @@ class ReportPushRunMixin:
|
|||||||
receive_id=receive_id,
|
receive_id=receive_id,
|
||||||
receive_id_type=receive_id_type,
|
receive_id_type=receive_id_type,
|
||||||
status=status,
|
status=status,
|
||||||
|
task_id=task_id,
|
||||||
actor=actor,
|
actor=actor,
|
||||||
queued_at=utc_now(),
|
queued_at=utc_now(),
|
||||||
idempotency_key=idempotency_key,
|
idempotency_key=idempotency_key,
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from sqlalchemy.orm import Session
|
|||||||
|
|
||||||
from app.core.background.task_queue import enqueue_risk_event_generation
|
from app.core.background.task_queue import enqueue_risk_event_generation
|
||||||
from app.core.database import get_db
|
from app.core.database import get_db
|
||||||
from app.core.security import ApiPrincipal, require_api_key, require_operations_enabled
|
from app.core.security import ApiPrincipal, require_api_key
|
||||||
from app.modules.risk.constants import RiskEventActionKey, RiskGenerationResultKey
|
from app.modules.risk.constants import RiskEventActionKey, RiskGenerationResultKey
|
||||||
from app.modules.risk.schemas import (
|
from app.modules.risk.schemas import (
|
||||||
RiskAssignRequest,
|
RiskAssignRequest,
|
||||||
@@ -94,7 +94,6 @@ def assign_risk_event(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).assign_event(
|
return RiskService(db).assign_event(
|
||||||
event_id,
|
event_id,
|
||||||
assigned_to=payload.assigned_to,
|
assigned_to=payload.assigned_to,
|
||||||
@@ -110,7 +109,6 @@ def comment_risk_event(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).comment_event(
|
return RiskService(db).comment_event(
|
||||||
event_id,
|
event_id,
|
||||||
comment=payload.comment,
|
comment=payload.comment,
|
||||||
@@ -126,7 +124,6 @@ def resolve_risk_event(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).resolve_event(
|
return RiskService(db).resolve_event(
|
||||||
event_id,
|
event_id,
|
||||||
comment=payload.comment,
|
comment=payload.comment,
|
||||||
@@ -142,7 +139,6 @@ def close_risk_event(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).close_event(
|
return RiskService(db).close_event(
|
||||||
event_id,
|
event_id,
|
||||||
closed_reason=payload.closed_reason,
|
closed_reason=payload.closed_reason,
|
||||||
@@ -158,7 +154,6 @@ def reopen_risk_event(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).reopen_event(
|
return RiskService(db).reopen_event(
|
||||||
event_id,
|
event_id,
|
||||||
comment=payload.comment,
|
comment=payload.comment,
|
||||||
@@ -171,7 +166,6 @@ def generate_risk_events(
|
|||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return RiskService(db).generate_events(actor=principal.actor)
|
return RiskService(db).generate_events(actor=principal.actor)
|
||||||
|
|
||||||
|
|
||||||
@@ -179,5 +173,4 @@ def generate_risk_events(
|
|||||||
def enqueue_risk_events(
|
def enqueue_risk_events(
|
||||||
principal: ApiPrincipal = Depends(require_api_key),
|
principal: ApiPrincipal = Depends(require_api_key),
|
||||||
) -> dict:
|
) -> dict:
|
||||||
require_operations_enabled()
|
|
||||||
return enqueue_risk_event_generation(actor=principal.actor)
|
return enqueue_risk_event_generation(actor=principal.actor)
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ from typing import Any
|
|||||||
|
|
||||||
|
|
||||||
from app.core.constants import ActorValue
|
from app.core.constants import ActorValue
|
||||||
from app.core.security import ensure_business_mutations_enabled
|
|
||||||
from app.core.utils.time import utc_now
|
from app.core.utils.time import utc_now
|
||||||
from app.modules.audit.constants import (
|
from app.modules.audit.constants import (
|
||||||
AuditAction,
|
AuditAction,
|
||||||
@@ -182,7 +181,6 @@ class RiskActionMixin:
|
|||||||
comment: str | None,
|
comment: str | None,
|
||||||
payload: dict[str, Any],
|
payload: dict[str, Any],
|
||||||
) -> RiskEventAction:
|
) -> RiskEventAction:
|
||||||
ensure_business_mutations_enabled()
|
|
||||||
action_record = RiskEventAction(
|
action_record = RiskEventAction(
|
||||||
code=f"RISK-ACTION-{utc_now():%Y%m%d%H%M%S%f}",
|
code=f"RISK-ACTION-{utc_now():%Y%m%d%H%M%S%f}",
|
||||||
risk_event_id=record.id,
|
risk_event_id=record.id,
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user