panda/agent_jrxml
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
46 Commits 2 Branches 0 Tags
4e1433403016316f07eb64131e5c80fb529b4978
Commit Graph

20 Commits

Author SHA1 Message Date
panda 4e14334030 fix: per-node max_tokens + validation 502 guard + correct_jrxml output validity 
- backend/llm.py: per-node max_tokens via get_llm(max_tokens=N), LLM_MAX_TOKENS env var (default 8192)
- agent/nodes.py: 5 generation nodes use max_tokens=32768, generate_skeleton retries at 65536
- agent/nodes.py: fix ns:field regex (<field → <[\w:]*field) to handle namespace prefixes
- agent/nodes.py: fix correct_jrxml never writing back to state["current_jrxml"]
- agent/nodes.py: correct_jrxml rejects non-JRXML output (no <jasperReport tag)
- agent/nodes.py: _strip_continuation_wrapper strips markdown/prefixes from continuation rounds
- agent/nodes.py: _extract_jrxml iterates multiple markdown code blocks, skips fragments
- agent/graph.py: route_after_validate skips correction loop when service_unavailable
- agent/graph.py: route_after_save skips validation for empty JRXML
- backend/validation.py: returns service_unavailable: True for ConnectError and HTTP 5xx
- Docs: CLAUDE.md v14 changelog, README.md LLM_MAX_TOKENS, .env.example LLM_MAX_TOKENS
 2026-05-24 15:20:25 +08:00
panda e362f530ea chore: remove 13 stale files and clean up project structure 
Removed:
- app.py (deprecated Streamlit UI, replaced by api_server.py + frontend/)
- start_agent_jrxml.py (old launcher, replaced by start.py)
- test_reorder.py, e2e_test.py (ad-hoc/outdated test scripts)
- ocr_raw_positions.json (debug output)
- ARCHITECTURE.md, CODE_GUIDE.md, RAG_INTEGRATION.md, ROADMAP.md (superseded by CLAUDE.md)
- EVALUATION_REPORT.md (auto-generated)
- scripts/init_kb.py (replaced by init_default_kb.py)
- validation_service/validate.bat (redundant, start.py covers it)
- sessions/*.json (34 test session files, already gitignored)

Updated:
- CLAUDE.md: removed stale file entries from key mapping table
- README.md: updated init script reference and removed validate.bat
- .gitignore: removed EVALUATION_REPORT.md entry
 2026-05-24 09:07:15 +08:00
panda bd5bfbac2d fix: band-level windowed refine_layout + programmatic map_fields to prevent 91.5% content loss 
Root cause: LLM receiving full 34k-char JRXML would regenerate from scratch
instead of modifying coordinates in-place, shrinking output to ~3k chars.

Solution (programmatic node control, not prompt engineering):

- New agent/jrxml_windower.py: decompose JRXML into header (never sent to
  LLM) + individual bands. Split bands >4000 chars at element boundaries.
  Reassemble with element count validation (>10% change = rollback).

- Rewrite refine_layout: per-band windowed LLM processing (~2-4k chars
  each). LLM cannot "reimagine" the entire report.

- Rewrite map_fields: 100% programmatic regex $F{field_N} -> real name
  replacement. Zero LLM calls, zero content loss.

- _sanitize_field_name: non-ASCII chars escaped to _uXXXX_ format for
  valid JRXML identifiers.

- Tests: 48 new unit tests (windower 28 + map_fields 20). All passing.
  Full suite 385 tests, zero regressions.
 2026-05-24 08:55:38 +08:00
panda bb6cc6e241 feat: add Java JRXML-to-PNG rendering pipeline with pixel-level SSIM comparison 
- lib/java/: Java renderer (JrxmlRenderer) using JasperReports 6.21.0
  - JrxmlDebug for diagnostics, JrxmlGen for format reference
  - download_jars.sh for one-time dependency setup
- agent/nodes.py: _render_jrxml_to_png() and _compute_pixel_similarity()
  - Pixel comparison integrates into validate node (SSIM < 0.4 fails)
  - Pixel fidelity context injected into correct_jrxml for targeted fixes
- tests/test_pixel_comparison.py: 15 unit tests (render, SSIM, integration)
- .gitignore: exclude lib/java/*.jar, lib/java/*.class, tmp/
- CLAUDE.md: v11 changelog documenting the rendering pipeline
- All non-LLM tests pass (97/97)
 2026-05-23 15:09:55 +08:00
panda 1210b926c3 fix: MAX_RETRY 5 + rolling continuation + namespace-aware JRXML extraction 
- MAX_RETRY: 3→5 (graph.py:35, nodes.py:25) with env override
- Rolling continuation: _generate_with_continuation() auto-detects
  truncated JRXML and sends anchor-based continuation, max 3 rounds
- JRXML extraction: regex/end-tag now namespace-prefix aware
  (ns0:jasperReport, ns:jasperReport, etc.)
- All 5 generation nodes refactored to use continuation helper
- Tests updated: scenario1 accepts ns-prefixed root, max_retry
  verifies graph termination
- stop_reason capture + WARNING log on max_tokens truncation
- Correction prompt now injects OCR context + layout schema
 2026-05-23 10:58:46 +08:00
panda 1952d75f13 test: add unit/integration/E2E test suites, fix create_session bug, update docs 
- Unit tests: test_session.py (27), test_error_kb.py (24), test_agent.py hardened
- Integration tests: test_api_integration.py (25) with FastAPI TestClient
- E2E tests: main-flows.spec.ts (8) with Playwright + API mocking
- Bug fix: backend/session.py create_session() missing session_id parameter
- Config: frontend/playwright.config.ts, npm run test:e2e
- Docs: update CLAUDE.md v9, .gitignore for test artifacts/eval reports
 2026-05-23 08:38:29 +08:00
panda b444303055 docs: CLAUDE.md v8 — prompt escape fix + installed plugins/skills reference 2026-05-22 23:01:59 +08:00
panda 1144a86d02 fix: session persistence, multi-turn memory, OCR pipeline, download UX (v7) 
- graph.stream() state fix: agent_state now properly accumulates node updates

- atomic session save (tempfile + os.replace)

- uploaded_file_path injection for OcrExtractor + annotation_detector

- download section always visible; refreshFromApi auto-reloads after generation

- node_start/complete unfiltered for full progress visibility

- modification_request without status=='pass' check
 2026-05-22 11:13:25 +08:00
panda a364e1de81 feat: 5-issue fix — OCR image parse bug + Vue frontend feature parity + streaming UX 
Fix 1 (CRITICAL): file_parser.py suffix normalization ".jpg", api_server.py Path.suffix
Fix 2: Sidebar version history download, ProcessSection replaces old components
Fix 3: OCR content/position layer structured logging in agent/nodes.py
Fix 4: collapsible process sections with per-section stream routing + auto-fold
Fix 5: agent_complete total_duration_ms, SummaryCard duration display

- backend/file_parser.py: normalize suffix to always include leading dot
- api_server.py: step_index in node_start, total_duration_ms in agent_complete
- agent/nodes.py: _log_ocr_layers() for [内容层]/[位置层]/[合并] logging
- frontend: ProcessSection.vue (NEW), chat.ts sections model, Sidebar versions
- CLAUDE.md: updated component list and v6 changelog
 2026-05-21 23:43:21 +08:00
panda 83c7da7517 fix: system env vars silently overriding .env — load_dotenv(override=True) 
Root cause: load_dotenv() default override=False meant system-level
ANTHROPIC_BASE_URL (https://api.deepseek.com/anthropic) took precedence
over .env's OPENAI_BASE_URL (https://api.minimaxi.com/anthropic). All
Anthropic API calls went to DeepSeek with a MiniMax key, causing 401.

Changes:
- backend/llm.py: load_dotenv(override=True) — .env always wins
- .env.example: add explicit ANTHROPIC_API_KEY + ANTHROPIC_BASE_URL
- CLAUDE.md: document env var priority pitfall
 2026-05-21 22:36:43 +08:00
panda 7c1aa7d934 docs: update architecture docs for Vue 3 + FastAPI separation, add one-click start.bat 
- CLAUDE.md: remove duplicate architecture section, fix MAX_RETRY 5→3
- README.md: update architecture diagram to 3-tier, add start.bat instructions
- ROADMAP.md: add 阶段六 layered generation v5 (items 16-20)
- start.bat: one-click startup with auto port-kill and path-with-spaces fix
- package-lock.json: updated from npm install
 2026-05-21 22:10:22 +08:00
panda 74f3f03d2c feat: 前后端分离架构 — FastAPI SSE后端 + Vue 3前端 
将单体 Streamlit 应用拆分为三层架构：
- api_server.py: FastAPI SSE 流式后端 (端口 8000)
- frontend/: Vue 3 + Vite + Pinia 聊天前端 (端口 5173)
- agent/graph.py: 新增 node_start 回调支持
- 更新启动脚本为三服务模式

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-21 20:04:27 +08:00
panda 2befd44430 Merge remote v4/v5 features (multimodal chat input, layered generation, annotation detection) with local v3 features (dialog file upload, XLSX support, session fix) 
Key resolutions:
- agent/nodes.py: Merged session_id exclusion fix with new persistable fields (ocr_extraction_result, annotation_result, layout_schema, ocr_elements)
- app.py: Adopted st-multimodal-chatinput for unified paste/drop/upload, removed custom JS paste bridge
- backend/file_parser.py: Kept local XLSX parser, added remote XLS/DOC parsers
- CLAUDE.md + CODE_GUIDE.md: Merged documentation from both branches

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-21 10:05:43 +08:00
panda 43a0542a11 feat: layered precise generation for A4 report images 
3-phase pipeline to solve LLM prompt overflow from too many OCR elements:
Phase 1 (generate_skeleton): compressed layout schema → skeleton JRXML
Phase 2 (refine_layout): sampled coordinates → pixel-level position tuning
Phase 3 (map_fields): OCR field names → replace $F{field_N} placeholders

Only triggered when layout_schema.total_rows > 0 on initial_generation intent.
Text requests and all other intents are unaffected (zero behavior change).
 2026-05-21 08:34:32 +08:00
panda 9bb011e429 feat: v4 multimodal chat input, multi-format support, and annotation detection 
- Replace st.chat_input with st-multimodal-chatinput (Ctrl+V paste, drag-drop, file button)
- Extract _process_uploaded_file() shared handler (eliminates ~70 duplicated lines)
- Add XLSX (openpyxl), XLS (xlrd), DOC (olefile) parsers to file_parser.py
- Add backend/annotation_detector.py: circle detection (HoughCircles) + arrow detection (HoughLinesP clustering) + OCR correlation + LLM context formatting
- Add annotation_result field to AgentState with session persistence
- Wire annotation detection into process_input and _format_ocr_context
- Add 11 new tests: 7 annotation detector + 4 multi-format parser
- Update all docs: CLAUDE.md, README.md, CODE_GUIDE.md, ROADMAP.md
 2026-05-20 23:43:16 +08:00
panda 87ead4fa6a feat: 对话区域文件上传(粘贴/拖拽) + XLSX支持 + 会话切换无限循环修复 
- 对话区域: st.file_uploader + 全局 paste/drop 事件监听 + sessionStorage 桥接
- 文件预览芯片: 上传后显示在对话区域，可逐文件移除
- OCR 双层解析全面接入: file_parser(文字) + ocr_extractor(字段提取)
- XLSX 解析: openpyxl 逐工作表/逐行读取
- 修复: create_session 强制写入 agent_state.session_id
- 修复: load_session_node 不再从磁盘覆盖 session_id
- 修复: 切换会话 _last_switched_to 哨兵防止无限 rerun

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-20 12:04:02 +08:00
panda da79640259 fix: OCR字段提取集成修复 + 会话切换无限循环修复 + 一键启动脚本 
- process_input 传入17个默认中文字段（修复空列表导致零字段提取）
- OCR提取结果自动注入 LLM 上下文
- save_session_node/load_session_node 持久化 session_id（修复切换会话无限 rerun）
- app.py 会话切换后显式设置 session_id（纵深防御）
- 新增 start.bat / stop.bat 一键启动/停止脚本
- 更新 CLAUDE.md + CODE_GUIDE.md 文档

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-20 10:17:05 +08:00
panda 067880bf2e feat: 添加结构化日志系统，更新LLM配置与全部文档 
新增:
- backend/logger.py — 集中日志模块 (JSON格式 + trace_id + 独立llm.log)
- @log_node / @_log_route 装饰器覆盖17个节点和8个路由

改进:
- backend/llm.py — _LLMLoggingWrapper 自动记录LLM输入输出
- backend/llm.py — API Key优先读ANTHROPIC_API_KEY，模型名改为MiniMax-M2.7
- backend/llm.py — get_llm() 新增caller参数标识调用来源
- backend/validation.py — 新增验证结果/连接失败日志
- backend/session.py — 新增会话创建/删除日志
- app.py — 新增用户交互日志 (输入/执行/异常/会话操作)
- app.py — 提前导入torchvision抑制transformers懒加载报错
- .env.example — 新增LOG_DIR/LOG_LEVEL/ANTHROPIC_API_KEY等配置项
- .gitignore — 新增logs/和db/忽略规则

文档:
- ROADMAP.md — 新增阶段四: 可观测性
- README.md — 补充日志架构/LLM配置/项目结构
- CLAUDE.md — 同步最新配置/日志/MAX_RETRY(3)
- CODE_GUIDE.md — 新增第15章日志系统，更新架构图/LLM/配置
 2026-05-19 23:40:01 +08:00
panda 6467fd4ae5 feat: v3 robustness upgrade — EasyOCR, failure recovery, minimum content check 
- OCR: EasyOCR (primary, ch_sim+en) with PaddleOCR fallback for Windows compatibility
- Validation: _check_minimum_content() rejects empty-shell JRXML (no band/textField)
- Retry: MAX_RETRY 3→5, exhaustion records pending_failure_context for next-turn auto-injection
- Finalize: only saves jrxml_versions on pass, preserves last good final_jrxml on fail
- Extract JRXML: improved empty markdown block handling and XML fragment fallback
- UI: real-time node progress via placeholder updates, initial "analyzing" feedback
- UI: use agent_state (full) instead of node_state (partial) for summary card routing
- UI: unknown template_type now gives LLM meaningful image context instead of metadata
- Docs: updated CLAUDE.md and CODE_GUIDE.md to reflect all v3 changes

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-19 19:15:30 +08:00
panda 70614dff5e feat: comprehensive v2 upgrade — streaming, error KB, file upload, layout analysis 
Major changes:
- Streaming: LLM统一 _BaseLLM 接口 (invoke + stream), generate/modify/correct
  节点使用 get_stream_writer() 实现逐字输出, UI 节点平铺展开自动折叠
- Prompt外部化: 7个prompt拆分到 prompts/*.md, loader.py 支持热重载
- 错误自增长: backend/error_kb.py — 指纹去重 + ChromaDB持久化,
  correct_jrxml→validate 通过时自动入库, retrieve同时搜索错误KB
- 文件上传: backend/file_parser.py — PDF/DOCX/图片/文本解析,
  侧边栏多文件上传, 文本自动注入下一条消息
- A4模板识别: backend/layout_analyzer.py — 三种模式(完整A4/行片段修改/行片段新建),
  PaddleOCR元素提取 + 行分组 + JRXML section匹配
- 会话历史下载: jrxml_versions版本追踪 + 侧边栏历史版本下载按钮
- 预览修复: route_after_save跳过预览/导出意图的验证循环
- Ctrl+C修复: JS注入拦截Streamlit裸c键清缓存

Docs: CLAUDE.md (完整项目文档), ROADMAP.md (改进路线图)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-19 15:02:53 +08:00