老葛
64b94d79f9
* feat(xhs): 集成xhshow库优化签名生成与请求参数 - 引入xhshow库用于小红书API签名生成 - 替换原有的seccore_signv2_playwright签名校验方式 - 支持GET和POST请求的差异化签名处理 - 增加对b1值从localStorage获取的容错处理 - 更新x-t时间戳为毫秒级精度 - 在获取博主笔记接口中增加xsec_token和xsec_source参数- 支持通过配置传递验证token和渠道来源 - 更新依赖文件引入xhshow库- 调整配置示例适配新的token参数要求 * Delete MindSpider/DeepSentimentCrawling/MediaCrawler/config/xhs_config.py 移除配置文件 * Add xhs_config.py for Xiaohongshu platform settings 恢复错误删除的文件 --------- Co-authored-by: gehongbin <gehongbin@autohome.com.cn> Co-authored-by: Doiiars <doiiars@qq.com>
87 lines
1.7 KiB
Plaintext
87 lines
1.7 KiB
Plaintext
# ========================================
|
|
# Weibo舆情分析多智能体系统依赖包
|
|
# 适用于Windows环境 + Conda部署
|
|
# ========================================
|
|
|
|
# ===== 核心Web框架 =====
|
|
flask==2.3.3
|
|
flask-socketio==5.3.6
|
|
streamlit==1.28.1
|
|
python-socketio==5.8.0
|
|
eventlet==0.33.3
|
|
|
|
# ===== HTTP请求和异步 =====
|
|
requests==2.31.0
|
|
httpx==0.28.1
|
|
socksio==1.0.0
|
|
aiofiles==23.2.1
|
|
aiohttp>=3.8.0
|
|
PySocks>=1.7.1
|
|
|
|
# ===== LLM接口 =====
|
|
openai>=1.3.0
|
|
# deepseek-ai>=0.1.0 # 使用OpenAI格式
|
|
|
|
# ===== 搜索API =====
|
|
tavily-python>=0.3.0
|
|
|
|
# ===== 数据处理 =====
|
|
pandas>=2.0.0
|
|
numpy>=1.24.0
|
|
regex>=2023.8.8
|
|
jieba==0.42.1
|
|
|
|
# ===== 数据库 =====
|
|
pymysql==1.1.0
|
|
aiomysql==0.2.0
|
|
aiosqlite==0.21.0
|
|
redis>=4.6.0
|
|
SQLAlchemy==2.0.35
|
|
asyncpg==0.29.0
|
|
cryptography==42.0.7
|
|
|
|
# ===== 爬虫相关 =====
|
|
playwright==1.45.0
|
|
Pillow==9.5.0
|
|
opencv-python>=4.8.0
|
|
beautifulsoup4>=4.12.0
|
|
lxml>=4.9.0
|
|
parsel==1.9.1
|
|
pyexecjs==1.5.1
|
|
xhshow>=0.1.3
|
|
|
|
# ===== 可视化 =====
|
|
plotly>=5.17.0
|
|
matplotlib==3.9.0
|
|
wordcloud==1.9.3
|
|
|
|
# ===== PDF生成 =====
|
|
weasyprint>=60.0 # PDF导出，支持Python 3.9-3.13
|
|
|
|
# ===== 机器学习（可选，用于情感分析，不安装也没事写了容错程序） =====
|
|
torch>=2.0.0 # CPU版本
|
|
transformers>=4.30.0
|
|
scikit-learn>=1.3.0
|
|
xgboost>=2.0.0
|
|
# NOTE：如果要安装GPU版本的torch，指令为pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
|
|
|
|
# ===== 工具库 =====
|
|
python-dotenv>=1.0.0
|
|
python-dateutil>=2.8.2
|
|
pytz>=2023.3
|
|
tqdm>=4.65.0
|
|
tenacity==8.2.2
|
|
loguru>=0.7.0
|
|
pydantic==2.5.2
|
|
pydantic-settings==2.2.1
|
|
json-repair==0.53.0
|
|
|
|
# ===== 开发工具（可选） =====
|
|
pytest>=7.4.0
|
|
black>=23.0.0
|
|
flake8>=6.0.0
|
|
|
|
# ===== Web服务器 =====
|
|
fastapi==0.110.2
|
|
uvicorn==0.29.0
|