脚本
This commit is contained in:
@@ -0,0 +1,513 @@
|
||||
# 竞品系统数据导出脚本说明文档
|
||||
|
||||
## 目录说明
|
||||
|
||||
本目录包含多个竞品系统的数据导出脚本，用于从不同竞品系统中导出客户信息、车辆信息、会员卡信息、历史维修记录等数据。
|
||||
|
||||
---
|
||||
|
||||
## 系统列表
|
||||
|
||||
### 1. i店系统
|
||||
- **文件**: `i.py`, `i店.ipynb`, `i店客户车辆导出.ipynb`
|
||||
- **功能**:
|
||||
- 材料信息导出
|
||||
- 客户信息导出
|
||||
- 客户车辆信息导出
|
||||
- 根据车牌查询fid
|
||||
- 根据fid查询卡信息
|
||||
- 储值卡信息导出
|
||||
- 套餐卡信息导出
|
||||
- **接口**: `http://www.idsz.xin:7070/posapi_invoke`
|
||||
- **主要API**:
|
||||
- `goodscommon_list`: 材料列表
|
||||
- `kpi_customerDetailQuery_new`: 客户详情查询
|
||||
- `member_customer_query_v1`: 根据车牌查询客户
|
||||
- `member_person_query_by_id_v1`: 根据fid查询客户信息
|
||||
- `kpi_memberVerifiAndSurplusQuery`: 会员卡信息查询
|
||||
|
||||
### 2. 京东云修系统
|
||||
- **文件**: `京东云修.py`, `京东云修客户信息排序.ipynb`, `京东云修工单子表单.ipynb`, `京东云修系统导出.ipynb`
|
||||
- **功能**:
|
||||
- 客户信息导出（支持排序）
|
||||
- 车辆信息导出
|
||||
- 工单信息导出
|
||||
- 工单子表单数据导出（服务项目、配件项目）
|
||||
- **接口**: `https://www.yunxiu.com`, `https://jch1.yunxiu.com`
|
||||
- **特点**:
|
||||
- 支持按时间范围查询（可回溯10年）
|
||||
- 支持多种排序方式
|
||||
- 支持子单数据提取
|
||||
|
||||
### 3. 优米系统
|
||||
- **文件**: `优米系统导出.ipynb`
|
||||
- **功能**: 会员卡明细导出
|
||||
- **接口**: `https://www.aibiyme.com`
|
||||
- **特点**: 需要先导出会员表，之后做处理
|
||||
|
||||
### 4. 全智通系统
|
||||
- **文件**: `全智通系统数据导出.ipynb`
|
||||
- **功能**: 数据导出
|
||||
|
||||
### 5. 华胜云系统
|
||||
- **文件**: `华胜云.py`
|
||||
- **功能**: 使用Selenium自动化操作导出数据
|
||||
- **特点**: 需要登录，使用浏览器自动化
|
||||
|
||||
### 6. 大唛云系统
|
||||
- **文件**: `大唛云.py`, `大唛云历史维修记录.ipynb`, `大唛云管理平台.ipynb`, `新大唛云历史维修记录导出.ipynb`
|
||||
- **功能**:
|
||||
- 订单明细导出
|
||||
- 历史维修记录导出
|
||||
- 销售明细导出
|
||||
- 库存查询
|
||||
- **接口**: `http://rp.chezizhu.com`, `http://sp.chezizhu.com`
|
||||
- **特点**:
|
||||
- 使用FineReport报表系统
|
||||
- 需要提取SessionID
|
||||
- 支持订单明细的消费项目拆分
|
||||
|
||||
### 7. 大大汽修系统
|
||||
- **文件**: `大大汽修token登录(1).ipynb`, `大大汽修点击导出(1).ipynb`
|
||||
- **功能**:
|
||||
- Token登录
|
||||
- 导出维修历史单据里程、备注
|
||||
- 导出维修历史明细
|
||||
- 导出客户车辆信息
|
||||
- 导出套餐卡信息
|
||||
- **接口**: `https://api-qixiu.dada365.com`
|
||||
- **特点**: 使用Token认证，支持批量导出
|
||||
|
||||
### 8. 好店长系统
|
||||
- **文件**: `好店长.ipynb`
|
||||
- **功能**:
|
||||
- 消费流水导出
|
||||
- 剩余套餐导出
|
||||
- **接口**: `https://api.365hdz.com`
|
||||
- **特点**: 需要配置storeid和authorization
|
||||
|
||||
### 9. 客户无忧系统
|
||||
- **文件**: `客户无忧.ipynb`
|
||||
- **功能**: 客户信息导出
|
||||
- **接口**: `https://s19.kehu51.com`
|
||||
- **特点**: 使用BeautifulSoup解析HTML表格
|
||||
|
||||
### 10. 客管家系统
|
||||
- **文件**: `客管家数据导出(1).ipynb`
|
||||
- **功能**:
|
||||
- 客户车辆信息导出
|
||||
- 储值卡导出
|
||||
- 套餐卡导出（注意：不能翻页，需要一页导出）
|
||||
- 库存查询
|
||||
- 维修历史导出
|
||||
- **接口**: `https://apicrm.kgjsoft.com`
|
||||
- **特点**: 使用Bearer Token认证
|
||||
|
||||
### 11. 快修哥系统
|
||||
- **文件**: `快修哥脚本.ipynb`
|
||||
- **功能**: 客户信息导出
|
||||
- **接口**: `http://www.kuaixiuge.com`
|
||||
- **特点**: 使用ASP.NET ViewState机制，需要处理分页
|
||||
|
||||
### 12. 快录系统
|
||||
- **文件**: `快录.ipynb`
|
||||
- **功能**: 数据导出
|
||||
|
||||
### 13. 快维系统
|
||||
- **文件**: `快维系统导出.ipynb`, `快维辅助程序.ipynb`
|
||||
- **功能**: 系统数据导出
|
||||
|
||||
### 14. 掌上车店系统
|
||||
- **文件**: `掌上车店.py`
|
||||
- **功能**: 数据导出
|
||||
- **接口**: `https://qfy.lycjh.com`
|
||||
|
||||
### 15. 财神驾到系统
|
||||
- **文件**: `接口抓取数据_财神驾到.ipynb`
|
||||
- **功能**: 通过接口抓取数据
|
||||
|
||||
### 16. 普盛系统
|
||||
- **文件**: `普盛系统导出.ipynb`
|
||||
- **功能**: 会员卡明细导出
|
||||
- **接口**: `https://www.600vip.cn`
|
||||
- **特点**: 需要先导出会员表，对会员卡号进行处理
|
||||
|
||||
### 17. 有礼云系统
|
||||
- **文件**: `有礼云数据导出.ipynb`
|
||||
- **功能**:
|
||||
- 车辆信息导出
|
||||
- 客户信息导出（含详情）
|
||||
- 历史维修记录导出
|
||||
- 历史维修记录数据处理（解析PHP序列化数据）
|
||||
- **接口**: `https://www.4008778515.com`
|
||||
- **特点**:
|
||||
- 支持DataTables分页
|
||||
- 需要处理PHP序列化的OthersList、PartsList、Carmaintainget字段
|
||||
|
||||
### 18. 枫车系统
|
||||
- **文件**: `枫车系统导出.ipynb`
|
||||
- **功能**:
|
||||
- 车辆信息导出
|
||||
- 会员卡信息导出（储值卡、套餐卡）
|
||||
- 历史维修记录导出
|
||||
- **接口**: `https://ssapp-api.carisok.com`, `https://cms.carisok.com`
|
||||
- **特点**:
|
||||
- 使用Token认证
|
||||
- 支持重试机制
|
||||
- 支持断点续传
|
||||
|
||||
### 19. 汽服云系统
|
||||
- **文件**: `汽服云.py`
|
||||
- **功能**: 车辆信息导出
|
||||
- **接口**: `https://qfy.lycjh.com/applet.php`
|
||||
|
||||
### 20. 瀚都系统
|
||||
- **文件**: `瀚都系统会员信息导出.ipynb`
|
||||
- **功能**: 会员信息导出
|
||||
|
||||
### 21. 爱车店系统
|
||||
- **文件**: `爱车店.ipynb`, `爱车店.py`, `爱车店导出数据处理.py`, `爱车店应收账款数据导出.py`, `爱车店新.ipynb`, `爱车店新版.py`
|
||||
- **功能**:
|
||||
- 订单数据导出（使用Selenium）
|
||||
- 应收账款数据导出
|
||||
- 数据拆分处理（服务项目、销售产品、支付记录）
|
||||
- **特点**:
|
||||
- 使用Selenium自动化浏览器操作
|
||||
- 需要登录和选择门店
|
||||
- 支持按日期范围查询
|
||||
- 数据需要拆分处理
|
||||
|
||||
### 22. 特价轮胎经销商系统
|
||||
- **文件**: `特价轮胎经销商系统.ipynb`, `特价轮胎经销商系统1.ipynb`
|
||||
- **功能**: 经销商系统数据导出
|
||||
|
||||
### 23. 百少侠系统
|
||||
- **文件**: `百少侠.ipynb`
|
||||
- **功能**: 历史维修记录导出
|
||||
- **接口**: `https://erp.byjy168.cn`
|
||||
- **特点**: 请求过于频繁会挂掉，需要控制请求频率
|
||||
|
||||
### 24. 神汽链系统
|
||||
- **文件**: `神汽链导出.ipynb`
|
||||
- **功能**: 历史维修记录导出
|
||||
- **接口**: `https://www.sqzone.com`
|
||||
- **特点**: 支持按月范围查询，可回溯多年数据
|
||||
|
||||
### 25. 程浩美车系统
|
||||
- **文件**: `程浩美车数据导出.ipynb`
|
||||
- **功能**: 会员卡明细导出
|
||||
- **接口**: `http://yqhm.vip5968.com`
|
||||
- **特点**: 需要提取CustCardID，然后逐个查询明细
|
||||
|
||||
### 26. 稻田系统
|
||||
- **文件**: `稻田系统导出.ipynb`
|
||||
- **功能**:
|
||||
- 车辆信息导出
|
||||
- 工单信息导出
|
||||
- 工单明细导出
|
||||
- **接口**: `https://api.shuidao.com`
|
||||
- **特点**: 使用Bearer Token认证
|
||||
|
||||
### 27. 米其林驰加系统
|
||||
- **文件**: `米其林驰加系统导出.ipynb`
|
||||
- **功能**:
|
||||
- 车辆信息导出
|
||||
- 历史维修记录列表（结算单）
|
||||
- 历史维修记录明细（结算单明细）
|
||||
- **接口**: `https://teds.tyreplus.com.cn`
|
||||
- **特点**:
|
||||
- 支持多门店查询
|
||||
- 明细数据分为服务明细、产品明细、支付方式
|
||||
|
||||
### 28. 美孚（孚创）系统
|
||||
- **文件**: `美孚（孚创）导出.ipynb`
|
||||
- **功能**:
|
||||
- 客户信息导出（含解密手机号）
|
||||
- 客户明细导出
|
||||
- **接口**: `https://store.fuchuang.com`
|
||||
- **特点**: 手机号需要解密
|
||||
|
||||
### 29. 谷涂（黑谷前身）系统
|
||||
- **文件**: `谷涂（黑谷前身）系统数据导出.ipynb`, `谷涂会员卡消费记录导出.py`
|
||||
- **功能**:
|
||||
- 有效会员卡导出
|
||||
- 会员卡消费记录导出
|
||||
- **接口**: `http://crm.zhongtukj.com`
|
||||
- **特点**: 使用Playwright进行浏览器自动化
|
||||
|
||||
### 30. 车仆系统
|
||||
- **文件**: `车仆.ipynb`
|
||||
- **功能**: 数据导出
|
||||
|
||||
### 31. 车好了系统
|
||||
- **文件**: `车好了.ipynb`
|
||||
- **功能**: 会员信息导出
|
||||
- **接口**: `http://www.chezhanggui.com:92`
|
||||
- **特点**: 支持一次性导出大量数据（pageSize=5000）
|
||||
|
||||
### 32. 车工坊系统
|
||||
- **文件**: `车工坊.ipynb`
|
||||
- **功能**: 历史维修记录导出
|
||||
- **接口**: `https://caremore.saic-gm.com`
|
||||
- **特点**:
|
||||
- 按30天间隔下载数据
|
||||
- 自动下载Excel文件
|
||||
- 支持合并多个Excel文件
|
||||
|
||||
### 33. 车店无忧（车店通）系统
|
||||
- **文件**: `车店无忧(车店通)会员卡明细.py`, `车店通.ipynb`
|
||||
- **功能**: 会员卡明细导出
|
||||
- **接口**: `https://auto.51autoshop.com`, `https://autoexpert.pli-petronas.com.cn`
|
||||
- **特点**:
|
||||
- 需要提取CustCardID
|
||||
- 逐个查询会员卡明细
|
||||
- 使用lxml解析HTML
|
||||
|
||||
### 34. 途虎养车系统
|
||||
- **文件**: `途虎养车脚本导出.ipynb`
|
||||
- **功能**:
|
||||
- 客户信息导出
|
||||
- 客户手机号解密
|
||||
- 历史维修记录导出
|
||||
- 供应商信息导出（含图片下载）
|
||||
- **接口**: `https://shop-gateway.tuhu.cn`
|
||||
- **特点**:
|
||||
- 使用Bearer Token认证
|
||||
- Token存在过期时间，需要经常更换
|
||||
- 支持图片下载和超链接生成
|
||||
- 订单号需要正则处理（TH前缀）
|
||||
|
||||
---
|
||||
|
||||
## 通用技术栈
|
||||
|
||||
### 主要使用的库
|
||||
- **requests**: HTTP请求
|
||||
- **pandas**: 数据处理和Excel导出
|
||||
- **selenium**: 浏览器自动化（部分系统）
|
||||
- **BeautifulSoup/lxml**: HTML解析
|
||||
- **tqdm**: 进度条显示
|
||||
- **openpyxl**: Excel文件操作
|
||||
- **phpserialize**: PHP序列化数据解析（有礼云系统）
|
||||
|
||||
### 常见功能模式
|
||||
|
||||
1. **API接口调用**
|
||||
- 大部分系统通过HTTP API获取数据
|
||||
- 需要配置cookies、headers、token等认证信息
|
||||
- 支持分页查询
|
||||
|
||||
2. **浏览器自动化**
|
||||
- 部分系统使用Selenium或Playwright
|
||||
- 需要处理登录、选择门店等操作
|
||||
- 适合无法直接调用API的系统
|
||||
|
||||
3. **数据导出格式**
|
||||
- 主要导出为Excel文件（.xlsx）
|
||||
- 部分导出为CSV文件
|
||||
- 保存路径通常在`文件输出`目录
|
||||
|
||||
4. **错误处理**
|
||||
- 大部分脚本包含重试机制
|
||||
- 使用try-except处理异常
|
||||
- 记录失败订单/记录
|
||||
|
||||
---
|
||||
|
||||
## 使用注意事项
|
||||
|
||||
### 1. 认证信息
|
||||
- **Token/Cookie**: 大部分脚本中的token和cookie会过期，需要定期更新
|
||||
- **登录状态**: 使用浏览器自动化的脚本需要保持登录状态
|
||||
- **权限**: 确保账号有相应的数据导出权限
|
||||
|
||||
### 2. 请求频率控制
|
||||
- 大部分脚本包含`time.sleep()`延迟，避免请求过于频繁
|
||||
- 部分系统（如百少侠）明确标注"请求过于频繁会挂掉"
|
||||
- 建议根据实际情况调整延迟时间
|
||||
|
||||
### 3. 数据量处理
|
||||
- 大量数据导出时建议分批处理
|
||||
- 部分脚本支持断点续传（如枫车系统）
|
||||
- 注意内存占用，及时保存中间结果
|
||||
|
||||
### 4. 特殊处理
|
||||
- **有礼云系统**: 需要解析PHP序列化数据（OthersList、PartsList、Carmaintainget）
|
||||
- **爱车店系统**: 数据需要拆分处理（服务项目、销售产品、支付记录）
|
||||
- **大唛云系统**: 需要提取SessionID，使用FineReport报表系统
|
||||
- **车工坊系统**: 按时间段下载多个Excel文件，需要合并
|
||||
|
||||
### 5. 文件路径
|
||||
- 输出文件路径通常使用绝对路径
|
||||
- 注意路径中的反斜杠需要使用原始字符串（r''）或双反斜杠
|
||||
|
||||
---
|
||||
|
||||
## 脚本分类
|
||||
|
||||
### 按技术实现分类
|
||||
|
||||
#### API接口调用类
|
||||
- i店系统
|
||||
- 京东云修
|
||||
- 优米系统
|
||||
- 客管家
|
||||
- 有礼云
|
||||
- 枫车系统
|
||||
- 稻田系统
|
||||
- 米其林驰加
|
||||
- 美孚（孚创）
|
||||
- 车好了
|
||||
- 途虎养车
|
||||
- 等大部分系统
|
||||
|
||||
#### 浏览器自动化类
|
||||
- 华胜云（Selenium）
|
||||
- 大大汽修（部分功能）
|
||||
- 爱车店（Selenium）
|
||||
- 谷涂系统（Playwright）
|
||||
- 车工坊（部分功能）
|
||||
|
||||
#### HTML解析类
|
||||
- 客户无忧（BeautifulSoup）
|
||||
- 快修哥（BeautifulSoup + ViewState）
|
||||
- 车店无忧（lxml）
|
||||
|
||||
---
|
||||
|
||||
## 数据导出类型
|
||||
|
||||
### 1. 客户信息
|
||||
- 客户姓名、手机号、车牌号、车型等基本信息
|
||||
- 部分系统支持客户详情导出
|
||||
|
||||
### 2. 车辆信息
|
||||
- 车牌号、品牌、型号、VIN码、里程等
|
||||
|
||||
### 3. 会员卡信息
|
||||
- **储值卡**: 余额、开卡时间、到期时间等
|
||||
- **套餐卡**: 套餐内容、剩余次数、使用记录等
|
||||
- **会员卡明细**: 消费记录、充值记录等
|
||||
|
||||
### 4. 历史维修记录
|
||||
- 工单基本信息
|
||||
- 服务项目明细
|
||||
- 配件/产品明细
|
||||
- 支付信息
|
||||
- 部分系统支持子单数据导出
|
||||
|
||||
### 5. 其他数据
|
||||
- 库存信息
|
||||
- 销售明细
|
||||
- 供应商信息
|
||||
- 材料信息
|
||||
|
||||
---
|
||||
|
||||
## 常见问题
|
||||
|
||||
### 1. Token过期
|
||||
**解决方案**: 重新登录获取新的token，更新脚本中的authorization字段
|
||||
|
||||
### 2. Cookie失效
|
||||
**解决方案**: 重新登录获取新的cookie，更新脚本中的cookies字典
|
||||
|
||||
### 3. 请求失败/超时
|
||||
**解决方案**:
|
||||
- 增加重试机制
|
||||
- 增加请求间隔时间
|
||||
- 检查网络连接
|
||||
|
||||
### 4. 数据不完整
|
||||
**解决方案**:
|
||||
- 检查分页逻辑是否正确
|
||||
- 确认总页数计算
|
||||
- 检查数据过滤条件
|
||||
|
||||
### 5. 编码问题
|
||||
**解决方案**:
|
||||
- 使用UTF-8编码
|
||||
- Excel导出时使用`encoding='utf-8-sig'`（CSV文件）
|
||||
|
||||
---
|
||||
|
||||
## 维护建议
|
||||
|
||||
1. **定期更新认证信息**: Token和Cookie会过期，需要定期更新
|
||||
2. **监控脚本运行**: 大量数据导出时注意监控进度和错误
|
||||
3. **备份重要数据**: 导出前确认输出路径，避免覆盖重要数据
|
||||
4. **版本控制**: 系统更新可能导致脚本失效，需要及时调整
|
||||
5. **文档更新**: 系统变更时及时更新脚本和文档
|
||||
|
||||
---
|
||||
|
||||
## 文件结构
|
||||
|
||||
```
|
||||
竞品系统数据导出/
|
||||
├── i.py # i店系统 - 简单测试脚本
|
||||
├── i店.ipynb # i店系统 - 完整导出脚本
|
||||
├── i店客户车辆导出.ipynb # i店系统 - 客户车辆导出
|
||||
├── 京东云修.py # 京东云修 - Python脚本
|
||||
├── 京东云修客户信息排序.ipynb # 京东云修 - 客户信息排序导出
|
||||
├── 京东云修工单子表单.ipynb # 京东云修 - 工单子表单导出
|
||||
├── 京东云修系统导出.ipynb # 京东云修 - 系统导出
|
||||
├── 优米系统导出.ipynb # 优米系统导出
|
||||
├── 全智通系统数据导出.ipynb # 全智通系统导出
|
||||
├── 华胜云.py # 华胜云系统（Selenium）
|
||||
├── 大唛云.py # 大唛云系统
|
||||
├── 大唛云历史维修记录.ipynb # 大唛云历史维修记录
|
||||
├── 大唛云管理平台.ipynb # 大唛云管理平台
|
||||
├── 大大汽修token登录(1).ipynb # 大大汽修Token登录
|
||||
├── 大大汽修点击导出(1).ipynb # 大大汽修点击导出
|
||||
├── 好店长.ipynb # 好店长系统
|
||||
├── 客户无忧.ipynb # 客户无忧系统
|
||||
├── 客管家数据导出(1).ipynb # 客管家数据导出
|
||||
├── 快修哥脚本.ipynb # 快修哥脚本
|
||||
├── 快录.ipynb # 快录系统
|
||||
├── 快维系统导出.ipynb # 快维系统导出
|
||||
├── 快维辅助程序.ipynb # 快维辅助程序
|
||||
├── 掌上车店.py # 掌上车店系统
|
||||
├── 接口抓取数据_财神驾到.ipynb # 财神驾到接口抓取
|
||||
├── 新大唛云历史维修记录导出.ipynb # 新大唛云历史维修记录
|
||||
├── 普盛系统导出.ipynb # 普盛系统导出
|
||||
├── 有礼云数据导出.ipynb # 有礼云数据导出
|
||||
├── 枫车系统导出.ipynb # 枫车系统导出
|
||||
├── 汽服云.py # 汽服云系统
|
||||
├── 测试.ipynb # 测试脚本
|
||||
├── 瀚都系统会员信息导出.ipynb # 瀚都系统会员信息
|
||||
├── 爱车店.ipynb # 爱车店系统（旧版）
|
||||
├── 爱车店.py # 爱车店系统（Python版）
|
||||
├── 爱车店导出数据处理.py # 爱车店数据拆分处理
|
||||
├── 爱车店应收账款数据导出.py # 爱车店应收账款导出
|
||||
├── 爱车店新.ipynb # 爱车店系统（新版）
|
||||
├── 爱车店新版.py # 爱车店系统（新版Python）
|
||||
├── 特价轮胎经销商系统.ipynb # 特价轮胎经销商系统
|
||||
├── 特价轮胎经销商系统1.ipynb # 特价轮胎经销商系统（版本1）
|
||||
├── 百少侠.ipynb # 百少侠系统
|
||||
├── 神汽链导出.ipynb # 神汽链导出
|
||||
├── 程浩美车数据导出.ipynb # 程浩美车数据导出
|
||||
├── 稻田系统导出.ipynb # 稻田系统导出
|
||||
├── 米其林驰加系统导出.ipynb # 米其林驰加系统导出
|
||||
├── 美孚（孚创）导出.ipynb # 美孚（孚创）导出
|
||||
├── 谷涂（黑谷前身）系统数据导出.ipynb # 谷涂系统导出
|
||||
├── 谷涂会员卡消费记录导出.py # 谷涂会员卡消费记录
|
||||
├── 车仆.ipynb # 车仆系统
|
||||
├── 车好了.ipynb # 车好了系统
|
||||
├── 车工坊.ipynb # 车工坊系统
|
||||
├── 车店无忧(车店通)会员卡明细.py # 车店无忧会员卡明细
|
||||
├── 车店通.ipynb # 车店通系统
|
||||
└── 途虎养车脚本导出.ipynb # 途虎养车脚本导出
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 更新日志
|
||||
|
||||
- 2025-01-27: 创建说明文档，整理所有脚本功能
|
||||
|
||||
---
|
||||
|
||||
## 联系方式
|
||||
|
||||
如有问题或需要更新脚本，请联系脚本维护人员。
|
||||
@@ -0,0 +1,18 @@
|
||||
import requests
|
||||
|
||||
cookies = {
|
||||
'JSESSIONID': '9C45E1317CD259B2847DFE880A214989',
|
||||
}
|
||||
|
||||
headers = {
|
||||
'Accept': 'application/json, text/javascript, */*; q=0.01',
|
||||
'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',
|
||||
'Connection': 'keep-alive',
|
||||
'Referer': 'http://www.idsz.xin:7070/report_member_verifi_list?detailtype=1&key=totalBalance&datafrom=2025-01-01&datato=2025-11-19&sshopId=&type=1',
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',
|
||||
'X-Requested-With': 'XMLHttpRequest',
|
||||
# 'Cookie': 'JSESSIONID=9C45E1317CD259B2847DFE880A214989',
|
||||
}
|
||||
|
||||
response = requests.get('http://www.idsz.xin:7070/posapi_invoke?apiname=kpi_memberVerifiAndSurplusQuery&detailtype=1&startTime=2025-01-01&endTime=2025-11-19&key=totalBalance&sshopId=×CardId=&option=&page=0&pageSize=50', cookies=cookies, headers=headers, verify=False)
|
||||
print(response.text)
|
||||
@@ -0,0 +1,971 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "获取材料",
|
||||
"id": "be89024828eeb339"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-23T03:52:55.446076Z",
|
||||
"start_time": "2025-06-23T03:52:49.281182Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"cookies = {\n",
|
||||
" 'Hm_lvt_a856165bfa0ce84d1fb04dc98d558ead': '1750648887',\n",
|
||||
" 'Hm_lpvt_a856165bfa0ce84d1fb04dc98d558ead': '1750648887',\n",
|
||||
" 'HMACCOUNT': 'ABFCA62083E00432',\n",
|
||||
" 'JSESSIONID': 'C719B94B2515D8D6582B9D4AE91903FF',\n",
|
||||
" 'td_cookie': '3361803543',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/setting_commodity_information_new',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'Hm_lvt_a856165bfa0ce84d1fb04dc98d558ead=1750648887; Hm_lpvt_a856165bfa0ce84d1fb04dc98d558ead=1750648887; HMACCOUNT=ABFCA62083E00432; JSESSIONID=C719B94B2515D8D6582B9D4AE91903FF; td_cookie=3361803543',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"topGroupId_list = [\n",
|
||||
" '974e032e-0052-42b0-9d98-2d83f9ee7d0a',\n",
|
||||
" '81f51c1d-f33b-4ec7-ab79-34af0cb4a006',\n",
|
||||
" 'b17589f3-ad29-429a-9f15-6077d1cbafbb',\n",
|
||||
" '25eb6487-0bf9-4fa2-8fcf-5c7e1fedf700',\n",
|
||||
" 'be723274-157e-4da5-a7cd-dae8874a3bf5',\n",
|
||||
" '29ae4eb1-cd7d-4521-a4cf-6feaae449f2f',\n",
|
||||
" '96a1f351-ae12-46dd-b6b3-57dd96a316a1',\n",
|
||||
" \"72278a29-7c9b-4c54-9eec-809542a2327b\"\n",
|
||||
"] # 手动获取\n",
|
||||
"\n",
|
||||
"for topGroupId in topGroupId_list:\n",
|
||||
" total = 0\n",
|
||||
" params = {\n",
|
||||
" 'apiname': 'goodscommon_list',\n",
|
||||
" 'page': '0',\n",
|
||||
" 'rows': '50',\n",
|
||||
" 'topGroupId': topGroupId, # 不同类目需要手动变更\n",
|
||||
" 'subGroupId': '',\n",
|
||||
" 'queryStr': '',\n",
|
||||
" 'goodsType': '',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" response = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
" total = response.json().get('total')\n",
|
||||
"\n",
|
||||
" rows_list = []\n",
|
||||
" for i in range(total + 1):\n",
|
||||
" new_params = {\n",
|
||||
" 'apiname': 'goodscommon_list',\n",
|
||||
" 'page': f'{i}',\n",
|
||||
" 'rows': '50',\n",
|
||||
" 'topGroupId': topGroupId, # 不同类目需要手动变更\n",
|
||||
" 'subGroupId': '',\n",
|
||||
" 'queryStr': '',\n",
|
||||
" 'goodsType': '',\n",
|
||||
" }\n",
|
||||
" rows_list += requests.get('http://www.idsz.xin:7070/posapi_invoke', params=new_params, cookies=cookies,\n",
|
||||
" headers=headers, verify=False).json().get('rows')\n",
|
||||
" df = pd.DataFrame(rows_list)\n",
|
||||
" df.to_excel(f'{topGroupId}data.xlsx', index=False)"
|
||||
],
|
||||
"outputs": [],
|
||||
"execution_count": 16
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 客户信息",
|
||||
"id": "bcb243b547895096"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T03:39:42.773538Z",
|
||||
"start_time": "2025-12-05T03:38:31.538614Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"USER_AGENT = \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36\"\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '621467AB3BF49A8C806B9AA859C05C81',\n",
|
||||
" 'td_cookie': '438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',\n",
|
||||
" 'Origin': 'http://www.idsz.xin:7070',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report/memberservicelist?detailtype=3&key=4&startTime=2025-01-01&endTime=2025-12-05&sshopId=&type=1&typeFlag=0',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=621467AB3BF49A8C806B9AA859C05C81; td_cookie=438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'kpi_customerDetailQuery_new',\n",
|
||||
" 'detailtype': '4',\n",
|
||||
" 'startTime': '2015-01-01',\n",
|
||||
" 'endTime': '2025-12-31',\n",
|
||||
" 'key': 'cus',\n",
|
||||
" 'sshopId': '',\n",
|
||||
" 'option': '',\n",
|
||||
" 'page': 1,\n",
|
||||
" 'pageSize': '50',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"response = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
"\n",
|
||||
"# print(response.json())\n",
|
||||
"total = response.json().get('total')\n",
|
||||
"total = int(total)\n",
|
||||
"print( total)\n",
|
||||
"total_pages = total // 50 + 1\n",
|
||||
"for i in tqdm(range(total_pages)):\n",
|
||||
" params['page'] = i + 1\n",
|
||||
" response = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
" cus_list = response.json().get('rows')\n",
|
||||
" for cus in cus_list:\n",
|
||||
" all_data.append(cus)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel('customer_data.xlsx', index=False)"
|
||||
],
|
||||
"id": "2c30955bd49819cc",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"7701\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/155 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "0aa7e0c27bc045c0aeb870544edaa328"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T03:21:40.255194Z",
|
||||
"start_time": "2025-12-05T03:21:38.345894Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": "df.to_excel('customer_data.xlsx', index=False)",
|
||||
"id": "1209e7c233e5a37a",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"正在抓取会员卡信息: 18%|█▊ | 1347/7658 [1:30:48<7:05:27, 4.04s/it]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 37
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 根据车牌查询fid",
|
||||
"id": "f483cb1329585127"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-04T11:06:52.170254Z",
|
||||
"start_time": "2025-12-04T09:37:05.539650Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '621467AB3BF49A8C806B9AA859C05C81',\n",
|
||||
" 'td_cookie': '438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',\n",
|
||||
" 'Origin': 'http://www.idsz.xin:7070',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report/memberservicelist?detailtype=3&key=4&startTime=2025-01-01&endTime=2025-12-05&sshopId=&type=1&typeFlag=0',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=621467AB3BF49A8C806B9AA859C05C81; td_cookie=438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# 定义查询单个车牌信息的函数\n",
|
||||
"def query_vehicle_info(license_plate):\n",
|
||||
" params = {\n",
|
||||
" 'apiname': 'member_customer_query_v1',\n",
|
||||
" 'option': license_plate,\n",
|
||||
" 'tagId': '',\n",
|
||||
" 'page': '0',\n",
|
||||
" 'pageSize': '50',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" response = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies,\n",
|
||||
" headers=headers, verify=False)\n",
|
||||
" response.raise_for_status() # 如果请求失败（状态码不是200），抛出异常\n",
|
||||
" data = response.json()\n",
|
||||
" if 'userObject' in data and 'list' in data['userObject']:\n",
|
||||
" return data['userObject']['list']\n",
|
||||
" else:\n",
|
||||
" return []\n",
|
||||
" except requests.RequestException as e:\n",
|
||||
" print(f\"查询车牌 {license_plate} 时出现请求错误: {e}\")\n",
|
||||
" return []\n",
|
||||
" except ValueError as e:\n",
|
||||
" print(f\"查询车牌 {license_plate} 时解析JSON出错: {e}\")\n",
|
||||
" return []\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# 读取 Excel 文件\n",
|
||||
"excel_file = pd.ExcelFile('customer_data.xlsx')\n",
|
||||
"df = excel_file.parse()\n",
|
||||
"\n",
|
||||
"# 假设车牌列名为 '车牌'，如果不是请修改\n",
|
||||
"license_plates = df['carNo'].tolist()\n",
|
||||
"\n",
|
||||
"all_results = []\n",
|
||||
"for plate in tqdm(license_plates):\n",
|
||||
" result = query_vehicle_info(plate)\n",
|
||||
" if result:\n",
|
||||
" all_results.extend(result)\n",
|
||||
" time.sleep(0.5) # 适当延时，避免给服务器造成过大压力\n",
|
||||
"\n",
|
||||
"# 将结果转换为 DataFrame 并保存为 Excel 文件\n",
|
||||
"result_df = pd.DataFrame(all_results)\n",
|
||||
"result_df.to_excel(r'车牌查询结果.xlsx', index=False)"
|
||||
],
|
||||
"id": "2d27a314d2cc6b40",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/7650 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "f5e0feb761e8432494dec95b0ed6146e"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 根据fid查询卡信息",
|
||||
"id": "b31f79d06acb381a"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T01:46:27.497896Z",
|
||||
"start_time": "2025-12-05T01:46:22.721666Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '32C141F5A7A289491D34C50325D2D5A8',\n",
|
||||
" 'td_cookie': '431362035',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded',\n",
|
||||
" 'Origin': 'http://www.idsz.xin:7070',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/member_1_info?cusId=001ae236-99ee-4330-88eb-83532e605eab&type=0',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=32C141F5A7A289491D34C50325D2D5A8; td_cookie=431362035',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'member_person_query_by_id_v1',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"df = pd.read_excel('车牌查询结果.xlsx', sheet_name='Sheet1')\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"for index, item in tqdm(df.iterrows(), total=len(df)):\n",
|
||||
" data = {\n",
|
||||
" 'id': item['fid'],\n",
|
||||
" 'licensePlate': '',\n",
|
||||
" 'idType': '1',\n",
|
||||
" }\n",
|
||||
" retry = 0\n",
|
||||
" while retry < 5:\n",
|
||||
" response = requests.post(\n",
|
||||
" 'http://www.idsz.xin:7070/posapi_invoke',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" data=data,\n",
|
||||
" verify=False,\n",
|
||||
" )\n",
|
||||
" if not json_data.get(\"success\"):\n",
|
||||
" # 即使 success=False，也视为“成功响应”，不再重试\n",
|
||||
" return []\n",
|
||||
" try:\n",
|
||||
" cus = response.json().get('userObject')\n",
|
||||
" cus_card_list = cus.get('tiemsItems')\n",
|
||||
"\n",
|
||||
" time.sleep(1)\n",
|
||||
" for card in cus_card_list:\n",
|
||||
" all_data.append(card)\n",
|
||||
" break\n",
|
||||
" except:\n",
|
||||
" retry += 1\n",
|
||||
" time.sleep(1)\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_excel('会员卡信息1.xlsx', index=False)\n",
|
||||
"# df2 = pd.merge(df, df1, on='fid', how='left')\n",
|
||||
"# df2.to_excel('会员卡信息.xlsx', index=False)\n"
|
||||
],
|
||||
"id": "c56ad6be942a37b1",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/7658 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "3a8ef6406c414ec2a030c1f30471f262"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{'success': False, 'msg': '未找到符合条件的客户！', 'errorCode': -1, 'userObject': None}\n",
|
||||
"{'success': False, 'msg': '未找到符合条件的客户！', 'errorCode': -1, 'userObject': None}\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "KeyboardInterrupt",
|
||||
"evalue": "",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[31m---------------------------------------------------------------------------\u001B[39m",
|
||||
"\u001B[31mAttributeError\u001B[39m Traceback (most recent call last)",
|
||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[32]\u001B[39m\u001B[32m, line 51\u001B[39m\n\u001B[32m 50\u001B[39m cus = response.json().get(\u001B[33m'\u001B[39m\u001B[33muserObject\u001B[39m\u001B[33m'\u001B[39m)\n\u001B[32m---> \u001B[39m\u001B[32m51\u001B[39m cus_card_list = cus.get(\u001B[33m'\u001B[39m\u001B[33mtiemsItems\u001B[39m\u001B[33m'\u001B[39m)\n\u001B[32m 52\u001B[39m \u001B[38;5;28mprint\u001B[39m(cus)\n",
|
||||
"\u001B[31mAttributeError\u001B[39m: 'NoneType' object has no attribute 'get'",
|
||||
"\nDuring handling of the above exception, another exception occurred:\n",
|
||||
"\u001B[31mKeyboardInterrupt\u001B[39m Traceback (most recent call last)",
|
||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[32]\u001B[39m\u001B[32m, line 59\u001B[39m\n\u001B[32m 57\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m:\n\u001B[32m 58\u001B[39m retry += \u001B[32m1\u001B[39m\n\u001B[32m---> \u001B[39m\u001B[32m59\u001B[39m time.sleep(\u001B[32m1\u001B[39m)\n\u001B[32m 60\u001B[39m \u001B[38;5;28;01mcontinue\u001B[39;00m\n\u001B[32m 62\u001B[39m df1 = pd.DataFrame(all_data)\n",
|
||||
"\u001B[31mKeyboardInterrupt\u001B[39m: "
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 32
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T02:02:22.985849Z",
|
||||
"start_time": "2025-12-05T01:54:44.634161Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import asyncio\n",
|
||||
"import httpx\n",
|
||||
"import pandas as pd\n",
|
||||
"import random\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"# ===== 配置 =====\n",
|
||||
"CONCURRENCY = 5\n",
|
||||
"RETRY_LIMIT = 3\n",
|
||||
"TIMEOUT = 10\n",
|
||||
"MIN_DELAY = 0.2\n",
|
||||
"MAX_DELAY = 0.8\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '621467AB3BF49A8C806B9AA859C05C81',\n",
|
||||
" 'td_cookie': '438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',\n",
|
||||
" 'Origin': 'http://www.idsz.xin:7070',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report/memberservicelist?detailtype=3&key=4&startTime=2025-01-01&endTime=2025-12-05&sshopId=&type=1&typeFlag=0',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=621467AB3BF49A8C806B9AA859C05C81; td_cookie=438646777',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"params = {'apiname': 'member_person_query_by_id_v1'}\n",
|
||||
"\n",
|
||||
"async def fetch_card_data(client, fid, semaphore):\n",
|
||||
" async with semaphore:\n",
|
||||
" await asyncio.sleep(random.uniform(MIN_DELAY, MAX_DELAY))\n",
|
||||
" data = {'id': fid, 'licensePlate': '', 'idType': '1'}\n",
|
||||
" for attempt in range(RETRY_LIMIT):\n",
|
||||
" try:\n",
|
||||
" response = await client.post(\n",
|
||||
" 'http://www.idsz.xin:7070/posapi_invoke',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=base_headers,\n",
|
||||
" data=data,\n",
|
||||
" timeout=TIMEOUT\n",
|
||||
" )\n",
|
||||
" response.raise_for_status()\n",
|
||||
" json_data = response.json()\n",
|
||||
" if not json_data.get(\"success\"):\n",
|
||||
" # 即使 success=False，也视为“成功响应”，不再重试\n",
|
||||
" return []\n",
|
||||
" user_obj = json_data.get('userObject', {})\n",
|
||||
" card_list = user_obj.get('tiemsItems', [])\n",
|
||||
" return [{'fid': fid, **card} for card in card_list]\n",
|
||||
" except Exception as e:\n",
|
||||
" if attempt == RETRY_LIMIT - 1:\n",
|
||||
" print(f\"❌ 最终失败: fid={fid}, error={str(e)}\")\n",
|
||||
" return []\n",
|
||||
" await asyncio.sleep(random.uniform(0.5, 1.5))\n",
|
||||
" return []\n",
|
||||
"\n",
|
||||
"# ===== 改进版：精准进度条 =====\n",
|
||||
"async def run_main():\n",
|
||||
" df = pd.read_excel('车牌查询结果.xlsx', sheet_name='Sheet1')\n",
|
||||
" fids = df['fid'].tolist()\n",
|
||||
" semaphore = asyncio.Semaphore(CONCURRENCY)\n",
|
||||
"\n",
|
||||
" async with httpx.AsyncClient(verify=False) as client:\n",
|
||||
" # 创建所有任务\n",
|
||||
" tasks = [fetch_card_data(client, fid, semaphore) for fid in fids]\n",
|
||||
"\n",
|
||||
" # 使用 tqdm 手动管理进度\n",
|
||||
" all_results = []\n",
|
||||
" pbar = tqdm(total=len(tasks), desc=\"正在抓取会员卡信息\")\n",
|
||||
"\n",
|
||||
" # 使用 as_completed 逐个获取完成的任务\n",
|
||||
" for coro in asyncio.as_completed(tasks):\n",
|
||||
" result = await coro\n",
|
||||
" all_results.append(result)\n",
|
||||
" pbar.update(1) # 每完成一个，进度+1\n",
|
||||
"\n",
|
||||
" pbar.close()\n",
|
||||
"\n",
|
||||
" # 展平结果\n",
|
||||
" all_data = [item for sublist in all_results for item in sublist]\n",
|
||||
" df1 = pd.DataFrame(all_data)\n",
|
||||
" df1.to_excel('会员卡信息1.xlsx', index=False)\n",
|
||||
" print(f\"✅ 共获取 {len(all_data)} 条记录，已保存到 '会员卡信息1.xlsx'\")\n",
|
||||
"\n",
|
||||
"# 在 Jupyter 中运行\n",
|
||||
"await run_main()"
|
||||
],
|
||||
"id": "ad8733e5b29be5fc",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"正在抓取会员卡信息: 0%| | 0/7658 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "6125ba241801415a883750b643390f6d"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"✅ 共获取 0 条记录，已保存到 '会员卡信息1.xlsx'\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 35
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 储值卡信息（快速接口）",
|
||||
"id": "6344358f85945812"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T07:01:03.797372Z",
|
||||
"start_time": "2025-12-05T07:00:56.618957Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"import requests\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '052CB97F817616E954B542F4F7442E54',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Proxy-Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report_member_verifi_list?detailtype=1&key=totalBalance&datafrom=2025-01-01&datato=2025-12-05&sshopId=&type=1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=052CB97F817616E954B542F4F7442E54',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'kpi_memberVerifiAndSurplusQuery',\n",
|
||||
" 'detailtype': '1',\n",
|
||||
" 'startTime': '2015-01-01',\n",
|
||||
" 'endTime': '2025-12-21',\n",
|
||||
" 'key': 'totalBalance',\n",
|
||||
" 'sshopId': '',\n",
|
||||
" 'timesCardId': '',\n",
|
||||
" 'option': '',\n",
|
||||
" 'page': '0',\n",
|
||||
" 'pageSize': '50',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"res = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
"# print(res.json())\n",
|
||||
"\n",
|
||||
"total = res.json()['total']\n",
|
||||
"total = int(total)\n",
|
||||
"print(total)\n",
|
||||
"page_num = total // 50 + 1\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(page_num)):\n",
|
||||
" params['page'] = i\n",
|
||||
" res = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
" time.sleep(0.5)\n",
|
||||
" rows = res.json()['rows']\n",
|
||||
" for row in rows:\n",
|
||||
" all_data.append(row)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\储值卡.xlsx', index=False)"
|
||||
],
|
||||
"id": "5bdde2efbe9b0390",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"233\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/5 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "fb907a4cfddd4bb09a7a6305204e1ebd"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 12
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 套餐卡",
|
||||
"id": "93dfefe59f8f6a63"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T05:37:07.706976Z",
|
||||
"start_time": "2025-12-05T05:36:57.105851Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '705CFAA4C504D16F38A0379FD33DF300',\n",
|
||||
" 'td_cookie': '445761476',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report_member_verifi_list?detailtype=1&key=totalBalance&datafrom=2025-01-01&datato=2025-12-05&sshopId=&type=1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=705CFAA4C504D16F38A0379FD33DF300; td_cookie=445761476',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'kpi_memberVerifiAndSurplusQuery',\n",
|
||||
" 'detailtype': '4',\n",
|
||||
" 'startTime': '2015-01-01',\n",
|
||||
" 'endTime': '2025-12-31',\n",
|
||||
" 'key': 'totalCount',\n",
|
||||
" 'sshopId': '',\n",
|
||||
" 'timesCardId': '',\n",
|
||||
" 'option': '',\n",
|
||||
" 'page': '0',\n",
|
||||
" 'pageSize': '50',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"res = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
"# print(res.json())\n",
|
||||
"\n",
|
||||
"total = res.json()['total']\n",
|
||||
"total = int(total)\n",
|
||||
"page_num = total // 50 + 1\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(page_num)):\n",
|
||||
" params['page'] = i\n",
|
||||
" res = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n",
|
||||
" time.sleep(0.5)\n",
|
||||
" rows = res.json()['rows']\n",
|
||||
" for row in rows:\n",
|
||||
" all_data.append(row)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\套餐卡.xlsx', index=False)"
|
||||
],
|
||||
"id": "b2f7fb71b16faedf",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/14 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "af61f1ef2dca42d195d6bb70d4c93909"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 10
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 两个卡信息合并获取客户信息fid",
|
||||
"id": "95c8ba6456b099b"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T01:28:45.798092Z",
|
||||
"start_time": "2025-12-05T01:28:44.780987Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"df1 = pd.read_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\储值卡.xlsx\")\n",
|
||||
"df2 = pd.read_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\套餐卡.xlsx\")\n",
|
||||
"df3 = pd.concat([df1, df2], ignore_index=True)\n",
|
||||
"df3.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\全部卡信息.xlsx\", index=False)"
|
||||
],
|
||||
"id": "8d7315e912a17884",
|
||||
"outputs": [],
|
||||
"execution_count": 19
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 客户信息\n",
|
||||
"id": "564b68724ffc3cc6"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T01:35:53.740495Z",
|
||||
"start_time": "2025-12-05T01:34:15.936800Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '052CB97F817616E954B542F4F7442E54',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Proxy-Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report_member_verifi_list?detailtype=1&key=totalBalance&datafrom=2025-01-01&datato=2025-12-05&sshopId=&type=1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=052CB97F817616E954B542F4F7442E54',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"df = pd.read_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\全部卡信息.xlsx\", sheet_name=\"Sheet1\")\n",
|
||||
"\n",
|
||||
"# 去重\n",
|
||||
"df_dedup = df.drop_duplicates(subset=['fid'], keep='first')\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'member_person_query_by_id_v1',\n",
|
||||
"}\n",
|
||||
"all_data = []\n",
|
||||
"for index, row in tqdm(df_dedup.iterrows(), total=len(df_dedup)):\n",
|
||||
" data = {\n",
|
||||
" 'id': row[\"fid\"],\n",
|
||||
" 'licensePlate': '',\n",
|
||||
" 'idType': '1',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" response = requests.post(\n",
|
||||
" 'http://www.idsz.xin:7070/posapi_invoke',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" data=data,\n",
|
||||
" verify=False,\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # print(response.json())\n",
|
||||
" all_data.append(response.json().get(\"userObject\", {}))\n",
|
||||
"\n",
|
||||
"res_df = pd.DataFrame(all_data)\n",
|
||||
"res_df.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\有卡客户信息.xlsx\", index=False)"
|
||||
],
|
||||
"id": "52074a8276d30ee6",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/339 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "bf2e958e68dc40828eb424acd4e9d1c3"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"ename": "AttributeError",
|
||||
"evalue": "'NoneType' object has no attribute 'keys'",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[31m---------------------------------------------------------------------------\u001B[39m",
|
||||
"\u001B[31mAttributeError\u001B[39m Traceback (most recent call last)",
|
||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[21]\u001B[39m\u001B[32m, line 52\u001B[39m\n\u001B[32m 49\u001B[39m \u001B[38;5;66;03m# print(response.json())\u001B[39;00m\n\u001B[32m 50\u001B[39m all_data.append(response.json().get(\u001B[33m\"\u001B[39m\u001B[33muserObject\u001B[39m\u001B[33m\"\u001B[39m, {}))\n\u001B[32m---> \u001B[39m\u001B[32m52\u001B[39m res_df = pd.DataFrame(all_data)\n\u001B[32m 53\u001B[39m res_df.to_excel(\u001B[33mr\u001B[39m\u001B[33m\"\u001B[39m\u001B[33mD:\u001B[39m\u001B[33m\\\u001B[39m\u001B[33mIdea Project\u001B[39m\u001B[33m\\\u001B[39m\u001B[33mF6+宜搭+其它(1)\u001B[39m\u001B[33m\\\u001B[39m\u001B[33m张阳脚本\u001B[39m\u001B[33m\\\u001B[39m\u001B[33m文件输出\u001B[39m\u001B[33m\\\u001B[39m\u001B[33m有卡客户信息.xlsx\u001B[39m\u001B[33m\"\u001B[39m,index=\u001B[38;5;28;01mFalse\u001B[39;00m)\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\ProgramTools\\anaconda3\\envs\\f6\\Lib\\site-packages\\pandas\\core\\frame.py:851\u001B[39m, in \u001B[36mDataFrame.__init__\u001B[39m\u001B[34m(self, data, index, columns, dtype, copy)\u001B[39m\n\u001B[32m 849\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m columns \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[32m 850\u001B[39m columns = ensure_index(columns)\n\u001B[32m--> \u001B[39m\u001B[32m851\u001B[39m arrays, columns, index = nested_data_to_arrays(\n\u001B[32m 852\u001B[39m \u001B[38;5;66;03m# error: Argument 3 to \"nested_data_to_arrays\" has incompatible\u001B[39;00m\n\u001B[32m 853\u001B[39m \u001B[38;5;66;03m# type \"Optional[Collection[Any]]\"; expected \"Optional[Index]\"\u001B[39;00m\n\u001B[32m 854\u001B[39m data,\n\u001B[32m 855\u001B[39m columns,\n\u001B[32m 856\u001B[39m index, \u001B[38;5;66;03m# type: ignore[arg-type]\u001B[39;00m\n\u001B[32m 857\u001B[39m dtype,\n\u001B[32m 858\u001B[39m )\n\u001B[32m 859\u001B[39m mgr = arrays_to_mgr(\n\u001B[32m 860\u001B[39m arrays,\n\u001B[32m 861\u001B[39m columns,\n\u001B[32m (...)\u001B[39m\u001B[32m 864\u001B[39m typ=manager,\n\u001B[32m 865\u001B[39m )\n\u001B[32m 866\u001B[39m \u001B[38;5;28;01melse\u001B[39;00m:\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\ProgramTools\\anaconda3\\envs\\f6\\Lib\\site-packages\\pandas\\core\\internals\\construction.py:520\u001B[39m, in \u001B[36mnested_data_to_arrays\u001B[39m\u001B[34m(data, columns, index, dtype)\u001B[39m\n\u001B[32m 517\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m is_named_tuple(data[\u001B[32m0\u001B[39m]) \u001B[38;5;129;01mand\u001B[39;00m columns \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[32m 518\u001B[39m columns = ensure_index(data[\u001B[32m0\u001B[39m]._fields)\n\u001B[32m--> \u001B[39m\u001B[32m520\u001B[39m arrays, columns = to_arrays(data, columns, dtype=dtype)\n\u001B[32m 521\u001B[39m columns = ensure_index(columns)\n\u001B[32m 523\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m index \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\ProgramTools\\anaconda3\\envs\\f6\\Lib\\site-packages\\pandas\\core\\internals\\construction.py:837\u001B[39m, in \u001B[36mto_arrays\u001B[39m\u001B[34m(data, columns, dtype)\u001B[39m\n\u001B[32m 835\u001B[39m arr = _list_to_arrays(data)\n\u001B[32m 836\u001B[39m \u001B[38;5;28;01melif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(data[\u001B[32m0\u001B[39m], abc.Mapping):\n\u001B[32m--> \u001B[39m\u001B[32m837\u001B[39m arr, columns = _list_of_dict_to_arrays(data, columns)\n\u001B[32m 838\u001B[39m \u001B[38;5;28;01melif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(data[\u001B[32m0\u001B[39m], ABCSeries):\n\u001B[32m 839\u001B[39m arr, columns = _list_of_series_to_arrays(data, columns)\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\ProgramTools\\anaconda3\\envs\\f6\\Lib\\site-packages\\pandas\\core\\internals\\construction.py:917\u001B[39m, in \u001B[36m_list_of_dict_to_arrays\u001B[39m\u001B[34m(data, columns)\u001B[39m\n\u001B[32m 915\u001B[39m gen = (\u001B[38;5;28mlist\u001B[39m(x.keys()) \u001B[38;5;28;01mfor\u001B[39;00m x \u001B[38;5;129;01min\u001B[39;00m data)\n\u001B[32m 916\u001B[39m sort = \u001B[38;5;129;01mnot\u001B[39;00m \u001B[38;5;28many\u001B[39m(\u001B[38;5;28misinstance\u001B[39m(d, \u001B[38;5;28mdict\u001B[39m) \u001B[38;5;28;01mfor\u001B[39;00m d \u001B[38;5;129;01min\u001B[39;00m data)\n\u001B[32m--> \u001B[39m\u001B[32m917\u001B[39m pre_cols = lib.fast_unique_multiple_list_gen(gen, sort=sort)\n\u001B[32m 918\u001B[39m columns = ensure_index(pre_cols)\n\u001B[32m 920\u001B[39m \u001B[38;5;66;03m# assure that they are of the base dict class and not of derived\u001B[39;00m\n\u001B[32m 921\u001B[39m \u001B[38;5;66;03m# classes\u001B[39;00m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mpandas/_libs/lib.pyx:367\u001B[39m, in \u001B[36mpandas._libs.lib.fast_unique_multiple_list_gen\u001B[39m\u001B[34m()\u001B[39m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\ProgramTools\\anaconda3\\envs\\f6\\Lib\\site-packages\\pandas\\core\\internals\\construction.py:915\u001B[39m, in \u001B[36m<genexpr>\u001B[39m\u001B[34m(.0)\u001B[39m\n\u001B[32m 895\u001B[39m \u001B[38;5;250m\u001B[39m\u001B[33;03m\"\"\"\u001B[39;00m\n\u001B[32m 896\u001B[39m \u001B[33;03mConvert list of dicts to numpy arrays\u001B[39;00m\n\u001B[32m 897\u001B[39m \n\u001B[32m (...)\u001B[39m\u001B[32m 912\u001B[39m \u001B[33;03mcolumns : Index\u001B[39;00m\n\u001B[32m 913\u001B[39m \u001B[33;03m\"\"\"\u001B[39;00m\n\u001B[32m 914\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m columns \u001B[38;5;129;01mis\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m915\u001B[39m gen = (\u001B[38;5;28mlist\u001B[39m(x.keys()) \u001B[38;5;28;01mfor\u001B[39;00m x \u001B[38;5;129;01min\u001B[39;00m data)\n\u001B[32m 916\u001B[39m sort = \u001B[38;5;129;01mnot\u001B[39;00m \u001B[38;5;28many\u001B[39m(\u001B[38;5;28misinstance\u001B[39m(d, \u001B[38;5;28mdict\u001B[39m) \u001B[38;5;28;01mfor\u001B[39;00m d \u001B[38;5;129;01min\u001B[39;00m data)\n\u001B[32m 917\u001B[39m pre_cols = lib.fast_unique_multiple_list_gen(gen, sort=sort)\n",
|
||||
"\u001B[31mAttributeError\u001B[39m: 'NoneType' object has no attribute 'keys'"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 21
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-05T01:23:06.186557Z",
|
||||
"start_time": "2025-12-05T01:23:05.740048Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'JSESSIONID': '32C141F5A7A289491D34C50325D2D5A8',\n",
|
||||
" 'td_cookie': '430509658',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'http://www.idsz.xin:7070/report_member_verifi_list?detailtype=4&key=totalBalance&datafrom=2025-01-01&datato=2025-12-04&sshopId=&type=1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36 Edg/142.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'JSESSIONID=32C141F5A7A289491D34C50325D2D5A8; td_cookie=430509658',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'apiname': 'kpi_memberVerifiAndSurplusQuery',\n",
|
||||
" 'detailtype': '4',\n",
|
||||
" 'startTime': '2025-01-01',\n",
|
||||
" 'endTime': '2025-12-04',\n",
|
||||
" 'key': 'totalBalance',\n",
|
||||
" 'sshopId': '',\n",
|
||||
" 'timesCardId': '',\n",
|
||||
" 'option': '',\n",
|
||||
" 'page': '2',\n",
|
||||
" 'pageSize': '50',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"response = requests.get('http://www.idsz.xin:7070/posapi_invoke', params=params, cookies=cookies, headers=headers,\n",
|
||||
" verify=False)\n"
|
||||
],
|
||||
"id": "63713fc278b99d27",
|
||||
"outputs": [],
|
||||
"execution_count": 9
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,76 @@
|
||||
import requests
|
||||
import json
|
||||
import time
|
||||
import pandas as pd
|
||||
from tqdm import tqdm
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
headers = {
|
||||
"cookie":"__jdv=135409966|direct|-|none|-|1746495871709; flash=3_fmYwX89tb2teRGeQ4SfQL-pKDSmxglqPBgD7qIuZXmReygIf2eagag6hUJes8tPuEqjJP9l1l2qwsJs1XTuyRzJEICXIDkXx6lzysw2bis8pDhQYXP31QXbNPfa5XiJ4K1NeoIjs2jKwQARaXdqu8cg2xJrLj74CClfK1rz_kwOPy__dMtZGSq-S13Lc; pin=jd_gEFaqZlekSYM; unick=jd_28m3wapraiha3z; thor=9ADBD5C0241E6AA7507DB6E6ECED6657690C3A23BC73A7FDFF4C59CFB5DC19D09141D8C81ECE715B83D420382764AC40A51836A77298AA3BA872D825E72E86FB855E24FFB4A3D661C3FF1A5388EB47EECE339286A888E357FCE532D1A8DB26AC4195C7CDF94F989D5EF977BAA810BB8FE80486E89D9A34C4464471AE9CFCDB55975C9092DB060D12120A2A864B48F07D2EF235B6C34DC65D4D9A5CA6CF1AD0B5; light_key=AASBKE7rOxgWQziEhC_QY6ya8T1mkut_nO-Zdj9XQRxIxK-0ug40Vcgbs_HMus6W8doYmG03; JD_UUID=ed02c724-9458-4fef-b74e-ff66090efca9; yunxiupin=jd_gEFaqZlekSYM; UUID=b1715a1a-b8f8-4d6b-b29f-b13da6358062; SESSION_USER_NAME=%E4%BD%95%E4%BA%9A%E5%B3%B0; __jda=154799550.17464958717091671508260.1746495872.1746495872.1746499819.2; __jdc=154799550; __jdb=154799550.8.17464958717091671508260|2.1746499819",
|
||||
"referer": "https://www.yunxiu.com/legend/account",
|
||||
"sec-ch-ua": "\"Not(A:Brand\";v=\"99\", \"Microsoft Edge\";v=\"133\", \"Chromium\";v=\"133\"",
|
||||
"sec-ch-ua-mobile": "?0",
|
||||
"sec-ch-ua-platform": "\"Windows\"",
|
||||
"sec-fetch-dest": "empty",
|
||||
"sec-fetch-mode": "cors",
|
||||
"sec-fetch-site": "same-origin",
|
||||
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0",
|
||||
"x-requested-with": "XMLHttpRequest"
|
||||
}
|
||||
|
||||
all_data = []
|
||||
number = 1746500626772 # 从车辆查询页面中 date参数获取
|
||||
page = 1
|
||||
max_retries = 5 # 最大重试次数
|
||||
backoff_factor = 1 # 指数退避因子
|
||||
|
||||
current_date = datetime.now()
|
||||
years_back = 10
|
||||
|
||||
end_date = current_date - timedelta(days=365 * years_back)
|
||||
start_date = current_date
|
||||
|
||||
while start_date >= end_date:
|
||||
try:
|
||||
search_startTime = (start_date - timedelta(days=365)).strftime('%Y-%m-%d')
|
||||
search_endTime = start_date.strftime('%Y-%m-%d')
|
||||
print(f'search_startTime: {search_startTime}, search_endTime: {search_endTime}')
|
||||
start_date -= timedelta(days=365) # 每次减去一年
|
||||
exit_loop = False
|
||||
for page in tqdm(range(1,1001)):
|
||||
if exit_loop:
|
||||
break
|
||||
url = f"https://www.yunxiu.com/legend/archives/carinfo/locate/list?page={page}&size=12&search_startTime={search_startTime}&search_endTime={search_endTime}&_={number}"
|
||||
attempt = 0
|
||||
while attempt < max_retries:
|
||||
try:
|
||||
response = requests.get(url=url, headers=headers)
|
||||
response.raise_for_status() # 如果响应状态码不是200，则抛出HTTPError异常
|
||||
res = response.json().get("data", {}).get("content", [])
|
||||
max_len = response.json().get("data", {}).get("totalPages", []) + 1
|
||||
if page == max_len:
|
||||
print(max_len)
|
||||
exit_loop = True
|
||||
break # 成功后退出重试循环
|
||||
for item in res:
|
||||
base_info = {k: v for k, v in item.items() if k != "licenseList"}
|
||||
all_data.append(base_info)
|
||||
|
||||
break # 成功后退出重试循环
|
||||
except requests.exceptions.RequestException as e:
|
||||
print(f"第{page}页请求失败: {e}")
|
||||
if attempt < max_retries - 1:
|
||||
sleep_time = backoff_factor * (2 ** attempt) # 计算等待时间
|
||||
print(f"准备第{attempt + 1}次重试，等待{sleep_time}秒...")
|
||||
time.sleep(sleep_time) # 等待一段时间后重试
|
||||
else:
|
||||
print(f"达到最大重试次数，跳过第{page}页")
|
||||
attempt += 1
|
||||
|
||||
number += 1
|
||||
time.sleep(1)
|
||||
except:
|
||||
pass
|
||||
|
||||
df1 = pd.DataFrame(all_data)
|
||||
df1.to_csv(r"D:\Idea Project\F6+宜搭+其它(1)\new\文件输出\京东云修数据导出-13914209863 -车辆.csv")
|
||||
@@ -0,0 +1,116 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-30T01:21:11.014130Z",
|
||||
"start_time": "2025-05-30T01:01:24.119875Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import json\n",
|
||||
"import time\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" \"Cookie\": \"__jdv=195278372%7Cdirect%7C-%7Cnone%7C-%7C1748480028960; mba_muid=1748480028960904285855; QRCodeKey=AAEAIN1BeBof-i4Zw8hvr-aG7CQu5u87gNXaRTDWEIyObJwJ; wlfstk_smdl=0zlr8a4uh5xi9n7wneoe7f1dtg3gmf4m; flash=3_Ph0N57CFsjJX7qJrf3jFhR2KtGaskdDyz0uaOe7M5cv8HXomGD7XAONhQEW4AdogODQVOMcvo7cAhnCtVRgeA_KEbDojTjBLROXbXXsH4dVDIUml_To2aRXuy7ifznU4E7gFgs6ztENk6VvYDzHM4XM4tzN8JA6ZinxiQEuhb7shI3tgFrbSG3LyTAyh3c1PHbU7; pin=%E5%85%BB%E8%BD%A6%E9%A9%BF%E6%A0%88%E4%B8%87%E6%B1%9F%E5%BA%97; unick=3x31uabda90cvn; thor=5EE400E7603F4FFFA56C15D1CDCE1ED2D696342F9B7D93B6828A585300D929E228AC00481EB0071D930EDA11CE005878BA99AE6501BFB4FCD664ED18C983B851317700EABE9412AAD1B2EFDBEE79D68BA7935E2CFA41531DDA2495DF925EEB38DE0E6D565E0AEA561263D27E127BA1C5A5A265A54FF2134462E28C15625847DAEA9838BA7CC226743C3AF780F4B4B1BD; light_key=AASBKE7rOxgWQziEhC_QY6yatak8Gj94PuPDJNKicqqnLhtyKqgNkVJhNXp4h31u--XoD8uMz5ZWORXUv1A4IKSjEqNTig; JD_UUID=d86ad331-9452-4965-81c4-871c9f622c8b; yunxiupin=%E5%85%BB%E8%BD%A6%E9%A9%BF%E6%A0%88%E4%B8%87%E6%B1%9F%E5%BA%97; bmall_id=8C9AFA76560E12B6116B47CE33B5ADF63E6941F825D9D52F317A543390F95D20700ADD2EE1D0F1C4F46929C1CA9A55D64D024C865A5C48C5E46BAEDB85601A47B4EFD44CB97863068AB4C1C1C9C0E633705B049FDACD382DDB3EA269F0E2D8DA63F3B5C6443C3B2DB12D9130CF7E5685; UUID=3b5708c5-aae1-4b91-83bd-606bb5a4c08d; SESSION_USER_NAME=%E7%8E%8B%E6%AD%A6%E6%96%8C; mba_sid=17485072577312005509178.0; 3AB9D23F7A4B3C9B=U3G6MJASLF6FOBCVPW5GOBPXFAUBK6VWDNALTH6W6EYUEBTPGQXLGP6A4P5CTRP5C6TQMFZLHYTECKB72KSKAPDORM; __jda=241039512.1748480028960904285855.1748480028.1748489963.1748495804.3; __jdc=241039512; __jdb=241039512.135.1748480028960904285855|3.1748495804\",\n",
|
||||
" \"Referer\": \"https://jch1.yunxiu.com/legend/account?refer=float-menu\",\n",
|
||||
" \"Sec-Ch-Ua\": '\"Not=A?Brand\";v=\"99\", \"Chromium\";v=\"118\"',\n",
|
||||
" \"Sec-Ch-Ua-Mobile\": \"?0\",\n",
|
||||
" \"Sec-Ch-Ua-Platform\": '\"Windows\"',\n",
|
||||
" \"Sec-Fetch-Dest\": \"empty\",\n",
|
||||
" \"Sec-Fetch-Mode\": \"cors\",\n",
|
||||
" \"Sec-Fetch-Site\": \"same-origin\",\n",
|
||||
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.1.2 Safari/537.36 JiSu/118.0.1.2\",\n",
|
||||
" \"X-Requested-With\": \"XMLHttpRequest\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"number = 1748507284349\n",
|
||||
"page = 1\n",
|
||||
"max_retries = 5 # 最大重试次数\n",
|
||||
"backoff_factor = 1 # 指数退避因子\n",
|
||||
"# sort = \"sortByCardNumberType,ASC\"\n",
|
||||
"# sort = \"sortByCardNumberType,DESC\"\n",
|
||||
"# sort = \"sortByTotalBalanceType,ASC\"\n",
|
||||
"sort = \"sortByTotalBalanceType,DESC\"\n",
|
||||
"for page in tqdm(range(1,1001)):\n",
|
||||
" url = f\"https://jch1.yunxiu.com/legend/account/search?page={page}&sort={sort}&_={number}\" # 注意url\n",
|
||||
" data = f'page={page}&_={number}'\n",
|
||||
"\n",
|
||||
" attempt = 0\n",
|
||||
" while attempt < max_retries:\n",
|
||||
" try:\n",
|
||||
" response = requests.get(url=url, headers=headers, data=data)\n",
|
||||
" response.raise_for_status()\n",
|
||||
" # print(response.json())\n",
|
||||
" # break# 如果响应状态码不是200，则抛出HTTPError异常\n",
|
||||
" res = response.json().get(\"data\", {}).get(\"content\", [])\n",
|
||||
"\n",
|
||||
" for item in res:\n",
|
||||
" base_info = {k: v for k, v in item.items() if k != \"licenseList\"}\n",
|
||||
" if \"licenseList\" in item and isinstance(item[\"licenseList\"], list):\n",
|
||||
" for new_item in item[\"licenseList\"]:\n",
|
||||
" row = base_info.copy()\n",
|
||||
" row.update({\"car\": new_item})\n",
|
||||
" all_data.append(row)\n",
|
||||
" else:\n",
|
||||
" # 如果没有licenseList或者它不是列表，则直接添加基本信息\n",
|
||||
" all_data.append(base_info)\n",
|
||||
"\n",
|
||||
" break # 成功后退出重试循环\n",
|
||||
" except requests.exceptions.RequestException as e:\n",
|
||||
" print(f\"第{page}页请求失败: {e}\")\n",
|
||||
" if attempt < max_retries - 1:\n",
|
||||
" sleep_time = backoff_factor * (2 ** attempt) # 计算等待时间\n",
|
||||
" print(f\"准备第{attempt + 1}次重试，等待{sleep_time}秒...\")\n",
|
||||
" time.sleep(sleep_time) # 等待一段时间后重试\n",
|
||||
" else:\n",
|
||||
" print(f\"达到最大重试次数，跳过第{page}页\")\n",
|
||||
" attempt += 1\n",
|
||||
"\n",
|
||||
" number += 1\n",
|
||||
" time.sleep(1)\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_excel(fr\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-养车驿栈万江店{sort}.xlsx\")\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 1000/1000 [19:45<00:00, 1.19s/it]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 4
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,848 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 工单子表单",
|
||||
"id": "88666286230ca491"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-23T16:17:14.754376Z",
|
||||
"start_time": "2025-05-23T11:40:55.999863Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import json\n",
|
||||
"import time\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from datetime import datetime, timedelta\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" \"cookie\": '__jdv=154799550|direct|-|none|-|1747659054237; flash=3_BN5FlBXqwDExD0Hqb3B70F9P_8Auw1KoNHmmWpbTyc_Zf4ughqhQAf3Dn_SYPn5R1Y6WZAd537CdRMmeauNNAqGU8ZwRu5c78bo-JimpRh71cvcmU2N9aKNNe7-ZXmPzhlMsJQhul_B5V3Y1L1TK42h8C0JKQW3xo4Cmtf20_WfGfvlVIXvuG8FnwxnR; pin=jd_BtxRnLkqczoE; unick=%E7%88%B1%E5%B8%AE%E8%BD%A6%E6%9C%8D; thor=F1C2CE45C3BB0D6AFAD21E4048588D13AA86F90A17A75FB8C125D8BBEBCEA71C7F0A7FCB3C3B6F5CDA534E1EA85BF2093B2A93EEE733A7536F7B9CBB50AE8FDCCBED1C41AF1F7F1EC0E525A306CDF7B50EDFC0972CC51ECA2ADCC74EE79CF0FD5BA3EB3798699E847C3D51F381E0F386C7394AE98B1F408C1454AB21FBF87C9E55E01BBD0EB64221F817191FA238708A35E710FCD7238DC9D7A574F3CF85919E; light_key=AASBKE7rOxgWQziEhC_QY6yabH8n9ROlxsU0CDGt7G7DsmfOcYCIBCjtBcp8f2n2enKriWRO; JD_UUID=7254175f-6a89-4144-ba95-823dd36d0fff; yunxiupin=jd_BtxRnLkqczoE; UUID=1f7f9980-72d3-49b9-85bc-ae8b99f559e0; SESSION_USER_NAME=%E6%AC%A7%E9%98%B3%E8%BF%9B%E9%A3%9E; __jda=154799550.17476590542361552970576.1747659054.1747876397.1748000112.4; __jdc=154799550; __jdb=154799550.4.17476590542361552970576|4.1748000112',\n",
|
||||
" \"referer\": \"https://www.yunxiu.com/legend/account\",\n",
|
||||
" \"sec-ch-ua\": \"\\\"Not(A:Brand\\\";v=\\\"99\\\", \\\"Microsoft Edge\\\";v=\\\"133\\\", \\\"Chromium\\\";v=\\\"133\\\"\",\n",
|
||||
" \"sec-ch-ua-mobile\": \"?0\",\n",
|
||||
" \"sec-ch-ua-platform\": \"\\\"Windows\\\"\",\n",
|
||||
" \"sec-fetch-dest\": \"empty\",\n",
|
||||
" \"sec-fetch-mode\": \"cors\",\n",
|
||||
" \"sec-fetch-site\": \"same-origin\",\n",
|
||||
" \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0\",\n",
|
||||
" \"x-requested-with\": \"XMLHttpRequest\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"all_service_data = []\n",
|
||||
"all_parts_data = []\n",
|
||||
"all_base_ddata = []\n",
|
||||
"\n",
|
||||
"number = 1743152268544\n",
|
||||
"page = 1\n",
|
||||
"max_retries = 5 # 最大重试次数\n",
|
||||
"backoff_factor = 10 # 指数退避因子\n",
|
||||
"\n",
|
||||
"current_date = datetime.now()\n",
|
||||
"years_back = 20\n",
|
||||
"\n",
|
||||
"end_date = current_date - timedelta(days=365 * years_back)\n",
|
||||
"start_date = current_date\n",
|
||||
"\n",
|
||||
"while start_date >= end_date:\n",
|
||||
" try:\n",
|
||||
" search_startTime = (start_date - timedelta(days=365)).strftime('%Y-%m-%d')\n",
|
||||
" search_endTime = start_date.strftime('%Y-%m-%d')\n",
|
||||
" print(f'查询时间范围: {search_startTime} 至 {search_endTime}')\n",
|
||||
"\n",
|
||||
" start_date -= timedelta(days=365)\n",
|
||||
" should_break = False\n",
|
||||
"\n",
|
||||
" for page in tqdm(range(1, 1001)):\n",
|
||||
" if should_break: # 检查是否需要跳出\n",
|
||||
" break\n",
|
||||
" \n",
|
||||
" url = f\"https://www.yunxiu.com/legend/shop/order/order-list/list?page={page}&size=12&search_startTime={search_startTime}&search_endTime={search_endTime}&_={number}\"\n",
|
||||
"\n",
|
||||
" for attempt in range(max_retries):\n",
|
||||
" try:\n",
|
||||
" response = requests.get(url=url, headers=headers)\n",
|
||||
" response.raise_for_status()\n",
|
||||
"\n",
|
||||
" data = response.json().get(\"data\", {})\n",
|
||||
" res = data.get(\"content\", [])\n",
|
||||
" total_pages = data.get(\"totalPages\", 1)\n",
|
||||
"\n",
|
||||
" if page >= total_pages:\n",
|
||||
" should_break = True\n",
|
||||
" break\n",
|
||||
"\n",
|
||||
" for item in res:\n",
|
||||
" if not isinstance(item, dict):\n",
|
||||
" print(f\"跳过非字典项: {item}\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" base_info = {k: str(v) for k, v in item.items() if k != \"licenseList\"}\n",
|
||||
" all_data.append(base_info)\n",
|
||||
" order_id = item.get(\"id\")\n",
|
||||
" # order_id = 66752497\n",
|
||||
"\n",
|
||||
" if not order_id:\n",
|
||||
" print(\"订单ID缺失，跳过处理\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # 获取订单详情\n",
|
||||
" detail_url = f\"https://www.yunxiu.com/legend/shop/order/detail?orderId={order_id}&refer=order-list\"\n",
|
||||
" detail_resp = requests.get(detail_url, headers=headers)\n",
|
||||
" detail_soup = BeautifulSoup(detail_resp.text, 'html.parser')\n",
|
||||
"\n",
|
||||
" # 检查子单\n",
|
||||
" sub_order_link = detail_soup.find('a', class_='new-order')\n",
|
||||
" if sub_order_link and \"查看子单\" in sub_order_link.text:\n",
|
||||
" # print(\"存在子单\")\n",
|
||||
" base_info[\"子单\"] = \"存在\"\n",
|
||||
"\n",
|
||||
" # 获取子单内容\n",
|
||||
" sub_order_url = \"https://www.yunxiu.com\" + sub_order_link['href']\n",
|
||||
" sub_resp = requests.get(sub_order_url, headers=headers)\n",
|
||||
" sub_soup = BeautifulSoup(sub_resp.text, 'html.parser')\n",
|
||||
"\n",
|
||||
" # 提取服务项目\n",
|
||||
" tables = sub_soup.find_all('table', class_='yqx-table')\n",
|
||||
" if tables:\n",
|
||||
" service_table = tables[0]\n",
|
||||
" # 修正选择器：使用实际的tr class\n",
|
||||
" service_rows = service_table.select('tr.service-datatr')\n",
|
||||
"\n",
|
||||
" service_data = []\n",
|
||||
" # 调整header与实际列数匹配\n",
|
||||
" service_headers = [\"服务名称\", \"服务类别\", \"工时费\", \"工时\", \"金额\", \"优惠\",\n",
|
||||
" \"服务备注\"]\n",
|
||||
"\n",
|
||||
" for row in service_rows:\n",
|
||||
" # 从input标签提取数据\n",
|
||||
" service_item = {\n",
|
||||
" \"服务名称\": row.find('input', {'name': 'serviceName'}).get('value', ''),\n",
|
||||
" \"服务类别\": row.find('input', {'name': 'serviceCatName'}).get('value', ''),\n",
|
||||
" \"工时费\": row.find('input', {'name': 'servicePrice'}).get('value', ''),\n",
|
||||
" \"工时\": row.find('input', {'name': 'serviceHour'}).get('value', ''),\n",
|
||||
" \"金额\": row.find('input', {'name': 'serviceAmount'}).get('value', ''),\n",
|
||||
" \"优惠\": row.find('input', {'name': 'discount'}).get('value', ''),\n",
|
||||
" \"服务备注\": row.find('input', {'name': 'serviceNote'}).get('value', ''),\n",
|
||||
" \"id\": order_id\n",
|
||||
" }\n",
|
||||
" service_data.append(service_item)\n",
|
||||
"\n",
|
||||
" all_service_data.extend(service_data)\n",
|
||||
"\n",
|
||||
" # tables = sub_soup.find_all('table', class_='yqx-table')\n",
|
||||
" # service_table = tables[0]\n",
|
||||
" # service_rows = service_table.select('tr.form_item')\n",
|
||||
" # # print(tables[0])\n",
|
||||
" # print(service_rows)\n",
|
||||
" # service_data = []\n",
|
||||
" # service_headers = [\"服务名称\", \"服务类别\", \"工时费\", \"工时\", \"金额\", \"优惠\", \"维修工\", \"服务备注\"]\n",
|
||||
" #\n",
|
||||
" # for row in service_rows:\n",
|
||||
" # print(1)\n",
|
||||
" # cells = row.find_all('td')\n",
|
||||
" # if len(cells) == len(service_headers):\n",
|
||||
" # service_item = {service_headers[i]: cells[i].get_text(strip=True) for i in range(len(service_headers))}\n",
|
||||
" # service_item[\"id\"] = order_id\n",
|
||||
" # service_data.append(service_item)\n",
|
||||
" #\n",
|
||||
" # all_service_data.extend(service_data)\n",
|
||||
"\n",
|
||||
" # 提取配件项目\n",
|
||||
" parts_table = tables[1]\n",
|
||||
" parts_rows = parts_table.select('tr.goods-datatr')\n",
|
||||
" parts_data = []\n",
|
||||
" parts_headers = [\"零件号\", \"配件名称\", \"售价\", \"数量\", \"单位\", \"金额\", \"优惠\", \"库存\",\n",
|
||||
" \"销售员\", \"配件备注\"]\n",
|
||||
"\n",
|
||||
" for row in parts_rows:\n",
|
||||
" parts_item = {\n",
|
||||
" \"零件号\": row.find('input', {'name': 'goodsFormat'}).get('value', ''),\n",
|
||||
" \"配件名称\": row.find('input', {'name': 'goodsName'}).get('value', ''),\n",
|
||||
" \"售价\": row.find('input', {'name': 'goodsPrice'}).get('value', ''),\n",
|
||||
" \"数量\": row.find('input', {'name': 'goodsNumber'}).get('value', ''),\n",
|
||||
" \"单位\": row.find('input', {'name': 'measureUnit'}).get('value', ''),\n",
|
||||
" \"金额\": row.find('input', {'name': 'goodsAmount'}).get('value', ''),\n",
|
||||
" \"优惠\": row.find('input', {'name': 'discount'}).get('value', ''),\n",
|
||||
" \"库存\": row.find('input', {'name': 'inventoryPrice'}).get('value', ''),\n",
|
||||
" \"销售员\": row.find('input', {'name': 'saleId'}).get('value', ''),\n",
|
||||
" \"配件备注\": row.find('input', {'name': 'goodsNote'}).get('value', ''),\n",
|
||||
" \"id\": order_id\n",
|
||||
" }\n",
|
||||
" parts_data.append(parts_item)\n",
|
||||
"\n",
|
||||
" all_parts_data.extend(parts_data)\n",
|
||||
"\n",
|
||||
" # parts_table = tables[1]\n",
|
||||
" # parts_rows = parts_table.select('tr.goods-datatr')\n",
|
||||
" # parts_data = []\n",
|
||||
" # parts_headers = [\"零件号\", \"配件名称\", \"售价\", \"数量\", \"金额\", \"优惠\", \"库存\", \"销售员\", \"配件备注\"]\n",
|
||||
" # \n",
|
||||
" # for row in parts_rows:\n",
|
||||
" # cells = row.find_all('td')\n",
|
||||
" # if len(cells) == len(parts_headers):\n",
|
||||
" # parts_item = {parts_headers[i]: cells[i].get_text(strip=True) for i in range(len(parts_headers))}\n",
|
||||
" # parts_item[\"id\"] = order_id\n",
|
||||
" # parts_data.append(parts_item)\n",
|
||||
" # \n",
|
||||
" # all_parts_data.extend(parts_data)\n",
|
||||
"\n",
|
||||
" # 提取基本信息\n",
|
||||
" base_info1 = {}\n",
|
||||
" inputs = sub_soup.find_all('input')\n",
|
||||
" for input_tag in inputs:\n",
|
||||
" name = input_tag.get('name')\n",
|
||||
" value = input_tag.get('value', '').strip()\n",
|
||||
" if name:\n",
|
||||
" key = name.replace('orderInfo.', '')\n",
|
||||
" base_info1[key] = value\n",
|
||||
"\n",
|
||||
" base_info1[\"id\"] = order_id\n",
|
||||
" all_base_ddata.append(base_info1)\n",
|
||||
"\n",
|
||||
" else:\n",
|
||||
" base_info[\"子单\"] = \"不存在\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"处理订单{order_id}时出错: {str(e)}\")\n",
|
||||
" continue\n",
|
||||
" page += 1\n",
|
||||
"\n",
|
||||
" break # 请求成功，跳出重试循环\n",
|
||||
"\n",
|
||||
" except requests.exceptions.RequestException as e:\n",
|
||||
" print(f\"第{page}页请求失败(尝试{attempt + 1}): {str(e)}\")\n",
|
||||
" if attempt < max_retries - 1:\n",
|
||||
" time.sleep(backoff_factor * (2 ** attempt))\n",
|
||||
" else:\n",
|
||||
" print(f\"达到最大重试次数，跳过第{page}页\")\n",
|
||||
" break\n",
|
||||
"\n",
|
||||
" number += 1\n",
|
||||
" time.sleep(1)\n",
|
||||
"\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"处理日期范围时出错: {str(e)}\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df2 = pd.DataFrame(all_base_ddata)\n",
|
||||
"df3 = pd.DataFrame(all_service_data)\n",
|
||||
"df4 = pd.DataFrame(all_parts_data)\n",
|
||||
"df1.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-18186751313 -工单.csv\")\n",
|
||||
"df2.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-18186751313 -工单子单基本信息.csv\")\n",
|
||||
"df3.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-18186751313 -工单子单服务项目.csv\")\n",
|
||||
"df4.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-18186751313 -工单子单配件项目.csv\")\n"
|
||||
],
|
||||
"id": "8d6f11a2b11f8be4",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2024-05-23 至 2025-05-23\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 15%|█▍ | 148/1000 [14:31<1:24:49, 5.97s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单60248836时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=60248836&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F22A14DE50>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 16%|█▌ | 158/1000 [15:54<1:31:45, 6.54s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"第159页请求失败(尝试1): HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/order-list/list?page=159&size=12&search_startTime=2024-05-23&search_endTime=2025-05-23&_=1743152268702 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F22C4FDE50>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 16%|█▌ | 159/1000 [16:31<3:42:44, 15.89s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单59683769时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=59683769&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F22AA403E0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 27%|██▋ | 269/1000 [27:45<1:13:13, 6.01s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单55024316时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=55024316&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F22CD4DA60>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 47%|████▋ | 466/1000 [47:50<54:49, 6.16s/it] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2023-05-24 至 2024-05-23\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 30%|██▉ | 299/1000 [29:29<1:09:17, 5.93s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单39527167时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=39527167&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F231231520>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 36%|███▌ | 358/1000 [35:36<1:01:30, 5.75s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单38398035时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=38398035&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F231606D80>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 37%|███▋ | 370/1000 [37:11<1:06:33, 6.34s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单38121509时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=38121509&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F2317D58E0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 37%|███▋ | 374/1000 [37:56<1:27:39, 8.40s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单38000056时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=38000056&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F231817F50>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 49%|████▉ | 492/1000 [49:50<50:05, 5.92s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单35733059时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=35733059&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F232FB2690>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 57%|█████▋ | 570/1000 [57:29<43:22, 6.05s/it] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2022-05-24 至 2023-05-24\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 46%|████▌ | 455/1000 [42:13<51:13, 5.64s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单28770640时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=28770640&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F2373C4260>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 52%|█████▏ | 517/1000 [48:13<43:45, 5.44s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单28203564时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=28203564&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F237AD5790>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 53%|█████▎ | 530/1000 [49:47<43:38, 5.57s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单28025893时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=28025893&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F2378F6BA0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 54%|█████▎ | 536/1000 [50:41<51:26, 6.65s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单27977874时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=27977874&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F237CDBDA0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 66%|██████▌ | 658/1000 [1:02:26<31:20, 5.50s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单26668270时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=26668270&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F22AB0D6D0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 69%|██████▉ | 691/1000 [1:05:48<29:25, 5.71s/it] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2021-05-24 至 2022-05-24\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 56%|█████▌ | 555/1000 [51:49<42:13, 5.69s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单20781414时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=20781414&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F23CAFC200>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 57%|█████▋ | 568/1000 [53:22<40:50, 5.67s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单20614104时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=20614104&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F23CBAE900>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 57%|█████▋ | 572/1000 [54:06<54:52, 7.69s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单20573201时出错: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/shop/order/detail?orderId=20573201&refer=order-list (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F23C8D3680>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 68%|██████▊ | 679/1000 [1:04:27<30:28, 5.70s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2020-05-24 至 2021-05-24\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 44%|████▎ | 435/1000 [40:20<52:23, 5.56s/it] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2019-05-25 至 2020-05-24\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:47, 1.19s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2018-05-25 至 2019-05-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:09, 1.15s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2017-05-25 至 2018-05-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:23, 1.17s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2016-05-25 至 2017-05-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<18:42, 1.12s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2015-05-26 至 2016-05-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<18:46, 1.13s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2014-05-26 至 2015-05-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:06, 1.15s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2013-05-26 至 2014-05-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:12, 1.15s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2012-05-26 至 2013-05-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:14, 1.16s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2011-05-27 至 2012-05-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:21, 1.16s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2010-05-27 至 2011-05-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:56, 1.20s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2009-05-27 至 2010-05-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<18:50, 1.13s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2008-05-27 至 2009-05-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:23, 1.16s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2007-05-28 至 2008-05-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:11, 1.15s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2006-05-28 至 2007-05-28\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:21, 1.16s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2005-05-28 至 2006-05-28\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:16, 1.16s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"查询时间范围: 2004-05-28 至 2005-05-28\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:27, 1.17s/it]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 处理附表(废弃)",
|
||||
"id": "758592b039b5f41a"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-14T02:51:30.881079Z",
|
||||
"start_time": "2025-05-14T02:51:30.866080Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df2 = pd.DataFrame(all_base_ddata)\n",
|
||||
"df3 = pd.DataFrame(all_service_data)\n",
|
||||
"df4 = pd.DataFrame(all_parts_data)\n",
|
||||
"df1.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13986383030 -工单.csv\")\n",
|
||||
"df2.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13986383030 -工单子单基本信息.csv\")\n",
|
||||
"df3.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13986383030 -工单子单服务项目.csv\")\n",
|
||||
"df4.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13986383030 -工单子单配件项目.csv\")\n"
|
||||
],
|
||||
"id": "d9dc38b82db31903",
|
||||
"outputs": [],
|
||||
"execution_count": 50
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 附表子单总额计算",
|
||||
"id": "19be0dac78f1a3e"
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "code",
|
||||
"outputs": [],
|
||||
"execution_count": null,
|
||||
"source": "",
|
||||
"id": "8a1cc6d877124e86"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,730 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "客户信息导出",
|
||||
"id": "a10af68ca00de50a"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-28T06:27:39.309261100Z",
|
||||
"start_time": "2026-01-28T06:27:31.811575700Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import json\n",
|
||||
"import time\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" '__jdv': '241039512|direct|-|none|-|1769580195196',\n",
|
||||
" 'mba_muid': '17695801951961847696270',\n",
|
||||
" 'mba_sid': '1769580378119515622883.0',\n",
|
||||
" 'QRCodeKey': 'AAEAIGlfy0S9IVwOVAfE3NXgBILRP8R2qpHAikvosD3cyvar',\n",
|
||||
" 'wlfstk_smdl': 'drd3j1dlvwjz7o96orgqco4keigo6ipj',\n",
|
||||
" 'pin': '%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8',\n",
|
||||
" 'unick': 's2v2tk1w4w8eb9',\n",
|
||||
" 'yunxiupin': '%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8',\n",
|
||||
" 'bmall_id': '499220987560FEF36C9FB4C092693FFC98D6D74FB85924A6DC3E030ADD3477238C68C7B8E10ED66898B6EBEB4562ED893FBF19E4CD0030FECE3218317F39A2F5FC9FF25A11561368D02C95564D247AC2A151A086B1074D5EE362C7CB1BA34EC79B50B3252535972791A084C5D010158EF4B5C1EADC2808FFF3EB992EF34A9D14',\n",
|
||||
" 'UUID': '7c069d6f-2724-4008-b308-09f436240035',\n",
|
||||
" 'SESSION_USER_NAME': '%E6%9F%A5%E4%BA%AE',\n",
|
||||
" 'flash': '3_bBBCg0-OBy68xAANqduCSaKWV_6DKpxvObl29hEy-5Il0taBIimdGJfimCMZHKH-CUR09cTQwjAyGRKP9z9jHkm9UrWP2nTOg38vb68OEMqiiwdZRg1AqxMtUmDSWBQdVhGg5qQSbO5p1eyeBPlu7Htsgtf_TMouEaoqlijt7PqsLs4yBBM8Alki49TorJwr9E6mcmF96X-2O-E6',\n",
|
||||
" 'thor': '51C732866E8A756A4ED8B4738163D4C059D9063322E05745BA0D815A80FEBE3D550B447E8BB3C99207EE02D994108998DE070A571A758BED83B7535090CB6C2DF853F8994511D1708463D0C16274A620668FCE84D3B708BA56633C46432FABC8B4DA7688141595BA1C45C419160C151A638E6A118D22DA606EE420BE383B9D53FB842C2C90A6454E4CFED4E3A63A8ED5',\n",
|
||||
" 'light_key': 'AASBKE7rOxgWQziEhC_QY6yaLIBQgOT1dxVmsPRjRIB9ndtR9Aslstb1iQBYYZyMvLEJNsNDL52e7UA1dFCeHNIVWAseFg',\n",
|
||||
" 'JD_UUID': 'b9441bd1-a782-4d34-b2af-47b6f1e89bc6',\n",
|
||||
" '3AB9D23F7A4B3CSS': 'jdd03XH7UJPD6AWWSVFN6XP6T23OSWBA235SAMFFYPEW3JTTJRTT4TQTZLIAKR47DQ26GKGT266NM6WGBCEKT7R6RL2K34IAAAAM4ANDUGQYAAAAACSHCLFATJTAJKMX',\n",
|
||||
" '_gia_d': '1',\n",
|
||||
" 'sdtoken': 'AAbEsBpEIOVjqTAKCQtvQu17FLUpUb4CM0QtRGo9_y1DbK8h44YEOS2UiemwosOOgSgqPwHoDiJho0E69o2zgRTqSGouwgzZW2ByDMgJdY2S_IVvi4uyNneMFom6TCFDsAUG_C0BzQYsVWWU',\n",
|
||||
" '__jda': '200816309.17695801951961847696270.1769580195.1769580195.1769580195.1',\n",
|
||||
" '__jdc': '200816309',\n",
|
||||
" '3AB9D23F7A4B3C9B': 'KPOAZ5CBYQXJ25JOSSXGBQDYYOVGX7EMTPWFWDGVZ3KWTLNZU45HHPQ4MXYSIO37IV72PLAND3VYJH2ZOLD6LFWBDA',\n",
|
||||
" '__jdb': '200816309.53.17695801951961847696270|1.1769580195',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" # 'content-length': '0',\n",
|
||||
" 'origin': 'https://jch.yunxiu.com',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://jch.yunxiu.com/',\n",
|
||||
" 'sec-ch-ua': '\"Not(A:Brand\";v=\"8\", \"Chromium\";v=\"144\", \"Microsoft Edge\";v=\"144\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-site',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0',\n",
|
||||
" 'x-requested-with': 'XMLHttpRequest',\n",
|
||||
" # 'cookie': '__jdv=241039512|direct|-|none|-|1769580195196; mba_muid=17695801951961847696270; mba_sid=1769580378119515622883.0; QRCodeKey=AAEAIGlfy0S9IVwOVAfE3NXgBILRP8R2qpHAikvosD3cyvar; wlfstk_smdl=drd3j1dlvwjz7o96orgqco4keigo6ipj; pin=%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8; unick=s2v2tk1w4w8eb9; yunxiupin=%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8; bmall_id=499220987560FEF36C9FB4C092693FFC98D6D74FB85924A6DC3E030ADD3477238C68C7B8E10ED66898B6EBEB4562ED893FBF19E4CD0030FECE3218317F39A2F5FC9FF25A11561368D02C95564D247AC2A151A086B1074D5EE362C7CB1BA34EC79B50B3252535972791A084C5D010158EF4B5C1EADC2808FFF3EB992EF34A9D14; UUID=7c069d6f-2724-4008-b308-09f436240035; SESSION_USER_NAME=%E6%9F%A5%E4%BA%AE; flash=3_bBBCg0-OBy68xAANqduCSaKWV_6DKpxvObl29hEy-5Il0taBIimdGJfimCMZHKH-CUR09cTQwjAyGRKP9z9jHkm9UrWP2nTOg38vb68OEMqiiwdZRg1AqxMtUmDSWBQdVhGg5qQSbO5p1eyeBPlu7Htsgtf_TMouEaoqlijt7PqsLs4yBBM8Alki49TorJwr9E6mcmF96X-2O-E6; thor=51C732866E8A756A4ED8B4738163D4C059D9063322E05745BA0D815A80FEBE3D550B447E8BB3C99207EE02D994108998DE070A571A758BED83B7535090CB6C2DF853F8994511D1708463D0C16274A620668FCE84D3B708BA56633C46432FABC8B4DA7688141595BA1C45C419160C151A638E6A118D22DA606EE420BE383B9D53FB842C2C90A6454E4CFED4E3A63A8ED5; light_key=AASBKE7rOxgWQziEhC_QY6yaLIBQgOT1dxVmsPRjRIB9ndtR9Aslstb1iQBYYZyMvLEJNsNDL52e7UA1dFCeHNIVWAseFg; JD_UUID=b9441bd1-a782-4d34-b2af-47b6f1e89bc6; 3AB9D23F7A4B3CSS=jdd03XH7UJPD6AWWSVFN6XP6T23OSWBA235SAMFFYPEW3JTTJRTT4TQTZLIAKR47DQ26GKGT266NM6WGBCEKT7R6RL2K34IAAAAM4ANDUGQYAAAAACSHCLFATJTAJKMX; _gia_d=1; sdtoken=AAbEsBpEIOVjqTAKCQtvQu17FLUpUb4CM0QtRGo9_y1DbK8h44YEOS2UiemwosOOgSgqPwHoDiJho0E69o2zgRTqSGouwgzZW2ByDMgJdY2S_IVvi4uyNneMFom6TCFDsAUG_C0BzQYsVWWU; __jda=200816309.17695801951961847696270.1769580195.1769580195.1769580195.1; __jdc=200816309; 3AB9D23F7A4B3C9B=KPOAZ5CBYQXJ25JOSSXGBQDYYOVGX7EMTPWFWDGVZ3KWTLNZU45HHPQ4MXYSIO37IV72PLAND3VYJH2ZOLD6LFWBDA; __jdb=200816309.53.17695801951961847696270|1.1769580195',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"number = 1748507284349\n",
|
||||
"page = 1\n",
|
||||
"max_retries = 5 # 最大重试次数\n",
|
||||
"backoff_factor = 1 # 指数退避因子\n",
|
||||
"\n",
|
||||
"for page in tqdm(range(1,2)):\n",
|
||||
" url = f\"https://jch1.yunxiu.com/legend/account/search?page={page}&_={number}\" # 注意url\n",
|
||||
" data = f'page={page}&_={number}'\n",
|
||||
"\n",
|
||||
" attempt = 0\n",
|
||||
" while attempt < max_retries:\n",
|
||||
" try:\n",
|
||||
" response = requests.get(url=url, headers=headers, data=data)\n",
|
||||
" response.raise_for_status() \n",
|
||||
" # print(response.json())\n",
|
||||
" # break# 如果响应状态码不是200，则抛出HTTPError异常\n",
|
||||
" res = response.json().get(\"data\", {}).get(\"content\", [])\n",
|
||||
" \n",
|
||||
" for item in res:\n",
|
||||
" base_info = {k: v for k, v in item.items() if k != \"licenseList\"}\n",
|
||||
" if \"licenseList\" in item and isinstance(item[\"licenseList\"], list):\n",
|
||||
" for new_item in item[\"licenseList\"]:\n",
|
||||
" row = base_info.copy()\n",
|
||||
" row.update({\"car\": new_item})\n",
|
||||
" all_data.append(row)\n",
|
||||
" else:\n",
|
||||
" # 如果没有licenseList或者它不是列表，则直接添加基本信息\n",
|
||||
" all_data.append(base_info)\n",
|
||||
" \n",
|
||||
" break # 成功后退出重试循环\n",
|
||||
" except requests.exceptions.RequestException as e:\n",
|
||||
" print(f\"第{page}页请求失败: {e}\")\n",
|
||||
" if attempt < max_retries - 1:\n",
|
||||
" sleep_time = backoff_factor * (2 ** attempt) # 计算等待时间\n",
|
||||
" print(f\"准备第{attempt + 1}次重试，等待{sleep_time}秒...\")\n",
|
||||
" time.sleep(sleep_time) # 等待一段时间后重试\n",
|
||||
" else:\n",
|
||||
" print(f\"达到最大重试次数，跳过第{page}页\")\n",
|
||||
" attempt += 1\n",
|
||||
"\n",
|
||||
" number += 1\n",
|
||||
" time.sleep(1)\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\京东云修数据导出-养车驿栈万江店.xlsx\")\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "KeyboardInterrupt",
|
||||
"evalue": "",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[31m---------------------------------------------------------------------------\u001B[39m",
|
||||
"\u001B[31mKeyboardInterrupt\u001B[39m Traceback (most recent call last)",
|
||||
"\u001B[36mCell\u001B[39m\u001B[36m \u001B[39m\u001B[32mIn[1]\u001B[39m\u001B[32m, line 67\u001B[39m\n\u001B[32m 65\u001B[39m \u001B[38;5;28;01mwhile\u001B[39;00m attempt < max_retries:\n\u001B[32m 66\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m---> \u001B[39m\u001B[32m67\u001B[39m response = \u001B[43mrequests\u001B[49m\u001B[43m.\u001B[49m\u001B[43mget\u001B[49m\u001B[43m(\u001B[49m\u001B[43murl\u001B[49m\u001B[43m=\u001B[49m\u001B[43murl\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m=\u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mdata\u001B[49m\u001B[43m=\u001B[49m\u001B[43mdata\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 68\u001B[39m response.raise_for_status() \n\u001B[32m 69\u001B[39m \u001B[38;5;66;03m# print(response.json())\u001B[39;00m\n\u001B[32m 70\u001B[39m \u001B[38;5;66;03m# break# 如果响应状态码不是200，则抛出HTTPError异常\u001B[39;00m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\api.py:73\u001B[39m, in \u001B[36mget\u001B[39m\u001B[34m(url, params, **kwargs)\u001B[39m\n\u001B[32m 62\u001B[39m \u001B[38;5;28;01mdef\u001B[39;00m\u001B[38;5;250m \u001B[39m\u001B[34mget\u001B[39m(url, params=\u001B[38;5;28;01mNone\u001B[39;00m, **kwargs):\n\u001B[32m 63\u001B[39m \u001B[38;5;250m \u001B[39m\u001B[33mr\u001B[39m\u001B[33;03m\"\"\"Sends a GET request.\u001B[39;00m\n\u001B[32m 64\u001B[39m \n\u001B[32m 65\u001B[39m \u001B[33;03m :param url: URL for the new :class:`Request` object.\u001B[39;00m\n\u001B[32m (...)\u001B[39m\u001B[32m 70\u001B[39m \u001B[33;03m :rtype: requests.Response\u001B[39;00m\n\u001B[32m 71\u001B[39m \u001B[33;03m \"\"\"\u001B[39;00m\n\u001B[32m---> \u001B[39m\u001B[32m73\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43mrequest\u001B[49m\u001B[43m(\u001B[49m\u001B[33;43m\"\u001B[39;49m\u001B[33;43mget\u001B[39;49m\u001B[33;43m\"\u001B[39;49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43murl\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mparams\u001B[49m\u001B[43m=\u001B[49m\u001B[43mparams\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\api.py:59\u001B[39m, in \u001B[36mrequest\u001B[39m\u001B[34m(method, url, **kwargs)\u001B[39m\n\u001B[32m 55\u001B[39m \u001B[38;5;66;03m# By using the 'with' statement we are sure the session is closed, thus we\u001B[39;00m\n\u001B[32m 56\u001B[39m \u001B[38;5;66;03m# avoid leaving sockets open which can trigger a ResourceWarning in some\u001B[39;00m\n\u001B[32m 57\u001B[39m \u001B[38;5;66;03m# cases, and look like a memory leak in others.\u001B[39;00m\n\u001B[32m 58\u001B[39m \u001B[38;5;28;01mwith\u001B[39;00m sessions.Session() \u001B[38;5;28;01mas\u001B[39;00m session:\n\u001B[32m---> \u001B[39m\u001B[32m59\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[43msession\u001B[49m\u001B[43m.\u001B[49m\u001B[43mrequest\u001B[49m\u001B[43m(\u001B[49m\u001B[43mmethod\u001B[49m\u001B[43m=\u001B[49m\u001B[43mmethod\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43murl\u001B[49m\u001B[43m=\u001B[49m\u001B[43murl\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\sessions.py:589\u001B[39m, in \u001B[36mSession.request\u001B[39m\u001B[34m(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)\u001B[39m\n\u001B[32m 584\u001B[39m send_kwargs = {\n\u001B[32m 585\u001B[39m \u001B[33m\"\u001B[39m\u001B[33mtimeout\u001B[39m\u001B[33m\"\u001B[39m: timeout,\n\u001B[32m 586\u001B[39m \u001B[33m\"\u001B[39m\u001B[33mallow_redirects\u001B[39m\u001B[33m\"\u001B[39m: allow_redirects,\n\u001B[32m 587\u001B[39m }\n\u001B[32m 588\u001B[39m send_kwargs.update(settings)\n\u001B[32m--> \u001B[39m\u001B[32m589\u001B[39m resp = \u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43msend\u001B[49m\u001B[43m(\u001B[49m\u001B[43mprep\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43msend_kwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 591\u001B[39m \u001B[38;5;28;01mreturn\u001B[39;00m resp\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\sessions.py:724\u001B[39m, in \u001B[36mSession.send\u001B[39m\u001B[34m(self, request, **kwargs)\u001B[39m\n\u001B[32m 721\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m allow_redirects:\n\u001B[32m 722\u001B[39m \u001B[38;5;66;03m# Redirect resolving generator.\u001B[39;00m\n\u001B[32m 723\u001B[39m gen = \u001B[38;5;28mself\u001B[39m.resolve_redirects(r, request, **kwargs)\n\u001B[32m--> \u001B[39m\u001B[32m724\u001B[39m history = \u001B[43m[\u001B[49m\u001B[43mresp\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;28;43;01mfor\u001B[39;49;00m\u001B[43m \u001B[49m\u001B[43mresp\u001B[49m\u001B[43m \u001B[49m\u001B[38;5;129;43;01min\u001B[39;49;00m\u001B[43m \u001B[49m\u001B[43mgen\u001B[49m\u001B[43m]\u001B[49m\n\u001B[32m 725\u001B[39m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[32m 726\u001B[39m history = []\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\sessions.py:265\u001B[39m, in \u001B[36mSessionRedirectMixin.resolve_redirects\u001B[39m\u001B[34m(self, resp, req, stream, timeout, verify, cert, proxies, yield_requests, **adapter_kwargs)\u001B[39m\n\u001B[32m 263\u001B[39m \u001B[38;5;28;01myield\u001B[39;00m req\n\u001B[32m 264\u001B[39m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m265\u001B[39m resp = \u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43msend\u001B[49m\u001B[43m(\u001B[49m\n\u001B[32m 266\u001B[39m \u001B[43m \u001B[49m\u001B[43mreq\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 267\u001B[39m \u001B[43m \u001B[49m\u001B[43mstream\u001B[49m\u001B[43m=\u001B[49m\u001B[43mstream\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 268\u001B[39m \u001B[43m \u001B[49m\u001B[43mtimeout\u001B[49m\u001B[43m=\u001B[49m\u001B[43mtimeout\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 269\u001B[39m \u001B[43m \u001B[49m\u001B[43mverify\u001B[49m\u001B[43m=\u001B[49m\u001B[43mverify\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 270\u001B[39m \u001B[43m \u001B[49m\u001B[43mcert\u001B[49m\u001B[43m=\u001B[49m\u001B[43mcert\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 271\u001B[39m \u001B[43m \u001B[49m\u001B[43mproxies\u001B[49m\u001B[43m=\u001B[49m\u001B[43mproxies\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 272\u001B[39m \u001B[43m \u001B[49m\u001B[43mallow_redirects\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43;01mFalse\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[32m 273\u001B[39m \u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43madapter_kwargs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 274\u001B[39m \u001B[43m \u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 276\u001B[39m extract_cookies_to_jar(\u001B[38;5;28mself\u001B[39m.cookies, prepared_request, resp.raw)\n\u001B[32m 278\u001B[39m \u001B[38;5;66;03m# extract redirect url, if any, for the next loop\u001B[39;00m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\sessions.py:703\u001B[39m, in \u001B[36mSession.send\u001B[39m\u001B[34m(self, request, **kwargs)\u001B[39m\n\u001B[32m 700\u001B[39m start = preferred_clock()\n\u001B[32m 702\u001B[39m \u001B[38;5;66;03m# Send the request\u001B[39;00m\n\u001B[32m--> \u001B[39m\u001B[32m703\u001B[39m r = \u001B[43madapter\u001B[49m\u001B[43m.\u001B[49m\u001B[43msend\u001B[49m\u001B[43m(\u001B[49m\u001B[43mrequest\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43mkwargs\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 705\u001B[39m \u001B[38;5;66;03m# Total elapsed time of the request (approximately)\u001B[39;00m\n\u001B[32m 706\u001B[39m elapsed = preferred_clock() - start\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\requests\\adapters.py:644\u001B[39m, in \u001B[36mHTTPAdapter.send\u001B[39m\u001B[34m(self, request, stream, timeout, verify, cert, proxies)\u001B[39m\n\u001B[32m 641\u001B[39m timeout = TimeoutSauce(connect=timeout, read=timeout)\n\u001B[32m 643\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m644\u001B[39m resp = \u001B[43mconn\u001B[49m\u001B[43m.\u001B[49m\u001B[43murlopen\u001B[49m\u001B[43m(\u001B[49m\n\u001B[32m 645\u001B[39m \u001B[43m \u001B[49m\u001B[43mmethod\u001B[49m\u001B[43m=\u001B[49m\u001B[43mrequest\u001B[49m\u001B[43m.\u001B[49m\u001B[43mmethod\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 646\u001B[39m \u001B[43m \u001B[49m\u001B[43murl\u001B[49m\u001B[43m=\u001B[49m\u001B[43murl\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 647\u001B[39m \u001B[43m \u001B[49m\u001B[43mbody\u001B[49m\u001B[43m=\u001B[49m\u001B[43mrequest\u001B[49m\u001B[43m.\u001B[49m\u001B[43mbody\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 648\u001B[39m \u001B[43m \u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m=\u001B[49m\u001B[43mrequest\u001B[49m\u001B[43m.\u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 649\u001B[39m \u001B[43m \u001B[49m\u001B[43mredirect\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43;01mFalse\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[32m 650\u001B[39m \u001B[43m \u001B[49m\u001B[43massert_same_host\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43;01mFalse\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[32m 651\u001B[39m \u001B[43m \u001B[49m\u001B[43mpreload_content\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43;01mFalse\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[32m 652\u001B[39m \u001B[43m \u001B[49m\u001B[43mdecode_content\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43;01mFalse\u001B[39;49;00m\u001B[43m,\u001B[49m\n\u001B[32m 653\u001B[39m \u001B[43m \u001B[49m\u001B[43mretries\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mmax_retries\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 654\u001B[39m \u001B[43m \u001B[49m\u001B[43mtimeout\u001B[49m\u001B[43m=\u001B[49m\u001B[43mtimeout\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 655\u001B[39m \u001B[43m \u001B[49m\u001B[43mchunked\u001B[49m\u001B[43m=\u001B[49m\u001B[43mchunked\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 656\u001B[39m \u001B[43m \u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 658\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m (ProtocolError, \u001B[38;5;167;01mOSError\u001B[39;00m) \u001B[38;5;28;01mas\u001B[39;00m err:\n\u001B[32m 659\u001B[39m \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mConnectionError\u001B[39;00m(err, request=request)\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\connectionpool.py:787\u001B[39m, in \u001B[36mHTTPConnectionPool.urlopen\u001B[39m\u001B[34m(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, preload_content, decode_content, **response_kw)\u001B[39m\n\u001B[32m 784\u001B[39m response_conn = conn \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m release_conn \u001B[38;5;28;01melse\u001B[39;00m \u001B[38;5;28;01mNone\u001B[39;00m\n\u001B[32m 786\u001B[39m \u001B[38;5;66;03m# Make the request on the HTTPConnection object\u001B[39;00m\n\u001B[32m--> \u001B[39m\u001B[32m787\u001B[39m response = \u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43m_make_request\u001B[49m\u001B[43m(\u001B[49m\n\u001B[32m 788\u001B[39m \u001B[43m \u001B[49m\u001B[43mconn\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 789\u001B[39m \u001B[43m \u001B[49m\u001B[43mmethod\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 790\u001B[39m \u001B[43m \u001B[49m\u001B[43murl\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 791\u001B[39m \u001B[43m \u001B[49m\u001B[43mtimeout\u001B[49m\u001B[43m=\u001B[49m\u001B[43mtimeout_obj\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 792\u001B[39m \u001B[43m \u001B[49m\u001B[43mbody\u001B[49m\u001B[43m=\u001B[49m\u001B[43mbody\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 793\u001B[39m \u001B[43m \u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m=\u001B[49m\u001B[43mheaders\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 794\u001B[39m \u001B[43m \u001B[49m\u001B[43mchunked\u001B[49m\u001B[43m=\u001B[49m\u001B[43mchunked\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 795\u001B[39m \u001B[43m \u001B[49m\u001B[43mretries\u001B[49m\u001B[43m=\u001B[49m\u001B[43mretries\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 796\u001B[39m \u001B[43m \u001B[49m\u001B[43mresponse_conn\u001B[49m\u001B[43m=\u001B[49m\u001B[43mresponse_conn\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 797\u001B[39m \u001B[43m \u001B[49m\u001B[43mpreload_content\u001B[49m\u001B[43m=\u001B[49m\u001B[43mpreload_content\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 798\u001B[39m \u001B[43m \u001B[49m\u001B[43mdecode_content\u001B[49m\u001B[43m=\u001B[49m\u001B[43mdecode_content\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 799\u001B[39m \u001B[43m \u001B[49m\u001B[43m*\u001B[49m\u001B[43m*\u001B[49m\u001B[43mresponse_kw\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 800\u001B[39m \u001B[43m\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 802\u001B[39m \u001B[38;5;66;03m# Everything went great!\u001B[39;00m\n\u001B[32m 803\u001B[39m clean_exit = \u001B[38;5;28;01mTrue\u001B[39;00m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\connectionpool.py:464\u001B[39m, in \u001B[36mHTTPConnectionPool._make_request\u001B[39m\u001B[34m(self, conn, method, url, body, headers, retries, timeout, chunked, response_conn, preload_content, decode_content, enforce_content_length)\u001B[39m\n\u001B[32m 461\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m 462\u001B[39m \u001B[38;5;66;03m# Trigger any extra validation we need to do.\u001B[39;00m\n\u001B[32m 463\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m464\u001B[39m \u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43m_validate_conn\u001B[49m\u001B[43m(\u001B[49m\u001B[43mconn\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 465\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m (SocketTimeout, BaseSSLError) \u001B[38;5;28;01mas\u001B[39;00m e:\n\u001B[32m 466\u001B[39m \u001B[38;5;28mself\u001B[39m._raise_timeout(err=e, url=url, timeout_value=conn.timeout)\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\connectionpool.py:1093\u001B[39m, in \u001B[36mHTTPSConnectionPool._validate_conn\u001B[39m\u001B[34m(self, conn)\u001B[39m\n\u001B[32m 1091\u001B[39m \u001B[38;5;66;03m# Force connect early to allow us to validate the connection.\u001B[39;00m\n\u001B[32m 1092\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m conn.is_closed:\n\u001B[32m-> \u001B[39m\u001B[32m1093\u001B[39m \u001B[43mconn\u001B[49m\u001B[43m.\u001B[49m\u001B[43mconnect\u001B[49m\u001B[43m(\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 1095\u001B[39m \u001B[38;5;66;03m# TODO revise this, see https://github.com/urllib3/urllib3/issues/2791\u001B[39;00m\n\u001B[32m 1096\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m conn.is_verified \u001B[38;5;129;01mand\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m conn.proxy_is_verified:\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\connection.py:796\u001B[39m, in \u001B[36mHTTPSConnection.connect\u001B[39m\u001B[34m(self)\u001B[39m\n\u001B[32m 793\u001B[39m \u001B[38;5;66;03m# Remove trailing '.' from fqdn hostnames to allow certificate validation\u001B[39;00m\n\u001B[32m 794\u001B[39m server_hostname_rm_dot = server_hostname.rstrip(\u001B[33m\"\u001B[39m\u001B[33m.\u001B[39m\u001B[33m\"\u001B[39m)\n\u001B[32m--> \u001B[39m\u001B[32m796\u001B[39m sock_and_verified = \u001B[43m_ssl_wrap_socket_and_match_hostname\u001B[49m\u001B[43m(\u001B[49m\n\u001B[32m 797\u001B[39m \u001B[43m \u001B[49m\u001B[43msock\u001B[49m\u001B[43m=\u001B[49m\u001B[43msock\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 798\u001B[39m \u001B[43m \u001B[49m\u001B[43mcert_reqs\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mcert_reqs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 799\u001B[39m \u001B[43m \u001B[49m\u001B[43mssl_version\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mssl_version\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 800\u001B[39m \u001B[43m \u001B[49m\u001B[43mssl_minimum_version\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mssl_minimum_version\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 801\u001B[39m \u001B[43m \u001B[49m\u001B[43mssl_maximum_version\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mssl_maximum_version\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 802\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_certs\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mca_certs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 803\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_cert_dir\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mca_cert_dir\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 804\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_cert_data\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mca_cert_data\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 805\u001B[39m \u001B[43m \u001B[49m\u001B[43mcert_file\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mcert_file\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 806\u001B[39m \u001B[43m \u001B[49m\u001B[43mkey_file\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mkey_file\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 807\u001B[39m \u001B[43m \u001B[49m\u001B[43mkey_password\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mkey_password\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 808\u001B[39m \u001B[43m \u001B[49m\u001B[43mserver_hostname\u001B[49m\u001B[43m=\u001B[49m\u001B[43mserver_hostname_rm_dot\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 809\u001B[39m \u001B[43m \u001B[49m\u001B[43mssl_context\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43mssl_context\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 810\u001B[39m \u001B[43m \u001B[49m\u001B[43mtls_in_tls\u001B[49m\u001B[43m=\u001B[49m\u001B[43mtls_in_tls\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 811\u001B[39m \u001B[43m \u001B[49m\u001B[43massert_hostname\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43massert_hostname\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 812\u001B[39m \u001B[43m \u001B[49m\u001B[43massert_fingerprint\u001B[49m\u001B[43m=\u001B[49m\u001B[38;5;28;43mself\u001B[39;49m\u001B[43m.\u001B[49m\u001B[43massert_fingerprint\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 813\u001B[39m \u001B[43m \u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 814\u001B[39m \u001B[38;5;28mself\u001B[39m.sock = sock_and_verified.socket\n\u001B[32m 816\u001B[39m \u001B[38;5;66;03m# If an error occurs during connection/handshake we may need to release\u001B[39;00m\n\u001B[32m 817\u001B[39m \u001B[38;5;66;03m# our lock so another connection can probe the origin.\u001B[39;00m\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\connection.py:975\u001B[39m, in \u001B[36m_ssl_wrap_socket_and_match_hostname\u001B[39m\u001B[34m(sock, cert_reqs, ssl_version, ssl_minimum_version, ssl_maximum_version, cert_file, key_file, key_password, ca_certs, ca_cert_dir, ca_cert_data, assert_hostname, assert_fingerprint, server_hostname, ssl_context, tls_in_tls)\u001B[39m\n\u001B[32m 972\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m is_ipaddress(normalized):\n\u001B[32m 973\u001B[39m server_hostname = normalized\n\u001B[32m--> \u001B[39m\u001B[32m975\u001B[39m ssl_sock = \u001B[43mssl_wrap_socket\u001B[49m\u001B[43m(\u001B[49m\n\u001B[32m 976\u001B[39m \u001B[43m \u001B[49m\u001B[43msock\u001B[49m\u001B[43m=\u001B[49m\u001B[43msock\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 977\u001B[39m \u001B[43m \u001B[49m\u001B[43mkeyfile\u001B[49m\u001B[43m=\u001B[49m\u001B[43mkey_file\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 978\u001B[39m \u001B[43m \u001B[49m\u001B[43mcertfile\u001B[49m\u001B[43m=\u001B[49m\u001B[43mcert_file\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 979\u001B[39m \u001B[43m \u001B[49m\u001B[43mkey_password\u001B[49m\u001B[43m=\u001B[49m\u001B[43mkey_password\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 980\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_certs\u001B[49m\u001B[43m=\u001B[49m\u001B[43mca_certs\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 981\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_cert_dir\u001B[49m\u001B[43m=\u001B[49m\u001B[43mca_cert_dir\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 982\u001B[39m \u001B[43m \u001B[49m\u001B[43mca_cert_data\u001B[49m\u001B[43m=\u001B[49m\u001B[43mca_cert_data\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 983\u001B[39m \u001B[43m \u001B[49m\u001B[43mserver_hostname\u001B[49m\u001B[43m=\u001B[49m\u001B[43mserver_hostname\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 984\u001B[39m \u001B[43m \u001B[49m\u001B[43mssl_context\u001B[49m\u001B[43m=\u001B[49m\u001B[43mcontext\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 985\u001B[39m \u001B[43m \u001B[49m\u001B[43mtls_in_tls\u001B[49m\u001B[43m=\u001B[49m\u001B[43mtls_in_tls\u001B[49m\u001B[43m,\u001B[49m\n\u001B[32m 986\u001B[39m \u001B[43m\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 988\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m 989\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m assert_fingerprint:\n",
|
||||
"\u001B[36mFile \u001B[39m\u001B[32mD:\\Program Files\\anaconda3\\envs\\F6+宜搭+其它\\Lib\\site-packages\\urllib3\\util\\ssl_.py:461\u001B[39m, in \u001B[36mssl_wrap_socket\u001B[39m\u001B[34m(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data, tls_in_tls)\u001B[39m\n\u001B[32m 459\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m ca_certs \u001B[38;5;129;01mor\u001B[39;00m ca_cert_dir \u001B[38;5;129;01mor\u001B[39;00m ca_cert_data:\n\u001B[32m 460\u001B[39m \u001B[38;5;28;01mtry\u001B[39;00m:\n\u001B[32m--> \u001B[39m\u001B[32m461\u001B[39m \u001B[43mcontext\u001B[49m\u001B[43m.\u001B[49m\u001B[43mload_verify_locations\u001B[49m\u001B[43m(\u001B[49m\u001B[43mca_certs\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mca_cert_dir\u001B[49m\u001B[43m,\u001B[49m\u001B[43m \u001B[49m\u001B[43mca_cert_data\u001B[49m\u001B[43m)\u001B[49m\n\u001B[32m 462\u001B[39m \u001B[38;5;28;01mexcept\u001B[39;00m \u001B[38;5;167;01mOSError\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m e:\n\u001B[32m 463\u001B[39m \u001B[38;5;28;01mraise\u001B[39;00m SSLError(e) \u001B[38;5;28;01mfrom\u001B[39;00m\u001B[38;5;250m \u001B[39m\u001B[34;01me\u001B[39;00m\n",
|
||||
"\u001B[31mKeyboardInterrupt\u001B[39m: "
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-03-17T09:05:23.254492Z",
|
||||
"start_time": "2025-03-17T09:05:20.198477Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13576939998.xlsx\")"
|
||||
],
|
||||
"id": "3ec7e6bfa5d425ee",
|
||||
"outputs": [],
|
||||
"execution_count": 2
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "车辆信息",
|
||||
"id": "a405bccb4dfdf949"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-07-25T02:19:42.395112Z",
|
||||
"start_time": "2025-07-25T01:57:39.563741Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import json\n",
|
||||
"import time\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from datetime import datetime, timedelta\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" \"cookie\":'__jdv=135409966|direct|-|none|-|1753349284245; flash=3_EuI2ojX4uJ6bPhnrd3tKHPKia0BaRB7TCi-gdcbqrSIauxZmkgE_yZJ7WTnzwB4_5G2Tdhdi89A4xE8t1uUvWrpjg5EnWwwrMNKGl8mWCaH1Vlo0VlXNk6Eyv9VrOA_6v-MDACjpO5RN4yGkhYfYFH5968Y9NPLEbK-ARUVmdrTXu6y8_zq-chEIqCSnme**; pin=jd_4d1be17ac64d3; unick=jd_2tnscrz74er69g; thor=884B7391C2AC4631693332E7FA593E284097D0ECACAF2D9BD0A558BB6EB65EAAB7F276B2475E6F7271396EBE3D1C84832247536BA40EB75D5EBF9AF87797FDD41D2AC25CC5C5B8803729EF8374A9C2CCE2D36DB51AB971A07238FDB6EE68D2ADC973BD6F40501EB116A185BEB86917D5C9DA41A03B1E1794D9591D2861EE053015AACDC7C64065615BBCCE428FC7E09AB66E0E5EA6EE69B23323F013536FF8ED; light_key=AASBKE7rOxgWQziEhC_QY6ya_BdQfCqNzo11axkWQXqAq56Qf-aHwELmm75kijsmGBR7FBg2; JD_UUID=d4eef9de-02a9-41c6-85dd-7f656b5c2bac; yunxiupin=jd_4d1be17ac64d3; UUID=b54ae144-a7f5-4000-bcfb-19ac3838a879; SESSION_USER_NAME=%E6%9D%8E%E5%BB%BA%E4%BC%9A; __jda=154799550.17533492842442035336434.1753349284.1753349284.1753408464.2; __jdc=154799550; PUBLIC_NOTICE_TIME105188=\"2022-10-16 10:33:17\"; __jdb=154799550.4.17533492842442035336434|2.1753408464',\n",
|
||||
" \"referer\": \"https://www.yunxiu.com/legend/account\",\n",
|
||||
" \"sec-ch-ua\": \"\\\"Not(A:Brand\\\";v=\\\"99\\\", \\\"Microsoft Edge\\\";v=\\\"133\\\", \\\"Chromium\\\";v=\\\"133\\\"\",\n",
|
||||
" \"sec-ch-ua-mobile\": \"?0\",\n",
|
||||
" \"sec-ch-ua-platform\": \"\\\"Windows\\\"\",\n",
|
||||
" \"sec-fetch-dest\": \"empty\",\n",
|
||||
" \"sec-fetch-mode\": \"cors\",\n",
|
||||
" \"sec-fetch-site\": \"same-origin\",\n",
|
||||
" \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0\",\n",
|
||||
" \"x-requested-with\": \"XMLHttpRequest\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"number = 1746510018148 # 从车辆查询页面中 date参数获取\n",
|
||||
"page = 1\n",
|
||||
"max_retries = 5 # 最大重试次数\n",
|
||||
"backoff_factor = 1 # 指数退避因子\n",
|
||||
"\n",
|
||||
"current_date = datetime.now()\n",
|
||||
"years_back = 10\n",
|
||||
"\n",
|
||||
"end_date = current_date - timedelta(days=365 * years_back)\n",
|
||||
"start_date = current_date\n",
|
||||
"\n",
|
||||
"while start_date >= end_date:\n",
|
||||
" try:\n",
|
||||
" search_startTime = (start_date - timedelta(days=365)).strftime('%Y-%m-%d')\n",
|
||||
" search_endTime = start_date.strftime('%Y-%m-%d')\n",
|
||||
" print(f'search_startTime: {search_startTime}, search_endTime: {search_endTime}')\n",
|
||||
" start_date -= timedelta(days=365) # 每次减去一年\n",
|
||||
" exit_loop = False\n",
|
||||
" for page in tqdm(range(1,1001)):\n",
|
||||
" if exit_loop:\n",
|
||||
" break\n",
|
||||
" url = f\"https://www.yunxiu.com/legend/archives/carinfo/locate/list?page={page}&size=12&search_startTime={search_startTime}&search_endTime={search_endTime}&_={number}\"\n",
|
||||
" attempt = 0\n",
|
||||
" while attempt < max_retries:\n",
|
||||
" try:\n",
|
||||
" response = requests.get(url=url, headers=headers)\n",
|
||||
" response.raise_for_status() # 如果响应状态码不是200，则抛出HTTPError异常\n",
|
||||
" res = response.json().get(\"data\", {}).get(\"content\", [])\n",
|
||||
" max_len = response.json().get(\"data\", {}).get(\"totalPages\", []) + 1\n",
|
||||
" if page == max_len:\n",
|
||||
" print(max_len)\n",
|
||||
" exit_loop = True\n",
|
||||
" break # 成功后退出重试循环\n",
|
||||
" for item in res:\n",
|
||||
" base_info = {k: v for k, v in item.items() if k != \"licenseList\"}\n",
|
||||
" all_data.append(base_info)\n",
|
||||
"\n",
|
||||
" break # 成功后退出重试循环\n",
|
||||
" except requests.exceptions.RequestException as e:\n",
|
||||
" print(f\"第{page}页请求失败: {e}\")\n",
|
||||
" if attempt < max_retries - 1:\n",
|
||||
" sleep_time = backoff_factor * (2 ** attempt) # 计算等待时间\n",
|
||||
" print(f\"准备第{attempt + 1}次重试，等待{sleep_time}秒...\")\n",
|
||||
" time.sleep(sleep_time) # 等待一段时间后重试\n",
|
||||
" else:\n",
|
||||
" print(f\"达到最大重试次数，跳过第{page}页\")\n",
|
||||
" attempt += 1\n",
|
||||
"\n",
|
||||
" number += 1\n",
|
||||
" time.sleep(1)\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-15202993999 车辆.csv\")\n"
|
||||
],
|
||||
"id": "335a10b919a1f6b5",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2024-07-25, search_endTime: 2025-07-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<25:03, 1.50s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"2\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 2/1000 [00:02<22:00, 1.32s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2023-07-26, search_endTime: 2024-07-25\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2%|▏ | 20/1000 [00:31<24:34, 1.50s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"21\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2%|▏ | 21/1000 [00:32<25:15, 1.55s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2022-07-26, search_endTime: 2023-07-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 22%|██▏ | 216/1000 [05:25<20:51, 1.60s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"第217页请求失败: HTTPSConnectionPool(host='www.yunxiu.com', port=443): Max retries exceeded with url: /legend/archives/carinfo/locate/list?page=217&size=12&search_startTime=2022-07-26&search_endTime=2023-07-26&_=1746510018387 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000027CA36597C0>, 'Connection to www.yunxiu.com timed out. (connect timeout=None)'))\n",
|
||||
"准备第1次重试，等待1秒...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 23%|██▎ | 226/1000 [06:02<22:23, 1.74s/it] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"227\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 23%|██▎ | 227/1000 [06:03<20:37, 1.60s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2021-07-26, search_endTime: 2022-07-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 25%|██▍ | 248/1000 [06:09<18:34, 1.48s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"249\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 25%|██▍ | 249/1000 [06:10<18:36, 1.49s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2020-07-26, search_endTime: 2021-07-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 33%|███▎ | 333/1000 [08:16<16:34, 1.49s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"334\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 33%|███▎ | 334/1000 [08:17<16:32, 1.49s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2019-07-27, search_endTime: 2020-07-26\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 3%|▎ | 29/1000 [00:42<23:50, 1.47s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"30\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 3%|▎ | 30/1000 [00:43<23:37, 1.46s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2018-07-27, search_endTime: 2019-07-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/1000 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:02, 1.14s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2017-07-27, search_endTime: 2018-07-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/1000 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<20:22, 1.22s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2016-07-27, search_endTime: 2017-07-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<25:47, 1.55s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"2\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 2/1000 [00:02<22:49, 1.37s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2015-07-28, search_endTime: 2016-07-27\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<24:04, 1.45s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"2\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 2/1000 [00:02<21:36, 1.30s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"search_startTime: 2014-07-28, search_endTime: 2015-07-28\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/1000 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 1/1000 [00:01<19:09, 1.15s/it]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-15T08:31:40.647176Z",
|
||||
"start_time": "2025-05-15T08:31:40.210149Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\京东云修数据导出-13032256777 -车辆.csv\")"
|
||||
],
|
||||
"id": "e45e8757ed4ca425",
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "NameError",
|
||||
"evalue": "name 'pd' is not defined",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mNameError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[1], line 1\u001B[0m\n\u001B[1;32m----> 1\u001B[0m df1 \u001B[38;5;241m=\u001B[39m pd\u001B[38;5;241m.\u001B[39mDataFrame(all_data)\n\u001B[0;32m 2\u001B[0m df1\u001B[38;5;241m.\u001B[39mto_csv(\u001B[38;5;124mr\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mD:\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mIdea Project\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mF6+宜搭+其它(1)\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mnew\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m文件输出\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m京东云修数据导出-13032256777 -车辆.csv\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n",
|
||||
"\u001B[1;31mNameError\u001B[0m: name 'pd' is not defined"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "",
|
||||
"id": "bf951aee0e74b1d1"
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "新单据信息",
|
||||
"id": "eb09e38f6613b5e7"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-28T06:31:17.385432200Z",
|
||||
"start_time": "2026-01-28T06:31:16.760396700Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" '__jdv': '241039512|direct|-|none|-|1769580195196',\n",
|
||||
" 'mba_muid': '17695801951961847696270',\n",
|
||||
" 'mba_sid': '1769580378119515622883.0',\n",
|
||||
" 'QRCodeKey': 'AAEAIGlfy0S9IVwOVAfE3NXgBILRP8R2qpHAikvosD3cyvar',\n",
|
||||
" 'wlfstk_smdl': 'drd3j1dlvwjz7o96orgqco4keigo6ipj',\n",
|
||||
" 'pin': '%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8',\n",
|
||||
" 'unick': 's2v2tk1w4w8eb9',\n",
|
||||
" 'yunxiupin': '%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8',\n",
|
||||
" 'bmall_id': '499220987560FEF36C9FB4C092693FFC98D6D74FB85924A6DC3E030ADD3477238C68C7B8E10ED66898B6EBEB4562ED893FBF19E4CD0030FECE3218317F39A2F5FC9FF25A11561368D02C95564D247AC2A151A086B1074D5EE362C7CB1BA34EC79B50B3252535972791A084C5D010158EF4B5C1EADC2808FFF3EB992EF34A9D14',\n",
|
||||
" 'UUID': '7c069d6f-2724-4008-b308-09f436240035',\n",
|
||||
" 'SESSION_USER_NAME': '%E6%9F%A5%E4%BA%AE',\n",
|
||||
" 'flash': '3_bBBCg0-OBy68xAANqduCSaKWV_6DKpxvObl29hEy-5Il0taBIimdGJfimCMZHKH-CUR09cTQwjAyGRKP9z9jHkm9UrWP2nTOg38vb68OEMqiiwdZRg1AqxMtUmDSWBQdVhGg5qQSbO5p1eyeBPlu7Htsgtf_TMouEaoqlijt7PqsLs4yBBM8Alki49TorJwr9E6mcmF96X-2O-E6',\n",
|
||||
" 'thor': '51C732866E8A756A4ED8B4738163D4C059D9063322E05745BA0D815A80FEBE3D550B447E8BB3C99207EE02D994108998DE070A571A758BED83B7535090CB6C2DF853F8994511D1708463D0C16274A620668FCE84D3B708BA56633C46432FABC8B4DA7688141595BA1C45C419160C151A638E6A118D22DA606EE420BE383B9D53FB842C2C90A6454E4CFED4E3A63A8ED5',\n",
|
||||
" 'light_key': 'AASBKE7rOxgWQziEhC_QY6yaLIBQgOT1dxVmsPRjRIB9ndtR9Aslstb1iQBYYZyMvLEJNsNDL52e7UA1dFCeHNIVWAseFg',\n",
|
||||
" 'JD_UUID': 'b9441bd1-a782-4d34-b2af-47b6f1e89bc6',\n",
|
||||
" '3AB9D23F7A4B3CSS': 'jdd03XH7UJPD6AWWSVFN6XP6T23OSWBA235SAMFFYPEW3JTTJRTT4TQTZLIAKR47DQ26GKGT266NM6WGBCEKT7R6RL2K34IAAAAM4ANDUGQYAAAAACSHCLFATJTAJKMX',\n",
|
||||
" 'sdtoken': 'AAbEsBpEIOVjqTAKCQtvQu17FLUpUb4CM0QtRGo9_y1DbK8h44YEOS2UiemwosOOgSgqPwHoDiJho0E69o2zgRTqSGouwgzZW2ByDMgJdY2S_IVvi4uyNneMFom6TCFDsAUG_C0BzQYsVWWU',\n",
|
||||
" '__jda': '200816309.17695801951961847696270.1769580195.1769580195.1769580195.1',\n",
|
||||
" '__jdc': '200816309',\n",
|
||||
" '3AB9D23F7A4B3C9B': 'KPOAZ5CBYQXJ25JOSSXGBQDYYOVGX7EMTPWFWDGVZ3KWTLNZU45HHPQ4MXYSIO37IV72PLAND3VYJH2ZOLD6LFWBDA',\n",
|
||||
" '__jdb': '200816309.56.17695801951961847696270|1.1769580195',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'origin': 'https://jch.yunxiu.com',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://jch.yunxiu.com/',\n",
|
||||
" 'sec-ch-ua': '\"Not(A:Brand\";v=\"8\", \"Chromium\";v=\"144\", \"Microsoft Edge\";v=\"144\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-site',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0',\n",
|
||||
" 'x-requested-with': 'XMLHttpRequest',\n",
|
||||
" # 'cookie': '__jdv=241039512|direct|-|none|-|1769580195196; mba_muid=17695801951961847696270; mba_sid=1769580378119515622883.0; QRCodeKey=AAEAIGlfy0S9IVwOVAfE3NXgBILRP8R2qpHAikvosD3cyvar; wlfstk_smdl=drd3j1dlvwjz7o96orgqco4keigo6ipj; pin=%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8; unick=s2v2tk1w4w8eb9; yunxiupin=%E5%8D%97%E6%98%8C%E4%BD%B3%E4%B9%BE%E8%BD%A6%E6%9C%8D%E6%9C%89%E9%99%90%E5%85%AC%E5%8F%B8; bmall_id=499220987560FEF36C9FB4C092693FFC98D6D74FB85924A6DC3E030ADD3477238C68C7B8E10ED66898B6EBEB4562ED893FBF19E4CD0030FECE3218317F39A2F5FC9FF25A11561368D02C95564D247AC2A151A086B1074D5EE362C7CB1BA34EC79B50B3252535972791A084C5D010158EF4B5C1EADC2808FFF3EB992EF34A9D14; UUID=7c069d6f-2724-4008-b308-09f436240035; SESSION_USER_NAME=%E6%9F%A5%E4%BA%AE; flash=3_bBBCg0-OBy68xAANqduCSaKWV_6DKpxvObl29hEy-5Il0taBIimdGJfimCMZHKH-CUR09cTQwjAyGRKP9z9jHkm9UrWP2nTOg38vb68OEMqiiwdZRg1AqxMtUmDSWBQdVhGg5qQSbO5p1eyeBPlu7Htsgtf_TMouEaoqlijt7PqsLs4yBBM8Alki49TorJwr9E6mcmF96X-2O-E6; thor=51C732866E8A756A4ED8B4738163D4C059D9063322E05745BA0D815A80FEBE3D550B447E8BB3C99207EE02D994108998DE070A571A758BED83B7535090CB6C2DF853F8994511D1708463D0C16274A620668FCE84D3B708BA56633C46432FABC8B4DA7688141595BA1C45C419160C151A638E6A118D22DA606EE420BE383B9D53FB842C2C90A6454E4CFED4E3A63A8ED5; light_key=AASBKE7rOxgWQziEhC_QY6yaLIBQgOT1dxVmsPRjRIB9ndtR9Aslstb1iQBYYZyMvLEJNsNDL52e7UA1dFCeHNIVWAseFg; JD_UUID=b9441bd1-a782-4d34-b2af-47b6f1e89bc6; 3AB9D23F7A4B3CSS=jdd03XH7UJPD6AWWSVFN6XP6T23OSWBA235SAMFFYPEW3JTTJRTT4TQTZLIAKR47DQ26GKGT266NM6WGBCEKT7R6RL2K34IAAAAM4ANDUGQYAAAAACSHCLFATJTAJKMX; sdtoken=AAbEsBpEIOVjqTAKCQtvQu17FLUpUb4CM0QtRGo9_y1DbK8h44YEOS2UiemwosOOgSgqPwHoDiJho0E69o2zgRTqSGouwgzZW2ByDMgJdY2S_IVvi4uyNneMFom6TCFDsAUG_C0BzQYsVWWU; __jda=200816309.17695801951961847696270.1769580195.1769580195.1769580195.1; __jdc=200816309; 3AB9D23F7A4B3C9B=KPOAZ5CBYQXJ25JOSSXGBQDYYOVGX7EMTPWFWDGVZ3KWTLNZU45HHPQ4MXYSIO37IV72PLAND3VYJH2ZOLD6LFWBDA; __jdb=200816309.56.17695801951961847696270|1.1769580195',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"params = {\n",
|
||||
" 'page': '1',\n",
|
||||
" 'size': '12',\n",
|
||||
" 'btnType': 'search',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"response = requests.get('https://g.yunxiu.com/megatron/api/v1/order/list', params=params, cookies=cookies, headers=headers)\n"
|
||||
],
|
||||
"id": "ca85268200bda895",
|
||||
"outputs": [],
|
||||
"execution_count": 2
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-28T06:35:23.307514700Z",
|
||||
"start_time": "2026-01-28T06:35:23.261598100Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"response.json()\n",
|
||||
"content = response.json()['data'].get(\"content\")\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"df = pd.DataFrame(content)\n",
|
||||
"df.to_csv(\"京东云修单据信息demo.csv\")"
|
||||
],
|
||||
"id": "b18cdb2c1fb49e75",
|
||||
"outputs": [],
|
||||
"execution_count": 5
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 会员卡明细导出",
|
||||
"id": "1b14f94c5e74163b"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-14T06:45:24.475273Z",
|
||||
"start_time": "2025-05-14T06:44:16.062351Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"url = f\"https://www.aibiyme.com/base-admin/memberCard/selectUserCardList\"\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': '*/*',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate, br, zstd',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Length': '21', # 根据实际情况调整Content-Length\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded',\n",
|
||||
" 'Cookie': 'sid=e8ca5702-f938-437e-a37b-368088cdaff4',\n",
|
||||
" 'Host': 'www.aibiyme.com',\n",
|
||||
" 'Origin': 'https://www.aibiyme.com',\n",
|
||||
" 'Referer': 'https://www.aibiyme.com/web-admin/erp/procurement.html',\n",
|
||||
" 'Sec-Ch-Ua': '\"Chromium\";v=\"136\", \"Microsoft Edge\";v=\"136\", \"Not.A/Brand\";v=\"99\"',\n",
|
||||
" 'Sec-Ch-Ua-Mobile': '?0',\n",
|
||||
" 'Sec-Ch-Ua-Platform': '\"Windows\"',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest'\n",
|
||||
"}\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"data = \"pageSize=10&pageNum=1\"\n",
|
||||
"\n",
|
||||
"# 获取总记录数\n",
|
||||
"f_res = requests.post(url, data=data, headers=headers)\n",
|
||||
"f_res.raise_for_status() # 检查请求是否成功\n",
|
||||
"total = f_res.json().get(\"total\", 0)\n",
|
||||
"print(total)\n",
|
||||
"\n",
|
||||
"for i in tqdm(range(1, total + 1)):\n",
|
||||
" data = f\"pageSize=10&pageNum={i}\"\n",
|
||||
" \n",
|
||||
" res = requests.post(url, data=data, headers=headers)\n",
|
||||
" res.raise_for_status()\n",
|
||||
"\n",
|
||||
" data_list = res.json().get(\"obj\", {}).get(\"pageData\", [])\n",
|
||||
"\n",
|
||||
" for item in data_list:\n",
|
||||
" pk = item.get(\"pk\")\n",
|
||||
" pk = int(pk)\n",
|
||||
" if pk:\n",
|
||||
" # 获取详细信息\n",
|
||||
" pk_data = f\"cardPk={pk}\"\n",
|
||||
" \n",
|
||||
" detail_url = \"https://www.aibiyme.com/base-admin/memberCard/getCardInfoByPk\"\n",
|
||||
" pk_res = requests.post(detail_url, data=pk_data, headers=headers)\n",
|
||||
" pk_res.raise_for_status()\n",
|
||||
"\n",
|
||||
" detail_data = pk_res.json().get(\"obj\", {})\n",
|
||||
" item.update(detail_data)\n",
|
||||
"\n",
|
||||
" # 处理卡片明细\n",
|
||||
" card_details = detail_data.get(\"cardItemList\", [])\n",
|
||||
" for card_detail in card_details:\n",
|
||||
" item_copy = item.copy() # 创建副本避免修改原始数据\n",
|
||||
" item_copy.update(card_detail)\n",
|
||||
" all_data.append(item_copy)\n",
|
||||
"\n",
|
||||
"# 保存数据\n",
|
||||
"if all_data:\n",
|
||||
" pd.DataFrame(all_data).to_csv(\"./new/文件输出/会员卡明细.csv\", index=False)\n",
|
||||
"else:\n",
|
||||
" print(\"没有获取到任何数据\")\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"22\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 22/22 [01:06<00:00, 3.04s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "OSError",
|
||||
"evalue": "Cannot save file into a non-existent directory: 'new\\文件输出'",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mOSError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[34], line 70\u001B[0m\n\u001B[0;32m 68\u001B[0m \u001B[38;5;66;03m# 保存数据\u001B[39;00m\n\u001B[0;32m 69\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m all_data:\n\u001B[1;32m---> 70\u001B[0m pd\u001B[38;5;241m.\u001B[39mDataFrame(all_data)\u001B[38;5;241m.\u001B[39mto_csv(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m./new/文件输出/会员卡明细.csv\u001B[39m\u001B[38;5;124m\"\u001B[39m, index\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mFalse\u001B[39;00m)\n\u001B[0;32m 71\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n\u001B[0;32m 72\u001B[0m \u001B[38;5;28mprint\u001B[39m(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m没有获取到任何数据\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\util\\_decorators.py:333\u001B[0m, in \u001B[0;36mdeprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper\u001B[1;34m(*args, **kwargs)\u001B[0m\n\u001B[0;32m 327\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28mlen\u001B[39m(args) \u001B[38;5;241m>\u001B[39m num_allow_args:\n\u001B[0;32m 328\u001B[0m warnings\u001B[38;5;241m.\u001B[39mwarn(\n\u001B[0;32m 329\u001B[0m msg\u001B[38;5;241m.\u001B[39mformat(arguments\u001B[38;5;241m=\u001B[39m_format_argument_list(allow_args)),\n\u001B[0;32m 330\u001B[0m \u001B[38;5;167;01mFutureWarning\u001B[39;00m,\n\u001B[0;32m 331\u001B[0m stacklevel\u001B[38;5;241m=\u001B[39mfind_stack_level(),\n\u001B[0;32m 332\u001B[0m )\n\u001B[1;32m--> 333\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m func(\u001B[38;5;241m*\u001B[39margs, \u001B[38;5;241m*\u001B[39m\u001B[38;5;241m*\u001B[39mkwargs)\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\core\\generic.py:3967\u001B[0m, in \u001B[0;36mNDFrame.to_csv\u001B[1;34m(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, compression, quoting, quotechar, lineterminator, chunksize, date_format, doublequote, escapechar, decimal, errors, storage_options)\u001B[0m\n\u001B[0;32m 3956\u001B[0m df \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(\u001B[38;5;28mself\u001B[39m, ABCDataFrame) \u001B[38;5;28;01melse\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mto_frame()\n\u001B[0;32m 3958\u001B[0m formatter \u001B[38;5;241m=\u001B[39m DataFrameFormatter(\n\u001B[0;32m 3959\u001B[0m frame\u001B[38;5;241m=\u001B[39mdf,\n\u001B[0;32m 3960\u001B[0m header\u001B[38;5;241m=\u001B[39mheader,\n\u001B[1;32m (...)\u001B[0m\n\u001B[0;32m 3964\u001B[0m decimal\u001B[38;5;241m=\u001B[39mdecimal,\n\u001B[0;32m 3965\u001B[0m )\n\u001B[1;32m-> 3967\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m DataFrameRenderer(formatter)\u001B[38;5;241m.\u001B[39mto_csv(\n\u001B[0;32m 3968\u001B[0m path_or_buf,\n\u001B[0;32m 3969\u001B[0m lineterminator\u001B[38;5;241m=\u001B[39mlineterminator,\n\u001B[0;32m 3970\u001B[0m sep\u001B[38;5;241m=\u001B[39msep,\n\u001B[0;32m 3971\u001B[0m encoding\u001B[38;5;241m=\u001B[39mencoding,\n\u001B[0;32m 3972\u001B[0m errors\u001B[38;5;241m=\u001B[39merrors,\n\u001B[0;32m 3973\u001B[0m compression\u001B[38;5;241m=\u001B[39mcompression,\n\u001B[0;32m 3974\u001B[0m quoting\u001B[38;5;241m=\u001B[39mquoting,\n\u001B[0;32m 3975\u001B[0m columns\u001B[38;5;241m=\u001B[39mcolumns,\n\u001B[0;32m 3976\u001B[0m index_label\u001B[38;5;241m=\u001B[39mindex_label,\n\u001B[0;32m 3977\u001B[0m mode\u001B[38;5;241m=\u001B[39mmode,\n\u001B[0;32m 3978\u001B[0m chunksize\u001B[38;5;241m=\u001B[39mchunksize,\n\u001B[0;32m 3979\u001B[0m quotechar\u001B[38;5;241m=\u001B[39mquotechar,\n\u001B[0;32m 3980\u001B[0m date_format\u001B[38;5;241m=\u001B[39mdate_format,\n\u001B[0;32m 3981\u001B[0m doublequote\u001B[38;5;241m=\u001B[39mdoublequote,\n\u001B[0;32m 3982\u001B[0m escapechar\u001B[38;5;241m=\u001B[39mescapechar,\n\u001B[0;32m 3983\u001B[0m storage_options\u001B[38;5;241m=\u001B[39mstorage_options,\n\u001B[0;32m 3984\u001B[0m )\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\io\\formats\\format.py:1014\u001B[0m, in \u001B[0;36mDataFrameRenderer.to_csv\u001B[1;34m(self, path_or_buf, encoding, sep, columns, index_label, mode, compression, quoting, quotechar, lineterminator, chunksize, date_format, doublequote, escapechar, errors, storage_options)\u001B[0m\n\u001B[0;32m 993\u001B[0m created_buffer \u001B[38;5;241m=\u001B[39m \u001B[38;5;28;01mFalse\u001B[39;00m\n\u001B[0;32m 995\u001B[0m csv_formatter \u001B[38;5;241m=\u001B[39m CSVFormatter(\n\u001B[0;32m 996\u001B[0m path_or_buf\u001B[38;5;241m=\u001B[39mpath_or_buf,\n\u001B[0;32m 997\u001B[0m lineterminator\u001B[38;5;241m=\u001B[39mlineterminator,\n\u001B[1;32m (...)\u001B[0m\n\u001B[0;32m 1012\u001B[0m formatter\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mfmt,\n\u001B[0;32m 1013\u001B[0m )\n\u001B[1;32m-> 1014\u001B[0m csv_formatter\u001B[38;5;241m.\u001B[39msave()\n\u001B[0;32m 1016\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m created_buffer:\n\u001B[0;32m 1017\u001B[0m \u001B[38;5;28;01massert\u001B[39;00m \u001B[38;5;28misinstance\u001B[39m(path_or_buf, StringIO)\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\io\\formats\\csvs.py:251\u001B[0m, in \u001B[0;36mCSVFormatter.save\u001B[1;34m(self)\u001B[0m\n\u001B[0;32m 247\u001B[0m \u001B[38;5;250m\u001B[39m\u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m 248\u001B[0m \u001B[38;5;124;03mCreate the writer & save.\u001B[39;00m\n\u001B[0;32m 249\u001B[0m \u001B[38;5;124;03m\"\"\"\u001B[39;00m\n\u001B[0;32m 250\u001B[0m \u001B[38;5;66;03m# apply compression and byte/text conversion\u001B[39;00m\n\u001B[1;32m--> 251\u001B[0m \u001B[38;5;28;01mwith\u001B[39;00m get_handle(\n\u001B[0;32m 252\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mfilepath_or_buffer,\n\u001B[0;32m 253\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mmode,\n\u001B[0;32m 254\u001B[0m encoding\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mencoding,\n\u001B[0;32m 255\u001B[0m errors\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39merrors,\n\u001B[0;32m 256\u001B[0m compression\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mcompression,\n\u001B[0;32m 257\u001B[0m storage_options\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mstorage_options,\n\u001B[0;32m 258\u001B[0m ) \u001B[38;5;28;01mas\u001B[39;00m handles:\n\u001B[0;32m 259\u001B[0m \u001B[38;5;66;03m# Note: self.encoding is irrelevant here\u001B[39;00m\n\u001B[0;32m 260\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mwriter \u001B[38;5;241m=\u001B[39m csvlib\u001B[38;5;241m.\u001B[39mwriter(\n\u001B[0;32m 261\u001B[0m handles\u001B[38;5;241m.\u001B[39mhandle,\n\u001B[0;32m 262\u001B[0m lineterminator\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mlineterminator,\n\u001B[1;32m (...)\u001B[0m\n\u001B[0;32m 267\u001B[0m quotechar\u001B[38;5;241m=\u001B[39m\u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mquotechar,\n\u001B[0;32m 268\u001B[0m )\n\u001B[0;32m 270\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_save()\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\io\\common.py:749\u001B[0m, in \u001B[0;36mget_handle\u001B[1;34m(path_or_buf, mode, encoding, compression, memory_map, is_text, errors, storage_options)\u001B[0m\n\u001B[0;32m 747\u001B[0m \u001B[38;5;66;03m# Only for write methods\u001B[39;00m\n\u001B[0;32m 748\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mr\u001B[39m\u001B[38;5;124m\"\u001B[39m \u001B[38;5;129;01mnot\u001B[39;00m \u001B[38;5;129;01min\u001B[39;00m mode \u001B[38;5;129;01mand\u001B[39;00m is_path:\n\u001B[1;32m--> 749\u001B[0m check_parent_directory(\u001B[38;5;28mstr\u001B[39m(handle))\n\u001B[0;32m 751\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m compression:\n\u001B[0;32m 752\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m compression \u001B[38;5;241m!=\u001B[39m \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mzstd\u001B[39m\u001B[38;5;124m\"\u001B[39m:\n\u001B[0;32m 753\u001B[0m \u001B[38;5;66;03m# compression libraries do not like an explicit text-mode\u001B[39;00m\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\pandas\\io\\common.py:616\u001B[0m, in \u001B[0;36mcheck_parent_directory\u001B[1;34m(path)\u001B[0m\n\u001B[0;32m 614\u001B[0m parent \u001B[38;5;241m=\u001B[39m Path(path)\u001B[38;5;241m.\u001B[39mparent\n\u001B[0;32m 615\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m \u001B[38;5;129;01mnot\u001B[39;00m parent\u001B[38;5;241m.\u001B[39mis_dir():\n\u001B[1;32m--> 616\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m \u001B[38;5;167;01mOSError\u001B[39;00m(\u001B[38;5;124mrf\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mCannot save file into a non-existent directory: \u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mparent\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n",
|
||||
"\u001B[1;31mOSError\u001B[0m: Cannot save file into a non-existent directory: 'new\\文件输出'"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 34
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-14T06:47:28.647625Z",
|
||||
"start_time": "2025-05-14T06:47:28.625811Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"# 保存数据\n",
|
||||
"if all_data:\n",
|
||||
" pd.DataFrame(all_data).to_csv(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\会员卡明细.csv\", index=False)\n",
|
||||
"else:\n",
|
||||
" print(\"没有获取到任何数据\")"
|
||||
],
|
||||
"id": "584f989c494205d5",
|
||||
"outputs": [],
|
||||
"execution_count": 40
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,77 @@
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.support.ui import WebDriverWait
|
||||
from selenium.webdriver.support import expected_conditions as EC
|
||||
import time
|
||||
from urllib.parse import urljoin
|
||||
import pandas as pd
|
||||
from selenium.webdriver import Chrome
|
||||
from selenium.webdriver.chrome.service import Service
|
||||
from datetime import datetime
|
||||
from selenium.webdriver.chrome.options import Options
|
||||
from datetime import datetime, timedelta
|
||||
from selenium.common.exceptions import NoSuchElementException
|
||||
from tqdm import tqdm
|
||||
from selenium.common.exceptions import TimeoutException
|
||||
|
||||
# 设置Chrome选项
|
||||
chrome_options = Options()
|
||||
# 设置为无头模式（不打开浏览器窗口）
|
||||
# chrome_options.add_argument('--headless')
|
||||
chrome_options.add_argument('--disable-gpu')
|
||||
chrome_options.add_argument('--no-sandbox')
|
||||
|
||||
# 指定ChromeDriver路径
|
||||
service = Service(executable_path='D:\ProgramTools\chromedriver-win64\chromedriver.exe')
|
||||
|
||||
# 创建WebDriver对象
|
||||
driver = webdriver.Chrome(service=service, options=chrome_options)
|
||||
|
||||
# 目标网址
|
||||
url = 'https://ebc.cloud.harsonserver.com/#/user/login?redirect=/'
|
||||
url1 = ''
|
||||
username = '193WSX'
|
||||
password = '193WSX'
|
||||
# title = 'HGYH -- HGYH'
|
||||
|
||||
# 访问网页
|
||||
driver.get(url)
|
||||
|
||||
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id="login-container"]/div[4]/div[3]/div'))).click()
|
||||
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.element_to_be_clickable((By.ID, "form_item_loginAccount"))
|
||||
).send_keys(username)
|
||||
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.element_to_be_clickable((By.ID, "form_item_password"))
|
||||
).send_keys(password)
|
||||
time.sleep(2)
|
||||
|
||||
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id="login-container"]/div[4]/div[2]/form/div[5]/div/div/div/button'))).click()
|
||||
|
||||
|
||||
time.sleep(3)
|
||||
|
||||
new_url = "https://ebc.cloud.harsonserver.com/#/maintenance/frontManage/maintenanceReport/workOrderReport"
|
||||
driver.get(new_url)
|
||||
|
||||
start_date = "2025-09-22"
|
||||
end_date = "2025-11-22"
|
||||
|
||||
# 等待日期选择器容器出现
|
||||
date_picker = WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.CSS_SELECTOR, ".ant-picker-range"))
|
||||
)
|
||||
|
||||
# 定位两个 input（根据 placeholder）
|
||||
start_input = date_picker.find_element(By.XPATH, './/input[@placeholder="开始时间"]')
|
||||
end_input = date_picker.find_element(By.XPATH, './/input[@placeholder="结束时间"]')
|
||||
|
||||
# 使用 JavaScript 设置值并触发事件
|
||||
driver.execute_script(f"arguments[0].value = '{start_date}'; arguments[0].dispatchEvent(new Event('input'));", start_input)
|
||||
driver.execute_script(f"arguments[0].value = '{end_date}'; arguments[0].dispatchEvent(new Event('input'));", end_input)
|
||||
|
||||
# 可选：再触发 change 事件（某些框架需要）
|
||||
driver.execute_script("arguments[0].dispatchEvent(new Event('change'));", start_input)
|
||||
driver.execute_script("arguments[0].dispatchEvent(new Event('change'));", end_input)
|
||||
@@ -0,0 +1,41 @@
|
||||
import pandas as pd
|
||||
import requests
|
||||
from tqdm import tqdm
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
all_data_list = []
|
||||
keys = ["门店","订单编号","产品类型","产品名称","型号","数量","采购单价","销售单价",
|
||||
"采购金额","销售金额","毛利润","下单人","接单人","车牌号"]
|
||||
reversed_keys = list(reversed(keys))
|
||||
|
||||
for i in tqdm(range(1, 2)):
|
||||
url = f"http://rp.chezizhu.com/ReportServer?_=1745552902755&__boxModel__=true&op=page_content&sessionID=27200&pn={i}&__fit__=false"
|
||||
|
||||
header = {
|
||||
"Referer": "http://rp.chezizhu.com/ReportServer?reportlet=czz/storage/prod_sale_dtl_jm.cpt&servicer_id=B9GC8C8oZpTAxmUR1YEp9q&hideexport=true&database=chezizhu_14",
|
||||
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0",
|
||||
"X-Requested-With": "XMLHttpRequest"
|
||||
}
|
||||
|
||||
res = requests.post(url, headers=header)
|
||||
soup = BeautifulSoup(res.text, "html.parser")
|
||||
|
||||
# 定位真正的数据行（跳过标题行）
|
||||
for tr in soup.select('tr:not(:has(th))'): # 排除包含表头的行
|
||||
tds = [td.get_text(strip=True) for td in tr.find_all("td")]
|
||||
reversed_tds = list(reversed(tds))
|
||||
|
||||
# 如果 reversed_tds 长度不够，填充空字符串
|
||||
if len(reversed_tds) < len(reversed_keys):
|
||||
reversed_tds += [""] * (len(reversed_keys) - len(reversed_tds))
|
||||
elif len(reversed_tds) > len(reversed_keys):
|
||||
reversed_tds = reversed_tds[:len(reversed_keys)]
|
||||
|
||||
# 构建字典（反转后的 keys 和 tds）
|
||||
row_data = dict(zip(reversed_keys, reversed_tds))
|
||||
all_data_list.append(row_data)
|
||||
|
||||
# break
|
||||
print(all_data_list)
|
||||
df = pd.DataFrame(all_data_list)
|
||||
df.to_excel(r'D:\Idea Project\F6+宜搭+其它(1)\new\文件输出\大唛云销售明细.xlsx', index=False)
|
||||
@@ -0,0 +1,410 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-15T03:14:27.994103Z",
|
||||
"start_time": "2025-05-15T02:03:56.503917Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import re\n",
|
||||
"import time\n",
|
||||
"\n",
|
||||
"timestamp_ms = int(time.time() * 1000)\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"from urllib.parse import urlencode, quote\n",
|
||||
"import time\n",
|
||||
"from requests.exceptions import RequestException\n",
|
||||
"\n",
|
||||
"# 读取Excel文件\n",
|
||||
"df = pd.read_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\大唛云订单明细 (2).xlsx\")\n",
|
||||
"\n",
|
||||
"# 定义要提取的基本字段列表\n",
|
||||
"BASE_FIELDS = [\n",
|
||||
" '保险公司', '保险到期', '年检日期', '车架号',\n",
|
||||
" '购车日期', '服务日期', '服务里程',\n",
|
||||
" '下次服务日期', '下次服务里程'\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# 创建两个DataFrame分别存储不同类型的数据\n",
|
||||
"base_info_df = df.copy() # 存储基本信息\n",
|
||||
"service_items_df = pd.DataFrame() # 存储消费项目明细\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def make_request_with_retry(url, headers, max_retries=10, retry_delay=1):\n",
|
||||
" \"\"\"带重试机制的请求函数\"\"\"\n",
|
||||
" for attempt in range(max_retries):\n",
|
||||
" try:\n",
|
||||
" res = requests.get(url=url, headers=headers, timeout=10)\n",
|
||||
" res.raise_for_status()\n",
|
||||
" return res\n",
|
||||
" except RequestException as e:\n",
|
||||
" if attempt == max_retries - 1:\n",
|
||||
" raise\n",
|
||||
" print(f\"请求失败，第 {attempt + 1} 次重试... 错误: {str(e)}\")\n",
|
||||
" time.sleep(retry_delay)\n",
|
||||
" return None\n",
|
||||
" # print(res.text)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def extract_session_id(html_content):\n",
|
||||
" soup = BeautifulSoup(html_content, \"html.parser\")\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"\n",
|
||||
" session_ids = set()\n",
|
||||
" for script in scripts:\n",
|
||||
" # 匹配注册语句中的sessionid\n",
|
||||
" register_match = re.search(r\"FR\\.SessionMgr\\.register\\('(\\d+)'\", script.text)\n",
|
||||
" if register_match:\n",
|
||||
" session_ids.add(register_match.group(1))\n",
|
||||
"\n",
|
||||
" # 匹配变量赋值中的sessionid\n",
|
||||
" current_match = re.search(r\"currentSessionID\\s*=\\s*'(\\d+)'\", script.text)\n",
|
||||
" if current_match:\n",
|
||||
" session_ids.add(current_match.group(1))\n",
|
||||
"\n",
|
||||
" return list(session_ids)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for index, row in tqdm(list(df.iterrows()), desc=\"处理订单\"):\n",
|
||||
" order_number = row[\"单号\"]\n",
|
||||
" success = False\n",
|
||||
" # print(f\"正在处理订单 {order_number}\")\n",
|
||||
"\n",
|
||||
" timestamp_ms = int(time.time() * 1000)\n",
|
||||
" for attempt in range(10): # 最大重试次数\n",
|
||||
" try:\n",
|
||||
" url = (\n",
|
||||
" \"http://rp.chezizhu.com/ReportServer?reportlet=/czz/order/order.cpt&__parameters__={%22_%22:%22\" + f\"{timestamp_ms}\" + \"%22,%22PINGJIA%22:[5b][5d],%22DISCOUNT_DESC%22:%22%22,%22B%22:%222011-05-01%22,%22LABELPLATE_NUMBER%22:%22%25E8%25BD%25A6%25E7%2589%258C%25E5%258F%25B7%22,%22LABELUSER_NAME%22:%22%25E5%25A7%2593%25E5%2590%258D%22,%22E%22:%222025-05-07%22,%22SERVICER%22:[5b][5d],%22CAR_NAME%22:%22%22,%22ORDER_SERVICE_TYPE%22:%22%22,%22order_number%22:%22\" + order_number + \"%22,%22USER_MOBILE%22:%22%22,%22LABELUSER_STATE%22:%22%25E6%2596%25B0%25E8%2580%2581%25E7%2594%25A8%25E6%2588%25B7%22,%22LABELCAR_NAME%22:%22%25E8%25BD%25A6%25E5%259E%258B%22,%22SERVICER_ID%22:%22683XGWqnyn66mE1oa8Gkzb%22,%22LABELORDER_SOURCE%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E6%259D%25A5%25E6%25BA%2590%22,%22LABELE%22:%22%25E8%2587%25B3%22,%22LABELSERVER%22:%22%25E9%2597%25A8%25E5%25BA%2597%22,%22LABELGONGDAN_STATUS%22:%22%25E5%25B7%25A5%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22LABELORDER_STATUS%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22ORDER_STATUS%22:[5b][5d],%22CMD%22:%22parameters_d%22,%22PAY_METHOD%22:%22%22,%22LABELUSER_MOBILE%22:%22%25E5%25AE%25A2%25E6%2588%25B7%25E7%2594%25B5%25E8%25AF%259D%22,%22LABELPINGJIA%22:%22%25E8%25AF%2584%25E4%25BB%25B7%25E7%258A%25B6%25E6%2580%2581%22,%22USER_STATE%22:[5b][5d],%22LABELRP_TYPE%22:%22%25E6%258A%25A5%25E8%25A1%25A8%25E7%25B1%25BB%25E5%259E%258B%22,%22LABELPAY_METHOD%22:%22%25E7%25BB%2593%25E7%25AE%2597%25E6%2596%25B9%25E5%25BC%258F%22,%22USER_NAME%22:%22%22,%22REPORTNAME%22:%22czz%252Forder%252Forder_detail_md.cpt%22,%22DISCOUNT_TYPE%22:%22%22,%22DATE_TYPE%22:%22%25E5%25AE%258C%25E6%2588%2590%25E6%2597%25A5%25E6%259C%259F%22,%22LABELDISCOUNT_DESC%22:%22%25E6%258A%2598%25E6%2589%25A3%25E5%258E%259F%25E5%259B%25A0%22,%22GONGDAN_STATUS%22:[5b][5d],%22__STREAMCLOSED__%22:%22%22,%22__pi__%22:%22true%22,%22CHANNEL%22:%22%22,%22DATABASE%22:%22chezizhu_14%22,%22ORDER_SOURCE%22:[5b][5d],%22LABELORDER_NUMBER%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E5%258F%25B7%22,%22PLATE_NUMBER%22:%22%22,%22RP_TYPE%22:%22%25E9%25AB%2598%25E7%25BA%25A7%25E6%2590%259C%25E7%25B4%25A2%22}\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" headers = {\n",
|
||||
" 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Cache-Control': 'max-age=0',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Cookie': 'td_cookie=2912605941',\n",
|
||||
" 'Host': 'rp.chezizhu.com',\n",
|
||||
" 'Referer': 'http://rp.chezizhu.com/ReportServer?reportlet=czz/order/order_detail_md.cpt&servicer_id=683XGWqnyn66mE1oa8Gkzb&database=chezizhu_14',\n",
|
||||
" 'Upgrade-Insecure-Requests': '1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0'\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" res = requests.get(url, headers=headers)\n",
|
||||
"\n",
|
||||
" session_ids = extract_session_id(res.text)\n",
|
||||
" # print(\"提取到的SessionID:\", session_ids[0])\n",
|
||||
" # ========== 请求部分 ==========\n",
|
||||
" url = f\"http://rp.chezizhu.com/ReportServer?_={timestamp_ms}&__boxModel__=true&op=page_content&sessionID={session_ids[0]}&pn=1&__fit__=false\"\n",
|
||||
"\n",
|
||||
" headers = {\n",
|
||||
" 'Accept': '*/*',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Cookie': 'td_cookie=2912605941',\n",
|
||||
" 'Host': 'rp.chezizhu.com',\n",
|
||||
" \"Referer\": \"http://rp.chezizhu.com/ReportServer?reportlet=/czz/order/order.cpt&__parameters__={%22_%22:%22\" + f\"{timestamp_ms}\" + \"%22,%22PINGJIA%22:[5b][5d],%22DISCOUNT_DESC%22:%22%22,%22B%22:%222021-05-01%22,%22LABELPLATE_NUMBER%22:%22%25E8%25BD%25A6%25E7%2589%258C%25E5%258F%25B7%22,%22LABELUSER_NAME%22:%22%25E5%25A7%2593%25E5%2590%258D%22,%22E%22:%222025-05-06%22,%22SERVICER%22:[5b][5d],%22CAR_NAME%22:%22%22,%22ORDER_SERVICE_TYPE%22:%22%22,%22order_number%22:%22\"\n",
|
||||
" + order_number +\n",
|
||||
" \"%22,%22USER_MOBILE%22:%22%22,%22LABELUSER_STATE%22:%22%25E6%2596%25B0%25E8%2580%2581%25E7%2594%25A8%25E6%2588%25B7%22,%22LABELCAR_NAME%22:%22%25E8%25BD%25A6%25E5%259E%258B%22,%22SERVICER_ID%22:%22683XGWqnyn66mE1oa8Gkzb%22,%22LABELORDER_SOURCE%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E6%259D%25A5%25E6%25BA%2590%22,%22LABELE%22:%22%25E8%2587%25B3%22,%22LABELSERVER%22:%22%25E9%2597%25A8%25E5%25BA%2597%22,%22LABELGONGDAN_STATUS%22:%22%25E5%25B7%25A5%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22LABELORDER_STATUS%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22ORDER_STATUS%22:[5b][5d],%22CMD%22:%22parameters_d%22,%22PAY_METHOD%22:%22%22,%22LABELUSER_MOBILE%22:%22%25E5%25AE%25A2%25E6%2588%25B7%25E7%2594%25B5%25E8%25AF%259D%22,%22LABELPINGJIA%22:%22%25E8%25AF%2584%25E4%25BB%25B7%25E7%258A%25B6%25E6%2580%2581%22,%22USER_STATE%22:[5b][5d],%22LABELRP_TYPE%22:%22%25E6%258A%25A5%25E8%25A1%25A8%25E7%25B1%25BB%25E5%259E%258B%22,%22LABELPAY_METHOD%22:%22%25E7%25BB%2593%25E7%25AE%2597%25E6%2596%25B9%25E5%25BC%258F%22,%22USER_NAME%22:%22%22,%22REPORTNAME%22:%22czz%252Forder%252Forder_detail_md.cpt%22,%22DISCOUNT_TYPE%22:%22%22,%22DATE_TYPE%22:%22%25E5%25AE%258C%25E6%2588%2590%25E6%2597%25A5%25E6%259C%259F%22,%22LABELDISCOUNT_DESC%22:%22%25E6%258A%2598%25E6%2589%25A3%25E5%258E%259F%25E5%259B%25A0%22,%22GONGDAN_STATUS%22:[5b][5d],%22__STREAMCLOSED__%22:%22%22,%22__pi__%22:%22true%22,%22CHANNEL%22:%22%22,%22DATABASE%22:%22chezizhu_14%22,%22ORDER_SOURCE%22:[5b][5d],%22LABELORDER_NUMBER%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E5%258F%25B7%22,%22PLATE_NUMBER%22:%22%22,%22RP_TYPE%22:%22%25E9%25AB%2598%25E7%25BA%25A7%25E6%2590%259C%25E7%25B4%25A2%22}\",\n",
|
||||
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0\",\n",
|
||||
" \"X-Requested-With\": \"XMLHttpRequest\"\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # 使用带重试的请求函数\n",
|
||||
" res = make_request_with_retry(url, headers)\n",
|
||||
" # print(res.text)\n",
|
||||
"\n",
|
||||
" soup = BeautifulSoup(res.text, 'lxml')\n",
|
||||
"\n",
|
||||
" # ========== 1. 提取基本信息（单行） ==========\n",
|
||||
" base_info = {'单号': order_number} # 保留原始订单号\n",
|
||||
"\n",
|
||||
" for field in BASE_FIELDS:\n",
|
||||
" field_td = soup.find('td', string=f'{field}：')\n",
|
||||
" if field_td:\n",
|
||||
" value_td = field_td.find_next('td')\n",
|
||||
" base_info[field] = value_td.get_text(strip=True) if value_td else None\n",
|
||||
" else:\n",
|
||||
" base_info[field] = None\n",
|
||||
"\n",
|
||||
" # 更新基本信息DataFrame\n",
|
||||
" for col in base_info:\n",
|
||||
" if col not in base_info_df.columns:\n",
|
||||
" base_info_df[col] = None\n",
|
||||
" base_info_df.at[index, col] = base_info[col]\n",
|
||||
"\n",
|
||||
" # ========== 2. 提取消费项目（多行） ==========\n",
|
||||
" # 初始化 last_item 用于保存上一行数据\n",
|
||||
" last_item = {}\n",
|
||||
" \n",
|
||||
" # 定义字段顺序（从后往前）\n",
|
||||
" field_order = [\n",
|
||||
" ('合计', 10),\n",
|
||||
" ('工时', 9),\n",
|
||||
" ('产品小计', 8),\n",
|
||||
" ('单价', 7),\n",
|
||||
" ('单位', 6),\n",
|
||||
" ('规格', 5),\n",
|
||||
" ('产品型号', 3),\n",
|
||||
" ('产品名称', 2),\n",
|
||||
" ('服务项目', 1),\n",
|
||||
" ('套餐', 0),\n",
|
||||
" ]\n",
|
||||
" \n",
|
||||
" # 设置不继承的字段集合\n",
|
||||
" do_not_inherit_fields = {\n",
|
||||
" '产品小计', '单价',\n",
|
||||
" '单位', '规格', '产品型号', '产品名称'\n",
|
||||
" }\n",
|
||||
" \n",
|
||||
" consumption_header = soup.find(lambda tag: tag.name == 'td' and '消费项目' in tag.text)\n",
|
||||
" if consumption_header:\n",
|
||||
" table = consumption_header.find_parent('table')\n",
|
||||
" if table:\n",
|
||||
" rows = table.find_all('tr')[1:] # 跳过表头\n",
|
||||
" \n",
|
||||
" for row_idx, row in enumerate(rows, 1):\n",
|
||||
" cols = row.find_all('td')\n",
|
||||
" \n",
|
||||
" if len(cols) < 7:\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" # 初始化当前 item\n",
|
||||
" item = {\n",
|
||||
" '单号': order_number,\n",
|
||||
" '行号': row_idx,\n",
|
||||
" }\n",
|
||||
" \n",
|
||||
" # 倒序填充字段\n",
|
||||
" for field_name, col_index in field_order:\n",
|
||||
" if col_index < len(cols):\n",
|
||||
" # 当前列存在，正常提取\n",
|
||||
" try:\n",
|
||||
" item[field_name] = cols[col_index].get_text(strip=True)\n",
|
||||
" except Exception:\n",
|
||||
" item[field_name] = ''\n",
|
||||
" else:\n",
|
||||
" # 列不存在，尝试从上一行继承（除非被禁止）\n",
|
||||
" if field_name in do_not_inherit_fields:\n",
|
||||
" item[field_name] = '' # 不允许继承，留空\n",
|
||||
" else:\n",
|
||||
" item[field_name] = last_item.get(field_name, '') # 可以继承\n",
|
||||
" \n",
|
||||
" # 更新 last_item\n",
|
||||
" last_item = item.copy()\n",
|
||||
" \n",
|
||||
" # 添加到 DataFrame\n",
|
||||
" service_items_df = pd.concat([\n",
|
||||
" service_items_df,\n",
|
||||
" pd.DataFrame([item])\n",
|
||||
" ], ignore_index=True)\n",
|
||||
" \n",
|
||||
" success = True\n",
|
||||
" break # 成功则跳出重试循环\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" \n",
|
||||
"\n",
|
||||
" except Exception as e:\n",
|
||||
" if attempt == 9: # 最后一次尝试仍然失败\n",
|
||||
" print(f\"订单 {order_number} 处理失败，已达最大重试次数。错误: {str(e)}\")\n",
|
||||
" # 记录失败订单（可选）\n",
|
||||
" with open('failed_orders.txt', 'a') as f:\n",
|
||||
" f.write(f\"{order_number}\\n\")\n",
|
||||
" time.sleep(1) # 等待1秒后重试\n",
|
||||
"\n",
|
||||
" if not success:\n",
|
||||
" continue # 跳过处理失败的订单\n",
|
||||
" # break\n",
|
||||
"\n",
|
||||
"# ========== 3. 合并数据并保存 ==========\n",
|
||||
"# 方法1：保存为两个关联表（推荐）\n",
|
||||
"base_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\订单基本信息.xlsx'\n",
|
||||
"items_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\消费项目明细.xlsx'\n",
|
||||
"base_info_df.to_excel(base_output, index=False)\n",
|
||||
"service_items_df.to_excel(items_output, index=False)\n",
|
||||
"\n",
|
||||
"# 方法2：合并为一个宽表（每行重复基本信息）\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" merged_df = pd.merge(\n",
|
||||
" base_info_df,\n",
|
||||
" service_items_df,\n",
|
||||
" on='单号',\n",
|
||||
" how='left'\n",
|
||||
" )\n",
|
||||
" merged_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\历史维修记录明细-15635606677.xlsx'\n",
|
||||
" merged_df.to_excel(merged_output, index=False)\n",
|
||||
"\n",
|
||||
"print(\"\\n处理完成！\")\n",
|
||||
"print(f\"- 基本信息保存至: {base_output}\")\n",
|
||||
"print(f\"- 消费项目保存至: {items_output}\")\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" print(f\"- 合并数据保存至: {merged_output}\") # 输出：['57435']\n",
|
||||
" "
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 0%| | 0/7946 [00:00<?, ?it/s]C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_12220\\1424788091.py:48: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"处理订单: 0%| | 1/7946 [00:00<24:49, 5.34it/s]C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_12220\\1424788091.py:48: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"处理订单: 31%|███ | 2441/7946 [12:04<34:15, 2.68it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"请求失败，第 1 次重试... 错误: HTTPConnectionPool(host='rp.chezizhu.com', port=80): Max retries exceeded with url: /ReportServer?_=1747275363838&__boxModel__=true&op=page_content&sessionID=7058&pn=1&__fit__=false (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000024EEAFAB080>, 'Connection to rp.chezizhu.com timed out. (connect timeout=10)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 36%|███▌ | 2851/7946 [15:04<36:00, 2.36it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"请求失败，第 1 次重试... 错误: HTTPConnectionPool(host='rp.chezizhu.com', port=80): Max retries exceeded with url: /ReportServer?_=1747275543553&__boxModel__=true&op=page_content&sessionID=55535&pn=1&__fit__=false (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000024EEE97B0E0>, 'Connection to rp.chezizhu.com timed out. (connect timeout=10)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 48%|████▊ | 3824/7946 [22:42<34:53, 1.97it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"请求失败，第 1 次重试... 错误: HTTPConnectionPool(host='rp.chezizhu.com', port=80): Max retries exceeded with url: /ReportServer?_=1747276001354&__boxModel__=true&op=page_content&sessionID=78866&pn=1&__fit__=false (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000024EE97A0A10>, 'Connection to rp.chezizhu.com timed out. (connect timeout=10)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 95%|█████████▌| 7578/7946 [1:03:49<04:32, 1.35it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"请求失败，第 1 次重试... 错误: HTTPConnectionPool(host='rp.chezizhu.com', port=80): Max retries exceeded with url: /ReportServer?_=1747278468348&__boxModel__=true&op=page_content&sessionID=24705&pn=1&__fit__=false (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000024EEE909580>, 'Connection to rp.chezizhu.com timed out. (connect timeout=10)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 100%|██████████| 7946/7946 [1:08:39<00:00, 1.93it/s]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"处理完成！\n",
|
||||
"- 基本信息保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\订单基本信息.xlsx\n",
|
||||
"- 消费项目保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\消费项目明细.xlsx\n",
|
||||
"- 合并数据保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\历史维修记录明细-15635606677.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 4
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-05-09T02:12:23.607378Z",
|
||||
"start_time": "2025-05-09T02:12:23.588256Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"# ========== 3. 合并数据并保存 ==========\n",
|
||||
"# 方法1：保存为两个关联表（推荐）\n",
|
||||
"base_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\订单基本信息.xlsx'\n",
|
||||
"items_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\消费项目明细.xlsx'\n",
|
||||
"base_info_df.to_excel(base_output, index=False)\n",
|
||||
"service_items_df.to_excel(items_output, index=False)\n",
|
||||
"\n",
|
||||
"# 方法2：合并为一个宽表（每行重复基本信息）\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" merged_df = pd.merge(\n",
|
||||
" base_info_df,\n",
|
||||
" service_items_df,\n",
|
||||
" on='单号',\n",
|
||||
" how='left'\n",
|
||||
" )\n",
|
||||
" merged_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\历史维修记录明细-古交腾飞路店13593189858.xlsx'\n",
|
||||
" merged_df.to_excel(merged_output, index=False)\n",
|
||||
"\n",
|
||||
"print(\"\\n处理完成！\")\n",
|
||||
"print(f\"- 基本信息保存至: {base_output}\")\n",
|
||||
"print(f\"- 消费项目保存至: {items_output}\")\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" print(f\"- 合并数据保存至: {merged_output}\") # 输出：['57435']\n",
|
||||
" "
|
||||
],
|
||||
"id": "2dad90815fd16898",
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "NameError",
|
||||
"evalue": "name 'base_info_df' is not defined",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mNameError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[3], line 5\u001B[0m\n\u001B[0;32m 3\u001B[0m base_output \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mr\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mD:\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mIdea Project\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mF6+宜搭+其它(1)\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mnew\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m文件输出\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m订单基本信息.xlsx\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[0;32m 4\u001B[0m items_output \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mr\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mD:\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mIdea Project\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mF6+宜搭+其它(1)\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124mnew\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m文件输出\u001B[39m\u001B[38;5;124m\\\u001B[39m\u001B[38;5;124m消费项目明细.xlsx\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m----> 5\u001B[0m base_info_df\u001B[38;5;241m.\u001B[39mto_excel(base_output, index\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mFalse\u001B[39;00m)\n\u001B[0;32m 6\u001B[0m service_items_df\u001B[38;5;241m.\u001B[39mto_excel(items_output, index\u001B[38;5;241m=\u001B[39m\u001B[38;5;28;01mFalse\u001B[39;00m)\n\u001B[0;32m 8\u001B[0m \u001B[38;5;66;03m# 方法2：合并为一个宽表（每行重复基本信息）\u001B[39;00m\n",
|
||||
"\u001B[1;31mNameError\u001B[0m: name 'base_info_df' is not defined"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,555 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 订单明细",
|
||||
"id": "8bdaf70d574d1868"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-20T09:45:36.313844400Z",
|
||||
"start_time": "2026-01-20T09:38:55.425173700Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"import time\n",
|
||||
"\n",
|
||||
"all_data_list = []\n",
|
||||
"for i in tqdm(range(1, 1303)):\n",
|
||||
" time.sleep(0.2)\n",
|
||||
" import requests\n",
|
||||
"\n",
|
||||
" headers = {\n",
|
||||
" 'Accept': 'text/html, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://rp.chezizhu.com/ReportServer?reportlet=czz/order/order_detail_md.cpt&servicer_id=HyBKFgc2uCNtqs59aDZhc2&database=chezizhu_14',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" params = {\n",
|
||||
" '_': '1768901546535',\n",
|
||||
" '__boxModel__': 'true',\n",
|
||||
" 'op': 'page_content',\n",
|
||||
" 'sessionID': '99282',\n",
|
||||
" 'pn': i,\n",
|
||||
" '__fit__': 'false',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" res = requests.get('http://rp.chezizhu.com/ReportServer', params=params, headers=headers, verify=False)\n",
|
||||
"\n",
|
||||
" # print(res.text)\n",
|
||||
"\n",
|
||||
" keys = [\n",
|
||||
" \"单号\", \"用料\", \"图片\", \"客户姓名\", \"客户电话\", \"车牌号码\", \"车型\",\n",
|
||||
" \"产品金额\", \"服务费\", \"上门费\", \"折扣金额\", \"折扣原因\", \"实收金额\",\n",
|
||||
" \"结算方式\", \"新老用户\", \"订单来源\", \"下单方式\", \"服务类型\", \"服务商\",\n",
|
||||
" \"技师\", \"下单时间\", \"预约时间\", \"完成时间\", \"评价状态\", \"工单状态\",\n",
|
||||
" \"订单状态\", \"全车检测\", \"备注\"\n",
|
||||
" ]\n",
|
||||
" soup = BeautifulSoup(res.text, 'lxml')\n",
|
||||
" for tr in soup.find_all('tr'):\n",
|
||||
" tds = tr.find_all('td')\n",
|
||||
" if \"单号\" in tds[0].text or \"总计\" in tds[0].text or \"订单列表\" in tds[\n",
|
||||
" 0].text or \"注：红色订单为实收金额有误订单\" in tds[0].text:\n",
|
||||
" continue\n",
|
||||
" td_list = []\n",
|
||||
" for td in tds:\n",
|
||||
" td_list.append(td.text)\n",
|
||||
"\n",
|
||||
" order_dict = dict(zip(keys, td_list))\n",
|
||||
" # print(order_dict)\n",
|
||||
" all_data_list.append(order_dict)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\大唛云订单明细-18135844455.xlsx', index=False)\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 1302/1302 [06:34<00:00, 3.30it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 6
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-20T09:34:49.479362100Z",
|
||||
"start_time": "2026-01-20T09:34:49.180981Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\大唛云订单明细-18135844455.xlsx', index=False)"
|
||||
],
|
||||
"id": "a848bd6b4137743e",
|
||||
"outputs": [],
|
||||
"execution_count": 4
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 历史维修记录",
|
||||
"id": "271853920f6210f"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-07-14T11:44:43.656887Z",
|
||||
"start_time": "2025-07-14T09:38:43.922512Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import re\n",
|
||||
"import time\n",
|
||||
"\n",
|
||||
"timestamp_ms = int(time.time() * 1000)\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"from urllib.parse import urlencode, quote\n",
|
||||
"import time\n",
|
||||
"from requests.exceptions import RequestException\n",
|
||||
"\n",
|
||||
"# 读取Excel文件\n",
|
||||
"df = pd.read_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\大唛云订单明细-18135844455.xlsx\")\n",
|
||||
"\n",
|
||||
"# 定义要提取的基本字段列表\n",
|
||||
"BASE_FIELDS = [\n",
|
||||
" '保险公司', '保险到期', '年检日期', '车架号',\n",
|
||||
" '购车日期', '服务日期', '服务里程',\n",
|
||||
" '下次服务日期', '下次服务里程'\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# 创建两个DataFrame分别存储不同类型的数据\n",
|
||||
"base_info_df = df.copy() # 存储基本信息\n",
|
||||
"service_items_df = pd.DataFrame() # 存储消费项目明细\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def make_request_with_retry(url, headers, max_retries=10, retry_delay=1):\n",
|
||||
" \"\"\"带重试机制的请求函数\"\"\"\n",
|
||||
" for attempt in range(max_retries):\n",
|
||||
" try:\n",
|
||||
" res = requests.get(url=url, headers=headers, timeout=10)\n",
|
||||
" res.raise_for_status()\n",
|
||||
" return res\n",
|
||||
" except RequestException as e:\n",
|
||||
" if attempt == max_retries - 1:\n",
|
||||
" raise\n",
|
||||
" print(f\"请求失败，第 {attempt + 1} 次重试... 错误: {str(e)}\")\n",
|
||||
" time.sleep(retry_delay)\n",
|
||||
" return None\n",
|
||||
" # print(res.text)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def extract_session_id(html_content):\n",
|
||||
" soup = BeautifulSoup(html_content, \"html.parser\")\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"\n",
|
||||
" session_ids = set()\n",
|
||||
" for script in scripts:\n",
|
||||
" # 匹配注册语句中的sessionid\n",
|
||||
" register_match = re.search(r\"FR\\.SessionMgr\\.register\\('(\\d+)'\", script.text)\n",
|
||||
" if register_match:\n",
|
||||
" session_ids.add(register_match.group(1))\n",
|
||||
"\n",
|
||||
" # 匹配变量赋值中的sessionid\n",
|
||||
" current_match = re.search(r\"currentSessionID\\s*=\\s*'(\\d+)'\", script.text)\n",
|
||||
" if current_match:\n",
|
||||
" session_ids.add(current_match.group(1))\n",
|
||||
"\n",
|
||||
" return list(session_ids)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"for index, row in tqdm(list(df.iterrows()), desc=\"处理订单\"):\n",
|
||||
" order_number = row[\"单号\"]\n",
|
||||
" success = False\n",
|
||||
" # print(f\"正在处理订单 {order_number}\")\n",
|
||||
"\n",
|
||||
" timestamp_ms = int(time.time() * 1000)\n",
|
||||
" for attempt in range(10): # 最大重试次数\n",
|
||||
" try:\n",
|
||||
" url = (\n",
|
||||
" \"http://rp.chezizhu.com/ReportServer?reportlet=/czz/order/order.cpt&__parameters__={%22_%22:%22\" + f\"{timestamp_ms}\" + \"%22,%22PINGJIA%22:[5b][5d],%22DISCOUNT_DESC%22:%22%22,%22B%22:%222011-05-01%22,%22LABELPLATE_NUMBER%22:%22%25E8%25BD%25A6%25E7%2589%258C%25E5%258F%25B7%22,%22LABELUSER_NAME%22:%22%25E5%25A7%2593%25E5%2590%258D%22,%22E%22:%222025-05-07%22,%22SERVICER%22:[5b][5d],%22CAR_NAME%22:%22%22,%22ORDER_SERVICE_TYPE%22:%22%22,%22order_number%22:%22\" + order_number + \"%22,%22USER_MOBILE%22:%22%22,%22LABELUSER_STATE%22:%22%25E6%2596%25B0%25E8%2580%2581%25E7%2594%25A8%25E6%2588%25B7%22,%22LABELCAR_NAME%22:%22%25E8%25BD%25A6%25E5%259E%258B%22,%22SERVICER_ID%22:%22683XGWqnyn66mE1oa8Gkzb%22,%22LABELORDER_SOURCE%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E6%259D%25A5%25E6%25BA%2590%22,%22LABELE%22:%22%25E8%2587%25B3%22,%22LABELSERVER%22:%22%25E9%2597%25A8%25E5%25BA%2597%22,%22LABELGONGDAN_STATUS%22:%22%25E5%25B7%25A5%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22LABELORDER_STATUS%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22ORDER_STATUS%22:[5b][5d],%22CMD%22:%22parameters_d%22,%22PAY_METHOD%22:%22%22,%22LABELUSER_MOBILE%22:%22%25E5%25AE%25A2%25E6%2588%25B7%25E7%2594%25B5%25E8%25AF%259D%22,%22LABELPINGJIA%22:%22%25E8%25AF%2584%25E4%25BB%25B7%25E7%258A%25B6%25E6%2580%2581%22,%22USER_STATE%22:[5b][5d],%22LABELRP_TYPE%22:%22%25E6%258A%25A5%25E8%25A1%25A8%25E7%25B1%25BB%25E5%259E%258B%22,%22LABELPAY_METHOD%22:%22%25E7%25BB%2593%25E7%25AE%2597%25E6%2596%25B9%25E5%25BC%258F%22,%22USER_NAME%22:%22%22,%22REPORTNAME%22:%22czz%252Forder%252Forder_detail_md.cpt%22,%22DISCOUNT_TYPE%22:%22%22,%22DATE_TYPE%22:%22%25E5%25AE%258C%25E6%2588%2590%25E6%2597%25A5%25E6%259C%259F%22,%22LABELDISCOUNT_DESC%22:%22%25E6%258A%2598%25E6%2589%25A3%25E5%258E%259F%25E5%259B%25A0%22,%22GONGDAN_STATUS%22:[5b][5d],%22__STREAMCLOSED__%22:%22%22,%22__pi__%22:%22true%22,%22CHANNEL%22:%22%22,%22DATABASE%22:%22chezizhu_14%22,%22ORDER_SOURCE%22:[5b][5d],%22LABELORDER_NUMBER%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E5%258F%25B7%22,%22PLATE_NUMBER%22:%22%22,%22RP_TYPE%22:%22%25E9%25AB%2598%25E7%25BA%25A7%25E6%2590%259C%25E7%25B4%25A2%22}\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" headers = {\n",
|
||||
" 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Cache-Control': 'max-age=0',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Cookie': 'td_cookie=2912605941',\n",
|
||||
" 'Host': 'rp.chezizhu.com',\n",
|
||||
" 'Referer': 'http://rp.chezizhu.com/ReportServer?reportlet=czz/order/order_detail_md.cpt&servicer_id=CJRuYehLSqLqy4snS5bLoo&database=chezizhu_14',\n",
|
||||
" 'Upgrade-Insecure-Requests': '1',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36 Edg/136.0.0.0'\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" res = requests.get(url, headers=headers)\n",
|
||||
"\n",
|
||||
" session_ids = extract_session_id(res.text)\n",
|
||||
" # print(\"提取到的SessionID:\", session_ids[0])\n",
|
||||
" # ========== 请求部分 ==========\n",
|
||||
" url = f\"http://rp.chezizhu.com/ReportServer?_={timestamp_ms}&__boxModel__=true&op=page_content&sessionID={session_ids[0]}&pn=1&__fit__=false\"\n",
|
||||
"\n",
|
||||
" headers = {\n",
|
||||
" 'Accept': '*/*',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Cookie': 'td_cookie=2912605941',\n",
|
||||
" 'Host': 'rp.chezizhu.com',\n",
|
||||
" \"Referer\": \"http://rp.chezizhu.com/ReportServer?reportlet=/czz/order/order.cpt&__parameters__={%22_%22:%22\" + f\"{timestamp_ms}\" + \"%22,%22PINGJIA%22:[5b][5d],%22DISCOUNT_DESC%22:%22%22,%22B%22:%222010-07-01%22,%22LABELPLATE_NUMBER%22:%22%25E8%25BD%25A6%25E7%2589%258C%25E5%258F%25B7%22,%22LABELUSER_NAME%22:%22%25E5%25A7%2593%25E5%2590%258D%22,%22E%22:%222025-07-14%22,%22SERVICER%22:[5b][5d],%22CAR_NAME%22:%22%22,%22ORDER_SERVICE_TYPE%22:%22%22,%22order_number%22:%22\" + order_number + \"%22,%22USER_MOBILE%22:%22%22,%22LABELUSER_STATE%22:%22%25E6%2596%25B0%25E8%2580%2581%25E7%2594%25A8%25E6%2588%25B7%22,%22LABELCAR_NAME%22:%22%25E8%25BD%25A6%25E5%259E%258B%22,%22SERVICER_ID%22:%22CJRuYehLSqLqy4snS5bLoo%22,%22LABELORDER_SOURCE%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E6%259D%25A5%25E6%25BA%2590%22,%22LABELE%22:%22%25E8%2587%25B3%22,%22LABELSERVER%22:%22%25E9%2597%25A8%25E5%25BA%2597%22,%22LABELGONGDAN_STATUS%22:%22%25E5%25B7%25A5%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22LABELORDER_STATUS%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E7%258A%25B6%25E6%2580%2581%22,%22ORDER_STATUS%22:[5b][5d],%22CMD%22:%22parameters_d%22,%22PAY_METHOD%22:%22%22,%22LABELUSER_MOBILE%22:%22%25E5%25AE%25A2%25E6%2588%25B7%25E7%2594%25B5%25E8%25AF%259D%22,%22LABELPINGJIA%22:%22%25E8%25AF%2584%25E4%25BB%25B7%25E7%258A%25B6%25E6%2580%2581%22,%22USER_STATE%22:[5b][5d],%22LABELRP_TYPE%22:%22%25E6%258A%25A5%25E8%25A1%25A8%25E7%25B1%25BB%25E5%259E%258B%22,%22LABELPAY_METHOD%22:%22%25E7%25BB%2593%25E7%25AE%2597%25E6%2596%25B9%25E5%25BC%258F%22,%22USER_NAME%22:%22%22,%22REPORTNAME%22:%22czz%252Forder%252Forder_detail_md.cpt%22,%22DISCOUNT_TYPE%22:%22%22,%22DATE_TYPE%22:%22%25E5%25AE%258C%25E6%2588%2590%25E6%2597%25A5%25E6%259C%259F%22,%22LABELDISCOUNT_DESC%22:%22%25E6%258A%2598%25E6%2589%25A3%25E5%258E%259F%25E5%259B%25A0%22,%22GONGDAN_STATUS%22:[5b][5d],%22__STREAMCLOSED__%22:%22%22,%22__pi__%22:%22true%22,%22CHANNEL%22:%22%22,%22DATABASE%22:%22chezizhu_14%22,%22ORDER_SOURCE%22:[5b][5d],%22LABELORDER_NUMBER%22:%22%25E8%25AE%25A2%25E5%258D%2595%25E5%258F%25B7%22,%22PLATE_NUMBER%22:%22%22,%22RP_TYPE%22:%22%25E9%25AB%2598%25E7%25BA%25A7%25E6%2590%259C%25E7%25B4%25A2%22}\",\n",
|
||||
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0\",\n",
|
||||
" \"X-Requested-With\": \"XMLHttpRequest\"\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # 使用带重试的请求函数\n",
|
||||
" res = make_request_with_retry(url, headers)\n",
|
||||
" # print(res.text)\n",
|
||||
"\n",
|
||||
" soup = BeautifulSoup(res.text, 'lxml')\n",
|
||||
"\n",
|
||||
" # ========== 1. 提取基本信息（单行） ==========\n",
|
||||
" base_info = {'单号': order_number} # 保留原始订单号\n",
|
||||
"\n",
|
||||
" for field in BASE_FIELDS:\n",
|
||||
" field_td = soup.find('td', string=f'{field}：')\n",
|
||||
" if field_td:\n",
|
||||
" value_td = field_td.find_next('td')\n",
|
||||
" base_info[field] = value_td.get_text(strip=True) if value_td else None\n",
|
||||
" else:\n",
|
||||
" base_info[field] = None\n",
|
||||
"\n",
|
||||
" # 更新基本信息DataFrame\n",
|
||||
" for col in base_info:\n",
|
||||
" if col not in base_info_df.columns:\n",
|
||||
" base_info_df[col] = None\n",
|
||||
" base_info_df.at[index, col] = base_info[col]\n",
|
||||
"\n",
|
||||
" # ========== 2. 提取消费项目（多行） ==========\n",
|
||||
" consumption_header = soup.find(lambda tag: tag.name == 'td' and '消费项目' in tag.text)\n",
|
||||
" if consumption_header:\n",
|
||||
" table = consumption_header.find_parent('table')\n",
|
||||
" if table:\n",
|
||||
" rows = table.find_all('tr')[1:] # 跳过表头\n",
|
||||
"\n",
|
||||
" for row_idx, row in enumerate(rows, 1):\n",
|
||||
" cols = row.find_all('td')\n",
|
||||
" if len(cols) >= 6:\n",
|
||||
" item = {\n",
|
||||
" '单号': order_number,\n",
|
||||
" '行号': row_idx,\n",
|
||||
" '套餐': cols[0].get_text(strip=True) if len(cols) > 0 else '',\n",
|
||||
" '服务项目': cols[1].get_text(strip=True) if len(cols) > 1 else '',\n",
|
||||
" '产品名称': cols[2].get_text(strip=True) if len(cols) > 2 else '',\n",
|
||||
" '产品型号': cols[3].get_text(strip=True) if len(cols) > 3 else '',\n",
|
||||
" '数量': cols[4].get_text(strip=True) if len(cols) > 4 else '',\n",
|
||||
" '规格': cols[5].get_text(strip=True) if len(cols) > 5 else '',\n",
|
||||
" '单位': cols[6].get_text(strip=True) if len(cols) > 6 else '',\n",
|
||||
" '单价': cols[7].get_text(strip=True) if len(cols) > 7 else '',\n",
|
||||
" '产品小计': cols[8].get_text(strip=True) if len(cols) > 8 else '',\n",
|
||||
" '工时': cols[9].get_text(strip=True) if len(cols) > 9 else '',\n",
|
||||
" '合计': cols[10].get_text(strip=True) if len(cols) > 10 else ''\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" # 添加到消费项目DataFrame\n",
|
||||
" service_items_df = pd.concat([\n",
|
||||
" service_items_df,\n",
|
||||
" pd.DataFrame([item])\n",
|
||||
" ], ignore_index=True)\n",
|
||||
" # print(item)\n",
|
||||
"\n",
|
||||
" success = True\n",
|
||||
" break # 成功则跳出重试循环\n",
|
||||
"\n",
|
||||
" except Exception as e:\n",
|
||||
" if attempt == 9: # 最后一次尝试仍然失败\n",
|
||||
" print(f\"订单 {order_number} 处理失败，已达最大重试次数。错误: {str(e)}\")\n",
|
||||
" # 记录失败订单（可选）\n",
|
||||
" with open('failed_orders.txt', 'a') as f:\n",
|
||||
" f.write(f\"{order_number}\\n\")\n",
|
||||
" time.sleep(1) # 等待1秒后重试\n",
|
||||
"\n",
|
||||
" if not success:\n",
|
||||
" continue # 跳过处理失败的订单\n",
|
||||
" # break\n",
|
||||
"\n",
|
||||
"# ========== 3. 合并数据并保存 ==========\n",
|
||||
"# 方法1：保存为两个关联表（推荐）\n",
|
||||
"base_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\订单基本信息.xlsx'\n",
|
||||
"items_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\消费项目明细.xlsx'\n",
|
||||
"base_info_df.to_excel(base_output, index=False)\n",
|
||||
"service_items_df.to_excel(items_output, index=False)\n",
|
||||
"\n",
|
||||
"# 方法2：合并为一个宽表（每行重复基本信息）\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" merged_df = pd.merge(\n",
|
||||
" base_info_df,\n",
|
||||
" service_items_df,\n",
|
||||
" on='单号',\n",
|
||||
" how='left'\n",
|
||||
" )\n",
|
||||
" merged_output = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\历史维修记录明细-古交腾飞路店13593189858.xlsx'\n",
|
||||
" merged_df.to_excel(merged_output, index=False)\n",
|
||||
"\n",
|
||||
"print(\"\\n处理完成！\")\n",
|
||||
"print(f\"- 基本信息保存至: {base_output}\")\n",
|
||||
"print(f\"- 消费项目保存至: {items_output}\")\n",
|
||||
"if not service_items_df.empty:\n",
|
||||
" print(f\"- 合并数据保存至: {merged_output}\") # 输出：['57435']"
|
||||
],
|
||||
"id": "ed2755053dd426c",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 0%| | 0/10868 [00:00<?, ?it/s]C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_4532\\3409432529.py:48: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"处理订单: 0%| | 1/10868 [00:00<28:50, 6.28it/s]C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_4532\\3409432529.py:48: DeprecationWarning: The 'text' argument to find()-type methods is deprecated. Use 'string' instead.\n",
|
||||
" scripts = soup.find_all(\"script\", text=re.compile(r\"(FR\\.SessionMgr\\.register|currentSessionID)\"))\n",
|
||||
"处理订单: 18%|█▊ | 1974/10868 [09:20<50:33, 2.93it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"请求失败，第 1 次重试... 错误: HTTPConnectionPool(host='rp.chezizhu.com', port=80): Max retries exceeded with url: /ReportServer?_=1752486488556&__boxModel__=true&op=page_content&sessionID=26250&pn=1&__fit__=false (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x000002C3E3298E90>, 'Connection to rp.chezizhu.com timed out. (connect timeout=10)'))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"处理订单: 100%|██████████| 10868/10868 [2:03:15<00:00, 1.47it/s] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"处理完成！\n",
|
||||
"- 基本信息保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\订单基本信息.xlsx\n",
|
||||
"- 消费项目保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\消费项目明细.xlsx\n",
|
||||
"- 合并数据保存至: D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\历史维修记录明细-古交腾飞路店13593189858.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 12
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 库存 （页面已支持导出）",
|
||||
"id": "5e7ac3ee321a3549"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-04-16T09:51:20.409717Z",
|
||||
"start_time": "2025-04-16T09:51:13.116335Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"from urllib.parse import urlencode, quote\n",
|
||||
"import time\n",
|
||||
"from requests.exceptions import RequestException\n",
|
||||
"\n",
|
||||
"url = \"http://sp.chezizhu.com/alliance/store/list.htm\"\n",
|
||||
"header = {\n",
|
||||
" \"content-type\": \"application/x-www-form-urlencoded; charset=UTF-8\",\n",
|
||||
" \"cookie\": \"JSESSIONID=28FFBA865A6CAAA76CB8CDB6A34B5886\",\n",
|
||||
" \"host\": \"sp.chezizhu.com\",\n",
|
||||
" \"origin\": \"http://sp.chezizhu.com\",\n",
|
||||
" \"referer\": \"http://sp.chezizhu.com/alliance/index.htm\",\n",
|
||||
" \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0\",\n",
|
||||
" \"x-requested-with\": \"XMLHttpRequest\"\n",
|
||||
"}\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(1, 103)):\n",
|
||||
" data = f'pageNum={i}&pageSize=20&dataType=json&keywords='\n",
|
||||
"\n",
|
||||
" res = requests.post(url, headers=header, data=data)\n",
|
||||
" data_list = res.json().get(\"list\", [])\n",
|
||||
"\n",
|
||||
" for data in data_list:\n",
|
||||
" tp_number = data.get(\"tp_number\")\n",
|
||||
" damai_id = data.get(\"damai_id\")\n",
|
||||
" prod_name = data.get(\"prod_name\")\n",
|
||||
" price = data.get(\"price\")\n",
|
||||
" standard = data.get(\"standard\")\n",
|
||||
" unit = data.get(\"unit\")\n",
|
||||
" count = data.get(\"count\")\n",
|
||||
" check_count = data.get(\"check_count\")\n",
|
||||
" check_date = data.get(\"check_date\")\n",
|
||||
"\n",
|
||||
" all_data.append([tp_number, damai_id, prod_name, price, standard, unit, count, check_count, check_date])\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data,\n",
|
||||
" columns=[\"tp_number\", \"damai_id\", \"prod_name\", \"price\", \"standard\", \"unit\", \"count\", \"check_count\",\n",
|
||||
" \"check_date\"])\n",
|
||||
"df.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\库存.xlsx\", index=False)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n"
|
||||
],
|
||||
"id": "95616ee8b933e5ce",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 102/102 [00:07<00:00, 14.43it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 52
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "",
|
||||
"id": "70a45b885b2739c"
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 销售明细",
|
||||
"id": "7df1375e5debe404"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-07-15T01:34:31.895645Z",
|
||||
"start_time": "2025-07-15T01:32:17.223368Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"\n",
|
||||
"all_data_list = []\n",
|
||||
"keys = [\"门店\", \"订单编号\", \"产品类型\", \"产品名称\", \"型号\", \"数量\", \"采购单价\", \"销售单价\",\n",
|
||||
" \"采购金额\", \"销售金额\", \"毛利润\", \"下单人\", \"接单人\", \"车牌号\"]\n",
|
||||
"reversed_keys = list(reversed(keys))\n",
|
||||
"\n",
|
||||
"for i in tqdm(range(1, 1390)):\n",
|
||||
" url = f\"http://rp.chezizhu.com/ReportServer?_=1752541860351&__boxModel__=true&op=page_content&sessionID=91877&pn={i}&__fit__=false\"\n",
|
||||
"\n",
|
||||
" header = {\n",
|
||||
" \"Referer\": \"http://rp.chezizhu.com/ReportServer?reportlet=czz/storage/prod_sale_dtl_jm.cpt&servicer_id=CJRuYehLSqLqy4snS5bLoo&hideexport=true&database=chezizhu_14\",\n",
|
||||
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 Edg/135.0.0.0\",\n",
|
||||
" \"X-Requested-With\": \"XMLHttpRequest\"\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" while True:\n",
|
||||
" try:\n",
|
||||
" res = requests.post(url, headers=header)\n",
|
||||
" break\n",
|
||||
" except RequestException as e:\n",
|
||||
" print(f\"请求失败，正在重试... 错误信息：{str(e)}\")\n",
|
||||
" time.sleep(1)\n",
|
||||
" soup = BeautifulSoup(res.text, \"html.parser\")\n",
|
||||
"\n",
|
||||
" # 定位真正的数据行（跳过标题行）\n",
|
||||
" for tr in soup.select('tr:not(:has(th))'): # 排除包含表头的行\n",
|
||||
" tds = [td.get_text(strip=True) for td in tr.find_all(\"td\")]\n",
|
||||
" reversed_tds = list(reversed(tds))\n",
|
||||
"\n",
|
||||
" # 如果 reversed_tds 长度不够，填充空字符串\n",
|
||||
" if len(reversed_tds) < len(reversed_keys):\n",
|
||||
" reversed_tds += [\"\"] * (len(reversed_keys) - len(reversed_tds))\n",
|
||||
" elif len(reversed_tds) > len(reversed_keys):\n",
|
||||
" reversed_tds = reversed_tds[:len(reversed_keys)]\n",
|
||||
"\n",
|
||||
" # 构建字典（反转后的 keys 和 tds）\n",
|
||||
" row_data = dict(zip(reversed_keys, reversed_tds))\n",
|
||||
" all_data_list.append(row_data)\n",
|
||||
"\n",
|
||||
" # break\n",
|
||||
"# print(all_data_list)\n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\大唛云销售明细.xlsx', index=False)"
|
||||
],
|
||||
"id": "b3100f77cc1061b1",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 1389/1389 [02:09<00:00, 10.70it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 14
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-04-29T07:16:14.900349Z",
|
||||
"start_time": "2025-04-29T07:16:07.255065Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"df.to_excel(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\大唛云销售明细-古交腾飞路店13593189858.xlsx', index=False)"
|
||||
],
|
||||
"id": "7671bfd65cc413c7",
|
||||
"outputs": [],
|
||||
"execution_count": 9
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,448 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 客户车辆信息导出"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-07-08T01:22:38.302541Z",
|
||||
"start_time": "2025-07-08T01:21:21.261993Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"from selenium import webdriver\n",
|
||||
"from selenium.webdriver.support.ui import Select\n",
|
||||
"from selenium.webdriver import ActionChains\n",
|
||||
"import time\n",
|
||||
"import xlrd\n",
|
||||
"import xlwt\n",
|
||||
"import re\n",
|
||||
"import datetime\n",
|
||||
"from xlutils.copy import copy\n",
|
||||
"from selenium.webdriver.common.keys import Keys\n",
|
||||
"from selenium.webdriver.common.action_chains import ActionChains\n",
|
||||
"from selenium.webdriver.common.by import By\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"# 登录\n",
|
||||
"name = '13568162018'\n",
|
||||
"password = 'a123456'\n",
|
||||
"option_chrome = webdriver.ChromeOptions()\n",
|
||||
"option_chrome.add_argument('--headless')\n",
|
||||
"http='https://qixiu.dada365.com/login'\n",
|
||||
"from selenium import webdriver\n",
|
||||
"from selenium.webdriver.chrome.service import Service\n",
|
||||
"\n",
|
||||
"# 如果 ChromeDriver 已经添加到系统的 PATH 中，可以这样写：\n",
|
||||
"service = Service()\n",
|
||||
"\n",
|
||||
"# 如果 ChromeDriver 没有在 PATH 中，则需要指定其路径：\n",
|
||||
"service = Service(executable_path=r\"D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe\")\n",
|
||||
"\n",
|
||||
"# 创建一个新的 Chrome 浏览器实例\n",
|
||||
"driver = webdriver.Chrome(service=service)\n",
|
||||
"\n",
|
||||
"# 打开大大汽修平台登录页面\n",
|
||||
"driver.get(http)\n",
|
||||
"# driver = webdriver.Chrome(executable_path=r\"D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe\")\n",
|
||||
"driver.maximize_window()\n",
|
||||
"driver.implicitly_wait(1)\n",
|
||||
"driver.get(http)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[1]/div/div/input').send_keys(name)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[2]/div/div/input').send_keys(password)\n",
|
||||
"time.sleep(3)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[4]/div/button/span').click()\n",
|
||||
"time.sleep(3)\n",
|
||||
"# 点击车辆管理\n",
|
||||
"\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[1]/div[1]/ul/div/div[3]/li/div/span').click()\n",
|
||||
"# driver.find_element(By.XPATH,'(//div[@id=\"app\"]//ul/div[1]/div[3])[1]').click()\n",
|
||||
"time.sleep(2)\n",
|
||||
"# 点击车辆查询\n",
|
||||
"driver.find_element(By.XPATH,'/html/body/div[2]/ul/li[1]/span').click()\n",
|
||||
"time.sleep(3)\n",
|
||||
"# 点击500条每页\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[2]/div[3]/div/span[2]/div/div/span/span/i').click()\n",
|
||||
"\n",
|
||||
"time.sleep(3)\n",
|
||||
"driver.find_element(By.XPATH,'/html/body/div[3]/div[1]/div[1]/ul/li[6]/span').click()\n",
|
||||
"\n",
|
||||
"# 点击查看\n",
|
||||
"time.sleep(10)\n",
|
||||
"# for a in range(1,3):\n",
|
||||
"for i in range(1,501):\n",
|
||||
" try:\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[2]/div[2]/div/div[2]/div[3]/table/tbody/tr[%s]/td[13]/div/button[1]/span'%i).click()\n",
|
||||
" time.sleep(2)\n",
|
||||
" car_no=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/span[1]') \n",
|
||||
" name=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/span[2]')\n",
|
||||
" print(2)\n",
|
||||
"\n",
|
||||
" phone_num=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[1]/li[1]')\n",
|
||||
" print(3) \n",
|
||||
" identity_num=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[1]/li[2]')\n",
|
||||
" print(4)\n",
|
||||
" insur_com=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[1]/li[3]')\n",
|
||||
" print(5)\n",
|
||||
" \n",
|
||||
" insur_date=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[1]/li[4]')\n",
|
||||
"\n",
|
||||
" vin_num=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[2]/li[1]')\n",
|
||||
" print(6)\n",
|
||||
" car_modle=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[2]/li[2]/strong/span')\n",
|
||||
" print(7)\n",
|
||||
" engin_num=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[2]/li[3]')\n",
|
||||
" car_date=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[2]/li[4]')\n",
|
||||
" memo=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/div[4]/ul[5]/li/span[2]')\n",
|
||||
" print(8)\n",
|
||||
" print(car_no.text,\"@\",name.text,\"@\",phone_num.text,\"@\",identity_num.text,\"@\",insur_com.text,\"@\",vin_num.text,\"@\",car_modle.text,\"@\", memo.text)\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"tab-item-2\"]/span[2]').click()\n",
|
||||
" except:\n",
|
||||
" print(\"有误\")\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"tab-item-2\"]/span[2]').click()\n",
|
||||
" # driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[2]/div[3]/div/ul/li[%s]'%a).click()\n",
|
||||
" "
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SX3667 @ 何 @ 车主电话： 13982846671 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： JF1GP25C2DG065475 @ 斯巴鲁 XV XV 2.0 2012年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S339X6 @ 李 @ 车主电话： 18582830888 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LSVAK2185B2333118 @ 大众 朗逸 朗逸 1.6 2011年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SEZ717 @ 周欣 @ 车主电话： 15882942902 @ 身份证号码： 513001199312051019 @ 保险公司： 暂无 @ 车架号： LC0CG4DF7G0013987 @ 比亚迪 元 元 1.5 2016年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S250K0 @ 李 @ 车主电话： 15281866877 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LVSHFCAC5MS221747 @ 福特 锐际 锐际 2.0 2020年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SFT881 @ 唐艳 @ 车主电话： 15390125925 @ 身份证号码： 513023197405246746 @ 保险公司： 暂无 @ 车架号： LSVX065N9E2159128 @ 大众 途观 途观 1.8 2015年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SWZ316 @ 13982824053 @ 车主电话： 18780800015 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LHGCV1641L9001579 @ 本田 雅阁 第十代雅阁 1.5 2018年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S09F2T @ 袁远 @ 车主电话： 18481969666 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： WBA8X3103JBM92377 @ 宝马 3 Series [3系] GT 320i 2.0 2018年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川Y63V90 @ 阳 @ 车主电话： 18282745485 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LSVDU2BM6KN101225 @ 大众 凌渡 凌渡 1.4 2019年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S158V9 @ 魏 @ 车主电话： 15882940035 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LS4ASB2E7FA702842 @ 长安商用 欧诺 欧诺 1.5 2012年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SQ226F @ 雷亚军 @ 车主电话： 13547247727 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LZWDAAGA1C7215513 @ 五菱 荣光 荣光小卡 1.2 2010年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S97Q07 @ 卢姐 @ 车主电话： 13808241818 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： W1KUG5GB6LA533010 @ 奔驰 S-Class [S级] S 350 L 48V 3.0 2020年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S90E91 @ 杨 @ 车主电话： 13228276199 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LFPM4APE0J1A23984 @ 马自达 CX-4 CX-4 2.0 2018年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SYJ559 @ 李 @ 车主电话： 17760078550 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LGBM4AE44MS462782 @ 日产 奇骏 奇骏 2.0 2021年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S1D278 @ 罗 @ 车主电话： 13882895869 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LGWDBE194MB010263 @ 长城 风骏 风骏5 2.0 2021年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SEH608 @ 张 @ 车主电话： 15775628388 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LSV7N60T7M2026048 @ 大众 途观 途观L R-Line 2.0 2021年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"渝AFV6238 @ 王 @ 车主电话： 19160657666 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LB378LCZ3PB132953 @ 银河 L7 L7 1.5 2023年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川S8U220 @ 李 @ 车主电话： 18982881545 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LS5A3DEE9KA422866 @ 长安 CS55 CS55 1.5 2019年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川SD99133 @ 卢 @ 车主电话： 15228099005 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： LJ1E6A2UXRG009850 @ 蔚来 ES6 ES6 电动 2023年款 @ 暂无\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n",
|
||||
"川G8HE47 @ 孙 @ 车主电话： 18398080313 @ 身份证号码： 暂无 @ 保险公司： 暂无 @ 车架号： W0LLB6E02DB057005 @ 欧宝 Antara [安德拉] Antara [安德拉] 2.4 2013年款 @ 暂无\n",
|
||||
"有误\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "InvalidSessionIdException",
|
||||
"evalue": "Message: invalid session id\nStacktrace:\n\tGetHandleVerifier [0x0x7ff6d1fccda5+78885]\n\tGetHandleVerifier [0x0x7ff6d1fcce00+78976]\n\t(No symbol) [0x0x7ff6d1d899fc]\n\t(No symbol) [0x0x7ff6d1dd07df]\n\t(No symbol) [0x0x7ff6d1e08a52]\n\t(No symbol) [0x0x7ff6d1e03413]\n\t(No symbol) [0x0x7ff6d1e024d9]\n\t(No symbol) [0x0x7ff6d1d55d55]\n\tGetHandleVerifier [0x0x7ff6d22a4eed+3061101]\n\tGetHandleVerifier [0x0x7ff6d229f33d+3037629]\n\tGetHandleVerifier [0x0x7ff6d22be592+3165202]\n\tGetHandleVerifier [0x0x7ff6d1fe730e+186766]\n\tGetHandleVerifier [0x0x7ff6d1feeb3f+217535]\n\t(No symbol) [0x0x7ff6d1d54dca]\n\tGetHandleVerifier [0x0x7ff6d23c45e8+4238440]\n\tBaseThreadInitThunk [0x0x7ffa253de8d7+23]\n\tRtlUserThreadStart [0x0x7ffa25ddc34c+44]\n",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[6], line 65\u001B[0m\n\u001B[0;32m 64\u001B[0m time\u001B[38;5;241m.\u001B[39msleep(\u001B[38;5;241m2\u001B[39m)\n\u001B[1;32m---> 65\u001B[0m car_no\u001B[38;5;241m=\u001B[39mdriver\u001B[38;5;241m.\u001B[39mfind_element(By\u001B[38;5;241m.\u001B[39mXPATH,\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m//*[@id=\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mapp\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m]/div/div[2]/div[2]/div[3]/div[1]/span[1]\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[0;32m 66\u001B[0m \u001B[38;5;28mprint\u001B[39m(\u001B[38;5;241m1\u001B[39m) \n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:748\u001B[0m, in \u001B[0;36mWebDriver.find_element\u001B[1;34m(self, by, value)\u001B[0m\n\u001B[0;32m 746\u001B[0m value \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m[name=\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mvalue\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m]\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m--> 748\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mexecute(Command\u001B[38;5;241m.\u001B[39mFIND_ELEMENT, {\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124musing\u001B[39m\u001B[38;5;124m\"\u001B[39m: by, \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m: value})[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m]\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:354\u001B[0m, in \u001B[0;36mWebDriver.execute\u001B[1;34m(self, driver_command, params)\u001B[0m\n\u001B[0;32m 353\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m response:\n\u001B[1;32m--> 354\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39merror_handler\u001B[38;5;241m.\u001B[39mcheck_response(response)\n\u001B[0;32m 355\u001B[0m response[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m] \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_unwrap_value(response\u001B[38;5;241m.\u001B[39mget(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m, \u001B[38;5;28;01mNone\u001B[39;00m))\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py:229\u001B[0m, in \u001B[0;36mErrorHandler.check_response\u001B[1;34m(self, response)\u001B[0m\n\u001B[0;32m 228\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace, alert_text) \u001B[38;5;66;03m# type: ignore[call-arg] # mypy is not smart enough here\u001B[39;00m\n\u001B[1;32m--> 229\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace)\n",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m: Message: invalid session id: session deleted as the browser has closed the connection\nfrom disconnected: not connected to DevTools\n (Session info: chrome=138.0.7204.97)\nStacktrace:\n\tGetHandleVerifier [0x0x7ff6d1fccda5+78885]\n\tGetHandleVerifier [0x0x7ff6d1fcce00+78976]\n\t(No symbol) [0x0x7ff6d1d89bca]\n\t(No symbol) [0x0x7ff6d1d759b5]\n\t(No symbol) [0x0x7ff6d1d9a9ca]\n\t(No symbol) [0x0x7ff6d1e105e5]\n\t(No symbol) [0x0x7ff6d1e30b42]\n\t(No symbol) [0x0x7ff6d1e08963]\n\t(No symbol) [0x0x7ff6d1dd16b1]\n\t(No symbol) [0x0x7ff6d1dd2443]\n\tGetHandleVerifier [0x0x7ff6d22a4eed+3061101]\n\tGetHandleVerifier [0x0x7ff6d229f33d+3037629]\n\tGetHandleVerifier [0x0x7ff6d22be592+3165202]\n\tGetHandleVerifier [0x0x7ff6d1fe730e+186766]\n\tGetHandleVerifier [0x0x7ff6d1feeb3f+217535]\n\tGetHandleVerifier [0x0x7ff6d1fd59b4+114740]\n\tGetHandleVerifier [0x0x7ff6d1fd5b69+115177]\n\tGetHandleVerifier [0x0x7ff6d1fbc368+10728]\n\tBaseThreadInitThunk [0x0x7ffa253de8d7+23]\n\tRtlUserThreadStart [0x0x7ffa25ddc34c+44]\n",
|
||||
"\nDuring handling of the above exception, another exception occurred:\n",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[6], line 91\u001B[0m\n\u001B[0;32m 89\u001B[0m \u001B[38;5;28;01mexcept\u001B[39;00m:\n\u001B[0;32m 90\u001B[0m \u001B[38;5;28mprint\u001B[39m(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m有误\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n\u001B[1;32m---> 91\u001B[0m driver\u001B[38;5;241m.\u001B[39mfind_element(By\u001B[38;5;241m.\u001B[39mXPATH,\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m//*[@id=\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mtab-item-2\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m]/span[2]\u001B[39m\u001B[38;5;124m'\u001B[39m)\u001B[38;5;241m.\u001B[39mclick()\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:748\u001B[0m, in \u001B[0;36mWebDriver.find_element\u001B[1;34m(self, by, value)\u001B[0m\n\u001B[0;32m 745\u001B[0m by \u001B[38;5;241m=\u001B[39m By\u001B[38;5;241m.\u001B[39mCSS_SELECTOR\n\u001B[0;32m 746\u001B[0m value \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124m[name=\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mvalue\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m]\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m--> 748\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mexecute(Command\u001B[38;5;241m.\u001B[39mFIND_ELEMENT, {\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124musing\u001B[39m\u001B[38;5;124m\"\u001B[39m: by, \u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m: value})[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m]\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:354\u001B[0m, in \u001B[0;36mWebDriver.execute\u001B[1;34m(self, driver_command, params)\u001B[0m\n\u001B[0;32m 352\u001B[0m response \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mcommand_executor\u001B[38;5;241m.\u001B[39mexecute(driver_command, params)\n\u001B[0;32m 353\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m response:\n\u001B[1;32m--> 354\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39merror_handler\u001B[38;5;241m.\u001B[39mcheck_response(response)\n\u001B[0;32m 355\u001B[0m response[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m] \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_unwrap_value(response\u001B[38;5;241m.\u001B[39mget(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m, \u001B[38;5;28;01mNone\u001B[39;00m))\n\u001B[0;32m 356\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m response\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py:229\u001B[0m, in \u001B[0;36mErrorHandler.check_response\u001B[1;34m(self, response)\u001B[0m\n\u001B[0;32m 227\u001B[0m alert_text \u001B[38;5;241m=\u001B[39m value[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124malert\u001B[39m\u001B[38;5;124m\"\u001B[39m]\u001B[38;5;241m.\u001B[39mget(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mtext\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n\u001B[0;32m 228\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace, alert_text) \u001B[38;5;66;03m# type: ignore[call-arg] # mypy is not smart enough here\u001B[39;00m\n\u001B[1;32m--> 229\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace)\n",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m: Message: invalid session id\nStacktrace:\n\tGetHandleVerifier [0x0x7ff6d1fccda5+78885]\n\tGetHandleVerifier [0x0x7ff6d1fcce00+78976]\n\t(No symbol) [0x0x7ff6d1d899fc]\n\t(No symbol) [0x0x7ff6d1dd07df]\n\t(No symbol) [0x0x7ff6d1e08a52]\n\t(No symbol) [0x0x7ff6d1e03413]\n\t(No symbol) [0x0x7ff6d1e024d9]\n\t(No symbol) [0x0x7ff6d1d55d55]\n\tGetHandleVerifier [0x0x7ff6d22a4eed+3061101]\n\tGetHandleVerifier [0x0x7ff6d229f33d+3037629]\n\tGetHandleVerifier [0x0x7ff6d22be592+3165202]\n\tGetHandleVerifier [0x0x7ff6d1fe730e+186766]\n\tGetHandleVerifier [0x0x7ff6d1feeb3f+217535]\n\t(No symbol) [0x0x7ff6d1d54dca]\n\tGetHandleVerifier [0x0x7ff6d23c45e8+4238440]\n\tBaseThreadInitThunk [0x0x7ffa253de8d7+23]\n\tRtlUserThreadStart [0x0x7ffa25ddc34c+44]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 6
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# 维修历史导出"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from selenium import webdriver\n",
|
||||
"from selenium.webdriver.support.ui import Select\n",
|
||||
"from selenium.webdriver import ActionChains\n",
|
||||
"import time\n",
|
||||
"import xlrd\n",
|
||||
"import xlwt\n",
|
||||
"import re\n",
|
||||
"import datetime\n",
|
||||
"from xlutils.copy import copy\n",
|
||||
"from selenium.webdriver.common.keys import Keys\n",
|
||||
"from selenium.webdriver.common.action_chains import ActionChains\n",
|
||||
"from selenium.webdriver.common.by import By\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"# 登录\n",
|
||||
"name = '18132039923'\n",
|
||||
"password = 'jl123456'\n",
|
||||
"option_chrome = webdriver.ChromeOptions()\n",
|
||||
"option_chrome.add_argument('--headless')\n",
|
||||
"http='https://qixiu.dada365.com/login'\n",
|
||||
"driver = webdriver.Chrome(executable_path=r'C:\\chromedriver\\chromedriver_99\\chromedriver.exe')\n",
|
||||
"driver.maximize_window()\n",
|
||||
"driver.implicitly_wait(1)\n",
|
||||
"driver.get(http)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[1]/div/div/input').send_keys(name)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[2]/div/div/input').send_keys(password)\n",
|
||||
"time.sleep(1)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div/div/div/form/div[4]/div/button/span').click()\n",
|
||||
"time.sleep(4)\n",
|
||||
"# 点击工单管理\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[1]/div[1]/ul/div/div[2]/li/div/span').click()\n",
|
||||
"time.sleep(2)\n",
|
||||
"# 点击工单查询\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[1]/div[1]/div[1]/ul/li[1]/div/p').click()\n",
|
||||
"\n",
|
||||
"# '/html/body/div[3]/ul/li[2]/span'\n",
|
||||
"# 点击已收款\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"tab-paid\"]').click()\n",
|
||||
"# 点击500条每页\n",
|
||||
"time.sleep(2)\n",
|
||||
"driver.find_element(By.XPATH,'//*[@id=\"pane-paid\"]/div/div[2]/div/span[2]/div/div[1]/span/span/i').click()\n",
|
||||
"\n",
|
||||
"time.sleep(3)\n",
|
||||
"driver.find_element(By.XPATH,'//html/body/div[3]/div[1]/div[1]/ul/li[6]/span').click()\n",
|
||||
"\n",
|
||||
"# 点击查看\n",
|
||||
"for j in range(2,5):\n",
|
||||
" for h in range(1,501):\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"pane-paid\"]/div/div[1]/div/div[3]/table/tbody/tr[%s]/td[10]/div/div/button[1]/span'%h).click()\n",
|
||||
" time.sleep(2)\n",
|
||||
" GD_num=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[1]/ul/li[1]')\n",
|
||||
" start_date=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[2]/li[1]')\n",
|
||||
" car_no_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[1]/li[1]')\n",
|
||||
" name_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[1]/li[2]')\n",
|
||||
" phone_num_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[1]/li[3]')\n",
|
||||
" car_model_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[1]/li[4]/strong')\n",
|
||||
" mile=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[2]/li[2]')\n",
|
||||
" tech_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[2]/li[3]')\n",
|
||||
" vin_GD=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[2]/ul[2]/li[4]')\n",
|
||||
" print()\n",
|
||||
" print(GD_num.text,\"@\",start_date.text,\"@\",car_no_GD.text,\"@\",name_GD.text,\"@\",phone_num_GD.text,\"@\",car_model_GD.text,\"@\",mile.text,\"@\",tech_GD.text,\"@\",vin_GD.text,\"@\")\n",
|
||||
" try:\n",
|
||||
" for c in range(2,100):\n",
|
||||
" print(GD_num.text,end=\" 材料历史：\")\n",
|
||||
" for d in range(2,15):\n",
|
||||
" cl=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[3]/div[2]/table/tr[%s]/td[%s]'%(c,d))\n",
|
||||
" print(cl.text,end=\"@\")\n",
|
||||
" if d==14:\n",
|
||||
" print()\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
" print()\n",
|
||||
" try:\n",
|
||||
" for e in range(2,100):\n",
|
||||
" print(GD_num.text,end=\" 项目历史：\")\n",
|
||||
" for f in range(2,8):\n",
|
||||
" if cl.text!=\"\":\n",
|
||||
" xm=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div/div[3]/div[4]/table/tr[%s]/td[%s]'%(e,f))\n",
|
||||
" # //*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[3]/div[4]/table/tr[2]/td[2]/span\n",
|
||||
" # //*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[3]/div[4]/table/tr[2]/td[3]\n",
|
||||
" # //*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[3]/div[4]/table/tr[2]/td[4]\n",
|
||||
" # //*[@id=\"app\"]/div/div[2]/div[2]/div[3]/div[3]/div[4]/table/tr[2]/td[7]\n",
|
||||
" else:\n",
|
||||
" xm=driver.find_element(By.XPATH,'//*[@id=\"app\"]/div/div[2]/div[2]/div/div[4]/div[2]/table/tr[%s]/td[%s]/span'%(e,f))\n",
|
||||
" print(xm.text,end=\"@\")\n",
|
||||
" if f==7:\n",
|
||||
" print()\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"tab-item-2\"]/span[2]').click()\n",
|
||||
"# 翻页\n",
|
||||
" driver.find_element(By.XPATH,'//*[@id=\"pane-paid\"]/div/div[2]/div/ul/li[%s]'%j).click()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"interpreter": {
|
||||
"hash": "59310123ccd3619229d524789fb6029e22b3ae7206c94adf033078aa9a774872"
|
||||
},
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3.9.6 ('F6')",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.6"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,156 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1787\n",
|
||||
"数据已成功导出到car_data.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"pageSize = 10000\n",
|
||||
"url = 'https://s19.kehu51.com/app/Customers/CusTools.aspx?action=GetGridData'\n",
|
||||
"\n",
|
||||
"json = f'showID=3&viewName=mycus&gridName=cuslistnew2&configGuid=&whereSql=E513575924A85785&sortName=lastfollowtime&sortMode=desc&pageSize={pageSize}&pageIndex=1&recordCount=1787&isComb=false&combName=&combIntTemp=_comb&completeSql=E513575924A85785&customParam=%7B%22field%22%3A%22selectUserID%22%2C%22key%22%3A0%7D'\n",
|
||||
"# 构建请求头\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': '*/*',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate, br, zstd',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Content-Length': '299',\n",
|
||||
" 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',\n",
|
||||
" 'Cookie': 'kehu51_user=18735521522; kehu51_pass=Xx123456; kehu51_server_username=cc9c4b66f19ec7ae6d489e6caaaceb0d; kehu51_server_serverid=19; ASP.NET_SessionId=lvhtxdfjbuymrsryoczhn5mi; teamCreateManID=0jkmaY3RdeFj2HX4WrJN5Q==; kehu51cookie=6958BDD7DCEADF0A83775994EFD09E5D722AF537BFB0741BB8E35CE4807E32F48097A08E3C914A52730C3D4B09AF6C9524BAA937C4EF2AF773D396081027AE7EA29932D831A19FA4D145E78564F9A85D63C305EE0B8DD39B2178B00340B34D3BEF70D88C8EF4FE02BC0EF6601A6B1F0FB762CBDAE71DDE08C27E62F90CCF7E155E41C9983E3327F67C170BF17387753B3B26C7D353085F23A2111D62896C9EE93174D53C5EB23DF8569C5FF49E615D498D62CEAA6DB5C73176112ACEA5DB5501C1A1B71C0B470239831165C0EB939B753AB79B0323E9AA06',\n",
|
||||
" 'Origin': 'https://s19.kehu51.com',\n",
|
||||
" 'Priority': 'u=1, i',\n",
|
||||
" 'Referer': 'https://s19.kehu51.com/App/customers/cuslist.aspx?viewname=mycus&templateid=',\n",
|
||||
" 'Sec-Ch-Ua': '\"Chromium\";v=\"130\", \"Microsoft Edge\";v=\"130\", \"Not?A_Brand\";v=\"99\"',\n",
|
||||
" 'Sec-Ch-Ua-Mobile': '?0',\n",
|
||||
" 'Sec-Ch-Ua-Platform': '\"Windows\"',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest'\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"res = requests.post(url=url,data = json,headers=headers)\n",
|
||||
"\n",
|
||||
"data = res.json()\n",
|
||||
"\n",
|
||||
"gridHtml = data['gridHtml']\n",
|
||||
"\n",
|
||||
"soup = BeautifulSoup(gridHtml, 'lxml') # 使用lxml解析器\n",
|
||||
"\n",
|
||||
"tbody = soup.find_all('tbody')[0]\n",
|
||||
"\n",
|
||||
"data = []\n",
|
||||
"\n",
|
||||
"# 查找所有的行\n",
|
||||
"rows = tbody.find_all('tr', class_='TableLine1 k5-grid-tr')\n",
|
||||
"print(len(rows))\n",
|
||||
"for row in rows:\n",
|
||||
" td_tags=row.find_all('td')\n",
|
||||
" # 客户名称（车牌）\n",
|
||||
" customer_name = row.find('a', class_='A0').text.strip()\n",
|
||||
" # 手机号\n",
|
||||
" phone_number = row.find('a', class_='phonemenu-trigger')\n",
|
||||
" if phone_number:\n",
|
||||
" phone_number = phone_number.text.strip()\n",
|
||||
" else:\n",
|
||||
" phone_number = ''\n",
|
||||
" # 机油\n",
|
||||
" oil = row.find('a', href=lambda x: x and 'OpneQQDialog' in x)\n",
|
||||
" if oil:\n",
|
||||
" oil = oil.text.strip()\n",
|
||||
" else:\n",
|
||||
" oil = ''\n",
|
||||
" # 车型\n",
|
||||
" car_models =td_tags[5].get_text()\n",
|
||||
" if car_models:\n",
|
||||
" pass\n",
|
||||
" else:\n",
|
||||
" car_models=''\n",
|
||||
" # 公里数\n",
|
||||
" mileage = row.find('a', href=lambda x: x and 'OpenWangWang' in x)\n",
|
||||
" if mileage:\n",
|
||||
" mileage = mileage.text.strip()\n",
|
||||
" else:\n",
|
||||
" mileage = ''\n",
|
||||
" # 其他联系方式\n",
|
||||
" \n",
|
||||
" other_contact = td_tags[7].get_text()\n",
|
||||
" if other_contact:\n",
|
||||
" pass\n",
|
||||
" else:\n",
|
||||
" other_contact = ''\n",
|
||||
" # 日期\n",
|
||||
" date = row.find('td', id=lambda x: x and x.startswith('Customize1_'))\n",
|
||||
" if date:\n",
|
||||
" date = date.text.strip()\n",
|
||||
" else:\n",
|
||||
" date = ''\n",
|
||||
" # 价格\n",
|
||||
" price = row.find('td', id=lambda x: x and x.startswith('Customize2_'))\n",
|
||||
" if price:\n",
|
||||
" price=price.text.strip()\n",
|
||||
" else:\n",
|
||||
" price=''\n",
|
||||
" # 备注\n",
|
||||
" remarks = row.find('td', id=lambda x: x and x.startswith('Customize3_'))\n",
|
||||
" if remarks:\n",
|
||||
" remarks=remarks.text.strip()\n",
|
||||
" else:\n",
|
||||
" remarks=''\n",
|
||||
" # 姓名\n",
|
||||
" name = row.find('td', id=lambda x: x and x.startswith('Customize4_'))\n",
|
||||
" if name:\n",
|
||||
" name = name.text.strip()\n",
|
||||
" else:\n",
|
||||
" name = ''\n",
|
||||
"\n",
|
||||
" # 将数据添加到列表中\n",
|
||||
" data.append([customer_name, phone_number, oil,car_models, mileage, other_contact, date, price, remarks, name])\n",
|
||||
"\n",
|
||||
"# 创建DataFrame\n",
|
||||
"df = pd.DataFrame(data, columns=['客户名称（车牌）', '手机号', '机油','车型', '公里数', '其他联系方式', '日期', '价格', '备注', '姓名'])\n",
|
||||
"\n",
|
||||
"# 导出到Excel\n",
|
||||
"df.to_excel('car_data.xlsx', index=False)\n",
|
||||
"\n",
|
||||
"print(\"数据已成功导出到car_data.xlsx\")\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "base",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.12.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+11045
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"车辆等级 /html/body/div[5]/div[1]/div/div/div[3]/table[1]/tbody/tr[3]/td[4]\n",
|
||||
"开单时间 /html/body/div[5]/div[1]/div/div/div[3]/table[1]/tbody/tr[3]/td[2]/input 中的value"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import requests
|
||||
import pandas as pd
|
||||
|
||||
url = "https://czbbb.cn/mnt/czbbb/storeMember/czbbbApi.action"
|
||||
|
||||
|
||||
headers = {
|
||||
'user-agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0',
|
||||
"cookies":'JSESSIONID=0F4B4ACDA311E03EB20874676A234F31; Authorization=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6IjVkOTk0MmQwLWNhMGEtNGVkMC04ZTA2LWQ0MWYyNzUwMjNmZSIsImV4cCI6MTcyOTA2OTk5NSwiZ3JvdXBJZCI6bnVsbCwibmJmIjoxNzI4OTgzNTk1LCJ1c2VyTmFtZSI6IumSn-WHjOS6kSIsInN0b3JlSWQiOiJiNjMwNDI4Zi05MDM0LTRjMTktOTM0NS0xYTBiNjBiMzJmOTYiLCJ1c2VyVHlwZSI6IjAifQ.4pwrYnvbmQvL2I_xF-5PMI8TWKCovtIdGKzx_Y6797_8kv_aIHq7SGW5O54irjFy6V7HmUH_ikjl4pAYKmlyhg; Hm_lvt_678c2a986264dd9650b6a59042718858=1728983596; HMACCOUNT=ABFCA62083E00432; Hm_lpvt_678c2a986264dd9650b6a59042718858=1728983627'
|
||||
}
|
||||
|
||||
number = 1
|
||||
all_data =pd.DataFrame()
|
||||
|
||||
for number in range(1,1591):
|
||||
data = f'method=1239&data=%7B%22pageNo%22%3A{number}%2C%22pageSize%22%3A15%7D'
|
||||
|
||||
res = requests.post(url=url,headers=headers,data=data)
|
||||
|
||||
data = res.json()
|
||||
|
||||
for a in range(0,len(data['data'])):
|
||||
data_list = [data['data'][a]]
|
||||
df_new1 = pd.DataFrame(data_list)
|
||||
# 将当前接口的数据添加到总数据中
|
||||
all_data = all_data._append(df_new1, ignore_index=True)
|
||||
print(number)
|
||||
|
||||
all_data.to_excel(r'D:\输出.xlsx')
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,105 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"会员卡明细导出\n",
|
||||
"\n",
|
||||
"需要先导出会员表，之后做了处理 "
|
||||
],
|
||||
"id": "e5353530294464bc"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-03-04T03:47:56.685347Z",
|
||||
"start_time": "2025-03-04T03:46:22.450480Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"import json\n",
|
||||
"import math\n",
|
||||
"import xlrd\n",
|
||||
"from pprint import pprint\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"url = \"https://www.600vip.cn/Business/GoodsConsume/GetBuyCountList\"\n",
|
||||
"\n",
|
||||
"df = pd.read_excel(r\"C:\\Users\\Administrator.DESKTOP-7IC2USJ\\Downloads\\会员表 (2).xls\",sheet_name='Sheet1')\n",
|
||||
"headers = {\n",
|
||||
" 'authority': 'www.600vip.cn',\n",
|
||||
" 'method': 'POST',\n",
|
||||
" 'path': '/Business/GoodsConsume/GetBuyCountList',\n",
|
||||
" 'scheme': 'https',\n",
|
||||
" 'accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'accept-encoding': 'gzip, deflate, br, zstd',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'content-length': '27',\n",
|
||||
" 'content-type': 'application/x-www-form-urlencoded; charset=UTF-8',\n",
|
||||
" 'cookie': 'luckcode=15083577635; luckchain=uid=14871835413470208; rememberPassword=1; ucompcode=TC/rxRH2Nrb9Gamp9vSMxw==; uaccount=qZZOA1XVj9A=; upwd=LjbFhD/UoUs=; Hm_lvt_eb92647b72da97bebb9f81b44b7581a2=1740551044,1741059683; Hm_lpvt_eb92647b72da97bebb9f81b44b7581a2=1741059683; HMACCOUNT=ABFCA62083E00432; sid=f30ef4ef67994057ab41c58dc1385666; 15083577635IsLogonFirst=1; 15083577635LogonTime=202534',\n",
|
||||
" 'origin': 'https://www.600vip.cn',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://www.600vip.cn/Business/ConsumeCount/Index',\n",
|
||||
" 'sec-ch-ua': '\"Not(A:Brand\";v=\"99\", \"Microsoft Edge\";v=\"133\", \"Chromium\";v=\"133\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-origin',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36 Edg/133.0.0.0',\n",
|
||||
" 'x-requested-with': 'XMLHttpRequest'\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"for index,row in tqdm(df.iterrows(), total=len(df)):\n",
|
||||
" data = f\"page=1&rows=20&cardID={row['新会员卡号'].split('A')[1]}\" # 需要注意对会员卡号进行了处理\n",
|
||||
" # print(data)\n",
|
||||
" res = requests.post(url=url, headers=headers, data=data)\n",
|
||||
" res = res.json()[\"rows\"]\n",
|
||||
" for item in res:\n",
|
||||
" all_data.append(item)\n",
|
||||
" # break\n",
|
||||
"df1 = pd.DataFrame(all_data)\n",
|
||||
"df1.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\普盛数据导出.xlsx\")\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"870it [01:33, 9.26it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,520 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "车辆信息",
|
||||
"id": "7a8707c0ac4733a6"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-25T08:46:31.693364Z",
|
||||
"start_time": "2025-06-25T08:19:15.222863Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import time\n",
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Origin': 'https://cms.carisok.com',\n",
|
||||
" 'Referer': 'https://cms.carisok.com/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-site',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"137\", \"Chromium\";v=\"137\", \"Not/A)Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# 添加 retry 装饰器到 request 请求上\n",
|
||||
"@retry(\n",
|
||||
" stop=stop_after_attempt(3), # 最多重试 3 次\n",
|
||||
" wait=wait_exponential(multiplier=1, max=10), # 指数退避，最大等待 10 秒\n",
|
||||
" retry=retry_if_exception_type((requests.exceptions.RequestException,))\n",
|
||||
")\n",
|
||||
"def safe_get_request(url, params, headers):\n",
|
||||
" response = requests.get(url, params=params, headers=headers, timeout=10)\n",
|
||||
" response.raise_for_status() # 如果状态码不是 2xx 抛出异常\n",
|
||||
" return response\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"for i in tqdm(range(1, 227)): # 手动修改页码范围\n",
|
||||
" params = {\n",
|
||||
" 'page': str(i),\n",
|
||||
" 'mobile': '',\n",
|
||||
" 'car_no': '',\n",
|
||||
" 'page_size': '10',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': '014cc085-97d5-4be2-956b-df7ca4d87821-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" response = safe_get_request('https://ssapp-api.carisok.com/pc.php/CustomerManage/cars_list', params=params, headers=headers)\n",
|
||||
" car_list = response.json().get('data', {}).get('car_list', [])\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"第 {i} 页 cars_list 请求失败：{e}\")\n",
|
||||
" car_list = []\n",
|
||||
"\n",
|
||||
" for car in car_list:\n",
|
||||
" car_no = car.get('car_no')\n",
|
||||
"\n",
|
||||
" new_params = {\n",
|
||||
" 'car_no': car_no,\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': '91f56c62-9349-4cdd-97d7-d03e264c8249-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" response1 = safe_get_request('https://ssapp-api.carisok.com/pc.php/CustomerManage/car_detail', params=new_params, headers=headers)\n",
|
||||
" base_data = response1.json().get('data')\n",
|
||||
" car.update(base_data or {})\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"car_detail 请求失败（car_no={car_no}）：{e}\")\n",
|
||||
"\n",
|
||||
" params2 = {\n",
|
||||
" 'page': '1',\n",
|
||||
" 'pagesize': '5',\n",
|
||||
" 'car_no': car_no,\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': '433e8cab-65bb-4ceb-80e0-7fa4eb1a6f8b-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" response2 = safe_get_request('https://ssapp-api.carisok.com/pc.php/CustomerManage/get_user_list', params=params2, headers=headers)\n",
|
||||
" user_info = response2.json().get('data', {}).get('user_list', [])\n",
|
||||
" if user_info:\n",
|
||||
" for user in user_info:\n",
|
||||
" car.update(user)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"get_user_list 请求失败（car_no={car_no}）：{e}\")\n",
|
||||
"\n",
|
||||
" all_data.append(car)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\枫车系统车辆信息.csv', index=False, encoding='utf-8-sig')"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 226/226 [27:16<00:00, 7.24s/it]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 36
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "",
|
||||
"id": "f584c584ca1c4664"
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 会员卡信息",
|
||||
"id": "e4ff5eb93d99d851"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-26T02:54:45.667548Z",
|
||||
"start_time": "2025-06-26T02:32:15.205943Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from tenacity import retry, stop_after_attempt, wait_fixed, retry_if_exception_type\n",
|
||||
"\n",
|
||||
"# ================ 重试装饰器 ================\n",
|
||||
"@retry(\n",
|
||||
" stop=stop_after_attempt(3), # 最多尝试3次\n",
|
||||
" wait=wait_fixed(2), # 每次间隔2秒\n",
|
||||
" retry=retry_if_exception_type((requests.exceptions.RequestException, KeyError)),\n",
|
||||
")\n",
|
||||
"def safe_get_request(url, params, headers):\n",
|
||||
" response = requests.get(url, params=params, headers=headers)\n",
|
||||
" response.raise_for_status()\n",
|
||||
" return response.json()\n",
|
||||
"\n",
|
||||
"# ================ 请求头 ================\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Origin': 'https://cms.carisok.com',\n",
|
||||
" 'Referer': 'https://cms.carisok.com/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-site',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"137\", \"Chromium\";v=\"137\", \"Not/A)Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data1 = []\n",
|
||||
"all_data2 = []\n",
|
||||
"\n",
|
||||
"# ================ 主循环获取数据 ================\n",
|
||||
"for i in tqdm(range(1, 78)): # 注意修改\n",
|
||||
" params = {\n",
|
||||
" 'pagesize': '20',\n",
|
||||
" 'page': i,\n",
|
||||
" 'user_name': '',\n",
|
||||
" 'custome_level': '0',\n",
|
||||
" 'user_mobile': '',\n",
|
||||
" 'car_no': '',\n",
|
||||
" 'cost1': '',\n",
|
||||
" 'cost2': '',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c5cdb71a-e195-4390-aa70-21d31bd62a63-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" user_list_response = safe_get_request(\n",
|
||||
" 'https://ssapp-api.carisok.com/pc.php/CustomerManage/get_user_list',\n",
|
||||
" params=params,\n",
|
||||
" headers=headers\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" user_list = user_list_response.get(\"data\").get('user_list', [])\n",
|
||||
" \n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"获取用户列表失败: {e}\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for user in user_list:\n",
|
||||
" user_id = user.get('user_id')\n",
|
||||
" if not user_id:\n",
|
||||
" print(\"无效的 user_id，跳过该用户\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" # ================ 获取用户详情 ================\n",
|
||||
" params1 = {\n",
|
||||
" 'user_id': user_id,\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c5cdb71a-e195-4390-aa70-21d31bd62a63-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" base_data = safe_get_request(\n",
|
||||
" 'https://ssapp-api.carisok.com/pc.php/CustomerManage/get_user_info',\n",
|
||||
" params=params1,\n",
|
||||
" headers=headers\n",
|
||||
" ).get('data', {})\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"获取用户详情失败 (user_id={user_id}): {e}\")\n",
|
||||
" base_data = {}\n",
|
||||
"\n",
|
||||
" merged_user = {**user, **base_data}\n",
|
||||
"\n",
|
||||
" # ================ 获取储值卡 ================\n",
|
||||
" params2 = {\n",
|
||||
" 'user_id': user_id,\n",
|
||||
" 'pagesize': '20',\n",
|
||||
" 'page': '1',\n",
|
||||
" 'startDate': '',\n",
|
||||
" 'endDate': '',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c5cdb71a-e195-4390-aa70-21d31bd62a63-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" card_data1 = safe_get_request(\n",
|
||||
" 'https://ssapp-api.carisok.com/pc.php/CustomerManage/recharge_card_list',\n",
|
||||
" params=params2,\n",
|
||||
" headers=headers\n",
|
||||
" )\n",
|
||||
" cardList1 = card_data1.get('data', {}).get('cardList', [])\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"获取储值卡失败 (user_id={user_id}): {e}\")\n",
|
||||
" cardList1 = []\n",
|
||||
"\n",
|
||||
" for card1 in cardList1:\n",
|
||||
" all_data1.append({**merged_user, **card1})\n",
|
||||
"\n",
|
||||
" # ================ 获取套餐卡 ================\n",
|
||||
" params3 = {\n",
|
||||
" 'user_id': user_id,\n",
|
||||
" 'pagesize': '20',\n",
|
||||
" 'page': '1',\n",
|
||||
" 'keyword': '',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c5cdb71a-e195-4390-aa70-21d31bd62a63-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" card_data2 = safe_get_request(\n",
|
||||
" 'https://ssapp-api.carisok.com/pc.php/CustomerManage/sstore_card_list',\n",
|
||||
" params=params3,\n",
|
||||
" headers=headers\n",
|
||||
" )\n",
|
||||
" cardList2 = card_data2.get('data', {}).get('cardList', [])\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"获取套餐卡失败 (user_id={user_id}): {e}\")\n",
|
||||
" cardList2 = []\n",
|
||||
"\n",
|
||||
" for card2 in cardList2:\n",
|
||||
" all_data2.append({**merged_user, **card2})\n",
|
||||
"\n",
|
||||
"# ================ 写入 CSV 文件 ================\n",
|
||||
"df1 = pd.DataFrame(all_data1)\n",
|
||||
"df2 = pd.DataFrame(all_data2)\n",
|
||||
"\n",
|
||||
"output_path_1 = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\枫车储值卡.csv'\n",
|
||||
"output_path_2 = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\枫车套餐卡.csv'\n",
|
||||
"\n",
|
||||
"df1.to_csv(output_path_1, index=False)\n",
|
||||
"df2.to_csv(output_path_2, index=False)\n",
|
||||
"\n",
|
||||
"print(\"导出完成！\")"
|
||||
],
|
||||
"id": "80536c4cd53d5e48",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 77/77 [22:30<00:00, 17.54s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"导出完成！\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 64
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 历史维修记录",
|
||||
"id": "45fa307391e01382"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-26T08:38:32.921361Z",
|
||||
"start_time": "2025-06-26T06:58:33.455924Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from retrying import retry\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"import logging\n",
|
||||
"import os\n",
|
||||
"import json\n",
|
||||
"\n",
|
||||
"# 配置日志\n",
|
||||
"logging.basicConfig(\n",
|
||||
" filename='fetch_errors.log',\n",
|
||||
" level=logging.ERROR,\n",
|
||||
" format='%(asctime)s - %(levelname)s - %(message)s'\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# 请求头\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Origin': 'https://cms.carisok.com',\n",
|
||||
" 'Referer': 'https://cms.carisok.com/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-site',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"137\", \"Chromium\";v=\"137\", \"Not/A)Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# 存储数据的列表\n",
|
||||
"car_data = []\n",
|
||||
"operate_data = []\n",
|
||||
"order_data = []\n",
|
||||
"server_data = []\n",
|
||||
"settlement_data = []\n",
|
||||
"\n",
|
||||
"# 已抓取的订单 ID 文件（用于断点续传）\n",
|
||||
"PROCESSED_FILE = 'processed_orders.json'\n",
|
||||
"\n",
|
||||
"# 如果存在已处理文件则加载，否则初始化为空集合\n",
|
||||
"processed_orders = set()\n",
|
||||
"if os.path.exists(PROCESSED_FILE):\n",
|
||||
" with open(PROCESSED_FILE, 'r', encoding='utf-8') as f:\n",
|
||||
" processed_orders = set(json.load(f))\n",
|
||||
"\n",
|
||||
"# 失败重试装饰器\n",
|
||||
"@retry(stop_max_attempt_number=3, wait_fixed=2000)\n",
|
||||
"def fetch_url(url, params=None):\n",
|
||||
" response = requests.get(url, headers=headers, params=params, timeout=10)\n",
|
||||
" response.raise_for_status()\n",
|
||||
" return response.json()\n",
|
||||
"\n",
|
||||
"@retry(stop_max_attempt_number=3, wait_fixed=2000)\n",
|
||||
"def fetch_order_detail(orderId):\n",
|
||||
" detail_url = 'https://ssapp-api.carisok.com/pc.php/GainRecord/get_order_detail'\n",
|
||||
" params = {\n",
|
||||
" 'orderId': orderId,\n",
|
||||
" 'businessType': '1',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c41f197b-ef5c-4813-8d97-8947f4289243-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" response = requests.get(detail_url, params=params, headers=headers, timeout=10)\n",
|
||||
" response.raise_for_status()\n",
|
||||
" return response.json()\n",
|
||||
"\n",
|
||||
"@retry(stop_max_attempt_number=3, wait_fixed=2000)\n",
|
||||
"def fetch_page(i):\n",
|
||||
" list_url = f'https://ssapp-api.carisok.com/pc.php/PcPickcarOrder/lists?name=&mobile=&orderState=0&workState=0&businessType=0&page={i}&pagesize=20&carNo=&orderNo=&operate_count=&companyName=&startDate&endDate&api_version=4.00&__trace_id=6f72ab8e-f569-4b5d-aa7b-c890eec6756d-store&token=4d62a33aa979d6d242d6b0198afbbf27'\n",
|
||||
" return fetch_url(list_url)\n",
|
||||
"\n",
|
||||
"# 主循环\n",
|
||||
"for i in tqdm(range(1, 350), desc=\"正在抓取页面\"):\n",
|
||||
" try:\n",
|
||||
" response = fetch_page(i)\n",
|
||||
" history_list = response.get('data', {}).get(\"listData\", [])\n",
|
||||
" except Exception as e:\n",
|
||||
" logging.error(f\"第{i}页抓取失败: {e}\")\n",
|
||||
" print(f\"⚠️ 第{i}页抓取失败\")\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for history in history_list:\n",
|
||||
" orderId = history.get('orderId')\n",
|
||||
"\n",
|
||||
" # 断点续传：跳过已处理的订单\n",
|
||||
" if str(orderId) in processed_orders:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" data = fetch_order_detail(orderId)\n",
|
||||
" data_content = data.get('data', {})\n",
|
||||
"\n",
|
||||
" carInfo = data_content.get('carInfo', [])\n",
|
||||
" operate = data_content.get('operate', [])\n",
|
||||
" orderInfo = data_content.get('orderInfo', [])\n",
|
||||
" serverInfo = data_content.get('serverInfo', [])\n",
|
||||
" settlementInfo = data_content.get('settlementInfo', [])\n",
|
||||
"\n",
|
||||
" # 数据合并到对应表中\n",
|
||||
" def process_data(data_list, target_list):\n",
|
||||
" if isinstance(data_list, dict):\n",
|
||||
" data_list[\"orderid\"] = orderId\n",
|
||||
" target_list.append(data_list)\n",
|
||||
" elif isinstance(data_list, list):\n",
|
||||
" for item in data_list:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" target_list.append(item)\n",
|
||||
"\n",
|
||||
" process_data(carInfo, car_data)\n",
|
||||
" process_data(operate, operate_data)\n",
|
||||
" process_data(orderInfo, order_data)\n",
|
||||
" process_data(serverInfo, server_data)\n",
|
||||
" process_data(settlementInfo, settlement_data)\n",
|
||||
"\n",
|
||||
" # 记录已处理的订单ID\n",
|
||||
" processed_orders.add(str(orderId))\n",
|
||||
" with open(PROCESSED_FILE, 'w', encoding='utf-8') as f:\n",
|
||||
" json.dump(list(processed_orders), f)\n",
|
||||
"\n",
|
||||
" except requests.exceptions.RequestException as req_err:\n",
|
||||
" logging.error(f\"请求失败 (orderId={orderId}): {req_err}\")\n",
|
||||
" print(f\"⚠️ 请求失败 (orderId={orderId}): {req_err}\")\n",
|
||||
" except Exception as e:\n",
|
||||
" logging.error(f\"未知错误 (orderId={orderId}): {e}\")\n",
|
||||
" print(f\"❌ 未知错误 (orderId={orderId}): {e}\")\n",
|
||||
"\n",
|
||||
"# 写入 CSV 文件\n",
|
||||
"output_path = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\\\'\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(car_data)\n",
|
||||
"df2 = pd.DataFrame(operate_data)\n",
|
||||
"df3 = pd.DataFrame(order_data)\n",
|
||||
"df4 = pd.DataFrame(server_data)\n",
|
||||
"df5 = pd.DataFrame(settlement_data)\n",
|
||||
"\n",
|
||||
"df1.to_csv(output_path + 'car.csv', index=False)\n",
|
||||
"df2.to_csv(output_path + 'operate.csv', index=False)\n",
|
||||
"df3.to_csv(output_path + 'order.csv', index=False)\n",
|
||||
"df4.to_csv(output_path + 'server.csv', index=False)\n",
|
||||
"df5.to_csv(output_path + 'settlement.csv', index=False)\n",
|
||||
"\n",
|
||||
"print(\"✅ 数据写入完成\")"
|
||||
],
|
||||
"id": "f8b29d309bd85d7b",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"正在抓取页面: 100%|██████████| 349/349 [1:39:59<00:00, 17.19s/it]\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"✅ 数据写入完成\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 2
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import requests
|
||||
from tqdm import tqdm
|
||||
import pandas as pd
|
||||
import time
|
||||
|
||||
cookies = {
|
||||
'plum_session_manage': '557ja766db2on5fvet5nhvnavt',
|
||||
}
|
||||
|
||||
headers = {
|
||||
'accept': 'application/json, text/plain, */*',
|
||||
'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',
|
||||
'priority': 'u=1, i',
|
||||
'referer': 'https://qfy.lycjh.com/public/h5/carWeb/index.html',
|
||||
'sec-ch-ua': '"Microsoft Edge";v="143", "Chromium";v="143", "Not A(Brand";v="24"',
|
||||
'sec-ch-ua-mobile': '?0',
|
||||
'sec-ch-ua-platform': '"Windows"',
|
||||
'sec-fetch-dest': 'empty',
|
||||
'sec-fetch-mode': 'cors',
|
||||
'sec-fetch-site': 'same-origin',
|
||||
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',
|
||||
# 'cookie': 'plum_session_manage=557ja766db2on5fvet5nhvnavt',
|
||||
}
|
||||
all_data = []
|
||||
for i in tqdm(range(0, 52)):
|
||||
params = {
|
||||
'map': 'applet_car_vehicle_list',
|
||||
'query': '',
|
||||
'page': i,
|
||||
'searchValue': '',
|
||||
'useCount': '200',
|
||||
'sourceType': 'web',
|
||||
'token': '37209C2A406D3B40068394CF3C27EE0C',
|
||||
'appid': 'wx5bc8c5b57e2223b9',
|
||||
'suid': 'hcpza678h9',
|
||||
'version': '1',
|
||||
}
|
||||
# 皖BF1821
|
||||
response = requests.get('https://qfy.lycjh.com/applet.php', params=params, cookies=cookies, headers=headers)
|
||||
data_list = response.json().get('data')
|
||||
all_data.extend(data_list)
|
||||
time.sleep(0.3)
|
||||
|
||||
df = pd.DataFrame(all_data)
|
||||
df.to_csv(fr"D:\Idea Project\F6+宜搭+其它(1)\张阳脚本\文件输出\汽服云车辆信息.csv")
|
||||
@@ -0,0 +1,304 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-26T05:39:09.305879Z",
|
||||
"start_time": "2025-06-26T05:39:09.248395Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from retrying import retry\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"# 设置重试策略\n",
|
||||
"@retry(stop_max_attempt_number=3, wait_fixed=2000) # 最多重试3次，每次间隔2秒\n",
|
||||
"def fetch_url(url, params=None):\n",
|
||||
" response = requests.get(url, headers=headers, params=params, timeout=10)\n",
|
||||
" response.raise_for_status() # 抛出 HTTP 错误（如 500）\n",
|
||||
" return response.json()\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Origin': 'https://cms.carisok.com',\n",
|
||||
" 'Referer': 'https://cms.carisok.com/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-site',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"137\", \"Chromium\";v=\"137\", \"Not/A)Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"car_data = []\n",
|
||||
"operate_data = []\n",
|
||||
"order_data = []\n",
|
||||
"server_data = []\n",
|
||||
"settlement_data = []\n",
|
||||
"\n",
|
||||
"for i in tqdm(range(1, 350)):\n",
|
||||
" list_url = f'https://ssapp-api.carisok.com/pc.php/PcPickcarOrder/lists?name=&mobile=&orderState=0&workState=0&businessType=0&page={i}&pagesize=20&carNo=&orderNo=&operate_count=&companyName=&startDate&endDate&api_version=4.00&__trace_id=6f72ab8e-f569-4b5d-aa7b-c890eec6756d-store&token=4d62a33aa979d6d242d6b0198afbbf27'\n",
|
||||
"\n",
|
||||
" response = fetch_url(list_url)\n",
|
||||
"\n",
|
||||
" history_list = response.get('data').get(\"listData\")\n",
|
||||
" \n",
|
||||
" for history in history_list:\n",
|
||||
" orderId = history.get('orderId')\n",
|
||||
"\n",
|
||||
" params = {\n",
|
||||
" 'orderId': orderId,\n",
|
||||
" 'businessType': '1',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c41f197b-ef5c-4813-8d97-8947f4289243-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
" }\n",
|
||||
" response = requests.get('https://ssapp-api.carisok.com/pc.php/GainRecord/get_order_detail', params=params, headers=headers)\n",
|
||||
"\n",
|
||||
" data = response.json().get('data')\n",
|
||||
" \n",
|
||||
" carInfo = data.get('carInfo', [])\n",
|
||||
" operate = data.get('operate', [])\n",
|
||||
" orderInfo = data.get('orderInfo', [])\n",
|
||||
" serverInfo = data.get('serverInfo', [])\n",
|
||||
" settlementInfo = data.get('settlementInfo', [])\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" if isinstance(carInfo, dict):\n",
|
||||
" carInfo[\"orderid\"] = orderId\n",
|
||||
" car_data.append(carInfo)\n",
|
||||
" elif isinstance(carInfo, list):\n",
|
||||
" for item in carInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" car_data.append(item)\n",
|
||||
" \n",
|
||||
" # 处理 orderInfo（可能是字典或列表）\n",
|
||||
" if isinstance(orderInfo, dict):\n",
|
||||
" orderInfo[\"orderid\"] = orderId\n",
|
||||
" order_data.append(orderInfo)\n",
|
||||
" elif isinstance(orderInfo, list):\n",
|
||||
" for item in orderInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" order_data.append(item)\n",
|
||||
" \n",
|
||||
" # 处理 operate（通常是列表，但也可能不是）\n",
|
||||
" if isinstance(operate, list):\n",
|
||||
" for item in operate:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" operate_data.append(item)\n",
|
||||
" \n",
|
||||
" # 处理 serverInfo（可能是字典或列表）\n",
|
||||
" if isinstance(serverInfo, dict):\n",
|
||||
" serverInfo[\"orderid\"] = orderId\n",
|
||||
" server_data.append(serverInfo)\n",
|
||||
" elif isinstance(serverInfo, list):\n",
|
||||
" for item in serverInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" server_data.append(item)\n",
|
||||
" \n",
|
||||
" # 处理 settlementInfo（可能是字典或列表）\n",
|
||||
" if isinstance(settlementInfo, dict):\n",
|
||||
" settlementInfo[\"orderid\"] = orderId\n",
|
||||
" settlement_data.append(settlementInfo)\n",
|
||||
" elif isinstance(settlementInfo, list):\n",
|
||||
" for item in settlementInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" settlement_data.append(item)\n",
|
||||
" \n",
|
||||
" \n",
|
||||
"\n",
|
||||
"# 写入 CSV 文件\n",
|
||||
"df1 = pd.DataFrame(car_data)\n",
|
||||
"df2 = pd.DataFrame(operate_data)\n",
|
||||
"df3 = pd.DataFrame(order_data)\n",
|
||||
"df4 = pd.DataFrame(server_data)\n",
|
||||
"df5 = pd.DataFrame(settlement_data)\n",
|
||||
"\n",
|
||||
"output_path = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\\\'\n",
|
||||
"\n",
|
||||
"df1.to_csv(output_path + 'car.csv', index=False)\n",
|
||||
"df2.to_csv(output_path + 'operate.csv', index=False)\n",
|
||||
"df3.to_csv(output_path + 'order.csv', index=False)\n",
|
||||
"df4.to_csv(output_path + 'server.csv', index=False)\n",
|
||||
"df5.to_csv(output_path + 'settlement.csv', index=False)\n",
|
||||
"\n",
|
||||
"print(\"✅ 数据写入完成\")"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"ename": "NameError",
|
||||
"evalue": "name 'tqdm' is not defined",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mNameError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[48], line 33\u001B[0m\n\u001B[0;32m 30\u001B[0m server_data \u001B[38;5;241m=\u001B[39m []\n\u001B[0;32m 31\u001B[0m settlement_data \u001B[38;5;241m=\u001B[39m []\n\u001B[1;32m---> 33\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m i \u001B[38;5;129;01min\u001B[39;00m tqdm(\u001B[38;5;28mrange\u001B[39m(\u001B[38;5;241m1\u001B[39m, \u001B[38;5;241m350\u001B[39m)):\n\u001B[0;32m 34\u001B[0m list_url \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mhttps://ssapp-api.carisok.com/pc.php/PcPickcarOrder/lists?name=&mobile=&orderState=0&workState=0&businessType=0&page=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00mi\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m&pagesize=20&carNo=&orderNo=&operate_count=&companyName=&startDate&endDate&api_version=4.00&__trace_id=6f72ab8e-f569-4b5d-aa7b-c890eec6756d-store&token=4d62a33aa979d6d242d6b0198afbbf27\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[0;32m 36\u001B[0m response \u001B[38;5;241m=\u001B[39m fetch_url(list_url)\n",
|
||||
"\u001B[1;31mNameError\u001B[0m: name 'tqdm' is not defined"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 48
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-26T05:35:06.024711Z",
|
||||
"start_time": "2025-06-26T05:35:05.187886Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Origin': 'https://cms.carisok.com',\n",
|
||||
" 'Referer': 'https://cms.carisok.com/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-site',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36 Edg/137.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"137\", \"Chromium\";v=\"137\", \"Not/A)Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"orderId = '1334846'\n",
|
||||
"params = {\n",
|
||||
" 'orderId': orderId,\n",
|
||||
" 'businessType': '1',\n",
|
||||
" 'api_version': '4.00',\n",
|
||||
" '__trace_id': 'c41f197b-ef5c-4813-8d97-8947f4289243-store',\n",
|
||||
" 'token': '4d62a33aa979d6d242d6b0198afbbf27',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"response = requests.get('https://ssapp-api.carisok.com/pc.php/GainRecord/get_order_detail', params=params, headers=headers)\n",
|
||||
"\n",
|
||||
"data = response.json().get('data')\n",
|
||||
"\n",
|
||||
"carInfo = data.get('carInfo', {})\n",
|
||||
"operate = data.get('operate', [])\n",
|
||||
"serverInfo = data.get('serverInfo', [])\n",
|
||||
"orderInfo = data.get('orderInfo', {})\n",
|
||||
"settlementInfo = data.get('settlementInfo', [])\n",
|
||||
"\n",
|
||||
"car_data=[]\n",
|
||||
"operate_data=[]\n",
|
||||
"order_data = []\n",
|
||||
"server_data = []\n",
|
||||
"settlement_data = []\n",
|
||||
"# 处理 carInfo（可能是字典或列表）\n",
|
||||
"if isinstance(carInfo, dict):\n",
|
||||
" carInfo[\"orderid\"] = orderId\n",
|
||||
" car_data.append(carInfo)\n",
|
||||
"elif isinstance(carInfo, list):\n",
|
||||
" for item in carInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" car_data.append(item)\n",
|
||||
"\n",
|
||||
"# 处理 orderInfo（可能是字典或列表）\n",
|
||||
"if isinstance(orderInfo, dict):\n",
|
||||
" orderInfo[\"orderid\"] = orderId\n",
|
||||
" order_data.append(orderInfo)\n",
|
||||
"elif isinstance(orderInfo, list):\n",
|
||||
" for item in orderInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" order_data.append(item)\n",
|
||||
"\n",
|
||||
"# 处理 operate（通常是列表，但也可能不是）\n",
|
||||
"if isinstance(operate, list):\n",
|
||||
" for item in operate:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" operate_data.append(item)\n",
|
||||
"\n",
|
||||
"# 处理 serverInfo（可能是字典或列表）\n",
|
||||
"if isinstance(serverInfo, dict):\n",
|
||||
" serverInfo[\"orderid\"] = orderId\n",
|
||||
" server_data.append(serverInfo)\n",
|
||||
"elif isinstance(serverInfo, list):\n",
|
||||
" for item in serverInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" server_data.append(item)\n",
|
||||
"\n",
|
||||
"# 处理 settlementInfo（可能是字典或列表）\n",
|
||||
"if isinstance(settlementInfo, dict):\n",
|
||||
" settlementInfo[\"orderid\"] = orderId\n",
|
||||
" settlement_data.append(settlementInfo)\n",
|
||||
"elif isinstance(settlementInfo, list):\n",
|
||||
" for item in settlementInfo:\n",
|
||||
" if isinstance(item, dict):\n",
|
||||
" item[\"orderid\"] = orderId\n",
|
||||
" settlement_data.append(item)\n",
|
||||
" \n",
|
||||
" \n",
|
||||
" \n",
|
||||
" \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(car_data)\n",
|
||||
"df2 = pd.DataFrame(operate_data)\n",
|
||||
"df3 = pd.DataFrame(order_data)\n",
|
||||
"df4 = pd.DataFrame(server_data)\n",
|
||||
"df5 = pd.DataFrame(settlement_data)\n",
|
||||
"\n",
|
||||
"df1.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\car.csv', index=False)\n",
|
||||
"df2.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\operate.csv', index=False)\n",
|
||||
"df3.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\order.csv', index=False)\n",
|
||||
"df4.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\server.csv', index=False)\n",
|
||||
"df5.to_csv(r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\settlement.csv', index=False)\n",
|
||||
"print(\"数据写入完成\")"
|
||||
],
|
||||
"id": "e1ca5734b455a269",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"数据写入完成\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 42
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,251 @@
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.support.ui import WebDriverWait
|
||||
from selenium.webdriver.support import expected_conditions as EC
|
||||
import time
|
||||
from urllib.parse import urljoin
|
||||
import pandas as pd
|
||||
from selenium.webdriver import Chrome
|
||||
from selenium.webdriver.chrome.service import Service
|
||||
from datetime import datetime
|
||||
from selenium.webdriver.chrome.options import Options
|
||||
from datetime import datetime, timedelta
|
||||
from selenium.common.exceptions import NoSuchElementException
|
||||
from tqdm import tqdm
|
||||
from selenium.common.exceptions import TimeoutException
|
||||
|
||||
# 设置Chrome选项
|
||||
chrome_options = Options()
|
||||
# 设置为无头模式（不打开浏览器窗口）
|
||||
# chrome_options.add_argument('--headless')
|
||||
chrome_options.add_argument('--disable-gpu')
|
||||
chrome_options.add_argument('--no-sandbox')
|
||||
|
||||
# 指定ChromeDriver路径
|
||||
service = Service(executable_path='D:\ProgramTools\chromedriver-win64\chromedriver.exe')
|
||||
|
||||
# 创建WebDriver对象
|
||||
driver = webdriver.Chrome(service=service, options=chrome_options)
|
||||
|
||||
# 目标网址
|
||||
url = 'http://best.aichedian.com/login/'
|
||||
url1 = ''
|
||||
username = '15307259977'
|
||||
password = 'juanzi810119'
|
||||
title = 'HGYH -- HGYH'
|
||||
|
||||
# 访问网页
|
||||
driver.get(url)
|
||||
|
||||
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id="username"]'))).send_keys(username)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[4]/div/input'))).click()
|
||||
time.sleep(2)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select-dom"]'))).click()
|
||||
# time.sleep(200)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select_box"]/div/div[1]/div'))).click()
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select_box"]/div/div[1]/div/span[1]'))).click()
|
||||
time.sleep(5)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[3]/div[2]/div/input'))).send_keys(password)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[4]/div/input'))).click()
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
# 设置起始日期为今天，结束日期为两年前 # 需要修改日期
|
||||
end_date = datetime.now()
|
||||
start_date = end_date - timedelta(days=1465) # 1358
|
||||
# demo_date =end_date - timedelta(days=786)
|
||||
current_date = start_date
|
||||
all_data = []
|
||||
|
||||
# 定义表头
|
||||
headers = [
|
||||
'订单号',
|
||||
'开单时间', '入账时间', '车辆', '车主', '订单详情',
|
||||
'车牌号码', '车辆品牌', '会员卡号', '车主姓名', '联系方式',
|
||||
'服务名称', '销售人员', '施工人员', '应付金额', '施工时间', '完工时间', '服务评分',
|
||||
'产品名称', '型号', '单价', '数量', '总价', '销售人员', '销售时间',
|
||||
'支付方式', '账号', '金额', '时间', '备注'
|
||||
]
|
||||
|
||||
# 进度条
|
||||
date_range = pd.date_range(start=start_date, end=end_date)
|
||||
# date_range = pd.date_range(start=start_date, end=demo_date)
|
||||
with tqdm(total=len(date_range), desc="处理日期") as pbar:
|
||||
no_data_count = 0 # 记录连续没有数据的次数
|
||||
for current_date in date_range:
|
||||
date_str = current_date.strftime('%Y/%m/%d')
|
||||
# http://best.aichedian.com/report/order-count-detail/?dt=2024/12/01
|
||||
url1 = f'http://best.aichedian.com/report/order-count-detail/?dt={date_str}'
|
||||
|
||||
driver.get(url1) # 获取每日订单
|
||||
# 获取指定 XPath 下的所有 <tr> 元素
|
||||
xpath = '//*[@id="x-single-content"]/table[2]/tbody/tr'
|
||||
try:
|
||||
rows = WebDriverWait(driver, 5).until(
|
||||
EC.presence_of_all_elements_located((By.XPATH, xpath))
|
||||
)
|
||||
if not rows:
|
||||
raise NoSuchElementException # 如果没有找到任何行，抛出异常
|
||||
no_data_count = 0 # 重置计数器
|
||||
except TimeoutException:
|
||||
no_data_count += 1
|
||||
if no_data_count >= 1000:
|
||||
print("连续1000天没有数据，退出循环")
|
||||
break
|
||||
pbar.update(1)
|
||||
continue
|
||||
|
||||
for row in rows:
|
||||
# 获取每一行中的所有 <td> 元素
|
||||
cells = row.find_elements(By.TAG_NAME, 'td')
|
||||
row_data = [cell.text for cell in cells] # 将一行中的所有单元格文本存入列表
|
||||
start_dj_data = row_data[0] # 开始时间
|
||||
get_money_data = row_data[1] # 入账时间
|
||||
car_number = row_data[2] # 车牌名称
|
||||
customer = row_data[3] # 假设第二列是客户名称
|
||||
order_details = '\n'.join(row_data) # 将行数据合并为一个字符串，每列之间用换行符分隔
|
||||
|
||||
for cell in cells:
|
||||
try:
|
||||
base_url = 'http://xlsf.aichedian.com/'
|
||||
link = cell.find_element(By.XPATH, './/a[@class="underline-styled"]')
|
||||
relative_url = link.get_attribute('href')
|
||||
absolute_url = urljoin(base_url, relative_url) # 明细url
|
||||
details_id = absolute_url.split('/')[-2]
|
||||
driver.get(absolute_url) # 获取订单明细
|
||||
|
||||
# 获取基本信息
|
||||
base_details = WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="x-single-content"]/ul'))
|
||||
)
|
||||
li_items = base_details.find_elements(By.TAG_NAME, 'li')
|
||||
base_info = {li.text.split(':')[0].strip(): li.text.split(':')[1].strip() if ':' in li.text else ''
|
||||
for li in li_items}
|
||||
|
||||
# 获取具体信息，如果不存在则返回空字符串
|
||||
license_plate = base_info.get('车牌号', '')
|
||||
customer_name = base_info.get('客户名称', '')
|
||||
order_id = base_info.get('订单编号', '')
|
||||
|
||||
# 获取服务项目
|
||||
service_info = []
|
||||
try:
|
||||
h3_element = WebDriverWait(driver, 0.2).until(
|
||||
EC.presence_of_element_located((By.XPATH, '//h3[text()="服务项目"]'))
|
||||
)
|
||||
if h3_element.text == "服务项目":
|
||||
table = h3_element.find_element(By.XPATH, './following-sibling::table')
|
||||
rows = table.find_elements(By.TAG_NAME, 'tr')
|
||||
for row1 in rows[1:]:
|
||||
li_items = row1.find_elements(By.TAG_NAME, 'td')
|
||||
service_info.append({
|
||||
'服务名称': li_items[0].text,
|
||||
'销售人员': li_items[1].text,
|
||||
'施工人员': li_items[2].text,
|
||||
'应付金额': li_items[3].text,
|
||||
'施工时间': li_items[4].text,
|
||||
'完工时间': li_items[5].text,
|
||||
'服务评分': li_items[6].text
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
# 获取销售产品
|
||||
product_info = []
|
||||
try:
|
||||
h3_element = WebDriverWait(driver, 0.2).until(
|
||||
EC.presence_of_element_located((By.XPATH, '//h3[text()="销售产品"]'))
|
||||
)
|
||||
if h3_element.text == "销售产品":
|
||||
table = h3_element.find_element(By.XPATH, './following-sibling::table')
|
||||
rows = table.find_elements(By.TAG_NAME, 'tr')
|
||||
for row2 in rows[1:]:
|
||||
li_items = row2.find_elements(By.TAG_NAME, 'td')
|
||||
product_info.append({
|
||||
'产品名称': li_items[0].text,
|
||||
'型号': li_items[1].text,
|
||||
'单价': li_items[2].text,
|
||||
'数量': li_items[3].text,
|
||||
'总价': li_items[4].text,
|
||||
'销售人员': li_items[5].text,
|
||||
'销售时间': li_items[6].text
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
# 获取支付记录
|
||||
payment_info = []
|
||||
try:
|
||||
h3_element = WebDriverWait(driver, 0.2).until(
|
||||
EC.presence_of_element_located((By.XPATH, '//h3[text()="支付记录"]'))
|
||||
)
|
||||
if h3_element.text == "支付记录":
|
||||
table = h3_element.find_element(By.XPATH, './following-sibling::table')
|
||||
rows = table.find_elements(By.TAG_NAME, 'tr')
|
||||
for row3 in rows[1:]:
|
||||
li_items = row3.find_elements(By.TAG_NAME, 'td')
|
||||
payment_info.append({
|
||||
'支付方式': li_items[0].text,
|
||||
'账号': li_items[1].text,
|
||||
'金额': li_items[2].text,
|
||||
'时间': li_items[3].text,
|
||||
'备注': li_items[4].text
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
# 将所有信息组合成一个字典
|
||||
order_info = {
|
||||
'订单号': details_id,
|
||||
'开单时间': start_dj_data,
|
||||
'入账时间': get_money_data,
|
||||
'车辆': car_number,
|
||||
'车主': customer,
|
||||
'车牌号码': base_info.get('车牌号码', ''),
|
||||
'车辆品牌': base_info.get('车辆品牌', ''),
|
||||
'会员卡号': base_info.get('会员卡号', ''),
|
||||
'车主姓名': base_info.get('车主姓名', ''),
|
||||
'联系方式': base_info.get('联系方式', ''),
|
||||
# '订单详情': order_details,
|
||||
'服务名称': '\n'.join([item['服务名称'] for item in service_info]),
|
||||
'销售人员': '\n'.join([item['销售人员'] for item in service_info]),
|
||||
'施工人员': '\n'.join([item['施工人员'] for item in service_info]),
|
||||
'应付金额': '\n'.join([item['应付金额'] for item in service_info]),
|
||||
'施工时间': '\n'.join([item['施工时间'] for item in service_info]),
|
||||
'完工时间': '\n'.join([item['完工时间'] for item in service_info]),
|
||||
'服务评分': '\n'.join([item['服务评分'] for item in service_info]),
|
||||
'产品名称': '\n'.join([item['产品名称'] for item in product_info]),
|
||||
'型号': '\n'.join([item['型号'] for item in product_info]),
|
||||
'单价': '\n'.join([item['单价'] for item in product_info]),
|
||||
'数量': '\n'.join([item['数量'] for item in product_info]),
|
||||
'总价': '\n'.join([item['总价'] for item in product_info]),
|
||||
'销售人员': '\n'.join([item['销售人员'] for item in product_info]),
|
||||
'销售时间': '\n'.join([item['销售时间'] for item in product_info]),
|
||||
'支付方式': '\n'.join([item['支付方式'] for item in payment_info]),
|
||||
'账号': '\n'.join([item['账号'] for item in payment_info]),
|
||||
'金额': '\n'.join([item['金额'] for item in payment_info]),
|
||||
'时间': '\n'.join([item['时间'] for item in payment_info]),
|
||||
'备注': '\n'.join([item['备注'] for item in payment_info])
|
||||
}
|
||||
|
||||
all_data.append(order_info) # 将完整订单信息添加到总数据列表中
|
||||
time.sleep(0.1)
|
||||
driver.back()
|
||||
except NoSuchElementException:
|
||||
continue # 如果没有找到 <a> 标签，继续下一个 <td>
|
||||
|
||||
driver.back()
|
||||
pbar.update(1)
|
||||
|
||||
# 使用pandas将数据保存为Excel文件
|
||||
df = pd.DataFrame(all_data, columns=headers)
|
||||
df.to_excel('爱车店数据导出.xlsx', index=False) # 保存为Excel文件
|
||||
|
||||
# 关闭浏览器
|
||||
driver.quit()
|
||||
@@ -0,0 +1,141 @@
|
||||
import pandas as pd
|
||||
from tqdm import tqdm
|
||||
import openpyxl
|
||||
from openpyxl.styles import PatternFill
|
||||
|
||||
# 读取Excel文件
|
||||
df = pd.read_excel(r"D:\Idea Project\F6+宜搭+其它(1)\new\竞品系统数据导出\爱车店数据导出.xlsx")
|
||||
|
||||
# 初始化一个新的列表来存储结果
|
||||
result_list = []
|
||||
|
||||
|
||||
# 定义一个函数来安全地拆分字符串
|
||||
def safe_split(value, delimiter='\n'):
|
||||
if pd.notna(value):
|
||||
return value.split(delimiter)
|
||||
return ['']
|
||||
|
||||
|
||||
# 获取需要拆分的列名
|
||||
columns_to_split = {
|
||||
'产品名称组': ['产品名称', '型号', '单价', '数量', '总价', '销售人员', '销售时间'],
|
||||
'服务名称组': ['服务名称', '施工人员', '应付金额', '施工时间', '完工时间', '服务评分'],
|
||||
'支付方式组': ['支付方式', '账号', '金额', '时间', '备注']
|
||||
}
|
||||
|
||||
# 获取需要保留的列名
|
||||
columns_to_keep = ['订单号', '开单时间', '入账时间', '车辆', '车主', '订单详情']
|
||||
|
||||
# 使用 tqdm 包装 iterrows() 以显示进度条
|
||||
for index, row in tqdm(df.iterrows(), total=len(df), desc="处理数据"):
|
||||
# 存储每个组的拆分数据
|
||||
split_data_groups = {}
|
||||
|
||||
# 拆分每个组的数据
|
||||
for group_name, cols in columns_to_split.items():
|
||||
split_data = {col: safe_split(row[col]) for col in cols}
|
||||
max_length = max(len(split_data[col]) for col in cols)
|
||||
for col in cols:
|
||||
split_data[col] += [''] * (max_length - len(split_data[col]))
|
||||
split_data_groups[group_name] = split_data
|
||||
|
||||
# 检查是否有数据
|
||||
has_product_data = any(any(item.strip() != '' for item in split_data_groups['产品名称组'][col]) for col in
|
||||
columns_to_split['产品名称组'])
|
||||
has_service_data = any(any(item.strip() != '' for item in split_data_groups['服务名称组'][col]) for col in
|
||||
columns_to_split['服务名称组'])
|
||||
has_payment_data = any(any(item.strip() != '' for item in split_data_groups['支付方式组'][col]) for col in
|
||||
columns_to_split['支付方式组'])
|
||||
|
||||
if not (has_product_data or has_service_data or has_payment_data):
|
||||
continue
|
||||
|
||||
# 获取最大长度
|
||||
max_length = max(
|
||||
len(split_data_groups['产品名称组'][col][0]) for col in columns_to_split['产品名称组']
|
||||
) + max(
|
||||
len(split_data_groups['服务名称组'][col][0]) for col in columns_to_split['服务名称组']
|
||||
) + max(
|
||||
len(split_data_groups['支付方式组'][col][0]) for col in columns_to_split['支付方式组']
|
||||
)
|
||||
|
||||
# 创建新的行
|
||||
for i in range(max_length):
|
||||
new_row = {}
|
||||
|
||||
# 添加需要保留的列的数据（开单时间组放在最前面）
|
||||
for col in columns_to_keep:
|
||||
new_row[col] = row[col]
|
||||
|
||||
# 添加产品名称组的数据
|
||||
for col in columns_to_split['产品名称组']:
|
||||
if i < len(split_data_groups['产品名称组'][col]):
|
||||
new_row[col] = split_data_groups['产品名称组'][col][i].strip()
|
||||
else:
|
||||
new_row[col] = ''
|
||||
|
||||
# 添加服务名称组的数据
|
||||
for col in columns_to_split['服务名称组']:
|
||||
if i < len(split_data_groups['服务名称组'][col]):
|
||||
new_row[col] = split_data_groups['服务名称组'][col][i].strip()
|
||||
else:
|
||||
new_row[col] = ''
|
||||
|
||||
# 添加支付方式组的数据
|
||||
for col in columns_to_split['支付方式组']:
|
||||
if i < len(split_data_groups['支付方式组'][col]):
|
||||
new_row[col] = split_data_groups['支付方式组'][col][i].strip()
|
||||
else:
|
||||
new_row[col] = ''
|
||||
|
||||
# 检查新行是否有数据
|
||||
has_data = any(new_row[col] != '' for col in columns_to_split['产品名称组']) or \
|
||||
any(new_row[col] != '' for col in columns_to_split['服务名称组']) or \
|
||||
any(new_row[col] != '' for col in columns_to_split['支付方式组'])
|
||||
|
||||
if has_data:
|
||||
# 将新行添加到结果列表中
|
||||
result_list.append(new_row)
|
||||
|
||||
# 将结果列表转换为DataFrame
|
||||
result_df = pd.DataFrame(result_list)
|
||||
|
||||
# 保存结果到新的Excel文件
|
||||
file_path = 'result.xlsx'
|
||||
result_df.to_excel(file_path, index=False)
|
||||
|
||||
# 加载Excel文件并设置背景颜色
|
||||
wb = openpyxl.load_workbook(file_path)
|
||||
ws = wb.active
|
||||
|
||||
# 定义背景颜色
|
||||
color_dict = {
|
||||
'开单时间': 'FFFFFF', # 白色
|
||||
'产品名称': 'FFFF00', # 黄色
|
||||
'服务名称': '00FF00', # 绿色
|
||||
'支付方式': '0000FF' # 蓝色
|
||||
}
|
||||
|
||||
# 设置背景颜色
|
||||
for row in ws.iter_rows(min_row=2, max_row=ws.max_row, min_col=1, max_col=ws.max_column):
|
||||
for cell in row:
|
||||
|
||||
if cell.column_letter in [ws[1][i].column_letter for i, col in enumerate(columns_to_keep)]:
|
||||
cell.fill = PatternFill(start_color=color_dict['开单时间'], end_color=color_dict['开单时间'],
|
||||
fill_type='solid')
|
||||
elif cell.column_letter in [ws[1][i].column_letter for i, col in enumerate(columns_to_split['产品名称组'])]:
|
||||
cell.fill = PatternFill(start_color=color_dict['产品名称'], end_color=color_dict['产品名称'],
|
||||
fill_type='solid')
|
||||
elif cell.column_letter in [ws[1][i].column_letter for i, col in enumerate(columns_to_split['服务名称组'])]:
|
||||
cell.fill = PatternFill(start_color=color_dict['服务名称'], end_color=color_dict['服务名称'],
|
||||
fill_type='solid')
|
||||
elif cell.column_letter in [ws[1][i].column_letter for i, col in enumerate(columns_to_split['支付方式组'])]:
|
||||
cell.fill = PatternFill(start_color=color_dict['支付方式'], end_color=color_dict['支付方式'],
|
||||
fill_type='solid')
|
||||
|
||||
# 保存Excel文件
|
||||
wb.save(file_path)
|
||||
wb.close()
|
||||
|
||||
print("数据拆分完成，已保存到result.xlsx")
|
||||
@@ -0,0 +1,199 @@
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.common.by import By
|
||||
from selenium.webdriver.support.ui import WebDriverWait
|
||||
from selenium.webdriver.support import expected_conditions as EC
|
||||
import time
|
||||
from urllib.parse import urljoin
|
||||
import pandas as pd
|
||||
from selenium.webdriver import Chrome
|
||||
from selenium.webdriver.chrome.service import Service
|
||||
from selenium.webdriver.chrome.options import Options
|
||||
from datetime import datetime, timedelta
|
||||
from selenium.common.exceptions import NoSuchElementException
|
||||
from tqdm import tqdm
|
||||
import re
|
||||
from selenium.common.exceptions import TimeoutException
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
# 设置Chrome选项
|
||||
chrome_options = Options()
|
||||
# 设置为无头模式（不打开浏览器窗口）
|
||||
# chrome_options.add_argument('--headless')
|
||||
chrome_options.add_argument('--disable-gpu')
|
||||
chrome_options.add_argument('--no-sandbox')
|
||||
|
||||
# 指定ChromeDriver路径
|
||||
service = Service(executable_path='D:\ProgramTools\chromedriver-win64\chromedriver.exe')
|
||||
|
||||
# 创建WebDriver对象
|
||||
driver = webdriver.Chrome(service=service, options=chrome_options)
|
||||
|
||||
# 目标网址
|
||||
url = 'http://best.aichedian.com/login/'
|
||||
url1 = ''
|
||||
username = '15307259977'
|
||||
password = 'juanzi810119'
|
||||
title = 'HGYH -- HGYH'
|
||||
|
||||
# 访问网页
|
||||
driver.get(url)
|
||||
|
||||
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id="username"]'))).send_keys(username)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[4]/div/input'))).click()
|
||||
time.sleep(2)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select-dom"]'))).click()
|
||||
# time.sleep(200)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select_box"]/div/div[1]/div'))).click()
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="select_box"]/div/div[1]/div/span[1]'))).click()
|
||||
time.sleep(5)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[3]/div[2]/div/input'))).send_keys(password)
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="login_form"]/div[4]/div/input'))).click()
|
||||
|
||||
time.sleep(2)
|
||||
|
||||
all_data = []
|
||||
page = 1
|
||||
|
||||
# 定义表头
|
||||
headers = [
|
||||
'订单号',
|
||||
'挂账时间', '挂账金额', # 入账订单列表
|
||||
'车牌号码', '车辆品牌', '车主姓名', '联系方式', '进站公里数', '业务类型', # 基础信息
|
||||
'服务名称', '销售人员', '施工人员', '应付金额', '施工时间', '完工时间', '服务评分', # 服务项目
|
||||
'产品名称', '型号', '单价', '数量', '总价', '销售人员', '销售时间',
|
||||
|
||||
]
|
||||
|
||||
for page in tqdm(range(1, 18)): # 循环页码
|
||||
url1 = f'http://best.aichedian.com/order/debt/?q=&page={page}'
|
||||
driver.get(url1) # 获取挂账列表
|
||||
|
||||
xpath = '//*[@id="data-table-1"]/tbody'
|
||||
|
||||
tbody = WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_all_elements_located((By.XPATH, xpath))
|
||||
)
|
||||
rows = tbody[0].find_elements(By.TAG_NAME, 'tr')
|
||||
if not rows:
|
||||
raise NoSuchElementException # 如果没有找到任何行，抛出异常
|
||||
|
||||
for row in rows: # 循环行
|
||||
cells = row.find_elements(By.TAG_NAME, 'td')
|
||||
|
||||
transaction_time = cells[0].text
|
||||
price_cell = cells[3].text
|
||||
# print(price_cell)
|
||||
match = re.search(r'￥(\d+\.\d{2})', price_cell)
|
||||
if match:
|
||||
price = float(match.group(1))
|
||||
else:
|
||||
price = "未匹配到金额"
|
||||
# print(transaction_time) # 打印挂账时间
|
||||
# print(price) # 打印挂账金额
|
||||
|
||||
detail_url = cells[4].find_element(By.TAG_NAME, 'a').get_attribute('href')
|
||||
# print(detail_url)
|
||||
driver.get(detail_url)
|
||||
details_id = detail_url.split('/')[-2]
|
||||
|
||||
# 获取基本信息
|
||||
base_details = WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.XPATH, f'//*[@id="x-single-content"]/ul'))
|
||||
)
|
||||
li_items = base_details.find_elements(By.TAG_NAME, 'li')
|
||||
base_info = {li.text.split(':')[0].strip(): li.text.split(':')[1].strip() if ':' in li.text else ''
|
||||
for li in li_items}
|
||||
# print(base_info)
|
||||
|
||||
# 获取具体信息，如果不存在则返回空字符串
|
||||
license_plate = base_info.get('车牌号', '')
|
||||
customer_name = base_info.get('客户名称', '')
|
||||
order_id = base_info.get('订单编号', '')
|
||||
|
||||
# 获取服务项目
|
||||
service_info = []
|
||||
try:
|
||||
h3_element = WebDriverWait(driver, 0.2).until(
|
||||
EC.presence_of_element_located((By.XPATH, '//h3[text()="服务项目"]'))
|
||||
)
|
||||
if h3_element.text == "服务项目":
|
||||
table = h3_element.find_element(By.XPATH, './following-sibling::table')
|
||||
rows = table.find_elements(By.TAG_NAME, 'tr')
|
||||
for row1 in rows[1:]:
|
||||
li_items = row1.find_elements(By.TAG_NAME, 'td')
|
||||
service_info.append({
|
||||
'服务名称': li_items[0].text,
|
||||
'销售人员': li_items[1].text,
|
||||
'施工人员': li_items[2].text,
|
||||
'应付金额': li_items[3].text,
|
||||
'施工时间': li_items[4].text,
|
||||
'完工时间': li_items[5].text,
|
||||
'服务评分': li_items[6].text
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
# 获取销售产品
|
||||
product_info = []
|
||||
try:
|
||||
h3_element = WebDriverWait(driver, 0.2).until(
|
||||
EC.presence_of_element_located((By.XPATH, '//h3[text()="销售产品"]'))
|
||||
)
|
||||
if h3_element.text == "销售产品":
|
||||
table = h3_element.find_element(By.XPATH, './following-sibling::table')
|
||||
rows = table.find_elements(By.TAG_NAME, 'tr')
|
||||
for row2 in rows[1:]:
|
||||
li_items = row2.find_elements(By.TAG_NAME, 'td')
|
||||
product_info.append({
|
||||
'产品名称': li_items[0].text,
|
||||
'型号': li_items[1].text,
|
||||
'单价': li_items[2].text,
|
||||
'数量': li_items[3].text,
|
||||
'总价': li_items[4].text,
|
||||
'销售人员': li_items[5].text,
|
||||
'销售时间': li_items[6].text
|
||||
})
|
||||
except:
|
||||
pass
|
||||
|
||||
# 将所有信息组合成一个字典
|
||||
order_info = {
|
||||
'订单号': details_id,
|
||||
'挂账时间': transaction_time,
|
||||
'挂账金额': price,
|
||||
'车牌号码': base_info.get('车牌号码', ''),
|
||||
'车辆品牌': base_info.get('车辆品牌', ''),
|
||||
# '会员卡号': base_info.get('会员卡号', ''),
|
||||
'车主姓名': base_info.get('车主姓名', ''),
|
||||
'联系方式': base_info.get('联系方式', ''),
|
||||
'进站公里数': base_info.get('进站公里数', ''),
|
||||
'业务类型': base_info.get('业务类型', ''),
|
||||
# '订单详情': order_details,
|
||||
'服务名称': '\n'.join([item['服务名称'] for item in service_info]),
|
||||
'销售人员': '\n'.join([item['销售人员'] for item in service_info]),
|
||||
'施工人员': '\n'.join([item['施工人员'] for item in service_info]),
|
||||
'应付金额': '\n'.join([item['应付金额'] for item in service_info]),
|
||||
'施工时间': '\n'.join([item['施工时间'] for item in service_info]),
|
||||
'完工时间': '\n'.join([item['完工时间'] for item in service_info]),
|
||||
'服务评分': '\n'.join([item['服务评分'] for item in service_info]),
|
||||
'产品名称': '\n'.join([item['产品名称'] for item in product_info]),
|
||||
'型号': '\n'.join([item['型号'] for item in product_info]),
|
||||
'单价': '\n'.join([item['单价'] for item in product_info]),
|
||||
'数量': '\n'.join([item['数量'] for item in product_info]),
|
||||
'总价': '\n'.join([item['总价'] for item in product_info]),
|
||||
'销售人员': '\n'.join([item['销售人员'] for item in product_info]),
|
||||
'销售时间': '\n'.join([item['销售时间'] for item in product_info]),
|
||||
}
|
||||
|
||||
all_data.append(order_info) # 将完整订单信息添加到总数据列表中
|
||||
|
||||
driver.back()
|
||||
|
||||
df = pd.DataFrame(all_data, columns=headers)
|
||||
df.to_excel('爱车店应收账款数据导出.xlsx', index=False) # 保存为Excel文件
|
||||
@@ -0,0 +1,978 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-23T05:28:31.780431Z",
|
||||
"start_time": "2025-06-23T05:26:25.738579Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"from selenium import webdriver\n",
|
||||
"from selenium.webdriver.common.by import By\n",
|
||||
"from selenium.webdriver.support.ui import WebDriverWait\n",
|
||||
"from selenium.webdriver.support import expected_conditions as EC\n",
|
||||
"import time\n",
|
||||
"from urllib.parse import urljoin\n",
|
||||
"import pandas as pd\n",
|
||||
"from selenium.webdriver import Chrome\n",
|
||||
"from selenium.webdriver.chrome.service import Service\n",
|
||||
"from datetime import datetime\n",
|
||||
"from selenium.webdriver.chrome.options import Options\n",
|
||||
"from datetime import datetime, timedelta\n",
|
||||
"from selenium.common.exceptions import NoSuchElementException\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from selenium.common.exceptions import TimeoutException\n",
|
||||
"\n",
|
||||
"# 设置Chrome选项\n",
|
||||
"chrome_options = Options()\n",
|
||||
"# 设置为无头模式（不打开浏览器窗口）\n",
|
||||
"# chrome_options.add_argument('--headless')\n",
|
||||
"chrome_options.add_argument('--disable-gpu')\n",
|
||||
"chrome_options.add_argument('--no-sandbox')\n",
|
||||
"\n",
|
||||
"# 指定ChromeDriver路径\n",
|
||||
"service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')\n",
|
||||
"\n",
|
||||
"# 创建WebDriver对象\n",
|
||||
"driver = webdriver.Chrome(service=service, options=chrome_options)\n",
|
||||
"\n",
|
||||
"# 目标网址\n",
|
||||
"# url = 'http://xlsf.aichedian.com/order/order-detail/1115935207959/ # 爱车店有不同的网址\n",
|
||||
"url = 'http://best.aichedian.com/order/order-detail/1115935207959/'\n",
|
||||
"username = '15307259977'\n",
|
||||
"password = 'juanzi810119'\n",
|
||||
"\n",
|
||||
"# 访问网页\n",
|
||||
"driver.get(url)\n",
|
||||
"\n",
|
||||
"WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id=\"username\"]'))).send_keys(username)\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[4]/div/input'))).click()\n",
|
||||
"time.sleep(5) # 提供时间选择门店\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[3]/div[2]/div/input'))).send_keys(password)\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[4]/div/input'))).click()\n",
|
||||
"\n",
|
||||
"time.sleep(2)\n",
|
||||
"\n",
|
||||
"# 设置起始日期为今天，结束日期为两年前 # 需要修改日期\n",
|
||||
"end_date = datetime.now()\n",
|
||||
"start_date = end_date - timedelta(days=1552) #786\n",
|
||||
"current_date = start_date\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"# 定义表头\n",
|
||||
"headers = [\n",
|
||||
" '订单号',\n",
|
||||
" '开单时间', '入账时间', '车辆', '车主', '订单详情',\n",
|
||||
" '车牌号码', '车辆品牌', '会员卡号', '车主姓名', '联系方式',\n",
|
||||
" '服务名称', '销售人员', '施工人员', '应付金额', '施工时间', '完工时间', '服务评分',\n",
|
||||
" '产品名称', '型号', '单价', '数量', '总价', '销售人员', '销售时间',\n",
|
||||
" '支付方式', '账号', '金额', '时间', '备注'\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# 起始日期和当前日期\n",
|
||||
"first_date = \"2021-04-03\"\n",
|
||||
"now_date = datetime.now().strftime('%Y-%m-%d')\n",
|
||||
"\n",
|
||||
"# 转换为 pandas 的 Timestamp 类型\n",
|
||||
"start = pd.to_datetime(first_date)\n",
|
||||
"end = pd.to_datetime(now_date)\n",
|
||||
"\n",
|
||||
"# 初始化当前时间指针\n",
|
||||
"current = start\n",
|
||||
"\n",
|
||||
"# 计算总共有多少年，用于进度条\n",
|
||||
"total_years = (end - start).days // 365 + 1\n",
|
||||
"\n",
|
||||
"all_url = []\n",
|
||||
"\n",
|
||||
"with tqdm(total=total_years, desc=\"处理时间段\") as pbar:\n",
|
||||
" no_data_count=0\n",
|
||||
" while current <= end:\n",
|
||||
" # 计算这一年的结束日期\n",
|
||||
" year_end = current + pd.DateOffset(years=1) - pd.Timedelta(days=1)\n",
|
||||
" if year_end > end:\n",
|
||||
" year_end = end\n",
|
||||
"\n",
|
||||
" # 格式化成 URL 需要的格式：YYYY/MM/DD\n",
|
||||
" url_start = current.strftime('%Y/%m/%d')\n",
|
||||
" url_end = year_end.strftime('%Y/%m/%d')\n",
|
||||
"\n",
|
||||
" # 构造 URL\n",
|
||||
" url = f'http://best.aichedian.com/report/order-stat/?start_date={url_start}&end_date={url_end}&tab=1'\n",
|
||||
"\n",
|
||||
" driver.get(url) # 获取每日订单\n",
|
||||
" # 获取指定 XPath 下的所有 <tr> 元素\n",
|
||||
" xpath = '//*[@id=\"order-profit-count\"]/table[2]'\n",
|
||||
" try:\n",
|
||||
" rows = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_all_elements_located((By.XPATH, xpath))\n",
|
||||
" )\n",
|
||||
" if not rows:\n",
|
||||
" raise NoSuchElementException # 如果没有找到任何行，抛出异常\n",
|
||||
" no_data_count = 0 # 重置计数器\n",
|
||||
" except TimeoutException:\n",
|
||||
" no_data_count += 1\n",
|
||||
" if no_data_count >= 30:\n",
|
||||
" print(\"连续30天没有数据，退出循环\")\n",
|
||||
" break\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for row in tqdm(rows):\n",
|
||||
" # 获取每一行中的所有 <td> 元素\n",
|
||||
" cells = row.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" row_data = [cell.text for cell in cells] # 将一行中的所有单元格文本存入列表\n",
|
||||
" start_dj_data = row_data[0] # 开始时间\n",
|
||||
" get_money_data = row_data[1] # 入账时间\n",
|
||||
" car_number = row_data[2] # 车牌名称\n",
|
||||
" customer = row_data[3] # 假设第二列是客户名称\n",
|
||||
" order_details = '\\n'.join(row_data) # 将行数据合并为一个字符串，每列之间用换行符分隔\n",
|
||||
"\n",
|
||||
" for cell in cells:\n",
|
||||
" try:\n",
|
||||
" links = cell.find_elements(By.TAG_NAME, 'a') # 获取当前单元格内的所有 <a> 标签\n",
|
||||
" absolute_url = None\n",
|
||||
" all_absolute_urls = []\n",
|
||||
" for link in links:\n",
|
||||
" if link.text == \"查看详细\":\n",
|
||||
" relative_url = link.get_attribute('href')\n",
|
||||
" absolute_url = urljoin('http://best.aichedian.com/', relative_url)\n",
|
||||
" all_absolute_urls.append(absolute_url)\n",
|
||||
" else:\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" if not all_absolute_urls:\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" for absolute_url in all_absolute_urls:\n",
|
||||
" \n",
|
||||
" driver.get(absolute_url) # 获取查看详情\n",
|
||||
" \n",
|
||||
" n_xpath = '//*[@id=\"x-single-content\"]'\n",
|
||||
" try:\n",
|
||||
" n_rows = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_all_elements_located((By.XPATH, n_xpath))\n",
|
||||
" )\n",
|
||||
" except:\n",
|
||||
" continue\n",
|
||||
" first_row = n_rows[0] # 此时为一个列表\n",
|
||||
" n_cells = first_row.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" \n",
|
||||
" for n_cell in n_cells:\n",
|
||||
" n_links = n_cell.find_elements(By.TAG_NAME, 'a') # 获取当前单元格内的所有 <a> 标签\n",
|
||||
" n_absolute_url = None\n",
|
||||
" details_id= \"\"\n",
|
||||
" for n_link in n_links:\n",
|
||||
" # print(n_link.text)\n",
|
||||
" if n_link.text == \"明细\":\n",
|
||||
" n_relative_url = n_link.get_attribute('href')\n",
|
||||
" n_absolute_url = urljoin('http://best.aichedian.com/', n_relative_url)\n",
|
||||
" details_id = n_absolute_url.split('/')[-2]\n",
|
||||
" all_url.append(n_absolute_url)\n",
|
||||
" else:\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" if not n_absolute_url:\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" \n",
|
||||
" time.sleep(0.1)\n",
|
||||
" driver.back()\n",
|
||||
"\n",
|
||||
" except :\n",
|
||||
" continue # 如果没有找到 <a> 标签，继续下一个 <td\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" # 返回页面\n",
|
||||
" driver.back()\n",
|
||||
" # 获取每日订单\n",
|
||||
" pbar.update(1)\n",
|
||||
" \n",
|
||||
"for url in all_url:\n",
|
||||
" driver.get(url) # 获取明细数据\n",
|
||||
"\n",
|
||||
" xpath1 = '//*[@id=\"x-single-content\"]/h2'\n",
|
||||
" h2 = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, xpath1))\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" if \"车辆检测单\" in h2.text or \"订单详情\" not in h2.text:\n",
|
||||
" driver.back()\n",
|
||||
" print(\"跳过\")\n",
|
||||
" time.sleep(\n",
|
||||
" 5\n",
|
||||
" )\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" # 获取基本信息\n",
|
||||
" try:\n",
|
||||
" base_details = WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"x-single-content\"]/ul'))\n",
|
||||
" )\n",
|
||||
" li_items = base_details.find_elements(By.TAG_NAME, 'li')\n",
|
||||
" base_info = {li.text.split(':')[0].strip(): li.text.split(':')[1].strip() if ':' in li.text else ''\n",
|
||||
" for li in li_items}\n",
|
||||
" # 获取具体信息，如果不存在则返回空字符串\n",
|
||||
" license_plate = base_info.get('车牌号', '')\n",
|
||||
" customer_name = base_info.get('客户名称', '')\n",
|
||||
" order_id = base_info.get('订单编号', '')\n",
|
||||
" except TimeoutException:\n",
|
||||
" continue # 获取不到基本信息\n",
|
||||
"\n",
|
||||
"\n",
|
||||
" # 获取服务项目\n",
|
||||
" service_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"服务项目\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"服务项目\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row1 in rows[1:]:\n",
|
||||
" li_items = row1.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" service_info.append({\n",
|
||||
" '服务名称': li_items[0].text,\n",
|
||||
" '销售人员': li_items[1].text,\n",
|
||||
" '施工人员': li_items[2].text,\n",
|
||||
" '应付金额': li_items[3].text,\n",
|
||||
" '施工时间': li_items[4].text,\n",
|
||||
" '完工时间': li_items[5].text,\n",
|
||||
" '服务评分': li_items[6].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" # 获取销售产品\n",
|
||||
" product_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"销售产品\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"销售产品\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row2 in rows[1:]:\n",
|
||||
" li_items = row2.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" product_info.append({\n",
|
||||
" '产品名称': li_items[0].text,\n",
|
||||
" '型号': li_items[1].text,\n",
|
||||
" '单价': li_items[2].text,\n",
|
||||
" '数量': li_items[3].text,\n",
|
||||
" '总价': li_items[4].text,\n",
|
||||
" '销售人员': li_items[5].text,\n",
|
||||
" '销售时间': li_items[6].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" # 获取支付记录\n",
|
||||
" payment_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"支付记录\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"支付记录\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row3 in rows[1:]:\n",
|
||||
" li_items = row3.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" payment_info.append({\n",
|
||||
" '支付方式': li_items[0].text,\n",
|
||||
" '账号': li_items[1].text,\n",
|
||||
" '金额': li_items[2].text,\n",
|
||||
" '时间': li_items[3].text,\n",
|
||||
" '备注': li_items[4].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
" # 将所有信息组合成一个字典\n",
|
||||
" order_info = {\n",
|
||||
" '订单号': details_id,\n",
|
||||
" '开单时间': start_dj_data,\n",
|
||||
" '入账时间': get_money_data,\n",
|
||||
" '车辆': car_number,\n",
|
||||
" '车主': customer,\n",
|
||||
" '车牌号码': base_info.get('车牌号码', ''),\n",
|
||||
" '车辆品牌': base_info.get('车辆品牌', ''),\n",
|
||||
" '会员卡号': base_info.get('会员卡号', ''),\n",
|
||||
" '车主姓名': base_info.get('车主姓名', ''),\n",
|
||||
" '联系方式': base_info.get('联系方式', ''),\n",
|
||||
" # '订单详情': order_details,\n",
|
||||
" '服务名称': '\\n'.join([item['服务名称'] for item in service_info]),\n",
|
||||
" '销售人员': '\\n'.join([item['销售人员'] for item in service_info]),\n",
|
||||
" '施工人员': '\\n'.join([item['施工人员'] for item in service_info]),\n",
|
||||
" '应付金额': '\\n'.join([item['应付金额'] for item in service_info]),\n",
|
||||
" '施工时间': '\\n'.join([item['施工时间'] for item in service_info]),\n",
|
||||
" '完工时间': '\\n'.join([item['完工时间'] for item in service_info]),\n",
|
||||
" '服务评分': '\\n'.join([item['服务评分'] for item in service_info]),\n",
|
||||
" '产品名称': '\\n'.join([item['产品名称'] for item in product_info]),\n",
|
||||
" '型号': '\\n'.join([item['型号'] for item in product_info]),\n",
|
||||
" '单价': '\\n'.join([item['单价'] for item in product_info]),\n",
|
||||
" '数量': '\\n'.join([item['数量'] for item in product_info]),\n",
|
||||
" '总价': '\\n'.join([item['总价'] for item in product_info]),\n",
|
||||
" '销售人员': '\\n'.join([item['销售人员'] for item in product_info]),\n",
|
||||
" '销售时间': '\\n'.join([item['销售时间'] for item in product_info]),\n",
|
||||
" '支付方式': '\\n'.join([item['支付方式'] for item in payment_info]),\n",
|
||||
" '账号': '\\n'.join([item['账号'] for item in payment_info]),\n",
|
||||
" '金额': '\\n'.join([item['金额'] for item in payment_info]),\n",
|
||||
" '时间': '\\n'.join([item['时间'] for item in payment_info]),\n",
|
||||
" '备注': '\\n'.join([item['备注'] for item in payment_info])\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" all_data.append(order_info) # 将完整订单信息添加到总数据列表中\n",
|
||||
"\n",
|
||||
"# 使用pandas将数据保存为Excel文件\n",
|
||||
"df = pd.DataFrame(all_data, columns=headers)\n",
|
||||
"df.to_excel('爱车店数据导出.xlsx', index=False) # 保存为Excel文件\n",
|
||||
"\n",
|
||||
"# 关闭浏览器\n",
|
||||
"driver.quit()"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
"<>:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
"C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_20808\\1843733566.py:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
" service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')\n",
|
||||
"处理时间段: 0%| | 0/5 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"100%|██████████| 1/1 [00:58<00:00, 58.80s/it]\u001B[A\n",
|
||||
"\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"100%|██████████| 1/1 [00:56<00:00, 56.86s/it]\u001B[A\n",
|
||||
"处理时间段: 0%| | 0/5 [01:56<?, ?it/s]\n",
|
||||
"C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_20808\\1843733566.py:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
" service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "InvalidSessionIdException",
|
||||
"evalue": "Message: invalid session id\nStacktrace:\n\tGetHandleVerifier [0x0x7ff728a4cda5+78885]\n\tGetHandleVerifier [0x0x7ff728a4ce00+78976]\n\t(No symbol) [0x0x7ff7288099fc]\n\t(No symbol) [0x0x7ff7288507df]\n\t(No symbol) [0x0x7ff728888a52]\n\t(No symbol) [0x0x7ff728883413]\n\t(No symbol) [0x0x7ff7288824d9]\n\t(No symbol) [0x0x7ff7287d5d55]\n\tGetHandleVerifier [0x0x7ff728d24eed+3061101]\n\tGetHandleVerifier [0x0x7ff728d1f33d+3037629]\n\tGetHandleVerifier [0x0x7ff728d3e592+3165202]\n\tGetHandleVerifier [0x0x7ff728a6730e+186766]\n\tGetHandleVerifier [0x0x7ff728a6eb3f+217535]\n\t(No symbol) [0x0x7ff7287d4dca]\n\tGetHandleVerifier [0x0x7ff728e445e8+4238440]\n\tBaseThreadInitThunk [0x0x7fff2f1de8d7+23]\n\tRtlUserThreadStart [0x0x7fff307bc34c+44]\n",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[49], line 95\u001B[0m\n\u001B[0;32m 92\u001B[0m \u001B[38;5;66;03m# 构造 URL\u001B[39;00m\n\u001B[0;32m 93\u001B[0m url \u001B[38;5;241m=\u001B[39m \u001B[38;5;124mf\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mhttp://best.aichedian.com/report/order-stat/?start_date=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00murl_start\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m&end_date=\u001B[39m\u001B[38;5;132;01m{\u001B[39;00murl_end\u001B[38;5;132;01m}\u001B[39;00m\u001B[38;5;124m&tab=1\u001B[39m\u001B[38;5;124m'\u001B[39m\n\u001B[1;32m---> 95\u001B[0m driver\u001B[38;5;241m.\u001B[39mget(url) \u001B[38;5;66;03m# 获取每日订单\u001B[39;00m\n\u001B[0;32m 96\u001B[0m \u001B[38;5;66;03m# 获取指定 XPath 下的所有 <tr> 元素\u001B[39;00m\n\u001B[0;32m 97\u001B[0m xpath \u001B[38;5;241m=\u001B[39m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124m//*[@id=\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124morder-profit-count\u001B[39m\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124m]/table[2]\u001B[39m\u001B[38;5;124m'\u001B[39m\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:363\u001B[0m, in \u001B[0;36mWebDriver.get\u001B[1;34m(self, url)\u001B[0m\n\u001B[0;32m 361\u001B[0m \u001B[38;5;28;01mdef\u001B[39;00m \u001B[38;5;21mget\u001B[39m(\u001B[38;5;28mself\u001B[39m, url: \u001B[38;5;28mstr\u001B[39m) \u001B[38;5;241m-\u001B[39m\u001B[38;5;241m>\u001B[39m \u001B[38;5;28;01mNone\u001B[39;00m:\n\u001B[0;32m 362\u001B[0m \u001B[38;5;250m \u001B[39m\u001B[38;5;124;03m\"\"\"Loads a web page in the current browser session.\"\"\"\u001B[39;00m\n\u001B[1;32m--> 363\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mexecute(Command\u001B[38;5;241m.\u001B[39mGET, {\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124murl\u001B[39m\u001B[38;5;124m\"\u001B[39m: url})\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\webdriver.py:354\u001B[0m, in \u001B[0;36mWebDriver.execute\u001B[1;34m(self, driver_command, params)\u001B[0m\n\u001B[0;32m 352\u001B[0m response \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39mcommand_executor\u001B[38;5;241m.\u001B[39mexecute(driver_command, params)\n\u001B[0;32m 353\u001B[0m \u001B[38;5;28;01mif\u001B[39;00m response:\n\u001B[1;32m--> 354\u001B[0m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39merror_handler\u001B[38;5;241m.\u001B[39mcheck_response(response)\n\u001B[0;32m 355\u001B[0m response[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m] \u001B[38;5;241m=\u001B[39m \u001B[38;5;28mself\u001B[39m\u001B[38;5;241m.\u001B[39m_unwrap_value(response\u001B[38;5;241m.\u001B[39mget(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m\"\u001B[39m, \u001B[38;5;28;01mNone\u001B[39;00m))\n\u001B[0;32m 356\u001B[0m \u001B[38;5;28;01mreturn\u001B[39;00m response\n",
|
||||
"File \u001B[1;32mD:\\ProgramTools\\Anaconda\\Lib\\site-packages\\selenium\\webdriver\\remote\\errorhandler.py:229\u001B[0m, in \u001B[0;36mErrorHandler.check_response\u001B[1;34m(self, response)\u001B[0m\n\u001B[0;32m 227\u001B[0m alert_text \u001B[38;5;241m=\u001B[39m value[\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124malert\u001B[39m\u001B[38;5;124m\"\u001B[39m]\u001B[38;5;241m.\u001B[39mget(\u001B[38;5;124m\"\u001B[39m\u001B[38;5;124mtext\u001B[39m\u001B[38;5;124m\"\u001B[39m)\n\u001B[0;32m 228\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace, alert_text) \u001B[38;5;66;03m# type: ignore[call-arg] # mypy is not smart enough here\u001B[39;00m\n\u001B[1;32m--> 229\u001B[0m \u001B[38;5;28;01mraise\u001B[39;00m exception_class(message, screen, stacktrace)\n",
|
||||
"\u001B[1;31mInvalidSessionIdException\u001B[0m: Message: invalid session id\nStacktrace:\n\tGetHandleVerifier [0x0x7ff728a4cda5+78885]\n\tGetHandleVerifier [0x0x7ff728a4ce00+78976]\n\t(No symbol) [0x0x7ff7288099fc]\n\t(No symbol) [0x0x7ff7288507df]\n\t(No symbol) [0x0x7ff728888a52]\n\t(No symbol) [0x0x7ff728883413]\n\t(No symbol) [0x0x7ff7288824d9]\n\t(No symbol) [0x0x7ff7287d5d55]\n\tGetHandleVerifier [0x0x7ff728d24eed+3061101]\n\tGetHandleVerifier [0x0x7ff728d1f33d+3037629]\n\tGetHandleVerifier [0x0x7ff728d3e592+3165202]\n\tGetHandleVerifier [0x0x7ff728a6730e+186766]\n\tGetHandleVerifier [0x0x7ff728a6eb3f+217535]\n\t(No symbol) [0x0x7ff7287d4dca]\n\tGetHandleVerifier [0x0x7ff728e445e8+4238440]\n\tBaseThreadInitThunk [0x0x7fff2f1de8d7+23]\n\tRtlUserThreadStart [0x0x7fff307bc34c+44]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 49
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-23T05:15:55.507074Z",
|
||||
"start_time": "2025-06-23T05:15:55.444172Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"# 使用pandas将数据保存为Excel文件\n",
|
||||
"df = pd.DataFrame(all_data, columns=headers)\n",
|
||||
"df.to_excel('爱车店数据导出.xlsx', index=False) # 保存为Excel文件"
|
||||
],
|
||||
"id": "911d116f7d50ffd2",
|
||||
"outputs": [],
|
||||
"execution_count": 44
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-23T11:53:26.818502Z",
|
||||
"start_time": "2025-06-23T06:29:37.293543Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"from selenium import webdriver\n",
|
||||
"from selenium.webdriver.common.by import By\n",
|
||||
"from selenium.webdriver.support.ui import WebDriverWait\n",
|
||||
"from selenium.webdriver.support import expected_conditions as EC\n",
|
||||
"import time\n",
|
||||
"from urllib.parse import urljoin\n",
|
||||
"import pandas as pd\n",
|
||||
"from selenium.webdriver import Chrome\n",
|
||||
"from selenium.webdriver.chrome.service import Service\n",
|
||||
"from datetime import datetime\n",
|
||||
"from selenium.webdriver.chrome.options import Options\n",
|
||||
"from datetime import datetime, timedelta\n",
|
||||
"from selenium.common.exceptions import NoSuchElementException\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from selenium.common.exceptions import TimeoutException\n",
|
||||
"\n",
|
||||
"# 设置Chrome选项\n",
|
||||
"chrome_options = Options()\n",
|
||||
"# 设置为无头模式（不打开浏览器窗口）\n",
|
||||
"# chrome_options.add_argument('--headless')\n",
|
||||
"chrome_options.add_argument('--disable-gpu')\n",
|
||||
"chrome_options.add_argument('--no-sandbox')\n",
|
||||
"\n",
|
||||
"# 指定ChromeDriver路径\n",
|
||||
"service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')\n",
|
||||
"\n",
|
||||
"# 创建WebDriver对象\n",
|
||||
"driver = webdriver.Chrome(service=service, options=chrome_options)\n",
|
||||
"\n",
|
||||
"# 目标网址\n",
|
||||
"# url = 'http://xlsf.aichedian.com/order/order-detail/1115935207959/ # 爱车店有不同的网址\n",
|
||||
"url = 'http://best.aichedian.com/order/order-detail/1115935207959/'\n",
|
||||
"username = '15307259977'\n",
|
||||
"password = 'juanzi810119'\n",
|
||||
"\n",
|
||||
"# 访问网页\n",
|
||||
"driver.get(url)\n",
|
||||
"\n",
|
||||
"WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, f'//*[@id=\"username\"]'))).send_keys(username)\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[4]/div/input'))).click()\n",
|
||||
"time.sleep(5) # 提供时间选择门店\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[3]/div[2]/div/input'))).send_keys(password)\n",
|
||||
"WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"login_form\"]/div[4]/div/input'))).click()\n",
|
||||
"\n",
|
||||
"time.sleep(2)\n",
|
||||
"\n",
|
||||
"# 设置起始日期为今天，结束日期为两年前 # 需要修改日期\n",
|
||||
"end_date = datetime.now()\n",
|
||||
"start_date = end_date - timedelta(days=1552) #786\n",
|
||||
"current_date = start_date\n",
|
||||
"all_data = []\n",
|
||||
"\n",
|
||||
"# 定义表头\n",
|
||||
"headers = [\n",
|
||||
" '订单号',\n",
|
||||
" '开单时间', '入账时间', '车辆', '车主', '订单详情',\n",
|
||||
" '车牌号码', '车辆品牌', '会员卡号', '车主姓名', '联系方式',\n",
|
||||
" '服务名称', '销售人员', '施工人员', '应付金额', '施工时间', '完工时间', '服务评分',\n",
|
||||
" '产品名称', '型号', '单价', '数量', '总价', '销售人员', '销售时间',\n",
|
||||
" '支付方式', '账号', '金额', '时间', '备注'\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"# 起始日期和当前日期\n",
|
||||
"first_date = \"2021-04-03\"\n",
|
||||
"now_date = datetime.now().strftime('%Y-%m-%d')\n",
|
||||
"\n",
|
||||
"# 转换为 pandas 的 Timestamp 类型\n",
|
||||
"start = pd.to_datetime(first_date)\n",
|
||||
"end = pd.to_datetime(now_date)\n",
|
||||
"\n",
|
||||
"# 初始化当前时间指针\n",
|
||||
"current = start\n",
|
||||
"\n",
|
||||
"# 计算总共有多少年，用于进度条\n",
|
||||
"total_years = (end - start).days // 365 + 1\n",
|
||||
"\n",
|
||||
"all_url = []\n",
|
||||
"\n",
|
||||
"with tqdm(total=total_years, desc=\"处理时间段\") as pbar:\n",
|
||||
" no_data_count=0\n",
|
||||
" while current <= end:\n",
|
||||
" # 计算这一年的结束日期\n",
|
||||
" year_end = current + pd.DateOffset(years=1) - pd.Timedelta(days=1)\n",
|
||||
" if year_end > end:\n",
|
||||
" year_end = end\n",
|
||||
"\n",
|
||||
" # 格式化成 URL 需要的格式：YYYY/MM/DD\n",
|
||||
" url_start = current.strftime('%Y/%m/%d')\n",
|
||||
" url_end = year_end.strftime('%Y/%m/%d')\n",
|
||||
"\n",
|
||||
" # 构造 URL\n",
|
||||
" url = f'http://best.aichedian.com/report/order-stat/?start_date={url_start}&end_date={url_end}&tab=1'\n",
|
||||
"\n",
|
||||
" driver.get(url) # 获取每日订单\n",
|
||||
"\n",
|
||||
" # 更新当前日期指针到下一年的开始\n",
|
||||
" current = year_end + pd.Timedelta(days=1)\n",
|
||||
" # 获取指定 XPath 下的所有 <tr> 元素\n",
|
||||
" xpath = '//*[@id=\"order-profit-count\"]/table[2]'\n",
|
||||
" try:\n",
|
||||
" rows = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_all_elements_located((By.XPATH, xpath))\n",
|
||||
" )\n",
|
||||
" if not rows:\n",
|
||||
" raise NoSuchElementException # 如果没有找到任何行，抛出异常\n",
|
||||
" no_data_count = 0 # 重置计数器\n",
|
||||
" except TimeoutException:\n",
|
||||
" no_data_count += 1\n",
|
||||
" if no_data_count >= 30:\n",
|
||||
" print(\"连续30天没有数据，退出循环\")\n",
|
||||
" break\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for row in tqdm(rows):\n",
|
||||
" # 获取每一行中的所有 <td> 元素\n",
|
||||
" cells = row.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" row_data = [cell.text for cell in cells] # 将一行中的所有单元格文本存入列表\n",
|
||||
" start_dj_data = row_data[0] # 开始时间\n",
|
||||
" get_money_data = row_data[1] # 入账时间\n",
|
||||
" car_number = row_data[2] # 车牌名称\n",
|
||||
" customer = row_data[3] # 假设第二列是客户名称\n",
|
||||
" order_details = '\\n'.join(row_data) # 将行数据合并为一个字符串，每列之间用换行符分隔\n",
|
||||
"\n",
|
||||
" for cell in cells:\n",
|
||||
" try:\n",
|
||||
" links = cell.find_elements(By.TAG_NAME, 'a') # 获取当前单元格内的所有 <a> 标签\n",
|
||||
" absolute_url = None\n",
|
||||
" all_absolute_urls = []\n",
|
||||
" for link in links:\n",
|
||||
" if link.text == \"查看详细\":\n",
|
||||
" relative_url = link.get_attribute('href')\n",
|
||||
" absolute_url = urljoin('http://best.aichedian.com/', relative_url)\n",
|
||||
" all_absolute_urls.append(absolute_url)\n",
|
||||
" else:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" if not all_absolute_urls:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for absolute_url in all_absolute_urls:\n",
|
||||
"\n",
|
||||
" driver.get(absolute_url) # 获取查看详情\n",
|
||||
"\n",
|
||||
" n_xpath = '//*[@id=\"x-single-content\"]'\n",
|
||||
" try:\n",
|
||||
" n_rows = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_all_elements_located((By.XPATH, n_xpath))\n",
|
||||
" )\n",
|
||||
" except:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" for row in n_rows:\n",
|
||||
" n_cells = row.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" for n_cell in n_cells:\n",
|
||||
" n_links = n_cell.find_elements(By.TAG_NAME, 'a')\n",
|
||||
" for n_link in n_links:\n",
|
||||
" if n_link.text == \"明细\":\n",
|
||||
" n_relative_url = n_link.get_attribute('href')\n",
|
||||
" n_absolute_url = urljoin('http://best.aichedian.com/', n_relative_url)\n",
|
||||
" all_url.append(n_absolute_url)\n",
|
||||
"\n",
|
||||
" time.sleep(0.1)\n",
|
||||
" driver.back()\n",
|
||||
" except :\n",
|
||||
" continue # 如果没有找到 <a> 标签，继续下一个 <td\n",
|
||||
" # 返回页面\n",
|
||||
" driver.back()\n",
|
||||
" # 获取每日订单\n",
|
||||
" pbar.update(1)\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_url)\n",
|
||||
"df1.to_csv('all_url.csv', index=False)\n",
|
||||
"for url in tqdm(all_url):\n",
|
||||
" try:\n",
|
||||
" driver.get(url) # 获取明细数据\n",
|
||||
" details_id = url.split('/')[-2]\n",
|
||||
" \n",
|
||||
" xpath1 = '//*[@id=\"x-single-content\"]/h2'\n",
|
||||
" h2 = WebDriverWait(driver, 5).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, xpath1))\n",
|
||||
" )\n",
|
||||
" \n",
|
||||
" if \"车辆检测单\" in h2.text or \"订单详情\" not in h2.text:\n",
|
||||
" driver.back()\n",
|
||||
" print(\"跳过\")\n",
|
||||
" time.sleep(\n",
|
||||
" 5\n",
|
||||
" )\n",
|
||||
" continue\n",
|
||||
" \n",
|
||||
" \n",
|
||||
" # 获取基本信息\n",
|
||||
" try:\n",
|
||||
" base_details = WebDriverWait(driver, 10).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, f'//*[@id=\"x-single-content\"]/ul'))\n",
|
||||
" )\n",
|
||||
" li_items = base_details.find_elements(By.TAG_NAME, 'li')\n",
|
||||
" base_info = {li.text.split(':')[0].strip(): li.text.split(':')[1].strip() if ':' in li.text else ''\n",
|
||||
" for li in li_items}\n",
|
||||
" # 获取具体信息，如果不存在则返回空字符串\n",
|
||||
" license_plate = base_info.get('车牌号', '')\n",
|
||||
" customer_name = base_info.get('客户名称', '')\n",
|
||||
" order_id = base_info.get('订单编号', '')\n",
|
||||
" except TimeoutException:\n",
|
||||
" continue # 获取不到基本信息\n",
|
||||
" \n",
|
||||
" \n",
|
||||
" # 获取服务项目\n",
|
||||
" service_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"服务项目\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"服务项目\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row1 in rows[1:]:\n",
|
||||
" li_items = row1.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" service_info.append({\n",
|
||||
" '服务名称': li_items[0].text,\n",
|
||||
" '销售人员': li_items[1].text,\n",
|
||||
" '施工人员': li_items[2].text,\n",
|
||||
" '应付金额': li_items[3].text,\n",
|
||||
" '施工时间': li_items[4].text,\n",
|
||||
" '完工时间': li_items[5].text,\n",
|
||||
" '服务评分': li_items[6].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
" \n",
|
||||
" # 获取销售产品\n",
|
||||
" product_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"销售产品\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"销售产品\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row2 in rows[1:]:\n",
|
||||
" li_items = row2.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" product_info.append({\n",
|
||||
" '产品名称': li_items[0].text,\n",
|
||||
" '型号': li_items[1].text,\n",
|
||||
" '单价': li_items[2].text,\n",
|
||||
" '数量': li_items[3].text,\n",
|
||||
" '总价': li_items[4].text,\n",
|
||||
" '销售人员': li_items[5].text,\n",
|
||||
" '销售时间': li_items[6].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
" \n",
|
||||
" # 获取支付记录\n",
|
||||
" payment_info = []\n",
|
||||
" try:\n",
|
||||
" h3_element = WebDriverWait(driver, 0.2).until(\n",
|
||||
" EC.presence_of_element_located((By.XPATH, '//h3[text()=\"支付记录\"]'))\n",
|
||||
" )\n",
|
||||
" if h3_element.text == \"支付记录\":\n",
|
||||
" table = h3_element.find_element(By.XPATH, './following-sibling::table')\n",
|
||||
" rows = table.find_elements(By.TAG_NAME, 'tr')\n",
|
||||
" for row3 in rows[1:]:\n",
|
||||
" li_items = row3.find_elements(By.TAG_NAME, 'td')\n",
|
||||
" payment_info.append({\n",
|
||||
" '支付方式': li_items[0].text,\n",
|
||||
" '账号': li_items[1].text,\n",
|
||||
" '金额': li_items[2].text,\n",
|
||||
" '时间': li_items[3].text,\n",
|
||||
" '备注': li_items[4].text\n",
|
||||
" })\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
" \n",
|
||||
" # 将所有信息组合成一个字典\n",
|
||||
" order_info = {\n",
|
||||
" '订单号': details_id,\n",
|
||||
" '开单时间': start_dj_data,\n",
|
||||
" '入账时间': get_money_data,\n",
|
||||
" '车辆': car_number,\n",
|
||||
" '车主': customer,\n",
|
||||
" '车牌号码': base_info.get('车牌号码', ''),\n",
|
||||
" '车辆品牌': base_info.get('车辆品牌', ''),\n",
|
||||
" '会员卡号': base_info.get('会员卡号', ''),\n",
|
||||
" '车主姓名': base_info.get('车主姓名', ''),\n",
|
||||
" '联系方式': base_info.get('联系方式', ''),\n",
|
||||
" # '订单详情': order_details,\n",
|
||||
" '服务名称': '\\n'.join([item['服务名称'] for item in service_info]),\n",
|
||||
" '销售人员': '\\n'.join([item['销售人员'] for item in service_info]),\n",
|
||||
" '施工人员': '\\n'.join([item['施工人员'] for item in service_info]),\n",
|
||||
" '应付金额': '\\n'.join([item['应付金额'] for item in service_info]),\n",
|
||||
" '施工时间': '\\n'.join([item['施工时间'] for item in service_info]),\n",
|
||||
" '完工时间': '\\n'.join([item['完工时间'] for item in service_info]),\n",
|
||||
" '服务评分': '\\n'.join([item['服务评分'] for item in service_info]),\n",
|
||||
" '产品名称': '\\n'.join([item['产品名称'] for item in product_info]),\n",
|
||||
" '型号': '\\n'.join([item['型号'] for item in product_info]),\n",
|
||||
" '单价': '\\n'.join([item['单价'] for item in product_info]),\n",
|
||||
" '数量': '\\n'.join([item['数量'] for item in product_info]),\n",
|
||||
" '总价': '\\n'.join([item['总价'] for item in product_info]),\n",
|
||||
" '销售人员': '\\n'.join([item['销售人员'] for item in product_info]),\n",
|
||||
" '销售时间': '\\n'.join([item['销售时间'] for item in product_info]),\n",
|
||||
" '支付方式': '\\n'.join([item['支付方式'] for item in payment_info]),\n",
|
||||
" '账号': '\\n'.join([item['账号'] for item in payment_info]),\n",
|
||||
" '金额': '\\n'.join([item['金额'] for item in payment_info]),\n",
|
||||
" '时间': '\\n'.join([item['时间'] for item in payment_info]),\n",
|
||||
" '备注': '\\n'.join([item['备注'] for item in payment_info])\n",
|
||||
" }\n",
|
||||
" \n",
|
||||
" all_data.append(order_info) # 将完整订单信息添加到总数据列表中\n",
|
||||
" except:\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"# 使用pandas将数据保存为Excel文件\n",
|
||||
"df = pd.DataFrame(all_data, columns=headers)\n",
|
||||
"df.to_excel('爱车店数据导出.xlsx', index=False) # 保存为Excel文件\n",
|
||||
"\n",
|
||||
"# 关闭浏览器\n",
|
||||
"driver.quit()"
|
||||
],
|
||||
"id": "3bdcde801e98b6bd",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
"<>:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
"C:\\Users\\Administrator.DESKTOP-7IC2USJ\\AppData\\Local\\Temp\\ipykernel_27060\\653923207.py:25: SyntaxWarning: invalid escape sequence '\\P'\n",
|
||||
" service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')\n",
|
||||
"处理时间段: 0%| | 0/5 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A\n",
|
||||
"100%|██████████| 1/1 [04:13<00:00, 253.29s/it]\u001B[A\n",
|
||||
"\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A\n",
|
||||
"100%|██████████| 1/1 [04:49<00:00, 289.93s/it]\u001B[A\n",
|
||||
"\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A\n",
|
||||
"100%|██████████| 1/1 [05:09<00:00, 309.01s/it]\u001B[A\n",
|
||||
"\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A\n",
|
||||
"100%|██████████| 1/1 [04:25<00:00, 265.08s/it]\u001B[A\n",
|
||||
"\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\u001B[A\n",
|
||||
"100%|██████████| 1/1 [00:02<00:00, 2.59s/it]\u001B[A\n",
|
||||
"处理时间段: 20%|██ | 1/5 [18:44<1:14:57, 1124.42s/it]\n",
|
||||
" 0%| | 39/14439 [00:37<3:54:19, 1.02it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 1%|▏ | 183/14439 [02:55<4:05:12, 1.03s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 1%|▏ | 184/14439 [03:00<9:07:41, 2.31s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 1%|▏ | 198/14439 [03:15<2:58:07, 1.33it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 1%|▏ | 199/14439 [03:20<8:21:44, 2.11s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 1%|▏ | 200/14439 [03:26<12:27:17, 3.15s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2%|▏ | 219/14439 [03:48<3:40:41, 1.07it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2%|▏ | 220/14439 [03:53<8:49:45, 2.24s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2%|▏ | 281/14439 [04:54<3:38:17, 1.08it/s] "
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 6%|▌ | 813/14439 [14:18<4:40:52, 1.24s/it]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 36%|███▌ | 5177/14439 [1:30:02<2:24:55, 1.07it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"跳过\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 14439/14439 [5:04:48<00:00, 1.27s/it] \n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 5
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-06-23T06:06:18.139653Z",
|
||||
"start_time": "2025-06-23T06:06:18.131895Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df1 = pd.DataFrame(all_url)\n",
|
||||
"df1.to_csv('all_url.csv', index=False)"
|
||||
],
|
||||
"id": "a68ff75a6184bae",
|
||||
"outputs": [],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
from selenium import webdriver
|
||||
from selenium.webdriver.chrome.service import Service
|
||||
from selenium.webdriver.chrome.options import Options
|
||||
from selenium.webdriver.common.by import By
|
||||
import time
|
||||
import os
|
||||
import pickle
|
||||
import json
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
# ==================== 配置 ====================
|
||||
COOKIE_FILE = "sina_cookies.pkl" # 保存 Cookies 的文件名
|
||||
LOGIN_URL = "https://tousu.sina.com.cn/"
|
||||
DOMAIN = "tousu.sina.com.cn"
|
||||
MAX_COOKIE_AGE_DAYS = 7 # Cookies 最大有效期（天）
|
||||
|
||||
def is_cookie_valid():
|
||||
"""检查 Cookies 是否存在且未过期"""
|
||||
if not os.path.exists(COOKIE_FILE):
|
||||
return False
|
||||
|
||||
with open(COOKIE_FILE, "rb") as f:
|
||||
cookies = pickle.load(f)
|
||||
|
||||
# 获取 Cookies 中的过期时间（假设有一个 Cookie 有 'expiry' 字段）
|
||||
for cookie in cookies:
|
||||
if 'expiry' in cookie:
|
||||
expiry_time = datetime.fromtimestamp(cookie['expiry'])
|
||||
if expiry_time < datetime.now():
|
||||
print("Cookies 已过期")
|
||||
return False
|
||||
return True
|
||||
|
||||
def save_cookies(driver):
|
||||
"""保存当前会话的 Cookies 到文件"""
|
||||
cookies = driver.get_cookies()
|
||||
with open(COOKIE_FILE, "wb") as f:
|
||||
pickle.dump(cookies, f)
|
||||
print("Cookies 已保存")
|
||||
|
||||
def load_cookies(driver):
|
||||
"""加载 Cookies 到当前浏览器会话"""
|
||||
driver.get(LOGIN_URL) # 必须先访问一次目标网站才能设置 Cookies
|
||||
with open(COOKIE_FILE, "rb") as f:
|
||||
cookies = pickle.load(f)
|
||||
for cookie in cookies:
|
||||
# 移除不必要的字段
|
||||
cookie.pop('sameSite', None)
|
||||
cookie.pop('expiry', None) if 'expiry' in cookie and cookie['expiry'] < time.time() else None
|
||||
driver.add_cookie(cookie)
|
||||
print("Cookies 加载成功")
|
||||
|
||||
def check_login_status(driver):
|
||||
"""检查用户是否登录"""
|
||||
try:
|
||||
WebDriverWait(driver, 10).until(
|
||||
EC.presence_of_element_located((By.CSS_SELECTOR, "#SI_User > div.ac-login.ac-logined"))
|
||||
)
|
||||
print("检测到已登录状态")
|
||||
return True
|
||||
except:
|
||||
print("未登录或登录状态失效")
|
||||
return False
|
||||
|
||||
def collect_complaints(driver, keyword, max_items=1000):
|
||||
search_url = f"https://tousu.sina.com.cn/index/search/?keywords={keyword}&t=1"
|
||||
driver.get(search_url)
|
||||
|
||||
collected_items = set()
|
||||
no_new_count = 0
|
||||
|
||||
while len(collected_items) < max_items and no_new_count < 3:
|
||||
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
|
||||
time.sleep(2)
|
||||
|
||||
complaints = driver.find_elements(By.CSS_SELECTOR, "#search_tab > div.blackcat-container > div.tab-con.tousu-list > div > a > h1")
|
||||
|
||||
current_count = len(collected_items)
|
||||
for complaint in complaints:
|
||||
text = complaint.text.strip()
|
||||
if text and text not in collected_items:
|
||||
collected_items.add(text)
|
||||
|
||||
if len(collected_items) == current_count:
|
||||
no_new_count += 1
|
||||
else:
|
||||
no_new_count = 0
|
||||
|
||||
return list(collected_items)
|
||||
|
||||
def main():
|
||||
# 配置 Chrome 浏览器选项
|
||||
chrome_options = Options()
|
||||
chrome_options.add_argument("--disable-blink-features=AutomationControlled")
|
||||
chrome_options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0 Safari/537.36")
|
||||
|
||||
# 启动浏览器
|
||||
chrome_service = Service(executable_path='D:\\ProgramTools\\chromedriver-win64\\chromedriver.exe')
|
||||
driver = webdriver.Chrome(service=chrome_service, options=chrome_options)
|
||||
|
||||
# 检查是否已有 Cookies
|
||||
if is_cookie_valid():
|
||||
print("使用保存的 Cookies 登录...")
|
||||
load_cookies(driver)
|
||||
driver.get(LOGIN_URL)
|
||||
if not check_login_status(driver):
|
||||
print("Cookies 无效，需要手动登录")
|
||||
input("请手动登录后按回车继续...")
|
||||
save_cookies(driver)
|
||||
else:
|
||||
print("首次登录，请手动登录")
|
||||
driver.get(LOGIN_URL)
|
||||
input("请手动登录后按回车继续...")
|
||||
save_cookies(driver)
|
||||
|
||||
# 获取搜索关键词
|
||||
keyword = input("请输入搜索关键词（例如：F6汽车科技）: ")
|
||||
|
||||
# 收集投诉内容
|
||||
complaints = collect_complaints(driver, keyword)
|
||||
|
||||
# 保存结果
|
||||
with open("tousu.txt", "w", encoding="utf-8") as f:
|
||||
for item in complaints:
|
||||
f.write(item + "\n")
|
||||
|
||||
print(f"成功收集到{len(complaints)}条投诉信息，已保存到tousu.txt")
|
||||
driver.quit()
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,216 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-29T03:33:47.044410Z",
|
||||
"start_time": "2025-12-29T03:31:04.703879800Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm # Jupyter 专属进度条\n",
|
||||
"\n",
|
||||
"# ---------------------- 配置请求参数 ----------------------\n",
|
||||
"cookies = {\n",
|
||||
" 'ECSCP[admin_id]': '108',\n",
|
||||
" 'ECSCP[admin_pass]': 'a7a7436dc3cdb9a0cf46db404553e361',\n",
|
||||
" 'ECS_LastCheckOrder': 'Mon%2C%2029%20Dec%202025%2003%3A26%3A30%20GMT',\n",
|
||||
" 'ECSCP[lastfilterfile]': '9028E76F',\n",
|
||||
" 'ECSCP[lastfilter]': 'a%253A12%253A%257Bs%253A9%253A%2522user_name%2522%253Bs%253A0%253A%2522%2522%253Bs%253A9%253A%2522note_name%2522%253Bs%253A0%253A%2522%2522%253Bs%253A9%253A%2522carnumber%2522%253Bs%253A0%253A%2522%2522%253Bs%253A5%253A%2522cates%2522%253Bs%253A0%253A%2522%2522%253Bs%253A12%253A%2522insurance_id%2522%253Bi%253A0%253Bs%253A6%253A%2522inyear%2522%253Bi%253A0%253Bs%253A6%253A%2522intime%2522%253Bi%253A0%253Bs%253A12%253A%2522record_count%2522%253Bs%253A4%253A%25222309%2522%253Bs%253A9%253A%2522page_size%2522%253Bi%253A15%253Bs%253A4%253A%2522page%2522%253Bi%253A1%253Bs%253A10%253A%2522page_count%2522%253Bd%253A154%253Bs%253A5%253A%2522start%2522%253Bi%253A0%253B%257D',\n",
|
||||
" 'ECSCP[lastfiltersql]': 'U0VMRUNUIHUuKiwgcC51c2VyX25hbWUgYXMgcGFyZW50X25hbWUsIGEuY2F0X25hbWUsIGIuY2F0X25hbWUgYXMgY2FyX25hbWUsIHUuc2VjdXJpdHlkYXRlLCBpLmluc3VyYW5jZV9uYW1lIEZST00gYHRlamlhbHVudGFpX2NvbWAuYGVjc191c2Vyc2AgYXMgdSBsZWZ0IGpvaW4gYHRlamlhbHVudGFpX2NvbWAuYGVjc191c2Vyc2AgYXMgcCBvbiBwLnVzZXJfaWQgPSB1LnBhcmVudF9pZCAgbGVmdCBqb2luIGB0ZWppYWx1bnRhaV9jb21gLmBlY3NfaW5zdXJhbmNlYCBhcyBpIG9uIGkuaW5zdXJhbmNlX2lkID0gdS5pbnN1cmFuY2VfaWQgIGxlZnQgam9pbiBgdGVqaWFsdW50YWlfY29tYC5gZWNzX2NhdGVnb3J5YCBhcyBhIG9uIGEuY2F0X2lkID0gdS5jc3J0X2lkICBsZWZ0IGpvaW4gYHRlamlhbHVudGFpX2NvbWAuYGVjc19jYXRlZ29yeWAgYXMgeSBvbiBhLnBhcmVudF9pZCA9IHkuY2F0X2lkICBsZWZ0IGpvaW4gYHRlamlhbHVudGFpX2NvbWAuYGVjc19jYXRlZ29yeWAgYXMgYyBvbiB5LnBhcmVudF9pZCA9IGMuY2F0X2lkICBsZWZ0IGpvaW4gYHRlamlhbHVudGFpX2NvbWAuYGVjc19jYXRlZ29yeWAgYXMgYiBvbiBjLnBhcmVudF9pZCA9IGIuY2F0X2lkICB3aGVyZSAxIEFORCB1LnN1cHBsaWVyc19pZCA9ICcnIE9SREVSIEJZIHVzZXJfaWQgREVTQyBMSU1JVCAwLDE1',\n",
|
||||
" 'real_ipd': '221.226.144.180',\n",
|
||||
" 'ECSCP_ID': '1be4bfb53679215381f4dac51482e1152b69f88a',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'cache-control': 'no-cache',\n",
|
||||
" 'content-type': 'application/x-www-form-urlencoded',\n",
|
||||
" 'origin': 'https://tejialuntai.com',\n",
|
||||
" 'pragma': 'no-cache',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://tejialuntai.com/admin2/suppliers_users.php?act=list',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-origin',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"params = {'is_ajax': '1'}\n",
|
||||
"base_data = {\n",
|
||||
" 'act': 'query',\n",
|
||||
" 'user_name': '',\n",
|
||||
" 'note_name': '',\n",
|
||||
" 'carnumber': '',\n",
|
||||
" 'cates': '0',\n",
|
||||
" 'insurance_id': '0',\n",
|
||||
" 'inyear': '0',\n",
|
||||
" 'intime': '0',\n",
|
||||
" 'record_count': '2309',\n",
|
||||
" 'page_size': '15',\n",
|
||||
" 'page_count': '154',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"# ---------------------- 定义数据提取函数 ----------------------\n",
|
||||
"def get_page_data(page):\n",
|
||||
" \"\"\"\n",
|
||||
" 请求指定页码的数据并解析\n",
|
||||
" :param page: 页码\n",
|
||||
" :return: 解析后的当前页数据列表, 总页数\n",
|
||||
" \"\"\"\n",
|
||||
" start = (page - 1) * 15\n",
|
||||
" data = base_data.copy()\n",
|
||||
" data.update({'page': str(page), 'start': str(start)})\n",
|
||||
"\n",
|
||||
" try:\n",
|
||||
" # 发送请求，添加超时和重试机制\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://tejialuntai.com/admin2/suppliers_users.php',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" data=data,\n",
|
||||
" timeout=15\n",
|
||||
" )\n",
|
||||
" response.raise_for_status() # 抛出HTTP错误\n",
|
||||
" res_json = response.json()\n",
|
||||
" except requests.exceptions.RequestException as e:\n",
|
||||
" print(f\"\\n请求第 {page} 页失败: {str(e)}\")\n",
|
||||
" return [], None\n",
|
||||
"\n",
|
||||
" # 解析HTML表格\n",
|
||||
" soup = BeautifulSoup(res_json['content'], 'html.parser')\n",
|
||||
" table = soup.find('table')\n",
|
||||
" if not table:\n",
|
||||
" print(f\"\\n第 {page} 页未找到数据表格\")\n",
|
||||
" return [], None\n",
|
||||
"\n",
|
||||
" rows = table.find_all('tr')[1:] # 跳过表头\n",
|
||||
" page_data = []\n",
|
||||
" for row in rows:\n",
|
||||
" cols = row.find_all('td')\n",
|
||||
" if len(cols) < 17:\n",
|
||||
" continue\n",
|
||||
"\n",
|
||||
" # 提取每列数据\n",
|
||||
" user_info = {\n",
|
||||
" '用户id': cols[0].text.strip(),\n",
|
||||
" '用户名称': cols[1].text.strip(),\n",
|
||||
" '备注名称': cols[2].text.strip(),\n",
|
||||
" '手机号码': cols[3].text.strip(),\n",
|
||||
" '注册时间': cols[4].text.strip(),\n",
|
||||
" '推荐人': cols[5].text.strip(),\n",
|
||||
" '车型': cols[6].text.strip(),\n",
|
||||
" '变速箱': cols[7].text.strip(),\n",
|
||||
" '机油信息': cols[8].text.strip(),\n",
|
||||
" '车牌号': cols[9].text.strip(),\n",
|
||||
" '微信号': cols[10].text.strip(),\n",
|
||||
" '余额': cols[11].text.strip(),\n",
|
||||
" '保险公司': cols[12].text.strip(),\n",
|
||||
" '保险到期': cols[13].text.strip(),\n",
|
||||
" '旗下车辆': cols[14].text.strip(),\n",
|
||||
" '车架号': cols[15].text.strip(),\n",
|
||||
" '操作': cols[16].text.strip()\n",
|
||||
" }\n",
|
||||
" page_data.append(user_info)\n",
|
||||
"\n",
|
||||
" return page_data, res_json.get('page_count', 154)\n",
|
||||
"\n",
|
||||
"# ---------------------- 主执行逻辑 ----------------------\n",
|
||||
"if __name__ == '__main__':\n",
|
||||
" all_data = []\n",
|
||||
" # 1. 获取第1页数据，确认总页数\n",
|
||||
" first_page_data, total_pages = get_page_data(1)\n",
|
||||
" if not first_page_data:\n",
|
||||
" print(\"无法获取第1页数据，程序终止\")\n",
|
||||
" else:\n",
|
||||
" all_data.extend(first_page_data)\n",
|
||||
" print(f\"第1页数据提取完成，共{len(first_page_data)}条\")\n",
|
||||
"\n",
|
||||
" # 2. 遍历剩余页码，使用进度条\n",
|
||||
" for page in tqdm(range(2, int(total_pages) + 1), desc=\"提取所有页面数据\"):\n",
|
||||
" page_data, _ = get_page_data(page)\n",
|
||||
" all_data.extend(page_data)\n",
|
||||
" time.sleep(0.5) # 延迟0.5秒，避免请求过快\n",
|
||||
"\n",
|
||||
" # 3. 转换为DataFrame并保存为xlsx\n",
|
||||
" if all_data:\n",
|
||||
" df = pd.DataFrame(all_data)\n",
|
||||
" # 保存到当前目录，文件名：会员数据全量.xlsx\n",
|
||||
" df.to_excel('会员数据全量.xlsx', index=False, engine='openpyxl')\n",
|
||||
" print(f\"\\n所有数据提取完成！共{len(all_data)}条\")\n",
|
||||
" print(f\"文件已保存为：会员数据全量.xlsx\")\n",
|
||||
" else:\n",
|
||||
" print(\"未提取到任何数据\")"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"第1页数据提取完成，共16条\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"提取所有页面数据: 0%| | 0/153 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "5c7aad4f6153456fa5d7347d3733eb5a"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n",
|
||||
"所有数据提取完成！共2463条\n",
|
||||
"文件已保存为：会员数据全量.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-11-22T08:49:41.787341Z",
|
||||
"start_time": "2024-11-22T08:09:20.489931Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"import time\n",
|
||||
"# 历史维修记录明细\n",
|
||||
"# 请求过于频繁会挂掉\n",
|
||||
"\n",
|
||||
"url = 'https://erp.byjy168.cn/top/statistics/fix/worksheetHistory/summary/new'\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': '*/*',\n",
|
||||
" 'Accept-Encoding': 'gzip, deflate, br, zstd',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Cookie': 'JSESSIONID=D0A01E7B7FA40CB76F1770BF36556A66',\n",
|
||||
" 'Host': 'erp.byjy168.cn',\n",
|
||||
" 'Referer': 'https://erp.byjy168.cn/dashboard/index/reportforms/history',\n",
|
||||
" 'Sec-Ch-Ua': '\"Microsoft Edge\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\"',\n",
|
||||
" 'Sec-Ch-Ua-Mobile': '?0',\n",
|
||||
" 'Sec-Ch-Ua-Platform': '\"Windows\"',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'Token':'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIzMDMxNyIsInVzZXJuYW1lIjoi5p2O54KcIiwibmFtZSI6IiIsImNvbXBhbnlJZCI6NDIzOSwib3BlcmF0b3JJZCI6MywicGhvbmUiOiIxODY4ODU1MjI3OCIsIm1lbnVUeXBlIjoiMSIsImNvbW5vIjoiQTAxIiwicGNvcmFwcCI6InBjIiwiZXhwIjoxNzMyNjk0NTkyfQ.EgVtyoVcOWZ4EmdYHv52k6Cruk8FExTF4Gp-u_4Ug3I',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0'\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data_list = []\n",
|
||||
"for i in tqdm(range(1, 655), desc='Processing pages', unit='page'):\n",
|
||||
" time.sleep(2)\n",
|
||||
" data = f'current={i}&size=10&status=&gridCode=GS011340&isOtherAccess=1&comno=A01&shopCode=A01'\n",
|
||||
"\n",
|
||||
" response = requests.post(url=url, headers=headers, data=data)\n",
|
||||
"\n",
|
||||
" if response.status_code == 200:\n",
|
||||
" data = response.json()\n",
|
||||
" data_list = data['data']['records']\n",
|
||||
"\n",
|
||||
" for data in data_list:\n",
|
||||
" work_sheet_code = data['worksheetCode']\n",
|
||||
" all_data_list.append(data)\n",
|
||||
" url1 = f'https://erp.byjy168.cn/api/repairServiceHistory/getListByCode?worksheetCode={work_sheet_code}&shopCode=A01'\n",
|
||||
" res = requests.get(url=url1, headers=headers)\n",
|
||||
" if res.status_code == 200:\n",
|
||||
" data1 = res.json()\n",
|
||||
" details = []\n",
|
||||
" for detail in data1['data']:\n",
|
||||
" all_data_list.append(detail)\n",
|
||||
" time.sleep(1)\n",
|
||||
" else:\n",
|
||||
" print(f\"Failed to get details for worksheetCode {work_sheet_code}, status code: {res.status_code}\")\n",
|
||||
" else:\n",
|
||||
" print(f\"Failed to retrieve page {i}, status code: {response.status_code}\")\n",
|
||||
" \n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"# 定义新列名\n",
|
||||
"new_columns = {\n",
|
||||
" 'inDate': '收银时间',\n",
|
||||
" 'customerName': '客户名称',\n",
|
||||
" 'registerNo': '车牌号',\n",
|
||||
" 'model': '车型',\n",
|
||||
" 'status': '状态',\n",
|
||||
" 'receptionSource': '客户来源',\n",
|
||||
" 'serviceType': '业务类型',\n",
|
||||
" 'relasalesName': '客户经理',\n",
|
||||
" 'recpersonName': '接待人',\n",
|
||||
" 'mainworkerName': '维修技师',\n",
|
||||
" 'recmoneyDate': '收银时间',\n",
|
||||
" 'timeAmount': '工时金额',\n",
|
||||
" 'partAmount': '配件金额',\n",
|
||||
" 'addAmount': '增值金额',\n",
|
||||
" 'amount': '合计金额',\n",
|
||||
" 'otherAmount': '其他金额',\n",
|
||||
" 'discountAmount': '减收金额',\n",
|
||||
" 'totalAmount': '实收金额',\n",
|
||||
" 'costAmount': '实收成本',\n",
|
||||
" 'cashAmount': '现金',\n",
|
||||
" 'arrearAmount': '挂帐金额',\n",
|
||||
" 'cardAmount': '刷卡',\n",
|
||||
" 'wxAmount': '微信支付',\n",
|
||||
" 'zfbAmount': '支付宝支付',\n",
|
||||
" 'accotherdetail': '其它支付明细',\n",
|
||||
" 'balanceAmount': '欠款余额',\n",
|
||||
" 'serviceCode': '项目编码',\n",
|
||||
" 'serviceName': '项目名称',\n",
|
||||
" 'workTime': '工时',\n",
|
||||
" 'timePrice': '项目单价',\n",
|
||||
" 'mainWorkerName': '主修员',\n",
|
||||
" # 继续添加更多的列名映射...\n",
|
||||
"}\n",
|
||||
"# 使用rename方法替换列名\n",
|
||||
"df.rename(columns=new_columns, inplace=True)\n",
|
||||
"# 保存到Excel文件\n",
|
||||
"file_path = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\百少侠导出数据.xlsx'\n",
|
||||
"df.to_excel(file_path, index=False)\n",
|
||||
"\n",
|
||||
"print(f\"数据已成功保存至{file_path}\") \n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Processing pages: 43%|████▎ | 282/654 [40:21<53:13, 8.59s/page] \n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "KeyError",
|
||||
"evalue": "'data'",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mKeyError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[17], line 45\u001B[0m\n\u001B[0;32m 43\u001B[0m data1 \u001B[38;5;241m=\u001B[39m res\u001B[38;5;241m.\u001B[39mjson()\n\u001B[0;32m 44\u001B[0m details \u001B[38;5;241m=\u001B[39m []\n\u001B[1;32m---> 45\u001B[0m \u001B[38;5;28;01mfor\u001B[39;00m detail \u001B[38;5;129;01min\u001B[39;00m data1[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mdata\u001B[39m\u001B[38;5;124m'\u001B[39m]:\n\u001B[0;32m 46\u001B[0m all_data_list\u001B[38;5;241m.\u001B[39mappend(detail)\n\u001B[0;32m 47\u001B[0m \u001B[38;5;28;01melse\u001B[39;00m:\n",
|
||||
"\u001B[1;31mKeyError\u001B[0m: 'data'"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 17
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,343 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "",
|
||||
"id": "647cfbcdeaa27eb5"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-01-17T03:08:53.359083Z",
|
||||
"start_time": "2025-01-17T03:08:53.166826Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"header = {\n",
|
||||
" \"Host\": \"yqhm.vip5968.com\",\n",
|
||||
" \"Origin\": \"http://yqhm.vip5968.com\",\n",
|
||||
" \"Referer\": \"http://yqhm.vip5968.com/Module/Member/MemInfo.aspx?PID=42&MemCard=810076\",\n",
|
||||
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0\",\n",
|
||||
" \"X-Requested-With\": \"XMLHttpRequest\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" \"ASP.NET_SessionId\": \"lzkml2bs2crzysshj2nutb5v\",\n",
|
||||
" \"TestCookie\": \"\",\n",
|
||||
" \"LoginState\": \"68AD1028E54520B1EDDB8584F2768630F73F7846610C020F0270B0E2C34FA16B27D3F2A5A659FD92\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"data = {\n",
|
||||
" \"Method\": \"MemInfoCountExpenseGet\",\n",
|
||||
" \"memid\": \"1185\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"url = 'http://yqhm.vip5968.com/AjaxService/Service.ashx?Method=MemInfoCountExpenseGet'\n",
|
||||
"\n",
|
||||
"res = requests.post(url=url, headers=header, cookies=cookies, data=data)\n",
|
||||
"data_list = res.json()\n",
|
||||
"\n",
|
||||
"for data in data_list:\n",
|
||||
" df = data.Dataframe()\n",
|
||||
" df.to_excel(f'成浩美车会员管理系统导出.xlsx')\n"
|
||||
],
|
||||
"id": "4930c73284248d36",
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'List': [{'OrderAccount': '2025010617115616',\n",
|
||||
" 'OrderEndTime': '2900/1/1 0:00:00',\n",
|
||||
" 'OrderDetailID': '9897',\n",
|
||||
" 'OrderID': '9445',\n",
|
||||
" 'OrderAccount1': '2025010617115616',\n",
|
||||
" 'GoodsID': '272',\n",
|
||||
" 'GoodsDetailBidPrice': '0.0000',\n",
|
||||
" 'OrderDetailPrice': '580.0000',\n",
|
||||
" 'OrderDetailDiscountPrice': '580.0000',\n",
|
||||
" 'OrderDetailItemPoint': '0.0000',\n",
|
||||
" 'OrderDetailPoint': '0',\n",
|
||||
" 'OrderDetailNumber': '5',\n",
|
||||
" 'OrderDetailType': '1',\n",
|
||||
" 'OrderDetailStaffID': '0',\n",
|
||||
" 'OrderDetailCreateTime': '2025/1/6 17:12:06',\n",
|
||||
" 'OrderDetaiEndTime': '2900/1/1 0:00:00',\n",
|
||||
" 'OrderDetailRemark': '',\n",
|
||||
" 'OrderDetailOldID': '0',\n",
|
||||
" 'OrderDetailCountNum': '5',\n",
|
||||
" 'DeductGoodsID': '0',\n",
|
||||
" 'GoodsID1': '272',\n",
|
||||
" 'GoodsCode': '190623082115',\n",
|
||||
" 'GoodsName': '隐形车衣保养',\n",
|
||||
" 'GoodsNameCode': 'YXCYBY',\n",
|
||||
" 'GoodsPrice': '580.0000',\n",
|
||||
" 'GoodsUnit': '块',\n",
|
||||
" 'GoodsSalePercet': '0',\n",
|
||||
" 'GoodsClassID': '12',\n",
|
||||
" 'CommissionType': '0',\n",
|
||||
" 'CommissionNumber': '0',\n",
|
||||
" 'GoodsPoint': '0',\n",
|
||||
" 'MinPercent': '0',\n",
|
||||
" 'GoodsType': 'True',\n",
|
||||
" 'GoodsBidPrice': '0.0000',\n",
|
||||
" 'GoodsRemark': '',\n",
|
||||
" 'GoodsPicture': '',\n",
|
||||
" 'CreateShopID': '3',\n",
|
||||
" 'CreateUserID': '4',\n",
|
||||
" 'CreateTime': '2022/10/12 10:51:49',\n",
|
||||
" 'GoodsNumberLowest': '0',\n",
|
||||
" 'GoodsWeiXinPrice': '0.0000',\n",
|
||||
" 'IsPutAway': 'False'}]}"
|
||||
]
|
||||
},
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"execution_count": 2
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "请求列表",
|
||||
"id": "cd5414f2e299d35a"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-01-18T02:28:51.436282Z",
|
||||
"start_time": "2025-01-18T02:28:51.300809Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" \"cookie\": \"td_cookie=2691013127; ASP.NET_SessionId=lzkml2bs2crzysshj2nutb5v; TestCookie=; LoginState=68AD1028E54520B1EDDB8584F2768630F73F7846610C020FB26F23CFB749BC377C96B23EE9919A65\",\n",
|
||||
" \"host\": \"yqhm.vip5968.com\",\n",
|
||||
" \"upgrade-insecure-requests\": \"1\",\n",
|
||||
" \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0\"\n",
|
||||
"}\n",
|
||||
"viewstate = None\n",
|
||||
"eventvalidation = None\n",
|
||||
"all_data_list = []\n",
|
||||
"for page in tqdm(range(1, 56)): # 总页数注意+1\n",
|
||||
" url = 'http://yqhm.vip5968.com/Module/Member/MemList.aspx?PID=3'\n",
|
||||
"\n",
|
||||
" data = {\n",
|
||||
" \"PID\": \"3\",\n",
|
||||
" \"__EVENTTARGET\": \"NetPager\",\n",
|
||||
" \"__EVENTARGUMENT\": page,\n",
|
||||
" \"__VIEWSTATE\": viewstate,\n",
|
||||
" \"__EVENTVALIDATION\": eventvalidation\n",
|
||||
"\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" res = requests.post(url=url, headers=headers, data=data)\n",
|
||||
" print(res.text)\n",
|
||||
" soup = BeautifulSoup(res.text, 'html.parser')\n",
|
||||
"\n",
|
||||
" viewstate = soup.find('input', {'id': '__VIEWSTATE'})['value']\n",
|
||||
" eventvalidation = soup.find('input', {'id': '__EVENTVALIDATION'})['value']\n",
|
||||
"\n",
|
||||
" input_tags = soup.find_all('input', {'id': lambda x: x and 'rptDataList_chkItem' in x})\n",
|
||||
"\n",
|
||||
" # 遍历找到的<input>标签并提取memid和memcard属性\n",
|
||||
" for tag in input_tags:\n",
|
||||
" memid = tag.get('memid')\n",
|
||||
" memcard = tag.get('memcard')\n",
|
||||
" # print(f\"MemID: {memid}, MemCard: {memcard}\")\n",
|
||||
" new_data = {\n",
|
||||
" \"Method\": \"MemInfoCountExpenseGet\",\n",
|
||||
" \"memid\": memid\n",
|
||||
" }\n",
|
||||
" print(memid)\n",
|
||||
" new_url = 'http://yqhm.vip5968.com/AjaxService/Service.ashx?Method=MemInfoCountExpenseGet'\n",
|
||||
" res = requests.post(url=new_url, headers=headers, data=new_data)\n",
|
||||
" list = res.json().get('List')\n",
|
||||
" for data in list:\n",
|
||||
" data['memcard'] = memcard\n",
|
||||
" all_data_list.append(data)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"\n",
|
||||
"df.to_excel(f'成浩美车会员管理系统导出.xlsx')\n",
|
||||
"\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/55 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"ename": "TypeError",
|
||||
"evalue": "'NoneType' object is not subscriptable",
|
||||
"output_type": "error",
|
||||
"traceback": [
|
||||
"\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
|
||||
"\u001B[1;31mTypeError\u001B[0m Traceback (most recent call last)",
|
||||
"Cell \u001B[1;32mIn[32], line 31\u001B[0m\n\u001B[0;32m 28\u001B[0m \u001B[38;5;28mprint\u001B[39m(res\u001B[38;5;241m.\u001B[39mtext)\n\u001B[0;32m 29\u001B[0m soup \u001B[38;5;241m=\u001B[39m BeautifulSoup(res\u001B[38;5;241m.\u001B[39mtext, \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mhtml.parser\u001B[39m\u001B[38;5;124m'\u001B[39m)\n\u001B[1;32m---> 31\u001B[0m viewstate \u001B[38;5;241m=\u001B[39m soup\u001B[38;5;241m.\u001B[39mfind(\u001B[38;5;124m'\u001B[39m\u001B[38;5;124minput\u001B[39m\u001B[38;5;124m'\u001B[39m, {\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mid\u001B[39m\u001B[38;5;124m'\u001B[39m: \u001B[38;5;124m'\u001B[39m\u001B[38;5;124m__VIEWSTATE\u001B[39m\u001B[38;5;124m'\u001B[39m})[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m'\u001B[39m]\n\u001B[0;32m 32\u001B[0m eventvalidation \u001B[38;5;241m=\u001B[39m soup\u001B[38;5;241m.\u001B[39mfind(\u001B[38;5;124m'\u001B[39m\u001B[38;5;124minput\u001B[39m\u001B[38;5;124m'\u001B[39m, {\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mid\u001B[39m\u001B[38;5;124m'\u001B[39m: \u001B[38;5;124m'\u001B[39m\u001B[38;5;124m__EVENTVALIDATION\u001B[39m\u001B[38;5;124m'\u001B[39m})[\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mvalue\u001B[39m\u001B[38;5;124m'\u001B[39m]\n\u001B[0;32m 34\u001B[0m input_tags \u001B[38;5;241m=\u001B[39m soup\u001B[38;5;241m.\u001B[39mfind_all(\u001B[38;5;124m'\u001B[39m\u001B[38;5;124minput\u001B[39m\u001B[38;5;124m'\u001B[39m, {\u001B[38;5;124m'\u001B[39m\u001B[38;5;124mid\u001B[39m\u001B[38;5;124m'\u001B[39m: \u001B[38;5;28;01mlambda\u001B[39;00m x: x \u001B[38;5;129;01mand\u001B[39;00m \u001B[38;5;124m'\u001B[39m\u001B[38;5;124mrptDataList_chkItem\u001B[39m\u001B[38;5;124m'\u001B[39m \u001B[38;5;129;01min\u001B[39;00m x})\n",
|
||||
"\u001B[1;31mTypeError\u001B[0m: 'NoneType' object is not subscriptable"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 32
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-01-18T02:35:24.666100Z",
|
||||
"start_time": "2025-01-18T02:35:24.666100Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from bs4 import BeautifulSoup\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" \"cookie\": \"td_cookie=2698739984; ASP.NET_SessionId=lzkml2bs2crzysshj2nutb5v; TestCookie=; td_cookie=2764366113; LoginState=68AD1028E54520B1EDDB8584F2768630F73F7846610C020FCDEE011B12E86E6E806A72CE142D4339\",\n",
|
||||
" \"host\": \"yqhm.vip5968.com\",\n",
|
||||
" \"upgrade-insecure-requests\": \"1\",\n",
|
||||
" \"user-agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0\"\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data_list = []\n",
|
||||
"# for page in tqdm(range(1, 56)): # 总页数注意+1\n",
|
||||
"url = 'http://yqhm.vip5968.com/Module/Member/MemList.aspx?PID=3'\n",
|
||||
"\n",
|
||||
"data = {\n",
|
||||
" \"PID\": \"3\",\n",
|
||||
" # \"__EVENTTARGET\": \"NetPager\",\n",
|
||||
" # \"__EVENTARGUMENT\": page,\n",
|
||||
" # \"__VIEWSTATE\": viewstate,\n",
|
||||
" # \"__EVENTVALIDATION\": eventvalidation\n",
|
||||
"\n",
|
||||
"}\n",
|
||||
"res = requests.post(url=url, headers=headers, data=data)\n",
|
||||
"print(res.text)\n",
|
||||
"soup = BeautifulSoup(res.text, 'html.parser')\n",
|
||||
"\n",
|
||||
"viewstate = soup.find('input', {'id': '__VIEWSTATE'})['value']\n",
|
||||
"eventvalidation = soup.find('input', {'id': '__EVENTVALIDATION'})['value']\n",
|
||||
"\n",
|
||||
"input_tags = soup.find_all('input', {'id': lambda x: x and 'rptDataList_chkItem' in x})\n",
|
||||
"\n",
|
||||
"# 遍历找到的<input>标签并提取memid和memcard属性\n",
|
||||
"for tag in input_tags:\n",
|
||||
" memid = tag.get('memid')\n",
|
||||
" memcard = tag.get('memcard')\n",
|
||||
" # print(f\"MemID: {memid}, MemCard: {memcard}\")\n",
|
||||
" new_data = {\n",
|
||||
" \"Method\": \"MemInfoCountExpenseGet\",\n",
|
||||
" \"memid\": memid\n",
|
||||
" }\n",
|
||||
" print(memid)\n",
|
||||
" new_url = 'http://yqhm.vip5968.com/AjaxService/Service.ashx?Method=MemInfoCountExpenseGet'\n",
|
||||
" res = requests.post(url=new_url, headers=headers, data=new_data)\n",
|
||||
" list = res.json().get('List')\n",
|
||||
" for data in list:\n",
|
||||
" data['memcard'] = memcard\n",
|
||||
" all_data_list.append(data)\n",
|
||||
"\n",
|
||||
"for page in tqdm(range(2, 56)): # 总页数注意+1\n",
|
||||
" url = 'http://yqhm.vip5968.com/Module/Member/MemList.aspx?PID=3'\n",
|
||||
"\n",
|
||||
" data = {\n",
|
||||
" \"PID\": \"3\",\n",
|
||||
" \"__EVENTTARGET\": \"NetPager\",\n",
|
||||
" \"__EVENTARGUMENT\": page,\n",
|
||||
" \"__VIEWSTATE\": viewstate,\n",
|
||||
" \"__EVENTVALIDATION\": eventvalidation\n",
|
||||
"\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" res = requests.post(url=url, headers=headers, data=data)\n",
|
||||
" # print(res.text)\n",
|
||||
" soup = BeautifulSoup(res.text, 'html.parser')\n",
|
||||
"\n",
|
||||
" viewstate = soup.find('input', {'id': '__VIEWSTATE'})['value']\n",
|
||||
" eventvalidation = soup.find('input', {'id': '__EVENTVALIDATION'})['value']\n",
|
||||
"\n",
|
||||
" input_tags = soup.find_all('input', {'id': lambda x: x and 'rptDataList_chkItem' in x})\n",
|
||||
"\n",
|
||||
" # 遍历找到的<input>标签并提取memid和memcard属性\n",
|
||||
" for tag in input_tags:\n",
|
||||
" memid = tag.get('memid')\n",
|
||||
" memcard = tag.get('memcard')\n",
|
||||
" # print(f\"MemID: {memid}, MemCard: {memcard}\")\n",
|
||||
" new_data = {\n",
|
||||
" \"Method\": \"MemInfoCountExpenseGet\",\n",
|
||||
" \"memid\": memid\n",
|
||||
" }\n",
|
||||
" print(memid)\n",
|
||||
" new_url = 'http://yqhm.vip5968.com/AjaxService/Service.ashx?Method=MemInfoCountExpenseGet'\n",
|
||||
" res = requests.post(url=new_url, headers=headers, data=new_data)\n",
|
||||
" list = res.json().get('List')\n",
|
||||
" for data in list:\n",
|
||||
" data['memcard'] = memcard\n",
|
||||
" all_data_list.append(data)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"\n",
|
||||
"df.to_excel(f'成浩美车会员管理系统导出.xlsx')\n",
|
||||
"\n"
|
||||
],
|
||||
"id": "4743835b87d92aa8",
|
||||
"outputs": [],
|
||||
"execution_count": null
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,361 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 车辆信息",
|
||||
"id": "f7881b6d57bf77d2"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-05T09:54:23.113999400Z",
|
||||
"start_time": "2026-01-05T09:52:58.415160Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'feehi_session': '695b7d067fd99',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjMwMTI2MDEwNTE1MjgxMDAwMSwic2NvcGVzIjoiYXQiLCJpc3MiOiJzaHVpZGFvLnNlcnZlciIsImF1ZCI6InNodWlkYW8uY2xpZW50IiwianRpIjoiZTkzNTUzIiwiaWF0IjoxNzY3NjAzODk1LjY1MTg3NSwibmJmIjoxNzY3NjAzODk1LjY1MTg3NSwiZXhwIjoxNzY3NjkwMjk1LjY1MTg3NX0.POSVuvUhvpWBoNqHypiMNL58eAzrSlHUmd-DOvt7-18',\n",
|
||||
" 'origin': 'https://daotian.shuidao.com',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://daotian.shuidao.com/',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-site',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" # 'cookie': 'feehi_session=695b7d067fd99',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(0, 174)):\n",
|
||||
" rtick = str(int(time.time() * 1000))\n",
|
||||
" params = {\n",
|
||||
" 'key': '',\n",
|
||||
" 'skip': 15 * i,\n",
|
||||
" 'limit': '15',\n",
|
||||
" 'rtick': rtick,\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" response = requests.get('https://api.shuidao.com/daotian_web/customer/customer-list', params=params,\n",
|
||||
" cookies=cookies, headers=headers)\n",
|
||||
" res_data = response.json().get('res').get('list')\n",
|
||||
" all_data.extend(res_data)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田车辆信息.xlsx\")\n",
|
||||
"\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\1310767323.py:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田车辆信息.xlsx\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/174 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "edadc9db42ef484c899c7d63c01dc3ce"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 7
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 工单信息",
|
||||
"id": "fcea901e3ea7ec4f"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T01:18:48.885923Z",
|
||||
"start_time": "2026-01-06T01:12:44.344552600Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'feehi_session': '695c609649781',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjMwMTI2MDEwNTE1MjgxMDAwMSwic2NvcGVzIjoiYXQiLCJpc3MiOiJzaHVpZGFvLnNlcnZlciIsImF1ZCI6InNodWlkYW8uY2xpZW50IiwianRpIjoiZTkzNTUzIiwiaWF0IjoxNzY3NjYxNzE4LjMwMDE2OSwibmJmIjoxNzY3NjYxNzE4LjMwMDE2OSwiZXhwIjoxNzY3NzQ4MTE4LjMwMDE2OX0.J_-dwMBaXiUEYLWcYC-sNfo4hc2CLCy-4w3qqxUJYyU',\n",
|
||||
" 'origin': 'https://daotian.shuidao.com',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://daotian.shuidao.com/',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-site',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" # 'cookie': 'feehi_session=695c609649781',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_bill_data = []\n",
|
||||
"for i in tqdm(range(0, 620)):\n",
|
||||
" rtick = str(int(time.time() * 1000))\n",
|
||||
" response = requests.get(\n",
|
||||
" f'https://api.shuidao.com/daotian_web/maintain/intention-list?key=&status=0,1&pay_status=-1&type=-2&skip={15 * i}&limit=15&rtick=1767661717801',\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" )\n",
|
||||
" res_data = response.json().get('res').get('list')\n",
|
||||
" all_bill_data.extend(res_data)\n",
|
||||
"\n",
|
||||
"df1 = pd.DataFrame(all_bill_data)\n",
|
||||
"df1.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\")"
|
||||
],
|
||||
"id": "c9bec35aa87c47cd",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:39: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:39: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\725055950.py:39: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/620 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "4effc6449f0a40dfbf1c23b82bfdaa5c"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 10
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T01:46:15.258049900Z",
|
||||
"start_time": "2026-01-06T01:46:04.200209200Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df1 = pd.DataFrame(all_bill_data)\n",
|
||||
"df1.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\")"
|
||||
],
|
||||
"id": "cd6a4b2fd5b30cd1",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\374022918.py:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df1.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\")\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 11
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 明细数据",
|
||||
"id": "f1cd6158da5b6b95"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T07:03:02.964073Z",
|
||||
"start_time": "2026-01-06T05:40:38.583462300Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'feehi_session': '695c609649781',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'accept': '*/*',\n",
|
||||
" 'accept-language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjMwMTI2MDEwNTE1MjgxMDAwMSwic2NvcGVzIjoiYXQiLCJpc3MiOiJzaHVpZGFvLnNlcnZlciIsImF1ZCI6InNodWlkYW8uY2xpZW50IiwianRpIjoiZTkzNTUzIiwiaWF0IjoxNzY3NjYyNjMyLjUzMzA1MiwibmJmIjoxNzY3NjYyNjMyLjUzMzA1MiwiZXhwIjoxNzY3NzQ5MDMyLjUzMzA1Mn0.hOQXKo5BgSf-oPNCRhdHtiDUkHNBBagE5YULNRmKLtM',\n",
|
||||
" 'origin': 'https://daotian.shuidao.com',\n",
|
||||
" 'priority': 'u=1, i',\n",
|
||||
" 'referer': 'https://daotian.shuidao.com/',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'sec-fetch-dest': 'empty',\n",
|
||||
" 'sec-fetch-mode': 'cors',\n",
|
||||
" 'sec-fetch-site': 'same-site',\n",
|
||||
" 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" # 'cookie': 'feehi_session=695c609649781',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"df1 = pd.read_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\", sheet_name=0)\n",
|
||||
"\n",
|
||||
"bill_detail_list =[]\n",
|
||||
"# start_index = 2055\n",
|
||||
"for index, row in tqdm(df1.iterrows(),total=len(df1)):\n",
|
||||
" params = {\n",
|
||||
" 'intention_id': row[\"_id\"],\n",
|
||||
" 'rtick': '1767664096987',\n",
|
||||
" }\n",
|
||||
" try:\n",
|
||||
" response = requests.get(\n",
|
||||
" 'https://api.shuidao.com/daotian_web/maintain/intention-item-list',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" )\n",
|
||||
" bill_detail_list.extend(response.json().get(\"res\").get(\"list\"))\n",
|
||||
" time.sleep(0.1)\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
" pass\n",
|
||||
"\n",
|
||||
"df2 = pd.DataFrame(bill_detail_list)\n",
|
||||
"\n",
|
||||
"df2.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单明细信息1.xlsx\")\n",
|
||||
"\n"
|
||||
],
|
||||
"id": "4fa2e08ed6bfb614",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:24: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:48: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:24: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:48: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\3065781067.py:24: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df1 = pd.read_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单信息.xlsx\", sheet_name=0)\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\3065781067.py:48: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df2.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单明细信息1.xlsx\")\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/9295 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "9cdf16f7030848218582f41071d2fec0"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
}
|
||||
],
|
||||
"execution_count": 22
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T02:28:33.164475100Z",
|
||||
"start_time": "2026-01-06T02:28:16.357244200Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df2 = pd.DataFrame(bill_detail_list)\n",
|
||||
"\n",
|
||||
"df2.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单明细信息.xlsx\")"
|
||||
],
|
||||
"id": "c7f67a5d2a9e0845",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:3: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:3: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_3692\\3122899901.py:3: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df2.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\稻田工单明细信息.xlsx\")\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 16
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,377 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 车辆信息导出",
|
||||
"id": "a738ca16021a9e3b"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-22T09:33:52.535439Z",
|
||||
"start_time": "2025-12-22T09:24:33.820278Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import time\n",
|
||||
"\n",
|
||||
"import pandas as pd\n",
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'acw_tc': '0a472f4517663729846515826e48e5fe00deb9589551f37ca83749c9ce043f',\n",
|
||||
" 'Hm_lvt_684c22b31d0037eca5a691cde16370ad': '1765863565,1766372987',\n",
|
||||
" 'Hm_lpvt_684c22b31d0037eca5a691cde16370ad': '1766372987',\n",
|
||||
" 'HMACCOUNT': '55F2182717FD6AE6',\n",
|
||||
" 'e_token': 'd6c91f1dbb754081948988a722287335',\n",
|
||||
" 'weixin_token': 'Y',\n",
|
||||
" 'e_c': 'MzY3NTI1OTM0OTg4NzQxMA==',\n",
|
||||
" 'e_i_o_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JTVCKzIwMDI3NDUrJTVEJTIy',\n",
|
||||
" 'e_i_p_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JUU1JUJBJTk3JUU0JUI4JUJCJTVCKzIwMjk4NjkrJTVEJTIy',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/plain, */*',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/json;charset=UTF-8',\n",
|
||||
" 'Irisclient': 'PC-web#tedspc',\n",
|
||||
" 'Origin': 'https://teds.tyreplus.com.cn',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'https://teds.tyreplus.com.cn/tedspc/index.html',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(1, 1144)):\n",
|
||||
" json_data = {\n",
|
||||
" 'pageStart': i,\n",
|
||||
" 'pageNums': 10,\n",
|
||||
" 'pageNum': 1,\n",
|
||||
" 'pageSize': 10,\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://teds.tyreplus.com.cn/api/v1/custom/vehicle/queryVehicleInfosByStore',\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" json=json_data,\n",
|
||||
" )\n",
|
||||
" data_list = response.json()['obj']['list']\n",
|
||||
" all_data += data_list\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\驰加车辆信息.xlsx\", index=False)\n",
|
||||
"\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 1143/1143 [08:56<00:00, 2.13it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 4
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 历史维修记录列表（结算单）",
|
||||
"id": "e166fa181fddc651"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-22T03:17:34.270096Z",
|
||||
"start_time": "2025-12-22T03:12:30.351185Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'acw_tc': '0a472f4517663729846515826e48e5fe00deb9589551f37ca83749c9ce043f',\n",
|
||||
" 'Hm_lvt_684c22b31d0037eca5a691cde16370ad': '1765863565,1766372987',\n",
|
||||
" 'Hm_lpvt_684c22b31d0037eca5a691cde16370ad': '1766372987',\n",
|
||||
" 'HMACCOUNT': '55F2182717FD6AE6',\n",
|
||||
" 'e_token': 'd6c91f1dbb754081948988a722287335',\n",
|
||||
" 'weixin_token': 'Y',\n",
|
||||
" 'e_c': 'MzY3NTI1OTM0OTg4NzQxMA==',\n",
|
||||
" 'e_i_o_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JTVCKzIwMDI3NDUrJTVEJTIy',\n",
|
||||
" 'e_i_p_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JUU1JUJBJTk3JUU0JUI4JUJCJTVCKzIwMjk4NjkrJTVEJTIy',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/plain, */*',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/json;charset=UTF-8',\n",
|
||||
" 'Irisclient': 'PC-web#tedspc',\n",
|
||||
" 'Origin': 'https://teds.tyreplus.com.cn',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'https://teds.tyreplus.com.cn/tedspc/index.html',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"history_all_data = []\n",
|
||||
"for i in tqdm(range(1, 848)):\n",
|
||||
" json_data = {\n",
|
||||
" # 记得修改门店list\n",
|
||||
" 'storeCodeList': [\n",
|
||||
" '302558',\n",
|
||||
" '366900',\n",
|
||||
" ],\n",
|
||||
" 'pageStart': i,\n",
|
||||
" 'pageNums': 10,\n",
|
||||
" 'pageNum': i,\n",
|
||||
" 'pageSize': 10,\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://teds.tyreplus.com.cn/api/aftersales/payment/queryPaymentSettlementListByCondition',\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" json=json_data,\n",
|
||||
" )\n",
|
||||
" history_data_list = response.json()['obj']['list']\n",
|
||||
" history_all_data += history_data_list\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(history_all_data)\n",
|
||||
"df.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\结算单.xlsx\", index=False)\n"
|
||||
],
|
||||
"id": "51f8f4b21505280a",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 847/847 [04:53<00:00, 2.89it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 2
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "历史维修记录明细（结算单明细）",
|
||||
"id": "44cdf0d8f5f01c7d"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2025-12-22T09:16:59.210944Z",
|
||||
"start_time": "2025-12-22T05:46:03.111327Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"import time\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'acw_tc': '0a472f4517663729846515826e48e5fe00deb9589551f37ca83749c9ce043f',\n",
|
||||
" 'Hm_lvt_684c22b31d0037eca5a691cde16370ad': '1765863565,1766372987',\n",
|
||||
" 'Hm_lpvt_684c22b31d0037eca5a691cde16370ad': '1766372987',\n",
|
||||
" 'HMACCOUNT': '55F2182717FD6AE6',\n",
|
||||
" 'e_token': 'd6c91f1dbb754081948988a722287335',\n",
|
||||
" 'weixin_token': 'Y',\n",
|
||||
" 'e_c': 'MzY3NTI1OTM0OTg4NzQxMA==',\n",
|
||||
" 'e_i_o_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JTVCKzIwMDI3NDUrJTVEJTIy',\n",
|
||||
" 'e_i_p_c_n': 'JTIyJUU5JUE5JUIwJUU1JThBJUEwJUU2JUIxJUJEJUU4JUJEJUE2JUU2JTlDJThEJUU1JThBJUExJUU0JUI4JUFEJUU1JUJGJTgzJUU5JUJDJThFJUU2JUI5JTk2JUU1JThDJUJBJUU1JTlEJTkxJUU1JThGJUEzJUU1JUJBJTk3JUU1JUJBJTk3JUU0JUI4JUJCJTVCKzIwMjk4NjkrJTVEJTIy',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/plain, */*',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Cache-Control': 'no-cache',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/json;charset=UTF-8',\n",
|
||||
" 'Irisclient': 'PC-web#tedspc',\n",
|
||||
" 'Origin': 'https://teds.tyreplus.com.cn',\n",
|
||||
" 'Pragma': 'no-cache',\n",
|
||||
" 'Referer': 'https://teds.tyreplus.com.cn/tedspc/index.html',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"df = pd.read_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\结算单.xlsx\", sheet_name='Sheet1')\n",
|
||||
"\n",
|
||||
"all_data_list = []\n",
|
||||
"all_service_data = []\n",
|
||||
"all_order_data = []\n",
|
||||
"all_payment_data = []\n",
|
||||
"\n",
|
||||
"for index, row in tqdm(df.iterrows(), total=len(df)):\n",
|
||||
" id = row[\"paymentNo\"]\n",
|
||||
" json_data = {\n",
|
||||
" 'paymentNo': id,\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" retry_count = 0\n",
|
||||
" success = False\n",
|
||||
"\n",
|
||||
" while retry_count < 5 and not success:\n",
|
||||
" try:\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://teds.tyreplus.com.cn/api/v2/aftersales/payment/queryPaymentSettlementDetail',\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" json=json_data,\n",
|
||||
" timeout=10 # 建议加上超时防止卡死\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" # 检查响应状态码和返回内容是否有效\n",
|
||||
" if response.status_code == 200:\n",
|
||||
" resp_json = response.json()\n",
|
||||
" if resp_json.get('code') == 'success' and 'obj' in resp_json:\n",
|
||||
" all_data = resp_json['obj']\n",
|
||||
" service_list = all_data.get('orderServiceList', [])\n",
|
||||
" order_list = all_data.get('orderSkuList', [])\n",
|
||||
" paymentMethodList = all_data.get('paymentMethodList', [])\n",
|
||||
"\n",
|
||||
" # 添加 paymentNo 字段\n",
|
||||
" all_data['paymentNo'] = id\n",
|
||||
" all_data_list.append(all_data)\n",
|
||||
"\n",
|
||||
" for service in service_list:\n",
|
||||
" service['paymentNo'] = id\n",
|
||||
" all_service_data.append(service)\n",
|
||||
"\n",
|
||||
" for order in order_list:\n",
|
||||
" order['paymentNo'] = id\n",
|
||||
" all_order_data.append(order)\n",
|
||||
"\n",
|
||||
" for payment in paymentMethodList:\n",
|
||||
" payment['paymentNo'] = id\n",
|
||||
" all_payment_data.append(payment)\n",
|
||||
"\n",
|
||||
" success = True # 成功获取数据，跳出重试循环\n",
|
||||
" else:\n",
|
||||
" print(f\"请求返回非成功状态: {resp_json.get('message', '未知错误')} (paymentNo: {id})\")\n",
|
||||
" else:\n",
|
||||
" print(f\"HTTP 请求失败，状态码: {response.status_code} (paymentNo: {id})\")\n",
|
||||
"\n",
|
||||
" except Exception as e:\n",
|
||||
" print(f\"请求异常: {e} (paymentNo: {id})\")\n",
|
||||
"\n",
|
||||
" if not success:\n",
|
||||
" retry_count += 1\n",
|
||||
" if retry_count < 5:\n",
|
||||
" time.sleep(2) # 等待 2 秒后重试\n",
|
||||
" else:\n",
|
||||
" print(f\"已达到最大重试次数，跳过 paymentNo: {id}\")\n",
|
||||
"\n",
|
||||
" time.sleep(1) # 正常请求之间仍保留 1 秒间隔\n",
|
||||
"\n",
|
||||
"# 创建DataFrame\n",
|
||||
"df = pd.DataFrame(all_data_list)\n",
|
||||
"df1 = pd.DataFrame(all_service_data)\n",
|
||||
"df2 = pd.DataFrame(all_order_data)\n",
|
||||
"df3 = pd.DataFrame(all_payment_data)\n",
|
||||
"\n",
|
||||
"# 保存到Excel\n",
|
||||
"df.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\历史维修记录明细.xlsx\", index=False)\n",
|
||||
"df1.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\历史维修记录明细-服务明细.xlsx\", index=False)\n",
|
||||
"df2.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\历史维修记录明细-产品明细.xlsx\", index=False)\n",
|
||||
"df3.to_excel(r\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\历史维修记录明细-支付方式.xlsx\", index=False)\n"
|
||||
],
|
||||
"id": "b7e0d7a4da6c9b03",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:37: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:37: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\zy187\\AppData\\Local\\Temp\\ipykernel_29896\\1397566597.py:37: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df = pd.read_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\结算单.xlsx\", sheet_name='Sheet1')\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/8463 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "6e51c0e7de2f4a7aab44e2c939cba811"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"HTTP 请求失败，状态码: 504 (paymentNo: SI2512141629588397)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2509131018744247)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2508260841398950)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2508171023222117)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2508091817076414)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2508081059041103)\n",
|
||||
"请求异常: HTTPSConnectionPool(host='teds.tyreplus.com.cn', port=443): Read timed out. (read timeout=10) (paymentNo: SI2508011823922622)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "## 客户信息导出(含解密手机号)",
|
||||
"id": "9c3ce59cd7160150"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T01:50:04.573176600Z",
|
||||
"start_time": "2026-01-06T01:41:33.836804400Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"cookies = {\n",
|
||||
" 'MD_accessToken': '6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/plain, */*',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Authorization': 'Bearer6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/json;charset=UTF-8',\n",
|
||||
" 'Origin': 'https://store.fuchuang.com',\n",
|
||||
" 'Referer': 'https://store.fuchuang.com/store/index.html',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" 'X-FC-TRACE-ID': '0924ec43dfae4ba48176700dee39833b',\n",
|
||||
" 'loginPlatform': 'store_pc',\n",
|
||||
" 'platform': 'store_pc',\n",
|
||||
" 'searchInfo': '%E5%AE%A2%E6%88%B7%E8%BD%A6%E8%BE%86,%E5%AE%A2%E6%88%B7%E7%AE%A1%E7%90%86,%E6%9F%A5%E8%AF%A2',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'user-info-wms': '{\"token\":\"Bearer6864ddf5-fa96-466f-b228-f2621516a2bf\",\"platform\":\"store_pc\"}',\n",
|
||||
" 'x-fc-client': 'web',\n",
|
||||
" # 'Cookie': 'MD_accessToken=6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"json_data = {\n",
|
||||
" 'phone': '',\n",
|
||||
" 'name': '',\n",
|
||||
" 'storeLabelIds': [],\n",
|
||||
" 'fcLabelIds': [],\n",
|
||||
" 'customerSource': None,\n",
|
||||
" 'customerType': None,\n",
|
||||
" 'sourceChannel': None,\n",
|
||||
" 'pageSize': 10,\n",
|
||||
" 'pageNum': 1,\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"all_data = []\n",
|
||||
"for i in tqdm(range(1, 758)):\n",
|
||||
" json_data['pageNum'] = i\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://store.fuchuang.com/api/store-crm/crm/customer/query',\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" json=json_data,\n",
|
||||
" )\n",
|
||||
" data_list= response.json()['data']['list']\n",
|
||||
" if len(data_list) == 0:\n",
|
||||
" break\n",
|
||||
" data_list= response.json()['data']['list']\n",
|
||||
" for data in data_list:\n",
|
||||
" all_data.append(data)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息.xlsx', index=False)\n"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:63: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:63: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_31392\\364108658.py:63: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息.xlsx', index=False)\n",
|
||||
"100%|██████████| 757/757 [08:25<00:00, 1.50it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 1
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 客户明细",
|
||||
"id": "d9b13e7006671ff2"
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T06:59:53.687706100Z",
|
||||
"start_time": "2026-01-06T05:40:36.873045Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import time\n",
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"import pandas as pd\n",
|
||||
"cookies = {\n",
|
||||
" 'MD_accessToken': '6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/plain, */*',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Authorization': 'Bearer6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Content-Type': 'application/json;charset=UTF-8',\n",
|
||||
" 'Origin': 'https://store.fuchuang.com',\n",
|
||||
" 'Referer': 'https://store.fuchuang.com/store/',\n",
|
||||
" 'Sec-Fetch-Dest': 'empty',\n",
|
||||
" 'Sec-Fetch-Mode': 'cors',\n",
|
||||
" 'Sec-Fetch-Site': 'same-origin',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0',\n",
|
||||
" 'X-FC-TRACE-ID': 'a26acd1fe3c5460e890e80c8ad88a6da',\n",
|
||||
" 'loginPlatform': 'store_pc',\n",
|
||||
" 'platform': 'store_pc',\n",
|
||||
" 'sec-ch-ua': '\"Microsoft Edge\";v=\"143\", \"Chromium\";v=\"143\", \"Not A(Brand\";v=\"24\"',\n",
|
||||
" 'sec-ch-ua-mobile': '?0',\n",
|
||||
" 'sec-ch-ua-platform': '\"Windows\"',\n",
|
||||
" 'user-info-wms': '{\"token\":\"Bearer6864ddf5-fa96-466f-b228-f2621516a2bf\",\"platform\":\"store_pc\"}',\n",
|
||||
" 'x-fc-client': 'web',\n",
|
||||
" # 'Cookie': 'MD_accessToken=6864ddf5-fa96-466f-b228-f2621516a2bf',\n",
|
||||
"}\n",
|
||||
"df1 = pd.read_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息.xlsx', sheet_name =0)\n",
|
||||
"detail_list = []\n",
|
||||
"start_index = 2658\n",
|
||||
"for index, row in tqdm(df1.iterrows(), total=len(df1)):\n",
|
||||
" params = {\n",
|
||||
" 'id': row[\"id\"],\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
" json_data = {}\n",
|
||||
" try:\n",
|
||||
" response = requests.post(\n",
|
||||
" 'https://store.fuchuang.com/api/store-crm/crm/customer/detail',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" json=json_data,\n",
|
||||
" )\n",
|
||||
" detail_list.append(response.json()['data'])\n",
|
||||
" except Exception as e:\n",
|
||||
" print(e)\n",
|
||||
" pass\n",
|
||||
" time.sleep(0.2)\n",
|
||||
"\n",
|
||||
"df2 = pd.DataFrame(detail_list)\n",
|
||||
"df2.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息明细1.xlsx', index=False)\n"
|
||||
],
|
||||
"id": "d096136584e77781",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:31: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:55: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:31: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:55: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_31392\\1770658545.py:31: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df1 = pd.read_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息.xlsx', sheet_name =0)\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_31392\\1770658545.py:55: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df2.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息明细1.xlsx', index=False)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
" 0%| | 0/7567 [00:00<?, ?it/s]"
|
||||
],
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"version_major": 2,
|
||||
"version_minor": 0,
|
||||
"model_id": "755b9172bf87470583e777ee60281e95"
|
||||
}
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data",
|
||||
"jetTransient": {
|
||||
"display_id": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"('Connection aborted.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None))\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 10
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-06T05:35:20.575883300Z",
|
||||
"start_time": "2026-01-06T05:35:18.435503400Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"df2 = pd.DataFrame(detail_list)\n",
|
||||
"df2.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息明细2.xlsx', index=False)"
|
||||
],
|
||||
"id": "71fe765d89e1fc8d",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_31392\\2944444433.py:2: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df2.to_excel('D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\美孚客户信息明细2.xlsx', index=False)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 7
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
from playwright.sync_api import sync_playwright
|
||||
import time
|
||||
import pandas as pd
|
||||
|
||||
|
||||
def extract_table_data(page):
|
||||
"""从当前页面提取表格数据"""
|
||||
rows = page.query_selector_all("table.dg tbody tr")
|
||||
data = []
|
||||
for row in rows:
|
||||
# 跳过表头和合计行
|
||||
if row.query_selector("th") or "合计" in row.text_content():
|
||||
continue
|
||||
|
||||
cells = row.query_selector_all("td")
|
||||
if len(cells) < 12:
|
||||
continue # 非数据行
|
||||
|
||||
record_id = row.query_selector("input[title]").get_attribute("title") or ""
|
||||
car_no = cells[2].text_content().strip()
|
||||
name = cells[3].text_content().strip()
|
||||
card_no = cells[4].text_content().strip()
|
||||
card_type = cells[5].text_content().strip()
|
||||
package = cells[6].text_content().strip()
|
||||
total_times = cells[7].text_content().strip()
|
||||
consumed = cells[8].text_content().strip()
|
||||
remaining = cells[9].text_content().strip()
|
||||
remaining_cost = cells[10].text_content().strip()
|
||||
expire_date = cells[11].text_content().strip()
|
||||
|
||||
data.append({
|
||||
"ID": record_id,
|
||||
"车牌": car_no,
|
||||
"姓名": name,
|
||||
"卡号": card_no,
|
||||
"卡类型": card_type,
|
||||
"套餐项目": package,
|
||||
"总次数": total_times,
|
||||
"消费": consumed,
|
||||
"剩余": remaining,
|
||||
"剩余成本": remaining_cost,
|
||||
"到期日期": expire_date
|
||||
})
|
||||
return data
|
||||
|
||||
|
||||
def main():
|
||||
# ====== 手动设置页码范围 ======
|
||||
start_page = 1 # 起始页（包含）
|
||||
end_page = 5 # 结束页（包含）
|
||||
# ============================
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.launch(headless=False, slow_mo=300)
|
||||
context = browser.new_context()
|
||||
|
||||
# 设置你的 Cookie（请根据实际情况更新）
|
||||
context.add_cookies([
|
||||
{"name": "ASP.NET_SessionId", "value": "knhs0hxsbmolk20gidmlis3j", "domain": "crm.zhongtukj.com",
|
||||
"path": "/"},
|
||||
{"name": "ztrjnew@4db97b96-12af-45b0-b232-fd1e9b7a672e@",
|
||||
"value": "PassWord=wZn2IuvdWeE=&RememberPwd=RXv90LpPskw=&UserId=nzK31b3ZYVQ=&CSID=VjfeyHPOjnU=&UserName=fDfTOArNJXHmbGEeaShOsw==&SID=nkNRF6dD83c=&RoleId=1X5bqQAfxQY=&GroupId=KUxCDdt69t4=",
|
||||
"domain": "crm.zhongtukj.com", "path": "/"}
|
||||
])
|
||||
|
||||
page = context.new_page()
|
||||
print(f"正在加载第 {start_page} 页（初始页）...")
|
||||
page.goto("http://crm.zhongtukj.com/Boss/Customer/CustomerPackageList.aspx")
|
||||
page.wait_for_load_state("networkidle")
|
||||
time.sleep(2)
|
||||
|
||||
all_data = []
|
||||
|
||||
for current_page in range(start_page, end_page + 1):
|
||||
if current_page == 1:
|
||||
# 第 1 页已加载，直接提取
|
||||
pass
|
||||
else:
|
||||
# 跳转到指定页码
|
||||
print(f"正在跳转到第 {current_page} 页...")
|
||||
page.evaluate(f"() => __doPostBack('AspNetPager', '{current_page}')")
|
||||
page.wait_for_load_state("networkidle")
|
||||
time.sleep(2) # 稳定等待
|
||||
|
||||
# 提取当前页数据
|
||||
data = extract_table_data(page)
|
||||
all_data.extend(data)
|
||||
print(f" 第 {current_page} 页提取 {len(data)} 条记录")
|
||||
|
||||
browser.close()
|
||||
|
||||
# 保存结果
|
||||
if all_data:
|
||||
df = pd.DataFrame(all_data)
|
||||
filename = f"D:\Idea Project\F6+宜搭+其它(1)\张阳脚本\文件输出\套餐卡_第{start_page}至{end_page}页.xlsx"
|
||||
df.to_excel(filename, index=False)
|
||||
print(f"\n✅ 共提取 {len(all_data)} 条记录，已保存到 '{filename}'")
|
||||
else:
|
||||
print("⚠️ 未提取到任何数据")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,112 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": "# 有效会员卡导出",
|
||||
"id": "ea10c552c30ccb5d"
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true,
|
||||
"ExecuteTime": {
|
||||
"end_time": "2026-01-27T02:00:26.466693100Z",
|
||||
"start_time": "2026-01-27T02:00:16.549976600Z"
|
||||
}
|
||||
},
|
||||
"source": [
|
||||
"import time\n",
|
||||
"\n",
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"cookies = {\n",
|
||||
" 'td_cookie': '628168942',\n",
|
||||
" 'td_cookie': '627897944',\n",
|
||||
" 'ASP.NET_SessionId': 'q5qzer2z51b4uzsxrhterzop',\n",
|
||||
" 'ztrjnew@4db97b96-12af-45b0-b232-fd1e9b7a672e@': 'PassWord=wZn2IuvdWeE=&RememberPwd=RXv90LpPskw=&UserId=nzK31b3ZYVQ=&CSID=VjfeyHPOjnU=&UserName=fDfTOArNJXHmbGEeaShOsw==&SID=nkNRF6dD83c=&RoleId=1X5bqQAfxQY=&GroupId=KUxCDdt69t4=',\n",
|
||||
"}\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Accept': 'application/json, text/javascript, */*; q=0.01',\n",
|
||||
" 'Accept-Language': 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6',\n",
|
||||
" 'Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://crm.zhongtukj.com/Boss/Customer/CustomerCardListMem.aspx',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/144.0.0.0 Safari/537.36 Edg/144.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest',\n",
|
||||
" # 'Cookie': 'td_cookie=628168942; td_cookie=627897944; ASP.NET_SessionId=q5qzer2z51b4uzsxrhterzop; ztrjnew@4db97b96-12af-45b0-b232-fd1e9b7a672e@=PassWord=wZn2IuvdWeE=&RememberPwd=RXv90LpPskw=&UserId=nzK31b3ZYVQ=&CSID=VjfeyHPOjnU=&UserName=fDfTOArNJXHmbGEeaShOsw==&SID=nkNRF6dD83c=&RoleId=1X5bqQAfxQY=&GroupId=KUxCDdt69t4=',\n",
|
||||
"}\n",
|
||||
"all_data= []\n",
|
||||
"for i in range(1,4):\n",
|
||||
" params = {\n",
|
||||
" 'action': 'GetList',\n",
|
||||
" 'groupId': '9',\n",
|
||||
" 'keyword': '',\n",
|
||||
" 'page': i,\n",
|
||||
" 'rows': '20',\n",
|
||||
" 'sort': 'ID',\n",
|
||||
" 'order': 'desc',\n",
|
||||
" }\n",
|
||||
" time.sleep(1)\n",
|
||||
"\n",
|
||||
" response = requests.get(\n",
|
||||
" 'http://crm.zhongtukj.com/Boss/Customer/CustomerCardListMem.aspx',\n",
|
||||
" params=params,\n",
|
||||
" cookies=cookies,\n",
|
||||
" headers=headers,\n",
|
||||
" verify=False,\n",
|
||||
" )\n",
|
||||
" rows = response.json().get(\"rows\")\n",
|
||||
" all_data.extend(rows)\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(all_data)\n",
|
||||
"df.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\谷途有效会员卡.xlsx\",index=False)"
|
||||
],
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"<>:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"<>:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
"C:\\Users\\hp_z66\\AppData\\Local\\Temp\\ipykernel_27516\\3060641384.py:42: SyntaxWarning: invalid escape sequence '\\I'\n",
|
||||
" df.to_excel(\"D:\\Idea Project\\F6+宜搭+其它(1)\\张阳脚本\\文件输出\\谷途有效会员卡.xlsx\",index=False)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
},
|
||||
{
|
||||
"metadata": {},
|
||||
"cell_type": "markdown",
|
||||
"source": [
|
||||
"# 会员卡消费记录\n",
|
||||
"\n",
|
||||
"通过py文件获取"
|
||||
],
|
||||
"id": "d91331ded096a8a5"
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"id": "initial_id",
|
||||
"metadata": {
|
||||
"collapsed": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-11-21T03:07:30.465213Z",
|
||||
"start_time": "2024-11-21T03:07:26.328734Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": [
|
||||
"import requests\n",
|
||||
"import pandas as pd\n",
|
||||
"\n",
|
||||
"number = '5000'\n",
|
||||
"\n",
|
||||
"url = f\"http://www.chezhanggui.com:92/SupplierBackstage/Customer/GetMemberList?pageSize={number}&page=1&searchkey=&isCardVip=&IsMoney=&cardName=&employeeName=&CompanyName=&isExpire=\"\n",
|
||||
"\n",
|
||||
"headers = {\n",
|
||||
" 'Cookie': 'ASP.NET_SessionId=q1f0j143i4ccqbfx1bpui2mp; LoginUserKey=D8B10FB06FD166E2B81949BB6DC5E35D65EFB7BD900C78247B2FCC443BB295017C07AD4FDD10B7528CC943B0FFF637916ADD46C705F76A935BF6BC39B966D1CB315BE55C2D4BEEDDC1B5CB070949F8DAF00E62BE916B55226C054C2443D06842A5E898BA1539F11B8B1AA65B3842D9FD33CB788A63C7C075E69F073BBC1CE4B8A91F57BA7D4DB975696A9F99FE389F2F623E9257DE9EA2E96E349D643A17A38CE67A365F96AD515DD547284758555B025664BFA21549999C9B7629FB823BD2E3AEABA0D9B945CACA47A174CE0953DD8975E1BA985F04E48C876EBD7FF8BF4E122CACD8880DB4220B9924159BBEDB56F10E19276C6920D2CB66C827D402101D15B5B6ED707C153C12A92AE6A0B6F7C4078EDC6655033B5AF2A1817777FABE4AF235DE687FF001CB4E9C99E132CF1E6886F840DDBADE5F2912CE6FF4FC47C44F132E31D310A28A15D49AB9EF0C1E822947B0362C0DA1E4CFF9F0506A3A36FB3065805163BA135EA6C49D7744213154EA2EBD9CE2E64F547473EE5CA550BC3033A9814788DBDCCE2F3BFFE90A17D82CC8A12684266CFD8F22D74E2F334114C7A557C13C989345EC803B36F20053897B22C7B080EB2A0176084629CF9E1700FB0C1B9C64852703C3AFB64FC265D071044C127DA663AA0EC9966E9F35D14D0960CE4C5A4CABAF7ABB8472404AA8D61B1DF9BDB251F12DEBE6D7E83B58F46314BCBB8F7BA2EE1B36848C1FA15257350AD7C97C461F6E41D21A2A2EAA2E38CF99B0985219EAB1FD2D682E732C1D723DC4B5F21AEE607D2BB6E6409CD9A7A57CC1C4C8BA5929782CC654AC9CE70D54A60E1AB3934A0A2C66CB033EF7E8532609A6364D90B0EC455B01F3C394D6B6367FB596900108093A50ADA3159559A0A713C9FEA230A1EE845C3C2D569A3803F8D5A2700A797CAEAB667C7809F066DD649CE26DEDC71C9DB841A5D83EF72B94ADE8AD5338E33E2D4D271B85CC27427A23EC8052A7DE782817940BD322819002D1D5BE1A07C0',\n",
|
||||
" 'Host': 'www.chezhanggui.com:92',\n",
|
||||
" 'Proxy-Connection': 'keep-alive',\n",
|
||||
" 'Referer': 'http://www.chezhanggui.com:92/SupplierBackstage/Customer/MemberInfoPage',\n",
|
||||
" 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36 Edg/130.0.0.0',\n",
|
||||
" 'X-Requested-With': 'XMLHttpRequest'\n",
|
||||
"}\n",
|
||||
"data = f'pageSize={number}&page=1&searchkey=&isCardVip=&IsMoney=&cardName=&employeeName=&CompanyName=&isExpire='\n",
|
||||
"\n",
|
||||
"response = requests.get(url=url, headers=headers, data=data)\n",
|
||||
"# print(response.json())\n",
|
||||
"all_data = response.json()\n",
|
||||
"\n",
|
||||
"data = all_data['Rows']\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(data)\n",
|
||||
"\n",
|
||||
"# 定义新列名\n",
|
||||
"new_columns = {\n",
|
||||
" 'CustomerName': '客户名称',\n",
|
||||
" 'CustomerMobile': '手机号',\n",
|
||||
" 'CustomerCarNumbers': '车牌',\n",
|
||||
" 'CustomerCreateTime': '创建时间',\n",
|
||||
" 'CardNum': '卡号',\n",
|
||||
" 'CardMoney': '卡上余额',\n",
|
||||
" 'CardName': '卡类型',\n",
|
||||
" 'CardCreateTime': '办卡日期',\n",
|
||||
" 'EmployeeName': '负责客服',\n",
|
||||
" 'EndDT': '过期时间',\n",
|
||||
" 'iswechat': '是否关注微信',\n",
|
||||
" 'Telephone': '其他联系方式',\n",
|
||||
" # 继续添加更多的列名映射...\n",
|
||||
"}\n",
|
||||
"# 使用rename方法替换列名\n",
|
||||
"df.rename(columns=new_columns, inplace=True)\n",
|
||||
"\n",
|
||||
"# 保存到Excel文件\n",
|
||||
"file_path = r'D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\车好了导出数据.xlsx'\n",
|
||||
"df.to_excel(file_path, index=False)\n",
|
||||
"\n",
|
||||
"print(f\"数据已成功保存至{file_path}\")\n"
|
||||
],
|
||||
"id": "ad55012a4aac09d6",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"数据已成功保存至D:\\Idea Project\\F6+宜搭+其它(1)\\new\\文件输出\\车好了导出数据.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 13
|
||||
},
|
||||
{
|
||||
"metadata": {
|
||||
"ExecuteTime": {
|
||||
"end_time": "2024-11-21T02:46:21.760427Z",
|
||||
"start_time": "2024-11-21T02:46:21.718012Z"
|
||||
}
|
||||
},
|
||||
"cell_type": "code",
|
||||
"source": "",
|
||||
"id": "f21b669a780e6800",
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"数据已成功保存至D:\\customer_data.xlsx\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"execution_count": 3
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 2
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython2",
|
||||
"version": "2.7.6"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,64 @@
|
||||
import requests
|
||||
import re
|
||||
import json
|
||||
|
||||
url = "https://auto.51autoshop.com/Customer/CustCardList"
|
||||
headers = {
|
||||
'Content-Type': 'application/x-www-form-urlencoded',
|
||||
'Cookie': 'Hm_lvt_b13246fd95aec7bfc57d9c415dfefe18=1733898115; __RequestVerificationToken=W1tpkb8doRLe28rYbhoVDiAF5kxH7vis6yUPWbL8CHIkABRM1QBjNpE3WV72mklwY9snog2sP0b_wz9NAevw5AV38-U1; HMACCOUNT=ABFCA62083E00432; LoginInfo=2412349ad7894c01b66cd10b894ff2d4; GetShopValidity=1; SERVERID=df24b70122439a42ff2eaa4b483d47a0|1733971685|1733934764; Hm_lpvt_b13246fd95aec7bfc57d9c415dfefe18=1733971687',
|
||||
'Origin': 'https://auto.51autoshop.com',
|
||||
'Priority': 'u=0, i',
|
||||
'Referer': 'https://auto.51autoshop.com/Customer/CustCardList',
|
||||
'Sec-Ch-Ua': '"Microsoft Edge";v="131", "Chromium";v="131", "Not_A Brand";v="24"',
|
||||
'Sec-Ch-Ua-Mobile': '?0',
|
||||
'Sec-Ch-Ua-Platform': '"Windows"',
|
||||
'Sec-Fetch-Dest': 'iframe',
|
||||
'Sec-Fetch-Mode': 'navigate',
|
||||
'Sec-Fetch-Site': 'same-origin',
|
||||
'Sec-Fetch-User': '?1',
|
||||
'Upgrade-Insecure-Requests': '1',
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0'
|
||||
}
|
||||
data = {
|
||||
'__RequestVerificationToken': '1OTyeT2u_1P6DQTBCMLDs1jTbEE4FmnXbz1d2N9-wox36wkGAkIKCcyXZkl9KW2BcGsW0lZjb6QDGcwQoryIiBme-Z81',
|
||||
'Keywords': '',
|
||||
'StartTime': '',
|
||||
'EndTime': '',
|
||||
'WCompanyID': '',
|
||||
'ExpiredOnly': 'false',
|
||||
'DetialUrl': '/Customer/CustCardDetial',
|
||||
'PageIndex': '1',
|
||||
'PageSize': '10000',
|
||||
'SortField': '',
|
||||
'SortDir': '',
|
||||
'IsExpired': '',
|
||||
'CompanyName': ''
|
||||
}
|
||||
|
||||
res = requests.post(url, data=data, headers=headers)
|
||||
html_text = res.text
|
||||
|
||||
|
||||
import re
|
||||
import pandas as pd
|
||||
text = '''
|
||||
"CustCardID": 710943,
|
||||
"OtherField": "SomeValue",
|
||||
"CustCardID": 123456,
|
||||
"CustCardID": 987654,
|
||||
'''
|
||||
|
||||
pattern = r'"CardCode":\s*"([^"]+)"'
|
||||
pattern1 = r'"CustCardID":(\d+)'
|
||||
|
||||
matches1 = re.findall(pattern1, html_text)
|
||||
matches = re.findall(pattern, html_text)
|
||||
print(matches) # 输出: ['710943', '123456', '987654']
|
||||
|
||||
# 将匹配结果转换为 DataFrame
|
||||
df = pd.DataFrame(matches, columns=['CardCode'])
|
||||
df.to_excel('CardCode.xlsx', index=False)
|
||||
df1 = pd.DataFrame(matches1, columns=['CustCardID'])
|
||||
|
||||
# 将 DataFrame 保存为 Excel 文件
|
||||
df1.to_excel('CardCode1.xlsx', index=False)
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user