zhuchaowe/maxkb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: 修复字符集不匹配的情况提取web的问题

Browse Source 
--bug=1048607 --user=刘瑞斌 【github#1577】有个网站导入web知识库报错 https://www.tapd.cn/57709429/s/1623295
This commit is contained in:
CaptainB 2024-12-03 14:57:54 +08:00 committed by 刘瑞斌
parent b572956da6
commit 695576bfbd
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
apps/common/util/fork.py
View File

@ -142,7 +142,10 @@ class Fork:
if len(charset_list) > 0: if len(charset_list) > 0:
charset = charset_list[0] charset = charset_list[0]
if charset != encoding: if charset != encoding:
try:
html_content = response.content.decode(charset) html_content = response.content.decode(charset)
except Exception as e:
logging.getLogger("max_kb").error(f'{e}')
return BeautifulSoup(html_content, "html.parser") return BeautifulSoup(html_content, "html.parser")
return beautiful_soup return beautiful_soup