zhuchaowe/maxkb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: When importing documents from a web knowledge base, if a failed document is encountered, the import will be interrupted (#2511)

Browse Source
This commit is contained in:
shaohuzhang1 2025-03-07 10:19:57 +08:00 committed by GitHub
parent 96ab89adf3
commit 8b9998a53d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 5 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
apps/dataset/task/sync.py
View File

@ -54,5 +54,8 @@ def sync_replace_web_dataset(dataset_id: str, url: str, selector: str):
def sync_web_document(dataset_id, source_url_list: List[str], selector: str): def sync_web_document(dataset_id, source_url_list: List[str], selector: str):
handler = get_sync_web_document_handler(dataset_id) handler = get_sync_web_document_handler(dataset_id)
for source_url in source_url_list: for source_url in source_url_list:
try:
result = Fork(base_fork_url=source_url, selector_list=selector.split(' ')).fork() result = Fork(base_fork_url=source_url, selector_list=selector.split(' ')).fork()
handler(source_url, selector, result) handler(source_url, selector, result)
except Exception as e:
pass