crawl4ai 大模型友好格式输入爬虫框架
参考：
https://github.com/unclecode/crawl4ai
底层用的微软的 playwright 爬虫架构
1、安装
# Install the package
pip install -U crawl4ai
# Run post-installation setup
crawl4ai-setup
2、使用
import asyncio
from crawl4ai import *
async def main():
async with AsyncWebCrawler() as crawler:
result = await crawler.arun(
url="https://sj.qq.com/appdetail/com.xingin.xhs",
)
print(result.markdown)
if __name__ == "__main__":
asyncio.run(main())
