ThinkChat🤖让你学习和工作更高效,注册即送10W Token,即刻开启你的AI之旅 广告
### 1.说明 selenium是一个自动化测试工具,可以利用它驱动浏览器执行特定的动作 中文文档:[http://selenium-python-zh.readthedocs.io/en/latest/](http://selenium-python-zh.readthedocs.io/en/latest/) 英文文档:[https://seleniumhq.github.io/selenium/docs/api/py/api.htm](https://seleniumhq.github.io/selenium/docs/api/py/api.html)l 节点操作:[http://selenium-python.readthedocs.io/api.html\#module-selenium.webdriver.remote.webelement](http://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.remote.webelement) 动作链:[http://selenium-python.readthedocs.io/api.html\#module-selenium.webdriver.common.action\_chains](http://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.common.action_chains) 等待条件参数及用法介绍:[http://selenium-python.readthedocs.io/api.html\#module-selenium.webdriver.support.expected\_conditions](http://selenium-python.readthedocs.io/api.html#module-selenium.webdriver.support.expected_conditions) 异常类:[http://selenium-python.readthedocs.io/api.html\#module-selenium.common.exceptions](http://selenium-python.readthedocs.io/api.html#module-selenium.common.exceptions) ### 2.准备 [安装selenium及其驱动](/1kai-fa-huan-jing-pei-zhi/12-qing-qiu-ku-de-an-zhuang/122-seleniumde-an-zhuang.md) ### 3.基本使用 ``` from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.wait import WebDriverWait browser = webdriver.Chrome() try: browser.get('https://www.baidu.com') input = browser.find_element_by_id('kw') input.send_keys("python") input.send_keys(Keys.ENTER) wait = WebDriverWait(browser,10) wait.until(EC.presence_of_element_located((By.ID,"content_left"))) print(browser.current_url) print(browser.get_cookies()) print(browser.page_source) except: browser.close() ``` ### 3.声明浏览器对象 ``` from selenium import webdriver browser = webdriver.Chrome() browser = webdriver.Firefox() browser = webdriver.Edge() browser = webdriver.PhantomJS() browser = webdriver.Safari() ``` ### 4.访问页面 调用get\(\)方法请求第一个网页,参数传入链接url,通过page\_source属性获取源码 ``` browser.get('https://www.baidu.com') print(browser.page_source) ``` ### 5.查找节点 #### 单个结点 获取淘宝搜索框的节点 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.taobao.com") # 根据id获取 input_first = browser.find_element_by_id("q") # 根据name获取 input_second = browser.find_element_by_name("q") # css选择器 input_third = browser.find_element_by_css_selector("#q") # xpath input_four = browser.find_element_by_xpath("//*[@id='q']") print(input_first,input_second,input_third,input_four,sep='\n') browser.close() ``` ``` find_element_by_id <==> find_element(ByID,id) find_element_by_name find_element_by_xpath find_element_by_link_text find_element_by_partial_link_text find_element_by_tag_name find_element_by_class_name find_element_by_css_selector ``` 多个节点 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.taobao.com") lis = browser.find_elements_by_css_selector(".service-bd li") print(lis) browser.close() ``` ``` find_elements_by_id find_elements_by_name find_elements_by_xpath find_elements_by_link_text find_elements_by_partial_link_text find_elements_by_tag_name find_elements_by_class_name find_elements_by_css_selector <==> find.elements(By.CSS_SELECTOR,'.service-bd li') ``` ### 6.节点交互 send\_key\(\):输入 clear\(\):清空 click\(\):点击 ``` from selenium import webdriver import time browser = webdriver.Chrome() browser.get("https://www.taobao.com/") # 选中输入框 input = browser.find_element_by_css_selector("#q") input.clear() # 输入 input.send_keys("苹果") # 等待时间 time.sleep(1) # 清空 input.clear() # 输入 input.send_keys("香蕉") # 选中按钮 button = browser.find_element_by_css_selector("#J_TSearchForm > div.search-button > button") # 点击 button.click() ``` ### 7.动作链 ``` from selenium import webdriver from selenium.webdriver import ActionChains browser = webdriver.Chrome() url = "http://www.runoob.com/try/try.php?filename=jqueryui-api-droppable" browser.get(url) # 根据id切换iframe # browser.switch_to_frame("iframeResult") # 最新的切换方法 browser.switch_to.frame("iframeResult") source = browser.find_element_by_css_selector("#draggable") target = browser.find_element_by_css_selector("#droppable") actions = ActionChains(browser) # 调用拖拽方法 actions.drag_and_drop(source,target) # 执行动作 actions.perform() ``` ### 8.执行javascript 模拟执行javascript,使用execute\_script\(\)方法实现 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.zhihu.com/explore") # 将进度条下拉到最底部 browser.execute_script('window.scrollTo(0, document.body.scrollHeight)') # 弹出提示框 browser.execute_script('alert("To Bottom")') ``` ### 9.获取节点信息 #### 获取属性 使用get\_attribute\(\)方法来获取节点的属性 ``` from selenium import webdriver browser = webdriver.Chrome() url = "https://www.zhihu.com/explore" browser.get(url) logo = browser.find_element_by_id("zh-top-link-logo") print(logo) print(logo.get_attribute("class")) ``` 运行结果: ``` <selenium.webdriver.remote.webelement.WebElement (session="036f1edccab08bbc5ca2b8f02accbb04", element="0.40649588367097245-1")> zu-top-link-logo ``` #### 获取文本值 通过text属性获取文本信息 ``` from selenium import webdriver browser = webdriver.Chrome() url = "https://www.zhihu.com/explore" browser.get(url) input = browser.find_element_by_id("zu-top-add-question") print(input.text) ``` 运行结果: ``` 提问 ``` #### 获取ID、位置、标签名、大小 id属性:获取节点id location:获取节点再页面中的相对位置 tag\_name:获取标签名称 size:获取节点的大小 ``` from selenium import webdriver browser = webdriver.Chrome() url = "https://www.zhihu.com/explore" browser.get(url) input = browser.find_element_by_id("zu-top-add-question") print(input.id) print(input.location) print(input.tag_name) print(input.size) ``` 运行结果: ``` 0.4994837841126192-1 {'x': 683, 'y': 7} button {'height': 32, 'width': 66} ``` ### 10.切换Frame 使用switch\_to.frame\(\)切换frame 示例:QQ邮箱登录 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://mail.qq.com/") # 等待三秒 browser.implicitly_wait(3) # 因为页面有很多iframe的内置框架,需要先逐级定位到登录元素所在的iframe才行 browser.switch_to.frame("login_frame") name = browser.find_element_by_name("u") name.send_keys("xxxx@qq.com") password = browser.find_element_by_name("p") password.send_keys("xxxx") button = browser.find_element_by_id("login_button") button.click() ``` ### 11.延时等待 #### 隐式等待 implicitly\_wait\(\):等待几秒 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://mail.qq.com/") browser.implicitly_wait(3) browser.switch_to.frame("login_frame") name = browser.find_element_by_name("u") name.send_keys("xxxx@qq.com") password = browser.find_element_by_name("p") password.send_keys("xxxx") button = browser.find_element_by_id("login_button") button.click() ``` #### 显示等待 ``` from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions from selenium.webdriver.common.by import By browser = webdriver.Chrome() browser.get("https://mail.qq.com/") # 创建WebDriverWait实例化对象,并指定最长等待时间 wait = WebDriverWait(browser,3) browser.switch_to.frame("login_frame") # presence_of_element_located:查询节点,参数为结点的定位元组 name = wait.until(expected_conditions.presence_of_element_located((By.ID,'u'))) name.send_keys("xxx@qq.com") password = browser.find_element_by_name("p") password.send_keys("xxx") # element_to_be_clickable:可执行点击 button = wait.until(expected_conditions.element_to_be_clickable((By.ID,'login_button'))) button.click() ``` #### 所有的加载条件 | 等待条件 | 含义 | | :--- | :--- | | title\_is | 标题是某内容 | | title\_contains | 标题包含某内容 | | presence\_of\_element\_located | 节点加载出,传入定位元组,如\(By.ID, 'p'\) | | visibility\_of\_element\_located | 节点可见,传入定位元组 | | visibility\_of | 可见,传入节点对象 | | presence\_of\_all\_elements\_located | 所有节点加载出 | | text\_to\_be\_present\_in\_element | 某个节点文本包含某文字 | | text\_to\_be\_present\_in\_element\_value | 某个节点值包含某文字 | | frame\_to\_be\_available\_and\_switch\_to\_it frame | 加载并切换 | | invisibility\_of\_element\_located | 节点不可见 | | element\_to\_be\_clickable | 节点可点击 | | staleness\_of | 判断一个节点是否仍在DOM,可判断页面是否已经刷新 | | element\_to\_be\_selected | 节点可选择,传节点对象 | | element\_located\_to\_be\_selected | 节点可选择,传入定位元组 | | element\_selection\_state\_to\_be | 传入节点对象以及状态,相等返回True,否则返回False | | element\_located\_selection\_state\_to\_be | 传入定位元组以及状态,相等返回True,否则返回False | | alert\_is\_present | 是否出现Alert | ### 12.前进后退 back\(\):后退 forward\(\):前进 ``` import time from selenium import webdriver browser = webdriver.Chrome() browser.get('https://www.baidu.com/') browser.get('https://www.taobao.com/') browser.get('https://www.python.org/') browser.back() time.sleep(1) browser.forward() browser.close() ``` ### 13.Cookies 对cookies进行操作 ``` from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.zhihu.com/explore") print(browser.get_cookies()) # 添加cookies browser.add_cookie({"name":"angle","value":"value"}) # 获取cookies print(browser.get_cookies()) # 删除所有cookies browser.delete_all_cookies() # 获取cookies print(browser.get_cookies()) ``` 运行结果: ``` [{'domain': '.zhihu.com', 'httpOnly': False, 'name': 'l_n_c', 'path': '/', 'secure': False, 'value': '1'}, {'domain': 'www.zhihu.com', 'expiry': 1535626235.952171, 'httpOnly': False, 'name': 'tgw_l7_route', 'path': '/', 'secure': False, 'value': '56f3b730f2eb8b75242a8095a22206f8'}, {'domain': '.zhihu.com', 'expiry': 1535627141, 'httpOnly': False, 'name': '__utmb', 'path': '/', 'secure': False, 'value': '51854390.0.10.1535625342'}, {'domain': '.zhihu.com', 'expiry': 1630233335.95244, 'httpOnly': False, 'name': 'q_c1', 'path': '/', 'secure': False, 'value': 'df753284d0ae4a2cb8bda8e41bc190e8|1535625336000|1535625336000'}, {'domain': 'www.zhihu.com', 'httpOnly': False, 'name': '_xsrf', 'path': '/', 'secure': False, 'value': '4c00cb0973af088c42d4a58468817612'}, {'domain': '.zhihu.com', 'expiry': 1538217335.95261, 'httpOnly': False, 'name': 'r_cap_id', 'path': '/', 'secure': False, 'value': '"OGEwOWQzNGYxYTZiNGU0ZDhlMTJmNGFhZmZhMjJjZTQ=|1535625336|37bd3be47d98ccd1123d7e35bf76cb1312d7143b"'}, {'domain': '.zhihu.com', 'expiry': 1538217335.952672, 'httpOnly': False, 'name': 'cap_id', 'path': '/', 'secure': False, 'value': '"MzJmZWJlNDc1YjY0NGQ0OGEzMGI4ZTUwMGE4OThhZmU=|1535625336|a4628b2bd1054ddfed0bb5f0b02cf7fbca1e5216"'}, {'domain': '.zhihu.com', 'expiry': 1538217335.952705, 'httpOnly': False, 'name': 'l_cap_id', 'path': '/', 'secure': False, 'value': '"ZGU4YTdlNjJhMWRiNDkyZGE3Y2I3OTYzYzkxN2IzMzg=|1535625336|dc1242668bfb5a0d13df384da1248835b6a1d7ba"'}, {'domain': '.zhihu.com', 'httpOnly': False, 'name': 'n_c', 'path': '/', 'secure': False, 'value': '1'}, {'domain': '.zhihu.com', 'expiry': 1630233340.416983, 'httpOnly': False, 'name': 'd_c0', 'path': '/', 'secure': False, 'value': '"AAAmfnWtIg6PTicvdBW52HutZ46oS_fCo1Y=|1535625341"'}, {'domain': '.zhihu.com', 'expiry': 1598697340, 'httpOnly': False, 'name': '_zap', 'path': '/', 'secure': False, 'value': '55900a43-f801-4abe-b643-1f67dd005a6e'}, {'domain': '.zhihu.com', 'expiry': 1598697341, 'httpOnly': False, 'name': '__utma', 'path': '/', 'secure': False, 'value': '51854390.1748932163.1535625342.1535625342.1535625342.1'}, {'domain': '.zhihu.com', 'httpOnly': False, 'name': '__utmc', 'path': '/', 'secure': False, 'value': '51854390'}, {'domain': '.zhihu.com', 'expiry': 1551393341, 'httpOnly': False, 'name': '__utmz', 'path': '/', 'secure': False, 'value': '51854390.1535625342.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'}, {'domain': '.zhihu.com', 'expiry': 1598697341, 'httpOnly': False, 'name': '__utmv', 'path': '/', 'secure': False, 'value': '51854390.000--|3=entry_date=20180830=1'}] [{'domain': '.zhihu.com', 'httpOnly': False, 'name': 'l_n_c', 'path': '/', 'secure': False, 'value': '1'}, {'domain': 'www.zhihu.com', 'expiry': 1535626235.952171, 'httpOnly': False, 'name': 'tgw_l7_route', 'path': '/', 'secure': False, 'value': '56f3b730f2eb8b75242a8095a22206f8'}, {'domain': '.zhihu.com', 'expiry': 1535627141, 'httpOnly': False, 'name': '__utmb', 'path': '/', 'secure': False, 'value': '51854390.0.10.1535625342'}, {'domain': '.zhihu.com', 'expiry': 1630233335.95244, 'httpOnly': False, 'name': 'q_c1', 'path': '/', 'secure': False, 'value': 'df753284d0ae4a2cb8bda8e41bc190e8|1535625336000|1535625336000'}, {'domain': 'www.zhihu.com', 'httpOnly': False, 'name': '_xsrf', 'path': '/', 'secure': False, 'value': '4c00cb0973af088c42d4a58468817612'}, {'domain': '.zhihu.com', 'expiry': 1538217335.95261, 'httpOnly': False, 'name': 'r_cap_id', 'path': '/', 'secure': False, 'value': '"OGEwOWQzNGYxYTZiNGU0ZDhlMTJmNGFhZmZhMjJjZTQ=|1535625336|37bd3be47d98ccd1123d7e35bf76cb1312d7143b"'}, {'domain': '.zhihu.com', 'expiry': 1538217335.952672, 'httpOnly': False, 'name': 'cap_id', 'path': '/', 'secure': False, 'value': '"MzJmZWJlNDc1YjY0NGQ0OGEzMGI4ZTUwMGE4OThhZmU=|1535625336|a4628b2bd1054ddfed0bb5f0b02cf7fbca1e5216"'}, {'domain': '.zhihu.com', 'expiry': 1538217335.952705, 'httpOnly': False, 'name': 'l_cap_id', 'path': '/', 'secure': False, 'value': '"ZGU4YTdlNjJhMWRiNDkyZGE3Y2I3OTYzYzkxN2IzMzg=|1535625336|dc1242668bfb5a0d13df384da1248835b6a1d7ba"'}, {'domain': '.zhihu.com', 'httpOnly': False, 'name': 'n_c', 'path': '/', 'secure': False, 'value': '1'}, {'domain': '.zhihu.com', 'expiry': 1630233340.416983, 'httpOnly': False, 'name': 'd_c0', 'path': '/', 'secure': False, 'value': '"AAAmfnWtIg6PTicvdBW52HutZ46oS_fCo1Y=|1535625341"'}, {'domain': '.zhihu.com', 'expiry': 1598697340, 'httpOnly': False, 'name': '_zap', 'path': '/', 'secure': False, 'value': '55900a43-f801-4abe-b643-1f67dd005a6e'}, {'domain': '.zhihu.com', 'expiry': 1598697341, 'httpOnly': False, 'name': '__utma', 'path': '/', 'secure': False, 'value': '51854390.1748932163.1535625342.1535625342.1535625342.1'}, {'domain': '.zhihu.com', 'httpOnly': False, 'name': '__utmc', 'path': '/', 'secure': False, 'value': '51854390'}, {'domain': '.zhihu.com', 'expiry': 1551393341, 'httpOnly': False, 'name': '__utmz', 'path': '/', 'secure': False, 'value': '51854390.1535625342.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)'}, {'domain': '.zhihu.com', 'expiry': 1598697341, 'httpOnly': False, 'name': '__utmv', 'path': '/', 'secure': False, 'value': '51854390.000--|3=entry_date=20180830=1'}, {'domain': 'www.zhihu.com', 'expiry': 2166345341, 'httpOnly': False, 'name': 'angle', 'path': '/', 'secure': True, 'value': 'value'}] [] ``` ### 14.选项卡管理 切换窗口:switch\_to.window\(browser.window.handles\[x\]\) ``` # from selenium import webdriver # # browser = webdriver.Chrome() # browser.get("https://www.zhihu.com/explore") # print(browser.get_cookies()) # # 添加cookies # browser.add_cookie({"name":"angle","value":"value"}) # # 获取cookies # print(browser.get_cookies()) # # 删除所有cookies # browser.delete_all_cookies() # # 获取cookies # print(browser.get_cookies()) import time from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.baidu.com") browser.execute_script("window.open()") print(browser.window_handles) # 切换选项卡 browser.switch_to.window(browser.window_handles[1]) browser.get("https://www.taobao.com") time.sleep(1) browser.switch_to.window(browser.window_handles[0]) browser.get("https://python.org")# from selenium import webdriver # # browser = webdriver.Chrome() # browser.get("https://www.zhihu.com/explore") # print(browser.get_cookies()) # # 添加cookies # browser.add_cookie({"name":"angle","value":"value"}) # # 获取cookies # print(browser.get_cookies()) # # 删除所有cookies # browser.delete_all_cookies() # # 获取cookies # print(browser.get_cookies()) import time from selenium import webdriver browser = webdriver.Chrome() browser.get("https://www.baidu.com") browser.execute_script("window.open()") print(browser.window_handles) # 切换选项卡 browser.switch_to.window(browser.window_handles[1]) browser.get("https://www.taobao.com") time.sleep(1) browser.switch_to.window(browser.window_handles[0]) browser.get("https://python.org") ``` 运行结果: ``` ['CDwindow-7A6C2399B267FCE1C8E4A69C49D515B3', 'CDwindow-A751A13C930C1383E6E228CD4C44CEA1'] ``` ### 15.异常处理 ``` from selenium import webdriver from selenium.common.exceptions import TimeoutException, NoSuchElementException browser = webdriver.Chrome() try: browser.get('https://www.baidu.com') except TimeoutException: print('Time Out') try: browser.find_element_by_id('hello') except NoSuchElementException: print('No Element') finally: browser.close() ``` 运行结果: ``` No Element ```