使用selenium抓取JS动态生成的页面

标签：selenium   浏览器   javascript   

Selenium是一个模拟浏览器的自动化测试工具，它提供一组API可以与真实的浏览器内核交互。

Java环境下的maven配置如下：

<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-java</artifactId>
   <version>2.46.0</version>
  </dependency>

System.getProperties().setProperty("webdriver.chrome.driver",
        "D:\\chromedriver\\chromedriver.exe");

获取整个页面

 public static void testChromeDriver() {
System.getProperties().setProperty("webdriver.chrome.driver",
        "D:\\chromedriver\\chromedriver.exe");
WebDriver webDriver = new ChromeDriver();
webDriver.get("http://picture.youth.cn/qtdb/201506/t20150625_6789707.htm");
String responseBody = webDriver.getPageSource();
System.out.println(responseBody);
webDriver.close();
}

public static void waitForSomthing(){
System.getProperties().setProperty("webdriver.chrome.driver",
        "D:\\chromedriver\\chromedriver.exe");
WebDriver driver = new ChromeDriver();
driver.get("http://news.sina.com.cn/c/2015-07-04/023532071740.shtml");
        WebDriverWait wait = new WebDriverWait(driver,10);  
        wait.until(new ExpectedCondition<Boolean>() {
            public Boolean apply(WebDriver webDriver) {
                System.out.println("Searching ...");
                return webDriver.findElement(By.id("commentCount1")).getText().length() != 0;
            }
        });
          
        WebElement element = driver.findElement(By.id("commentCount1"));  
        System.out.println("element="+element.getText());
}

使用selenium抓取JS动态生成的页面

标签：selenium   浏览器   javascript   

原文地址：http://blog.csdn.net/lgcssx/article/details/47174565