As mentioned in our previous article, we have a wdio-based automation project. We utilized Chrome v108 for our end-to-end functional automation testing. However, given the current browser version being > v115 and since Chrome For Testing was available for Chrome version >= v115, we decided it would be beneficial to upgrade to Chrome For Testing (CFT) in our automation project.
Since the introduction of Google's Chrome For Testing, a new Chrome flavor that specifically targets web app testing and automation use cases, in browser automation, it has emerged as the top choice for conducting automated browser tasks.
What is CFT?
Chrome For Testing is a flavor of chrome browser which is specifically created for running automated test cases. It is a browser which is similar to chrome browser but with few key differences. First off, there is no automatic updating for the CFT browser. This makes it possible to execute automated scripts against a specific version of Chrome. The command-line utility can be used to get the binaries for that particular version of CFT.
Note:- Chromedriver prior to version 115 are not integrated with chrome for testing infrastructure and need to be downloaded from Chromedriver Website
Setting up CFT
We explored using "Chrome For Testing" in our WebdriverIO automation project. As mentioned in the article Take a seat, WebdriverIO is driving for you!, Now that CFT is included into WebdriverIO projects, you can run your automation script on a specific version of CFT by specifying browser name and version in the browser capabilities as demonstrated in the example below:
wdio.conf.js
export const config = {
// ...
capabilities: [{
browserName: 'chrome',
browserVersion: 125.0.6422.141 // Alternatively you can also specify only the initial part of version number e.g. 125.
}]
}
With the above config, WDIO will download the designated CFT and chromedriver version and execute tests.
In doing so, we eliminated all browser drivers and explicitly defined the Chrome version in the WDIO capabilities. This allowed WDIO to execute tests on the CFT instead of actual browsers. However, during local testing, we observed unexpected prompts such as "Save password?" appearing on the CFT, which were absent in real Chrome browsers. Additionally, a warning was displayed in the terminal. Please see the screenshot below for reference.
Continue to use real Chrome Browsers
Due to the above behavior, we were uncertain whether employing CFT would be advantageous for our web application's end-to-end regression testing. So we dropped the idea of CFT and opted to utilize real Chrome browsers. However, we still had to address our issue with sudden Chrome browser upgrades, which were causing session creation failures with the message This chromedriver only supports version <version_number> 😞. Through the insights gained from this blog post, we comprehended that starting from WebdriverIO version v8.14.0 and beyond, we could eliminate the need for driver services.
We removed the following two dependencies from our package.json
- chromedriver
- wdio-chromedriver-service
To run the WebdriverIO project with Chrome installed on your computer, you only need to specify the browser name in the browser capabilities.
export const config = {
// ...
capabilities: [{
browserName: 'chrome',
}]
}
After removing the aforementioned dependencies and updating the browser capabilities, WDIO started searching for the currently installed Chrome browser on our machine and downloaded the appropriate chromedriver.
The issue remains unresolved 😞
After pushing this commit and executing our end-to-end regression tests according to the schedule, we encountered numerous spec failures with the invalid session id error. Our regression runs on CircleCI, as detailed in our previous blog post. Upon investigation, we identified that the latest Chrome browser version v121 (121.0.6167.164), was being utilized. Further investigation revealed that while Chrome v121 had been released, it was not sufficiently stable for automation purposes. Notably, it exhibited high memory usage due to memory leak, leading to frequent browser crashes.
We also encountered this issue on Chromium and found a similar problem here.
Use Specific Stable Chrome Version
In our CircleCI's configuration file, we specified Chrome version 118.0.5993.117
commands:
install:
steps:
- checkout
- run: sudo apt-get update
- node/install-packages
- browser-tools/install-chrome:
replace-existing: true
chrome-version: "118.0.5993.117"
With this adjustment, our automation specifications are now executing flawlessly on Chrome version 118.
Disclaimer:- When we looked into CFT, the latest browser version was 121. Since then, newer versions of Chrome have been released, so you may not encounter the same issues. The primary purpose of this blog is to guide you on how to use a specific Chrome version and run tests on real browsers if you prefer not to use CFT.