Fetch Page Load Metrics With C# Selenium WebDriver Testing Framework

Software testing in complex, large-scale software systems

When working in a scrum team, QAs spend as much time working on potential failure detection as the actual software testing. Keeping in mind the big picture product purpose and its users is a must to be able to dramatically improve product quality as a QA engineer.

Every now and then we ll encounter changes in software that are noticeable but too difficult to track. One of such changes is application performance degradation. This is not a rare occurrence in large-scale software systems development, so if you are currently testing a performance-intensive product, I believe you ll find this blog useful.

How to fight application performance degradation?

Imagine having a simple feature implemented in your testing framework, that will not bother you and you can pull it out whenever you need its help. Sounds good? I m going to show you how you can implement it in your C# Selenium WebDriver framework.

To begin, open a web browser and navigate to the AUT (application under test). For the demonstration purposes, I m going to use https://demo.nopcommerce.com/ page as AUT.

Press F12 to pull up the Developer Tools screen.

Now select Console and type in: window.performance.timing. This is what the properties will look like:

You ll notice there are loads of metrics to play with!

Navigation Timing is a JavaScript API used for measuring performance on the web page. The API provides a simple way to get accurate and detailed timing statistics.

The metrics are accessed via window.performance object properties. The properties are divided into two specifications:

• Navigation: This specification introduces an interface that provides Web applications with timing-related information. Note: this specification does not cover how Web applications leverage these interfaces to collect, store and report the provided information
• Timing: data for navigation and page load events

Chrome browser also provides a performance.memory property that gives access to JavaScript memory usage data.

The graph above illustrates the timing attributes defined by the PerformanceTiming and the PerformanceNavigation interface with or without a redirect, respectively. Underlined attributes may not be available in navigation involving documents from different origins. User agents may perform internal processing in between timings, which allow for non-normative intervals between timings (read more about this here).

Note: There s also a new type of page metrics, but keep in mind that it s not fully implemented.

From a QA standpoint, if we want to measure page loading time, we should take the time passed between Prompt for unload (navigationStart) and onLoad end (loadEventEnd).

Type the following to the console: “window.performance.timing.domContentLoadedEventEnd-window.performance.timing.navigationStart” and press enter.

I got the response of 1289 milliseconds (circa 1.3 seconds).

If we reload the page and execute the same command again, we’ll get a new value. That s due to the fact that we have loads of parameters that are dependent, such as network latency, CPU usage and similar.

Now let s fetch this data during the test execution and log it to output.

The “window.performance.timing.domContentLoadedEventEnd-window.performance.timing.navigationStart” command is written in JavaScript, so we need to execute it as follows:

IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
var ResponseTime = Convert.ToInt32(js.ExecuteScript("return window.performance.timing.domContentLoadedEventEnd-window.performance.timing.navigationStart;"));

We can also log it to console in order to see the result:

Console.WriteLine(string.Format("Page {0} loading time is {1} ms", driver.Title, ResponseTime));

My method looks like this:

public Home isAt()
{
 Assert.IsTrue(driver.Title.Equals("nopCommerce demo store"));
 IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
 var ResponseTime = Convert.ToInt32(js.ExecuteScript("return window.performance.timing.domContentLoadedEventEnd-window.performance.timing.navigationStart;"));
 Console.WriteLine(string.Format("Page {0} loading time is {1} ms", driver.Title, ResponseTime));
 return this;
}

After test execution, this is what the output looks like:

Looks nice, right? But the real question is “How we can benefit from it?” We should somehow store it and retrieve a page response time by test execution time, test name, host machine and similar as needed. Let s bring a database to action.

Required steps:

• Download and install SQL Server Express edition (get it here, how to install)
• Download and install SQL Management Studio (get it here, how to install)

Create a new database:

Name it and click on the OK button. I gave it the name “nopCommerceDemoMetrics”.

Right-click on the Table folder and select Table to create a new table:

Now define columns (make sure to set ID as a primary key and set Identity Specification to Yes in order for it to be auto-populated), data types and click on the Save button. I named the table “PageResponseTime”.

Note: You can simply execute SQL command below and it will create a new table.

CREATE TABLE [dbo].[PageResponseTime](
 [ID] [int] IDENTITY(1,1) NOT NULL,
 [Page] [nvarchar](150) NULL,
 [ResponseTime] [int] NULL,
 [Host] [nvarchar](50) NULL,
 [TestName] [nvarchar](150) NULL,
 [Date] [datetime] NULL,
 CONSTRAINT [PK_PageResponseTime] PRIMARY KEY CLUSTERED 
(
 [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

Create a DB user “demo” and enable SQL Server and Windows Authentication mode.

Bring up NuGet Package Manager in Visual Studio, search for log4net and install it to the project.

Add the following code to the App.config file:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
  </configSections>
  <appSettings>
    <add key="url" value="https://demo.nopcommerce.com" />
    <add key="browser" value="Firefox" />
  </appSettings>
  <log4net>
    <root>
      <level value="ALL"></level>
      <appender-ref ref="AdoNetAppender"></appender-ref>
    </root>
    <appender name="AdoNetAppender" type="log4net.Appender.AdoNetAppender">
      <bufferSize value="1" />
      <connectionType value="System.Data.SqlClient.SqlConnection,   
       System.Data, Version=1.0.3300.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <connectionStringName value="AutomationDBConnString" />
      <commandText value="INSERT INTO PageResponseTime ([Page],[ResponseTime],[Host],[TestName],[Date]) VALUES (@page, @responsetime, @host, @testname, @log_date)" />
      <parameter>
        <parameterName value="@page" />
        <dbType value="String" />
        <size value="150" />
        <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%property{page}" />
        </layout>
      </parameter>
      <parameter>
        <parameterName value="@responsetime" />
        <dbType value="Int32" />
        <size value="50" />
        <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%property{responsetime}" />
        </layout>
      </parameter>
      <parameter>
        <parameterName value="@host" />
        <dbType value="String" />
        <size value="50" />
        <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%property{host}" />
        </layout>
      </parameter>
      <parameter>
        <parameterName value="@testname" />
        <dbType value="String" />
        <size value="150" />
        <layout type="log4net.Layout.PatternLayout">
          <conversionPattern value="%property{testname}" />
        </layout>
      </parameter>
      <parameter>
        <parameterName value="@log_date" />
        <dbType value="DateTime" />
        <layout type="log4net.Layout.RawTimeStampLayout" />
      </parameter>
    </appender>
  </log4net>
  <connectionStrings>
    <add name="AutomationDBConnString" connectionString="Server=ENTER_HOST_NAME\SQLEXPRESS;Database=nopCommerceDemoMetric;User Id=demo;Password=demo;"  providerName="System.Data.SqlClient"  />
  </connectionStrings>
</configuration>

• <appSettings> node contains my test data
• <log4net> log4net configuration
• <connectionStrings> node contains database connection string.
  Remove ENTER_HOST_NAME with a host name where the database is installed.

Now, create a new class LogHelper.cs.

using log4net;
using OpenQA.Selenium;
using System;
using System.Collections;
using System.Collections.Generic;
namespace Test
{
    public class LogHelper
    {
        public LogHelper()
        {
            log4net.Config.XmlConfigurator.Configure();
        }
readonly ILog Logger = LogManager.GetLogger(Environment.MachineName);
        IList items = new List<KeyValuePair<int, string>>();
public void LogData(IWebDriver driver, string testname, int reponsetime)
        {
            LogicalThreadContext.Properties["page"] = driver.Title;
            LogicalThreadContext.Properties["responsetime"] = reponsetime;
            LogicalThreadContext.Properties["host"] = Environment.MachineName;
            LogicalThreadContext.Properties["testname"] = testname;
            Logger.Info("AutomationTesting");
        }
    }
}

Modify page classes and call the LogData method:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using SeleniumExtras.PageObjects;
using System;
namespace Test
{
    public class Home
    {
        public Home()
        {
            driver = null;
            log = null;
        }
        public Home(IWebDriver webDriver)
        {
            driver = webDriver;
            log = new LogHelper();
        }
//Driver
        IWebDriver driver;
        LogHelper log;
//Locators
        [FindsBy(How = How.XPath, Using = "//ul[@class='top-menu notmobile']//a[@href='/computers']")]
        private IWebElement ComputersLink;
//Actions
        public Home isAt(string testname)
        {
            Assert.IsTrue(driver.Title.Equals("nopCommerce demo store"));
            IJavaScriptExecutor js = (IJavaScriptExecutor)driver;
            int ResponseTime = Convert.ToInt32(js.ExecuteScript("return window.performance.timing.domContentLoadedEventEnd-window.performance.timing.navigationStart;"));
            Console.WriteLine(string.Format("Page {0} loading time is {1} ms", driver.Title, ResponseTime));
            log.LogData(driver, testname, ResponseTime);
            return this;
        }
        public Home GoToComputers()
        {
            ComputersLink.Click();
            return this;
        }
    }
}

Notice that I had to update isAt method in order to accept the test name parameter (which is pulled out from the TestContext).

Update all the page classes and run the test.

Bing up SQL Management Studio and select all records from our table:

After this, all the test runs will store page loading time metrics to our database. Later we ll be able to monitor page response time over time with simple queries.

---

I hope you ll find this short guide of use with performance degradation tracking. If you have any questions on the topic, feel free to ask!