Coded UI Test: Technologies Evaluation

As of this Christmas day, this is my assessment of the main technologies & products out there supporting Coded UI Testing. For those who don't know, Coded UI Testing is about programmers writing tests in code that automate functional testing by ways of replaying user actions on an application.

Product Comparison:

Category	MS Coded UI Tests	Selenium	Awesomium	PhantomJs	Watir	TestComplete	Unified Functional Testing (UFT)
Cross Browser Support	Not by default. Requires Selenium components to be installed.	Yes Requires WebDrivers.	No (Uses Chrome core)	No	Yes	Yes	No
Ease of programming	A bit clunky unless additional wrapper library installed: CodedUI CodeFirst	Fairly simple	Clunky and too low level. Missing documentation.	Cool if you like JS programming	Cool if you like Ruby. A bit hard to setup.	Very hard to use. Not a full programming language	Clunky, using VBScript
Testing Framework Support	TestTools.UITesting	NUnit fits well	Any .Net testing framework	Many	test-unit fits well	N\A	N\A
Cost	Requires VS Premium / Ultimate. We currently just have Professional.	Free (Apache 2.0)	Professional license ($$)	Free (BSD)	Free (BSD)	Expensive ($$)	Proprietary ($$)
Headless Support	No	Yes (HTMLUnit)	Offscreen view	Yes	Yes	No	No
Record and Playback	Yes (can record via IE only but can do cross browser playback)	Can be recorded with Selenium IDE (Firefox plugin)	No	Not built-in. Can be done in Resurrectio	Minimal	Yes but with issues when playback cross browser.	Yes

The order of the category is in decreasing importance. For my use case, cross browser support is the most important because the applications under test are websites requiring to be run on multiple browser types. In the mean time, record and playback is least important as this style of test generation create brittle tests that can be run on a browser not another, and changing the application UI results in broken tests that are hard to maintain.

In short, Selenium is the clear winner if you want the most mature Coded UI testing support, plus it is free. Microsoft's Coded UI test is a runner up for its close integration with Visual Studio. But price is a factor since this feature is only available in the premium or ultimate edition. Watir is third to me as it is somewhat hard to setup on Windows and the documentation is a bit sketchy. But if you are in the Ruby camp, you definitely should give this a try.

Getting Log4Net integration with Topshelf working

I was exploring a bit on logging in Topshelf using Log4Net. It's quite easy to use once I figured out how to pass in the log config file and having the correct content in the config file. Below is an example showing how to set this up with the bare minimum to get the logging to work.

Topshelf program:

public class Program
    {
        public static void Main(string[] args)
        {

            HostFactory.Run(hostConfigurator =>
                {
                    hostConfigurator.UseLog4Net("..\\..\\App.config");

                    hostConfigurator.Service<bird>(serviceConfigurator =>
                        {
                            serviceConfigurator.ConstructUsing(name => new Bird());
                            serviceConfigurator.WhenStarted(nm => nm.Start());
                        });
                });
        }
    }

    public class Bird
    {
        public void Start()
        {
            HostLogger.Get<bird>().Info("Chirp, chirp!");
        }
    }

App.config:

Unit testing with Moq

If you're using C# at work and are a little serious about unit testing, then the Moq framework can be your friend. It's simple to use and will do the job for both classical TDDer or a mockist TDDer.

I find this blog although written a while now, still is great as a starting point to understanding and using Moq: http://stephenwalther.com/archive/2008/06/12/tdd-introduction-to-moq

Referencing from the original blog, the following code completes the example so the mockist part is more correct and aligned with the real-world situation.

using System;
using System.Collections.Generic;
using System.Web;
using EasyAssertions;
using Moq;

namespace MoqTest
{
    class MoqTesting
    {
        static void Main()
        {
            Run();
        }

        private static void Run()
        {
            ClassicalTdd();
            MockistTdd();
        }

        private static void ClassicalTdd()
        {
            var newProduct = new Mock<IProduct>();
            newProduct.SetupGet(p => p.Id).Returns(1);
            newProduct.SetupGet(p => p.Name).Returns("Bushmills");

            newProduct.Object.Name.ShouldBe("Bushmills");

            var productRepository = new Mock<IProductRepository>();
            productRepository
                .Setup(p => p.Get(It.Is<int>(id => id > 0 && id < 6)))
                .Returns(newProduct.Object);

            var productReturned = productRepository.Object.Get(1);
            productReturned.Name.ShouldBe("Bushmills");
        }

        private static void MockistTdd()
        {
            var mockCache = new Mock<ProductCache>();
            var mockDatabase = new Mock<Database>();
            var mockProduct = new Mock<IProduct>();

            mockCache
                .Setup(c => c.Get(1))
                .Returns<IProduct>(null)
                .Verifiable();

            mockDatabase
                .Setup(d => d.Fetch(1))
                .Returns(mockProduct.Object)
                .Verifiable();

            mockCache
                .Setup(c => c.Set(1, mockProduct.Object))
                .Verifiable();

            ProductRepository sut = new ProductRepository(mockCache.Object, mockDatabase.Object);
            sut.GetProduct(1);

            mockCache.Verify();
        }
    }

    public interface IProductRepository
    {
        List<IProductRepository> Select();
        IProduct Get(int id);
    }

    public class ProductRepository : IProductRepository
    {
        private readonly ProductCache _cache;
        private readonly Database _db;

        public ProductRepository(ProductCache cache, Database db)
        {
            _cache = cache;
            _db = db;
        }

        public IProduct GetProduct(int id)
        {
            var product = _cache.Get(id);
            if (product == null)
            {
                product = Get(id);
                _cache.Set(id, product);
            }
            return product;
        }

        public List<IProductRepository> Select()
        {
            throw new NotImplementedException();
        }

        public IProduct Get(int id)
        {
            return _db.Fetch(id);
        }
    }

    public class ProductCache
    {
        public virtual IProduct Get(int id)
        {
            return (IProduct)HttpContext.Current.Cache["product_" + id];
        }

        public virtual void Set(int id, IProduct product)
        {
            HttpContext.Current.Cache["product_" + id] = product;
        }
    }

    public class Database
    {
        public virtual IProduct Fetch(int id)
        {
            return new Product { Id = id };
        }
    }

    public interface IProduct
    {
        int Id { get; set; }
        string Name { get; set; }
    }

    public class Product : IProduct
    {
        public int Id { get; set; }
        public string Name { get; set; }
    }
}

Functions of blackbird.js

In today's blog, I will go through each function of the blackbird.js as an exercise. Basically, I thought going through some JS code would be nice as a way to brush up and learn more about the beautiful and fluid language of JavaScript.

I find the naming of these functions are pretty evident in what it does. Looking at the code closer does helps one to grasp the branches and leaves of the code with greater confidence.

Below are the functions in more details:

generateMarkup: Generates the tool's content markup, including the header (filters and controls), main log body, and the footer (checkbox).

addMessage: Addes the log message to the output list.

clear: Clears the output list.

clickControl: Event handler for clicking to clear, resize, or hide log.

clickFilter: Event handler for filtering out logs based on type (debug, error, warning, etc)

clickVis: Event handler for setting to start tool on load.

scrollToBottom: Scrolls to the bottom of the log output list.

isVisible: This determines whether the display is visible or not by checking the css display style.

hide: This hides the display by setting the css style display attribute to none.

show: This gets the body element, then remove and add the tool content code into the body element. Also, it sets to display the tool.

reposition: Sets the position according the the position argument. If no argument is provided, use the default topRight or move to the next position as stored in the state. This calls setState after the classes have been set accordingly.

resize: Resizes the tool display to either small or large based on the argument, default or state value.

setState: Goes thru the state object and sets the property (props) array to be written to the cookie. Then it sets the class attribute for CSS styling.

getState: Retrieves the state object from cookie.

readKey: This is the event handler for keyup. Based on the F2 key combination with alt and shift, as well as the current visible state, the function clears the log, reposition or hide/show the tool display.

Examining blackbird.js

The code to blackbird.js can be found here. Today, I will examine the inner working of blackbird.js, line by line.

To start off with, let's understand the self-executing anonymous function.

(function () {
    ... some code
})();

The use of it, helps to provide function level scoping while making the code executing on its own. For more detailed explanation, refer to this nicely written blog.

Next, at the top level within the main function, we can see that there are some variables defined, including cache array, and objects like classes, profiler and messageTypes. At the same time, some inner functions are there: generateMarkup, clickControl, isVisible, getState, addEvent, and removeEvent. The window object is set with a namespace, i.e. the log object with methods such as toggle, clear, debug, warn and profile. These methods provide the API of the library. The initialization code is performed on window's load. This is achieved by calling addEvent with window, 'load' event and the callback function as the arguments. The addEvent & removeEvent code look a bit long but have good reasons. The main idea is to be cross-browser compatible.

The initialization code requires some describing.


addEvent(window, 'load',
/* initialize Blackbird when the page loads */
function () {
    var body = document.getElementsByTagName('BODY')[0];
    bbird = body.appendChild(generateMarkup());
    outputList = bbird.getElementsByTagName('OL')[0];

   backgroundImage();

   //add events
   addEvent(IDs.checkbox, 'click', clickVis);
   addEvent(IDs.filters, 'click', clickFilter);
   addEvent(IDs.controls, 'click', clickControl);
   addEvent(document, 'keyup', readKey);

   resize(state.size);
   reposition(state.pos);
   if (state.load) {
       show();
       document.getElementById(IDs.checkbox).checked = true;
   }

   scrollToBottom();

   window[NAMESPACE].init = function () {
       show();
       window[NAMESPACE].error(['', NAMESPACE, ' can only be initialized once']);
   }

   addEvent(window, 'unload', function () {
       removeEvent(IDs.checkbox, 'click', clickVis);
       removeEvent(IDs.filters, 'click', clickFilter);
       removeEvent(IDs.controls, 'click', clickControl);
       removeEvent(document, 'keyup', readKey);
   });
});

Line by line, first it grabs the body element, then append to it the generated markup. The outputList element is set. Next, events are registered for clicking the checkbox, filters, additional controls, and key ups. Base on the current state, the log display is resized and repositioned. If indicated to load, the display will be shown and the checkbox checked. Then the log content is scrolled to the bottom. The init method on window namespace object is defined. Finally, window on unload will remove the before-mentioned event registration.

In the next blog, we'll go and visit the individual functions and the relevant variable data.

JS logger: Blackbird

The Blackbird project is a neat little JS project that helps one to not write alert for debugging. Although you may argue much of the functionality has been superseded by developer tool in various browsers, this Blackbird logger tool is still nice in terms of its ease of use, cool UI and simplicity.

In the next post, we'll dissect its JS source code to understand how it works behind the scene as an exercise!

Code Kata: Coin Counters

Problem:

There are four types of common coins in US currency:
  quarters (25 cents)
  dimes (10 cents)
  nickels (5 cents)
  pennies (1 cent)

There are 6 ways to make change for 15 cents:
  A dime and a nickel;
  A dime and 5 pennies;
  3 nickels;
  2 nickels and 5 pennies;
  A nickel and 10 pennies;
  15 pennies.

How many ways are there to make change for a dollar
using these common coins? (1 dollar = 100 cents).

Solution:

The idea is to work out the composition of the coins to make up an amount from the higher to the lower value coins. The remainder amount can be determined recursively.

Source Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace _03_Count_Coins
{
    class CountCoins
    {
        private static readonly int[] CoinValues = { 25, 10, 5, 1 };
        private static int _waysToMakeUp;

        static void Main(string[] args)
        {
            int amountInCents = 100;
            Debug.WriteLine(string.Format("There are {0} ways to make up {1} cents", WaysToMakeUpValue(amountInCents), amountInCents));
        }

        private static int WaysToMakeUpValue(int cents)
        {
            MakeUpValueBy(cents, 0, "");
            return _waysToMakeUp;
        }

        private static void MakeUpValueBy(int cents, int coinValueIndex, string outputPrefix)
        {
            for (int i = coinValueIndex; i < CoinValues.Length; i++)
            {
                Coin coin = new Coin(CoinValues[i]);
                List<coinnumberandremainder> cnrList = coin.GetCoinNumberAndRemainders(cents);
                foreach (CoinNumberAndRemainder cnr in cnrList)
                {
                    if (cnr.remainder == 0)
                    {
                        coin.printLine(outputPrefix, cnr.coinNumber);
                        _waysToMakeUp++;
                    }
                    else
                        MakeUpValueBy(cnr.remainder, i + 1, coin.ToString(outputPrefix, cnr.coinNumber));
                }
            }
        }
    }

    class Coin
    {
        private int _value;

        public Coin(int value)
        {
            _value = value;
        }

        public List<coinnumberandremainder> GetCoinNumberAndRemainders(int amount)
        {
            int numberOfCoins = amount / _value;

            if (_value == 1)
                return new List<coinnumberandremainder>() { GenerateCoinNumberAndRemainder(amount, numberOfCoins) };

            List<coinnumberandremainder> list = new List<coinnumberandremainder>();

            for (int i = numberOfCoins; i > 0; i--)
                list.Add(GenerateCoinNumberAndRemainder(amount, i));

            return list;
        }

        private CoinNumberAndRemainder GenerateCoinNumberAndRemainder(int amount, int numberOfCoins)
        {
            int remainder = amount - (numberOfCoins * _value);
            CoinNumberAndRemainder cnr = new CoinNumberAndRemainder()
            {
                coinNumber = numberOfCoins,
                remainder = remainder
            };
            return cnr;
        }

        public void printLine(string prefix, int coinNumber)
        {
            Debug.WriteLine(ToString(prefix, coinNumber));
        }

        public string ToString(string prefix, int coinNumber)
        {
            if (!string.IsNullOrEmpty(prefix))
                prefix += ", ";

            return prefix + string.Format("{0} cents with {1} coins", _value, coinNumber);
        }
    }

    struct CoinNumberAndRemainder
    {
        public int coinNumber;
        public int remainder;
    }
}