Code structure for automated checks


I have been reading about how to structure my automation scripts for checking
scenarios, and this is what I have learnt.
 
The Test Pyramid
(Source: Test Pyramid)
First of all, we should know the test pyramid concept that was developed by Mike Cohn, and described in his book Succeeding with Agile.
In a nutshell, the concept is about having many more low-level unit tests than high-level end-to-end UI tests. UI tests are brittle, expensive to write, and time consuming to run. So, the automation should lean much more towards unit tests.
 test-pyramid
UI layer refers to the end-to-end checks using something like Selenium or Sahi. The Service layer refers to checks that act through a service layer of an application. (Eg. checking through an API layer)
And the final base of checks is the low-level unit tests.
 
Guidelines for structuring automated tests
#1 Structure – Prioritize creation of unit tests to cover most test scenarios over
any other types of tests. Unit tests should cover the bulk of your testing strategy.
Keep UI tests to a minimum, only covering critical happy flow scenarios & failures. Avoid making assertion on validation error message in UI level. It should just verify the error case by virtue of redirect to same page. All validation scenarios are already covered with messages in unit tests
#2 Speed – Team should collectively decide the acceptable time for running the test suite, define it for all test types – unit, integration and acceptance.
#3 Robustness – Avoid testing JavaScript behavior and scenarios heavily using
Selenium based acceptance tests. Instead, use Jasmine-based pure JavaScript unit testing, which are more robust and fast.
Create mocks for any external services that tests might be dependent on.
#4 Debugging – When it fails, it should point broken code.
 
Self Testing Code
Every object should be able to test itself. Write your code in such a way that typing a command should make the whole software system do a self-test.
Also, practice Test Driven Development while working.
This command will be run every time code is committed and pushed to Git develop branch. By running the test suite frequently, at least several times a day, you’re able to detect such bugs soon after they are introduced, so you can just look in the recent changes, which makes it much easier to find them.
Build a testing culture where developers are naturally thinking about writing code and tests together.
 
When Should a Test Be Automated?
(Source: Automate)
Do not over-automate. Only some tests should be automated.
Decision process would use following questions:
1. If I automate this test, what manual tests will I lose (not be able to execute in the time given)? How many bugs might I lose with them? What will be their severity?
2. An automated test has a finite lifetime, during which it must recoup that additional cost. Is this test likely to die sooner or later? What events are likely to end it?
When deciding whether to automate a test, you must estimate how many code changes it will survive. When the test breaks after code change, if it hasn’t repaid the automation effort by that point, you would have been better off leaving it as a
manual test
Create product-specific test libraries. After that many user interface changes will
require no changes to tests, only to the test library, thus, saving a lot of effort.
3. During its lifetime, how likely is this test to find additional bugs (beyond whatever bugs it found the first time it ran)? How does this uncertain benefit balance against the cost of automation?
If those questions don’t suffice for a decision, other minor considerations might tip the balance.
 
Note – The tests you need are often called task-driven tests, use-case tests, or scenario tests. Because scenario tests favor common tasks, they find the bugs most users will find.
Some of the secondary considerations
* Humans can notice bugs that automation ignores, especially GUI issues.
* But, while humans are good at noticing oddities, they’re bad at painstaking or precise checking of results.
* As humans can’t be precise about inputs, repeated runs of a manual test are often slightly different tests, which might lead to discovery of a support code bug.
For example, people make mistakes, back out, and retry inputs, thus sometimes stumbling across interactions between error-handling code and the code under test.
* Because test automation takes time, you often won’t report the first bugs back to the programmer as soon as you could in manual testing.
* Automated tests, if written well, can be run in sequence, and the ordering can vary from day to day
* An automated test might not pay for itself until next release. A manual test will find any bugs it finds this release. Bugs found now might be worth more than bugs found next release.
 
Page objects
Within your web app’s UI there are areas that your tests interact with. A Page Object simply models these as objects within the test code. This reduces the amount of duplicated code and means that if the UI changes, the fix need only be applied in one place.
Some other articles I read that could be useful –

Connecting to SQL in your Python script


pypyodbc python package

I had been wondering for such a long time on how to connect my python scripts with our MS SQL database. So, when I had a little break from my workload today, I decided to finally solve this problem.

I have tried searching for solutions before too, but it seems today was my lucky day. I found the pypyodbc module.  It’s written in python, and helps execute SQL in your database.

Here’s an example of the module’s usage, and a getting started guide.

Some other article’s  withe useful information are – article on the various methods available in moduleUsing stored procedures

I needed to execute an SP in the database through my automated test script to check the data displayed on the front-end. So, I included the  following function in my code.

It’s great that I finally know how to integrate python and database because I can do so much more.  🙂

import pypyodbc

connection_string = "driver={SQL Server Native Client 10.0};
 server=127.0.0.1; database=databaseName; 
  uid=user; pwd=pwd"

cnxn = pypyodbc.connect(connection_string)
cursor = cnxn.cursor()

# SET NOCOUNT ON should be added to prevent errors thrown by cursor

sql = """
SET NOCOUNT ON
DECLARE @var1 INT = 0
DECLARE @var2Out NVARCHAR(MAX)
EXEC [dbo].[storedProc] 1,5,'',0,0,2,@var1 OUTPUT,@var2Out OUTPUT
SELECT @var1, @var2Out 
"""
result = cursor.execute(sql)

print result.fetchall()

result.nextset()
print result.fetchall()
result.nextset()
print result.fetchall()

Check broken links and handle multiple windows through Selenium webdriver in Python


webdriver - handle multiple windows

Today I am going to post the code of a function I wrote to check broken links in a web page.

You can learn two things by going through it –

First, how to check broken links (I know that’s obvious :D).
Second, how to handle multiple windows in selenium webdriver + python. There are three useful functions for this –

1. webdriver.window_handles 
 => returns the handles of all windows within the 
    current webdriver session
2. webdriver.current_window_handle
 => returns the handle of the current window 
     (the window currently in focus)
3. webdriver.switch_to_window(window_name) 
 => switches focus to the window having specified 
    window_name or window_handle 
    (we can pass the window_handle instead of window_name
        as a parameter to this function)

To check broken links, my code navigates to the link given, and checks whether user lands up in the same page as expected.
While checking links on a page, I also find some links that, when clicked, open a page in a separate window. Most common example of such links are the links to social media sites on a page.

Eg. In the below code, I am testing the home page of “http://www.carwale.com/“.
This page has 4 links – the icons at the bottom of the page for facebook, youtube, google plus, and twitter – that when clicked, open the respective pages in a new window. Here, the section that handles multiple windows will come into play.

from selenium import webdriver
browser = webdriver.Firefox()
home_page = "http://www.carwale.com/"

def check_page_broken_links(self, url):
# Sample usage:
#     check_page_broken_links(self,"http://www.carwale.com/") 
#         will return empty list if all links in the page work fine
#         else it will return list of all the broken links 
#                                    (either link text, or link href)
#   Will check for -  i) "Page Not found" error
#                     ii) Redirects

    try:
        failed = []
        self.implicitly_wait(5)
        self.get(url)
        number_of_links = len(self.find_elements_by_tag_name('a'))

        for i in range(number_of_links):
            # Save current browser window handle
            initial_window = self.current_window_handle 
            ## print "initial_window_handle:      ", initial_window

            link = self.find_elements_by_tag_name('a')[i]
            link_address = link.get_attribute("href")
            link_name = link.text
            print "link checked: ",i,": ",link_name,": ",link_address

            if ((link_address is not None) 
                and ("google" not in link_address) 
                and ("mailto" not in link_address) 
                and is_link_element_displayed(self,element=link) is True):
                  link.click() # link clicked
                  open_windows = self.window_handles
                  ## print "window_handles:      ", open_windows

                  # Navigate to the browser window where 
                  #               latest page was opened
                  self.switch_to_window(open_windows[-1])
                  ## print "current_window_handle:"
                  ## print self.current_window_handle
                  time.sleep(5)
                  print "defined: ",link_address
                  print "current: ", self.current_url

                  if (link_address[-1] == "#" 
                       and self.current_url in 
                        [link_address, link_address[:-1],
                                      link_address[:-2],link_address+'/']):
                          # A "#" at the end means user 
                          # will stay on the same page.(Valid scenario) 
                          pass  
                  elif (self.current_url not in 
                        [link_address,
                         home_page + link_address[1:]]): 
                        # if user lands up in a page different 
                        #                    from that intended 
                        if link_name:
                          failed.append(link_name)
                        else:
                          failed.append(link_address)

                  if len(self.window_handles) > 1:  
                          # close newly opened window
                          self.close()

                  # Switch to main browser window
                  self.switch_to_window(open_windows[0]) 

            self.get(url)
    except Exception, e: 
           return ['Exception occurred while checking',e]
    return failed

# call defined function to check broken links in carwale.com home page
print check_page_broken_links(browser,"http://www.carwale.com/")

If there are any broken links in the URL you passed to the function, they will be printed out in a list at the end of program execution.

Reading/Writing Excel sheets in Python


excel_sheet

 Reading and Writing data from/to excel sheets is very easy with the python modules xlrd,  xlwt.
There is also an xlutils module available to help in copying data, filtering data, etc. But, I don’t use that module simply because such a need has not arisen yet. So, I’m going to focus on the xlrd and xlwt modules only.

Installing the modules
xlrd – https://pypi.python.org/pypi/xlrd
xlwt – https://pypi.python.org/pypi/xlwt

Download the modules, extract contents and run setup in a way similar to that described in the installation steps for selenium

Now, that you are ready to use the modules, let me list out the methods I find most useful while using these modules.

Writing data to Excel using xlwt

First we import the module into our script

import xlwt

Then, we create a workbook object to write data into. You can pass an optional parameter to specify encoding here, but it is not necessary (I normally don’t specify it).

wb = xlwt.Workbook(encoding="utf-8")

Next, we will add some sheets to the workbook. Again, there’s an optional parameter cell_overwrite_ok. I generally use it because I need to overwrite already existing excel sheets with new test results.
First parameter is the name you want to give the sheet being created.

ws1 = wb.add_sheet('Sheet 1',cell_overwrite_ok=True)
ws2 = wb.add_sheet('Sheet 2',cell_overwrite_ok=True)
ws3 = wb.add_sheet('Sheet 3',cell_overwrite_ok=True)

We can add the data to the workbook in various ways, as detailed below:

ws1.row(0).write(0, 'Data written in first cell of first sheet')

ws1.write(0, 0, 'Data overwritten in the first cell of 
first sheet')

ws2.write(0, 0, 'Data written in first cell of second
 sheet')

ws3.write(0, 0, 'Data written in first cell of third sheet')

ws1.write(0, 1, 'Data written in first row,second column of
 first sheet')

ws1.row(1).write(1, 'Data written in second row,second column 
of first sheet')

var = "Data from variable written in third row,second column of
 first sheet" 

ws1.row(2).write(1,var)

Like you might have already deduced by now, we first specify the row no. (0, for 1st row, and onwards), then the column no. (0, for 1st column, and onwards), and then you enter whatever data we want written (as a string, or maybe as a variable – as shown in the last statement)

The final step is to save this workbook object as an actual excel file. We pass the path to the file as a parameter. If a file already exists at the given path, the sheet will be overwritten.

wb.save('D:\\test_logs\\Spreadsheet_test.xls')

Reading data from Excel using xlrd

Now that we have saved the data in an Excel file successfully, it’s time to read the saved data from it.
We start by importing the xlrd module.

import xlrd

Then, we open the workbook to be read by passing the file path parameter to the open_workbook() method. In this case, we’ll be opening the file we just saved.

wb2 = xlrd.open_workbook('D:\\test_logs\\Spreadsheet_test.xls')


To access a sheet of the workbook object, we can use the sheet_by_name() or the sheet_by_index() method.These methods return a sheet object that can be used to access the cell contents.
There’s another method sheet_names() that returns the names of all the sheets in the file.

sheet1 = wb2.sheet_by_name('Sheet 1')
sheet2 = wb2.sheet_by_index(1)
sheet3 = wb2.sheet_by_name(wb2.sheet_names()[2])

After the sheet objects have been created, we can read the values using the following methods

print "Number of rows in sheet 1:" + sheet1.nrows
print "List of cell values in 1st row:" + sheet1.row_values(0)
print "Number of columns in sheet 1:" + sheet1.ncols
print "List of cell values in 2nd column:" + sheet1.col_values(1)
print "Data in 1st cell of sheet 1:" + sheet1.cell(0,0).value
print "Data in 3rd row,2nd col of sheet1:"+sheet1.cell(2,1).value
print "Data in 1st cell of sheet 2:" + sheet2.cell(0,0).value

The row_values() and col_values() methods return a list of data upto the final cell that contains data. Any empty cell in between will be read as an empty element. The nrows and ncols values will also be calculated accordingly.

Another thing to note is that while reading integer values from an excel, xlrd methods will return them as floats. So, in case you have a value “100” in the first cell of the first sheet, you will need to convert it back to integer while reading it.

print int(sheet1.cell(0,0).value)
Aside

Test Automation – Selenium webdriver with python


This is a long pending post. I had done the installation months back, and then forgotten the steps. I needed to re-install to be able to write down the below steps, and was not getting the time. Now, finally, I have done an installation again, and this time I am noting down the steps.

The below method is very simple. Contrary to what most of the Google search results have you believe, it is not necessary to install easy_install or pip to install the selenium python bindings.

So, first I will just introduce you to Selenium WebDriver (part of Selenium 2). It is a library for automating your test cases across browsers, and it can be used from a variety of language bindings. I prefer Python because I love the language and its no-nonsense format. So, I will be installing the Python language bindings along with the Selenium Framework.

The Selenium webdriver allows you to programmatically drive a browser and interact with web elements. It uses the browser’s native methods instead of javascript injection, and should be preferred over Selenium RC

Now, let us go through the installation steps:

1. Install Python in your machine

– Go to http://python.org/download/
– Download the latest MSI installer for Python 2.7 (32-bit)
– Run the installer

2. Set your PATH
– Go to your system’s environment variables, and add ‘C:\Python27’ to your PATH

(this will enable you to invoke ‘python.exe’ from any command line)

3. Go to https://pypi.python.org/pypi/selenium and download the .tar.gz file (the Selenium Python Client Driver)

4. Extract files from the .tar.gz file

5. Go through the extracted files. You will find a selenium-2.xx.x.tar file in the dist folder. (x denotes version number in the file name)

Extract files from this .tar file. Verify that the extracted folder contains a setup.py file

6.  Open the cmd Command prompt, and navigate to the location where setup.py is located (the folder from step 6):

 >> cd <location of the setup.py file for selenium bindings>

7. Enter the following command into the command prompt:

 >> python setup.py install

A “selenium” directory gets added in the C:\Python27\Lib folder.

Selenium is now installed in your machine with Python bindings.

The following code should work now:

#!/usr/bin/env python

from selenium import webdriver

browser = webdriver.Firefox()
browser.get("https://amybughunter.wordpress.com/")

This should open a Firefox browser session and navigate to https://amybughunter.wordpress.com/

A very useful doc to understand the Selenium Webdriver API for python is http://selenium-python.readthedocs.org/en/latest/api.html. It has helped me a lot while creating test scripts.

Following is a simple functional test in Python, using Selenium WebDriver and the unittest framework:

#!/usr/bin/env python

import unittest
from selenium import webdriver

class TestUdacityHomepage(unittest.TestCase):

    def setUp(self):
        self.browser = webdriver.Firefox()

    def testTitle(self):
        browser = self.browser
        browser.get('https://www.udacity.com/')
        title = browser.title
        self.assertIn('Udacity', title)

    def tearDown(self):
        self.browser.quit()

if __name__ == '__main__':
     unittest.main()

Output should be:

>>> 
.
----------------------------------------------------------------------
Ran 1 test in 15.478s

OK
Exit code:  False