Archive for the ‘Software Testing’ Category

Testing strategy for rails applications

Recently we had to design a test strategy for a ruby on rails medium to large scale application. We just wanted to have a lean but effective strategy in place. We went thro many standard templates available but finally came out with a new strategy that we think can help many reduce their turn around time for testing application. I will give a sneak preview for the strategy. Would really appreciate inputs on how your organization would go about this one. The following is a skelton of the test strategy document that we worked on: 1    Introduction The Test Strategy documents the overall strategy for testing the processes and applications to be delivered by the project. 2       Test Objectives [Describe the objectives of testing.  Testing should ensure that the future business processes, together with the enabling technology, provide the expected business benefits.  Testing objectives could include:
  • Verify products against their requirements (i.e. was the product built right?)
  • Validate that the product performs as expected (i.e. was the right product built?)
  • Ensure system components and business processes work end-to-end
  • Build a test model that can be used on an ongoing basis
  • Identify and resolve issues and risks.]
3       Identify Test Types [Describe the types of tests to be conducted to verify that requirements have been met and to validate that the system performs satisfactorily.  Consider the types of tests in the table below.]
Type of test Definition
Unit Testing Testing conducted to verify the implementation of the design for one software element (e.g., unit, module)
Integration Testing An orderly progression of testing in which software elements, hardware elements, or both are combined and tested until the entire system has been integrated
System Testing The process of testing an integrated hardware and software system to verify that the system meets its specified requirements
Acceptance Testing Formal testing conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system
Performance Testing Performed to confirm that the system meets performance goals such as turnaround times, maximum delays, peak performance, etc.
Volume Testing Tests the system to verify that the system can handle an expected volume profile
Stress Testing Tests the entire system to find the limits of performance
Configuration Testing Tests the product over all the possible configurations on which it is supposed to run
Operational Readiness Testing Tests the system to find defects that will prevent installation and deployment by the users
Data Conversion and Load Testing Performed to verify the correctness of automated or manual conversions and/or loads of data in preparation for implementing the new system
4       Scope of Testing Describe the scope of testing.  Consider the following when defining scope:
  • Test both business processes and the technical solution
  • Specify regions and sub-regions included in testing
  • Identify interfaces with other projects
  • Identify interfaces with external entities such as dealers, suppliers, and joint ventures
5    Test Preparation and Execution Process 5.1        Test Preparation Describe the steps for preparing for testing.  The purpose of Test Preparation is to verify that requirements are understood and prepare for Test Execution.  Steps for Test Preparation may include:
  • Identify test cases
  • Identify test cycles
  • Identify test data
  • Develop expected results
  • Develop test schedule (may be done as part of Test Plan)
  • Obtain signoff
5.2        Test Execution Describe the steps for executing tests. The purpose of Test Execution is to execute the test cycles and test cases created during the Test Preparation activity, compare actual results to expected results, and resolve any discrepancies.  Steps for Test Execution may include:
  • Verify entry criteria
  • Conduct tests
  • Compare actual results to expected results
  • Investigate and resolve discrepancies
  • Conduct regression test
  • Verify exit criteria
  • Obtain signoff
6    Test Data Management Describe the approach for identifying and managing test data.  Consider the following guidelines:
  • System and user acceptance tests – a subset of production data could be used to initialize the test environment.  Because the focus of these tests is to simulate the production environment and validate business transactions, data integrity is extremely critical.
  • Performance/volume/stress test – full size production files should be used to test the performance and volume aspects of the test.  Additional ‘dummy’ data will be created to stress the system.  Data integrity is not critical, as the test focuses on performance rather than the ability to conduct business transactions.
  • Operational readiness test – a copy of system/user acceptance test data could be used for the operational readiness test.  Since the focus of the test is on operational procedures, a low number of transactions will be required and data integrity is not critical.
7    Unit testing ruby on rails application We will use the baked in testing framework for testing our application since that’s the best tool available. Every Ruby on Rails application can be unit tested on the basic MVC premises: Testing Models: When you generate a model with the generate script, Rails also generates a unit test script for the model in the test directory. It also creates a fixture, a YAML file containing test data to be loaded into the test database. This is the data against which your unit tests will run. As you write code in the model classes, you’ll write corresponding tests in these files. Then you can easily specify records to be created using YAML in fixtures. With the sample data set we can then specify the unit test in the unit tests file. A successful unit test will give 0 errors and 0 failures. Testing controllers: Controller testing is also known as functional testing. Functional testing tests the following type of functionalities of the controllers:
  • Is the response redirected as expected?
  • Is the expected template rendered?
  • Is the routing as expected
  • Does the response contain the expected tags?
Rails framework supports 5 types of requests and to write a functional test, you need to simulate any of the five HTTP request types that your controller will process. :
  • get
  • post
  • put
  • head
  • delete
Request type “get” and “post” are the most commonly used in controller testing. All these methods take four arguments:
  • The action of a controller
  • An optional hash of request parameters
  • An optional session hash
  • An optional flash hash
When you generate a controller with generate, Rails creates a functional test script for the controller. As you write code in the controller classes, you’ll write corresponding tests in these files. Use the fixture as defined in testing models. Add the functions to test the existence of the views and run the unit tests to see the results. Using Rake for testing: You can use rake utility to test your applications. Here are few important commands.
  • $rake test – Test all unit tests and functional tests (and integration tests, if they exist).
  • $rake test:functionals – Run all functional tests.
  • $rake test:units – Run all unit tests.
  • $rake test:integration – Run all integration tests.
  • $rake test:plugins – Run all test in ./vendor/plugins/**/test.
  • $rake test:recent – Run tests for models and controllers that have been modified in the last 10 minutes:
  • $rake test:uncommitted – For projects in Subversion, run tests for models and controllers changes since last commit.
Would keep on updating this document as our testing strategy evolves.