• 1 Objective
• 1.1 Goals
• 1.2 Non-Goals 
• 2 Test Schedule
• 3 Test Environments
• 3.1 Mifos Test Server
• 3.2 Test Data
• 3.3 Local Test Environment
  
• 4 Test Approach and Execution
  
• 4.1 New feature testing
• 4.2 Regression testing
• 4.3 Performance and Stress testing
• 4.4 Release Candidate Preview testing
  
• 5 Known/Active bugs
• 6 Resolved bugs requiring verification
• 7. Test Status Monitoring
  
• 8. Release Criteria
  

Objective

For this release, we will perform system-level testing of the Mifos Rhino project enhancements to ensure the new features have been implemented as specified.  The Rhino release will also include bug fixes for issues identified against the v1.1 code base, so the functionality of the 1.1 release must be validated, with an emphasis on modifed areas of the code.  Testing efforts will be targeted based on project priorities and risks.

Testing Goals

For the Rhino project, the primary testing goals are as follows:

• Functional testing of Rhino enhancements using documented test plans and test cases.
• Functional verification of bug fixes.
• Acceptance and regression testing of 1.1 features to confirm no regressions have been introduced during the Rhino project.
• Provide a UAT candiate for new deployments.
• Maintain and expand automated acceptance test suite.
• Develop basic performance testing goals.
• As time and resources permit, add multi-user performance testing scenarios.

Testing Non-Goals

The following testing goals are considered to be beyond the scope of the Rhino testing effort.

• Uptime or reliability testing.  Standard architecture and platform being used is considered to be a proven, stable environment.
• Unit/integration testing is not part of this system testing effort.
• Verlfication of localized UI strings.
• Compatibility testing with other Web servers, databases, browsers will not be a focus of this release’s testing.
• Documentation will be reviewed by project team.
• Will avoid heavy investment in tools or techniques that won't apply to future Mifos releases which will have a different architecture.

Test Schedule

The testing effort for Rhino will be currently planned and executed with the following assumptions:

• Scheduling and priorization of testing efforts will be based on project team's priorization of Rhino changes as targeted during development iterations.  
• Risks and project team feedback will be used as a basis for feature and test case priorities.  Test results from initial test cycles may influence the priority of test cases in later test cycles.

Specific test milestones and dates will be maintained with the Rhino project schedule.

Test Environments

Standard software test environment for Rhino testing

Operating system: Windows 2003 server

Java: JRE 1.5.0_12

Application server: Tomcat 6.0.16

Database: MySQL 5.051b

Report Designer: BIRT 2.1.2

Browser: IE 6

Mifos Test Server - test.mifos.org/mifos

Currently, we have one test environment generally available for testing the latest code of Mifos.  This server is updated in an automated fashion with the latest build available on the Mifos Build Server.  This server also contains a set of test data which you can add to and modify for testing purposes.  Please note that access to this server is only provided through the application web URL.  We do not provide direct access to the server machine itself or the DB. 

Please view the Mifos Test Server page for more details about accessing and using the test server.  

Test Data

For basic functional testing, a standard set of test data will be used to ensure consistency and repeatability.  Test data sets are available from mifos.org's resource repository.

Local Test Environment

Testing of features that require non-default configuration or unique data can be done using a local system.  Deployment should include the standard software runtime environment configuration for Rhino as noted above.  Mifos war files are generated by the build server and are available via ftp server. 

Mifos can be installed on a local test environment based on the build and install instructions for developers.

Test Sandboxes

Test sandboxes will also be available for Rhino testing using internal systems.  These sandboxes allow flexibility to deploy new war files and db refreshes when most convenient for the tester.

 

Test Approach and Execution

New feature testing

New feature testing will be scheduled to following the development iteration process for Rhino.  Test plans and test cases will be documented either in spreadsheets and stored in SVN.   

Acceptance and Regression testing

A set of Selenium and manual test cases from the 1.1 release will be selected to produce a representative set of acceptance and regression tests.  These regression tests will be prioritized and executed as part of the overall Rhino test schedule.  Pass/fail results will be reported on each test milestone. 

Performance and Stress testing

Limited performance and stress testing for Rhino will be focused on typical customer load scenarios.  Completion of peformance/stress testing for Rhino release depends on testing resources available during project.  The performance testing details will be documented in a subsequent document as these plans are developed. 

UAT Candidate testing

The team will schedule a User Acceptance Testing candidate (UAT) build that will be available to Mifos existing and future deployments for evaluation and feedback. The UAT candidate will be a build that has passed acceptance testing for Rhino features.

Resolved bugs requiring verification

This list in our issue tracker is a list of all bugs that have been marked as 'resolved' in our issue tracker and are awaiting review and verification by a tester.  All bugs targeted for Rhino should be verified by project completion.  Please contact the Mifos Developer Mailing List if you would like to assist with verifying these bugs.

Test Status Execution and Monitoring

Test for new features and bug fixes will be executed in order of business priority and availability.  Within a feature area, the following priorities will be designated for test cases:

• 1 (High) - test cases that are run with any new build prior to release, regardless whether any changes were introduced to the feature.
  
• 2 (Medium) - test cases which test major functionality of the features.  These cases are executed with each major project milestone.
  
• 3 (Low) - test cases which will test additional aspects of a feature, but can be skipped later in a project release when a feature is considered stable.
  

We will report test results by monitoring the results of test case execution and issue tracking results. 

Test case pass/fail totals will be tracked for each test milestone, along with reference to issues reported on any test cases.  Test case results should also refer to the build used for test execution.

Issues reported during testing will be monitored to identify the trend of issues being detected and resolved. 

Release Criteria

The following release criteria will be met prior to completion of the project:

• All the features targeted for this release are complete and tested to the satisfaction of the entire team. 
• There are no remaining High (P1 or P2) priority open issues targeted for the Rhino release.  Some high priority issues identified during Rhino testing may deferred based on acceptable workarounds. 
• Planned accepance and regression tests have been executed, with all tests passing or having an acceptable workaround.
• Defined performance tests identify no blocking issues.
• Final release candidate build has been tested and has passed all acceptance and installation tests.