Qa test plan

[resplin]

Obsolete Pages{{Obsolete}}

The official documentation is at: http://docs.alfresco.com

QA 3.0
Back to Test Plan

---

Table of Contents

• 1 Alfresco Test Plan
  
  
  
  • 1.1 Revision History
  
  
  • 1.2 Introduction
    
    
    
    • 1.2.1 Test Plan Objectives
    
    
  
  
  • 1.3 Scope
    
    
    
    • 1.3.1 Data Entry
    
    
    • 1.3.2 File Transfer
    
    
    • 1.3.3 Security
    
    
  
  
  • 1.4 Test Strategy
    
    
    
    • 1.4.1 System Test
    
    
    • 1.4.2 API Tests
    
    
    • 1.4.3 Security Test
    
    
    • 1.4.4 Automated Test
    
    
    • 1.4.5 Stress and Performance Test
    
    
    • 1.4.6 Multiuser / Concurrency Test
    
    
    • 1.4.7 Longevity Test
    
    
    • 1.4.8 Recovery Test
    
    
    • 1.4.9 Upgrade Test
    
    
    • 1.4.10 Cluster Test
    
    
    • 1.4.11 Accessibility Test
    
    
    • 1.4.12 Documentation Test
    
    
    • 1.4.13 Beta Test
    
    
    • 1.4.14 User Acceptance Test
    
    
  
  
  • 1.5 Test Procedure
  
  
  • 1.6 Environment Requirements
  
  
  • 1.7 Control Procedures
    
    
    
    • 1.7.1 Reviews
    
    
    • 1.7.2 Bug Review meetings
    
    
    • 1.7.3 Change Request
    
    
    • 1.7.4 Defect Reporting
    
    
  
  
  • 1.8 Functions To Be Tested
  
  
  • 1.9 Resources and Responsibilities
    
    
    
    • 1.9.1 Resources
    
    
    • 1.9.2 Responsibilities
    
    
  
  
  • 1.10 Deliverables
  
  
  • 1.11 Suspension / Exit Criteria
  
  
  • 1.12 Resumption Criteria
  
  
  • 1.13 Dependencies
    
    
    
    • 1.13.1 Personnel Dependencies
    
    
    • 1.13.2 Software Dependencies
    
    
    • 1.13.3 Hardware Dependencies
    
    
    • 1.13.4 Test Data & Database
    
    
  
  
  • 1.14 Risks
    
    
    
    • 1.14.1 Schedule
    
    
    • 1.14.2 Technical
    
    
    • 1.14.3 Management
    
    
    • 1.14.4 Personnel
    
    
    • 1.14.5 Requirements
    
    
  
  
  • 1.15 Tools
  
  
  • 1.16 Documentation
  
  

Alfresco Test Plan

Revision History

Revision A - Initial Draft

Revision B - Alfresco 3.0 plan

Introduction

Alfresco is an open source, open-standards content repository built by the most experienced content management team that includes the co-founder of Documentum. The Alfresco product has a lean, modular component architecture that allows new functionality to be added without any system disruption and is significantly faster than proprietary commercial systems.

The Alfresco 3.0 product will include the existing JSF client along with a new client.
It is the new client and it's capabilities that the main bulk of the 3.0 testing will concentrate upon.

Information can be found about Alfresco 3.0 in the following locations:

Alfresco 3.0 Wiki Pages - Index of all relevant Alfresco 3.0 wiki pages.

Roadmap - You can also read more about our 2008 Roadmap and 3.0 plan

Test Plan Objectives

This Test Plan for the Alfresco CMS supports the following objectives:

• Define the activities required to prepare for and conduct System, Beta and User Acceptance testing.
  
• Communicate to all responsible parties the System Test strategy.
  
• Define deliverables and responsible parties.
  
• Communicate to all responsible parties the various Dependencies and Risks
  

Scope

Data Entry

Alfresco 3.0 provides a new web client targeted at performing specific tasks. The web client is engineered to run on Microsoft IE 6/7, Mozilla Firefox and Safari 3.
The product also provides several API's to access the functionality programmatically.

File Transfer

The web client provides an interface for adding, viewing and retrieving files from the system.
There is also a CIFS interface to the repository that acts like a standard shared directory on the network and can have all standard actions associated with a shared network directory applied to it, e.g. offline synchronisation.
The product also includes an FTP server for direct ftp access, and WebDAV.

Security

Each user will need a userid and password to login to the system. The management of these details can be configured to use Ldap, NTLM etc…

Test Strategy

The test strategy consists of a series of different tests that will fully exercise the Alfresco 3.0 product. The primary purpose of these tests is to uncover the systems limitations and measure its full capabilities. A list of the various planned tests and a brief explanation follows below.

System Test

The System tests will focus on the behavior of the system. User scenarios will be executed against the system as well as screen mapping and error message testing. Overall, the system tests will test the integrated system and verify that it meets the requirements defined in the Wiki (see above for links to relevant requirements).

API Tests

The API tests will focus on testing the REST API for interacting with the repository.

Security Test

Security tests will determine how secure the system is. The tests will verify that unauthorized user access to confidential data is prevented. This includes XSS and injection techniques.

Automated Test

A suite of automated tests will be developed to initially test the basic functionality of the system and allow regression testing to be performed on areas of the systems that previously had blocker/critical defects.

Stress and Performance Test

We will subject the system to high input conditions and a high data volumes. The System will ideally be stress tested using incrementally increasing volumes of both input and data to produce scaling graphs.

Multiuser / Concurrency Test

We will test the system with multiple users trying to interact with the same item concurrently to ensure there are no cases of data corruption, data loss or system failure. These tests will be conducted using an automated tool such as Jmeter or Selenium.

Longevity Test

We will subject the system to steady input conditions and steady data volumes over a long period to check for any potential issues with system uptime with a more realistic usage scenario. The System will ideally be tested using steady volumes of both input and data to produce system health graphs ofer the test period.

Recovery Test

Recovery tests will force the system to fail in a various ways and verify the recovery is properly performed. It is vitally important that all data is recovered after a system failure & no corruption of the data occurred. This is especially important in clustered deployments.

Upgrade Test

Upgrade testing will ensure that customers can upgrade from previous versions of Alfresco to Alfresco 3.0 and be able to use the JSF client without issues.
Any potential upgrade route that includes use of the 'new' client is still tbd.

Cluster Test

The default deployment of Alfresco 3.0 for testing will be clustered.

• Cluster layout / config tbd
  

Accessibility Test

The system will be validated against accessibility criteria, with a minimum level of matching section 508 guidelines. Further criteria might be validated against, such as W3C's Web Accessibility Initiative's WAI-ARIA and WCAG 2.0.

Documentation Test

Tests will be conducted to check the accuracy of the user documentation. These tests will ensure that no features are missing, and the contents can be easily understood.

Beta Test

We will beta test the new system by providing ‘Preview Releases’ to the community and reporting any defects that are found. This will subject the system to tests that could not be performed in our test environment.

User Acceptance Test

Once the system is ready for implementation, we will perform User Acceptance Testing. The purpose of these tests is to confirm that the system is developed according to the specified requirements and is ready for operational use.

Test Procedure

The test procedure is:

(For JSF Client)

• Test alfresco on each Stack
  
• Default configuration for DM is a 2 machine cluster
  
• Default configuration for WCM is a standalone server with remote FSR machines
  
• External authentication mechanism used by default.
  
• Issues found on DM cluster are revalidated on a standalone DM install to determine if they are cluster specific or not
  
• Issues found on 'exotic' stacks (i.e. one not used by the development team) are revalidated on the development stack of Win/Tomcat/MySQL to determine if they are generic issues across all stacks or specific to the 'exotic' stack.
  
• When issues are found they are logged in Jira
  
• When issues are fixed buy an engineer they are assigned to a QA member for retest.
  

(For New 3.0 Client)

• test procedure TBD
  

Environment Requirements

See Support for list of stacks

Control Procedures

Reviews

The project team will perform reviews for each Phase as required. A meeting notice, with related documents, will be emailed to each participant.

Bug Review meetings

Regular meetings will be held when appropriate to discuss reported defects. The development department will provide status/updates on all defects reported and the test department will provide addition defect information if needed. The support department will flag any issues raised as Support issues. A member of all teams will participate.

Change Request

Once testing begins, changes to the system are discouraged. If functional changes are required, these proposed changes will be discussed with the Change Control Board (CCB). The CCB will determine the impact of the change and if/when it should be implemented. The CCB consists of the QA Manager, the Development Manager, the Support Manager and Product Manager(s), with the CEO as final arbiter.

Defect Reporting

When defects are found, the testers will complete a defect report in Jira. The defect tracking system is accessible all members of Alfresco as well as any external party who has registered with the system. When a defect has been fixed or more information is needed, the developer will change the status of the defect to indicate the current state. Once a defect is verified as FIXED by the testers, the testers will close the defect report.

Functions To Be Tested

The list of functions that will be tested can be found in
Alfresco 3.0 Wiki Pages

Resources and Responsibilities

The QA Manager and Project Manager / Development Manager will determine when system test will start and end. The QA Manager will be responsible for coordinating schedules, equipment, & tools for the testers as well as writing/updating the Test Plan, Weekly Test Status reports during formal system testing phases and Final Test Summary report. The testers will be responsible for writing the test cases, creating the automated tests and executing the tests. With the help of the QA Manager, the other people involved in the project will be able to input into various aspects of testing such as beta and user acceptance testing.

Resources

The test team consists of:

• A Development Manager
  
• QA Manager
  
• Testers
  
• Usability Expert
  
• Engineers when needed.
  

Responsibilities

Development Manager - Responsible for Project schedules and the overall success of the project. Participate on CCB.

QA Manager - Ensures the overall success of the test cycles. Will coordinate any BRB meetings and will communicate the testing status to the project team. Participate on CCB.

Testers - Responsible for performing the actual system testing.

Usability Expert - He will help coordinate the Beta and User Acceptance testing efforts. Optional participation on CCB.

Engineers - Will write comprehensive unit tests, and will assist in performing testing when necessary.

Deliverables

Deliverable	Responsibility	Completion Date
Develop Test cases	Testers
Test Case Review	QA Manager, Dev. Lead, Testers
Develop Automated test suites	Testers
Requirements Validation Matrix	QA Manager
Execute manual and automated tests for product release	Testers
Complete Defect Reports	Everyone testing the product	On-going
Document and communicate test status/coverage	QA Manager	Weekly
Execute Beta tests	Community	Beta Release
Document and communicate Beta test status/coverage	Development Manager
Execute User Acceptance tests	Usability Expert	On-going
Document and communicate Acceptance test status/coverage	Usability Expert
Final Test Summary Report	QA Manager	End Of Project

Suspension / Exit Criteria

If any defects are found which seriously impact the test progress, the QA manager may choose to
Suspend testing. Criteria that will justify test suspension are:

• Hardware/software is not available at the times indicated in the project schedule.
  
• Product under test contains one or more critical defects, which seriously prevents or limits testing progress.
  
• Assigned test resources are not available when needed by the test team.
  

Resumption Criteria

If testing is suspended, resumption will only occur when the problem(s) that caused the suspension has been resolved. When a critical defect is the cause of the suspension, the “FIX' must be verified by the test department before testing is resumed.

Dependencies

Personnel Dependencies

The test team requires experience testers to develop, perform and validate tests. These testers must also be versed in automated testing.

Software Dependencies

The source code must be unit tested and provided within the scheduled time outlined in the Project Schedule.

Hardware Dependencies

The test servers and test client machines, as well as the LAN environment need to be available during normal working hours. Any downtime will affect the test schedule.

Test Data & Database

Test data should also be made available to the testers for use during testing.

Risks

Schedule

The schedule for each phase is very aggressive and could affect testing. A slip in the schedule in one of the other phases could result in a subsequent slip in the test phase. Close management is crucial to meeting the forecasted completion date.

Technical

We will be running our test applications on multiple servers so that if one server develops faults we can still perform testing against the other servers. If, however, the requirement to perform testing against a platform or software that we currently do not have then the set-up time to implement this needs to be considered. 

Management

Management support is required so when the project falls behind, the test schedule does not
get squeezed to make up for the delay. Management can reduce the risk of delays by supporting the test team throughout the testing phase and assigning people to this project with the required skills set.

Personnel

Due to the aggressive schedule, it is very important to have experienced testers on this project. Unexpected turnovers can impact the schedule. If attrition does happen, all efforts must be made to replace the experienced individual

Requirements

The test plan and test schedule are based on the current Requirements Document (or wiki plan). Any changes to the requirements could affect the test schedule and will need to be approved by the CCB.

Tools

The Mercury QuickTestPro Automated test tool will be used to help test the system. We have the licensed product onsite and installed.

Please Click here to view the information on the Automated Test Scripts (note, current legacy scripts can be found there).

Documentation

The following documentation will be available at the end of the test phase:

• Test Plan (this document)
  
• Automated Test Cases (see above)
  
• Test Case review (Jira)
  
• Requirements Validation Matrix (Wiki)
  
• Defect reports (Jira)
  
• Final Test Summary Report (Jira/Wiki)