Testing is a broad software discipline, with many sub-disciplines. ProServices offers testing services in a factory context. ProServices offers specialized services such as unit test generation and execution, and coverage testing execution. These are not commonly available in the customer's internal testing efforts, and include categories of testing that do not require any specialized domain knowledge of the customer's application.
These services can be purchased from proServices as standard engagements incorporating the areas of service strength where proServices provides an alternative source for testing equivalent to the customer's environment.
proServices' testing service provides machines, personnel, standardized testing processes, and critical testing techniques.
Major forms of white box testing (where proServices is performing the testing as a result of our expertise based on the ability to review the customer's source code) include:
- Unit Test Generation / Execution.
- Coverage Testing.
In some circumstances proServices performs the major forms of black box testing including:
- Setting up factory test environments and sample tests.
- Building tests to address any subset of the following: requirements traceability testing, feature testing, coverage testing, exception testing, integration / API testing, function testing, and unit testing.
- Performing testing to address any subset of the following: requirements traceability testing, feature testing, coverage testing, exception testing, integration / API testing, function testing, and unit testing.
- Load testing services.
Unit Test - A single test of a low-level program entity like a function or method on a class. Usually such a single test will execute one complete path from entry of the function / method to the exit. It requires a number of unit tests to cover all major paths to constitute a 'unit tested' function or method.
Path - This is a Low-level program category. To perform 'complete' testing, it requires all sequences of program activities (statements) from point A to point B to be executed for at least one set of data. This is loosely referred to as a path. The units / paths tested can be targeted to address specific areas of interest to the customer.
Unit Test Generation / Execution - Performed using standard industry techniques for automating unit testing or performed by compiling code with the called routines stubbed. The customer program is run using a debugger, setting breakpoints, starting the execution at the 'unit' routine and supplying input data in the debugger. Traditional cross-referencing can identify the input data. The testing machines can be available in the factory for all but the subset of unit testing that is machine specific function calls unable to be stubbed off meaningfully.
Coverage Testing - Coverage testing is applicable to a 'Single' application (can include a database). Coverage typically indicates:
- Current percentage of existing test suites coverage.
- A report indicating which functions are or are not covered from existing tests.
Objectives
Identify Unit Tests To Build / Execute
Certain aspects of an application will be more significant to test at a white box unit testing level than others. This service is performed in order of priority, the functions in an application to unit test and the data sets that will be used to obtain full branch coverage.
Identify Tests to Perform Test Coverage
Customer input is required to insure the correct set of customer tests is selected to perform test coverage analysis.
Execution of Tests
proServices performs the execution of unit tests, collects data about potential pass/ fail, and desired data regarding the state during unit testing execution. proServices also performs the execution of code coverage testing to collect data regarding profile outputs in running an application.
Interpretation of Tests
proServices evaluates the outputs of all unit and coverage tests executed to identify software defects, inadequate specification of expected program behavior (documentation), aspects of quality, insufficient testing, and areas of program weakness. In some cases, recommendations of improvements or additional efforts needing consideration will be delivered.
Deliver Analysis
Produce and deliver a report on the testing processes and the conclusions drawn from the interpretation of the results.
Deliverables
The testing results delivered in this service are:
- In unit test suites constructed.
- Results of execution of tests: may include pass/fail, data condition indications.
- Coverage output indicating degree of coverage achieved at function level running profiling techniques.
- Recommendations based on Interpretation process.
- Implications of tests or coverage on software quality.
- Software process implications of testing process results identified.
Customer Profiles
- Customers releasing a product on a critical path with minimal cycle time for thorough testing.
- Customers requiring unit or coverage testing and value improvement to their current software process enabling fault detection in an earlier lifecycle phase.
- Customers lacking experienced testing organizations.
Service Options Available For Testing
- e-Factory: Remote access to customer site to perform this service.
- Factory: Perform this service in proServices' Trenton, NJ Factory.
- On-Site: Perform this service at the customer's site.(not available)
Service Qualifications
- C, C++, Java
- Unit Testing Service Limitations - (F) The factory will not be able to unit test any system specific functions when the factory does not possess the same machine configuration as the customer.
- Coverage Testing Service Limitations - (F) factory services need identification that customer machines, operating systems and commercial of the shelf products (COTS) are on approved lists. This information is required for re-tooling and machine resource planning purposes.
