Software Testing Lifecycle

Overview

Lifecycle

Software Testing Lifecycle (STLC): structured process for testing software to ensure quality and functionality throughout its development life cycle

Stages

STLC Stage	Entry Criteria	Activity	Exit Criteria	Deliverables
Requirement Analysis	Requirements Document available (both functional and non functional) Acceptance criteria defined Application architectural document available	Analyse business functionality to know the business modules and module specific functionalities Identify all transactions in the modules Identify all the user profiles Gather user interface/authentication, geographic spread requirements Identify types of tests to be performed Gather details about testing priorities and focus Prepare Requirement Traceability Matrix (RTM) Identify test environment details where testing is supposed to be carried out Automation feasibility analysis	Signed off RTM Test automation feasibility report signed off by the client	RTM Automation feasibility report
Test Planning	Requirements Documents Requirement Traceability Matrix Test Automation Feasibility Document	Analyze various testing approaches available Finalize on the best-suited approach Preparation of test plan/strategy document for various types of testing Test tool selection Test effort estimation Resource planning and determining roles and responsibilities	Approved test plan/strategy document Effort estimation document signed off	Test plan/strategy document Effort estimation document
Test Case Development	Requirements Documents RTM and test plan Automation analysis report	Create test cases, test design, automation scripts Review and baseline test cases and scripts Create test data	Reviewed and signed test cases/scripts Reviewed and signed test data	Test cases/scripts Test data
Test Environment Setup	System Design and architecture documents are available Environment set-up plan is available	Understand the required architecture, environment set-up Prepare hardware and software development requirement list Finalize connectivity requirements Prepare environment setup checklist Setup test Environment and test data Perform smoke test on the build Accept/reject the build depending on smoke test result	Environment setup is working as per the plan and checklist Test data setup is complete Smoke test is successful	Environment ready with test data set up Smoke Test Results
Test Execution	Baselined RTM, Test Plan, Test case/scripts are available Test environment is ready Test data set up is done Unit/Integration test report for the build to be tested is available	Execute tests as per plan Document test results, and log defects for failed cases Update test plans/test cases, if necessary Map defects to test cases in RTM Retest the defect fixes Regression Testing of application Track the defects to closure	All tests planned are executed Defects logged and tracked to closure	Completed RTM with execution status Test cases updated with results Defect reports
Test Cycle Closure	Testing has been completed Test results are available Defect logs are available	Evaluate cycle completion criteria based on - Time, Test coverage, Cost, Software Quality, Critical Business Objectives Prepare test metrics based on the above parameters Document the learning out of the project Prepare Test closure report Qualitative and quantitative reporting of quality of the work product to the customer Test result analysis to find out the defect distribution by type and severity	Test Closure report signed off by client	Test Closure report Test metrics

Principles

• Defect Detection: Testing reveals defects but cannot guarantee their absence
• Limited Coverage: It's impossible to test every scenario; focus on high-risk areas
• Early Initiation: Begin testing as soon as possible to catch defects early
• Defect Clustering: A few modules usually contain most defects; focus testing there
• Pesticide Paradox: Repeated tests find the same defects; update tests regularly
• Context Sensitivity: Choose testing strategies based on project context
• No Absence-of-Errors Fallacy: No defects found doesn't mean there are none
• Risk-Based Approach: Prioritize testing in high-risk areas
• No Proof of Correctness: Testing can't prove software is error-free
• Continuous Improvement: Constantly improve testing processes
• Traceability: Ensure test cases trace back to requirements
• Independence: Conduct testing independently from development
• Realistic Environment: Test in environments resembling production
• Multiple Perspectives: Test for functionality, usability, performance
• Validation & Verification: Confirm requirements and specifications are met
• Feedback Loop: Provide timely feedback to stakeholders

Detailed

High Overview

Requirements

Aspect	Purpose	Characteristics	Key Activities
Functional Requirements	Define the specific functions the software must perform	Describes what the software should do from a user's perspective Defines input, processing, and output of the system	Eliciting requirements through interviews, workshops, surveys Documenting requirements in a structured format (e.g., use cases, user stories) Validating requirements with stakeholders
Non-Functional Requirements	Specify the quality attributes and constraints of the system	Specifies attributes such as performance, usability, reliability, security Typically expressed as constraints or qualities the system must possess	Identifying and categorizing non-functional requirements (NFRs) Quantifying NFRs wherever possible (e.g., response time should be under 2 seconds) Prioritizing NFRs based on criticality
Requirement Analysis	Understand and document the needs and expectations of stakeholders	Involves gathering, analyzing, and documenting requirements Requires collaboration with stakeholders to ensure all needs are captured	Identifying stakeholders and their roles Gathering initial requirements through various techniques (interviews, questionnaires) Analyzing requirements for completeness, consistency, and feasibility Resolving conflicts or ambiguities in requirements

Design

Component	Purpose	Characteristics	Key Activities
Architectural Review	To assess and validate the overall structure and design of the software system at a high level	Evaluates the system's architecture to ensure it aligns with requirements and standards Identifies potential risks and issues related to architecture early in the development process	Review architectural diagrams, such as high-level design documents and system architecture models Conduct discussions with architects and developers to understand design decisions Verify compliance with architectural principles and standards
Data Flow	To analyze and validate the flow of data within the system, ensuring that it meets functional requirements and operates efficiently	Focuses on understanding how data moves through the system Identifies potential bottlenecks or vulnerabilities in data flow	Create data flow diagrams illustrating the movement of data within the system Verify that data flows correctly between components and modules Assess data security measures to ensure compliance with regulations and protection against threats
Design Verification and Validation	To confirm that the software design meets the specified requirements and is feasible for implementation	Validates the design against requirements to ensure completeness and accuracy Verifies that the design is implementable within technical constraints	Conduct reviews of design documents to verify alignment with requirements Perform walkthroughs and inspections to identify design flaws and inconsistencies Use modeling tools to simulate and validate design behavior Collaborate with stakeholders to gather feedback and refine the design
High-Level Design Review	To evaluate and refine the high-level design of the software system before proceeding to detailed design	Focuses on the overall structure and organization of the software components Identifies potential design flaws or inefficiencies at a conceptual level	Review high-level design documents, such as architecture diagrams and system specifications Assess design patterns and architectural styles for suitability and coherence Evaluate scalability and performance considerations at the system level Identify dependencies and interfaces between system components
Interface	To design, document, and validate the interfaces between various system components and external entities	Defines communication protocols and data formats for interactions between system elements Ensures compatibility and consistency across interfaces	Design interface specifications detailing the methods, parameters, and protocols for communication Verify interoperability by testing interface interactions between components Document interface requirements and constraints for developers and stakeholders Conduct compatibility testing with external systems or APIs
Low-Level Design Review	To refine and finalize the detailed design of individual components or modules within the software system	Focuses on the implementation details and internal structure of system components Ensures that detailed designs adhere to architectural guidelines and standards	Review detailed design documents, such as class diagrams and module specifications Assess design patterns and coding conventions for consistency and maintainability Verify traceability between high-level and low-level design elements Conduct code walkthroughs and inspections to identify design defects and optimization opportunities
Prototype Testing	To evaluate a preliminary version of the software system to gather feedback, validate design assumptions, and identify areas for improvement	Involves testing a functional prototype or mock-up of the software system Focuses on user experience, functionality, and performance	Develop prototype versions of the software to demonstrate key features and interactions Conduct usability testing to gather feedback from end-users and stakeholders Evaluate system performance and scalability under simulated usage scenarios Iterate on the prototype based on user feedback and testing results
System Design Review	To review and validate the overall system design, encompassing both high-level architecture and detailed component designs	Integrates feedback from previous design stages to ensure coherence and completeness of the system design Identifies any remaining design issues or risks before proceeding to implementation	Conduct comprehensive review sessions covering all aspects of the system design Verify alignment with overall project goals and requirements Assess design trade-offs and risks to inform decision-making Prepare design documentation and artifacts for approval and reference Obtain stakeholder sign-off on the system design before proceeding to implementation

Development

Component	Purpose	Characteristics	Key Activities
Static Code Analysis	To analyze the source code without executing it, detecting potential defects, vulnerabilities, and adherence to coding standards	Automatically reviews code for bugs, security vulnerabilities, and compliance with coding standards Identifies potential performance bottlenecks and code smells Provides insights into code complexity and maintainability	Configure static code analysis tools to scan the codebase for issues Review analysis results to prioritize and address identified issues Set up rules and policies for coding standards and best practices Integrate static code analysis into the continuous integration/continuous deployment (CI/CD) pipeline

Testing

Component	Purpose	Characteristics	Key Activities
Accessibility Testing	To ensure that the software is usable by people with disabilities and conforms to accessibility standards	Evaluates software for compliance with accessibility guidelines such as WCAG Tests various aspects including keyboard navigation, screen reader compatibility, and color contrast	Conduct manual and automated accessibility audits Use assistive technologies to simulate user experiences Address accessibility issues found during testing
Acceptance Testing	To verify that the software meets the acceptance criteria and is ready for deployment	Performed by end-users or stakeholders to validate the software against business requirements Tests are typically conducted in a production-like environment	Define acceptance criteria based on user stories and requirements Execute test cases representing real-world scenarios Obtain user feedback and approval for the software
API Testing	To verify the functionality, reliability, performance, and security of the application programming interfaces (APIs)	Tests the integration points and communication between different software systems Validates inputs, outputs, and error handling of APIs	Design test cases covering various API functionalities Execute API calls and verify responses Perform security testing to identify vulnerabilities
Code Review	To identify defects, improve code quality, and ensure adherence to coding standards through manual inspection of source code	Reviews code for bugs, readability, maintainability, and performance Provides feedback to developers to improve code quality	Conduct peer code reviews using tools or manual inspection Identify potential improvements and areas for refactoring Address issues raised during code review
Component Testing	To validate the individual components or modules of the software system in isolation	Focuses on testing the functionality and behavior of individual units of code Uses stubs or mocks to simulate dependencies	Write test cases to verify the functionality of each component Execute component tests in an isolated environment Verify interactions with external dependencies using stubs or mocks
Compatibility Testing	To ensure that the software functions correctly across different devices, browsers, operating systems, and environments	Tests software compatibility with various hardware and software configurations Identifies any issues related to platform-specific features or behaviors	Identify target platforms and configurations for testing Execute test cases on different devices, browsers, and operating systems Verify functionality, layout, and performance across various environments
Compliance Testing	To ensure that the software complies with legal, regulatory, and industry standards	Verifies adherence to specific regulations, standards, or guidelines Includes testing for data privacy, security, and accessibility requirements	Identify relevant compliance requirements for the software Execute test cases to verify compliance with regulations and standards Document compliance test results for audit and regulatory purposes
Configuration Testing	To validate the software's behavior under different configurations and settings	Tests the impact of configuration changes on system functionality and performance Verifies compatibility with various hardware and software configurations	Identify key system configurations to test Create test cases covering different configuration scenarios Execute tests with varying configurations and settings
Data Retrieval/Restoration/Purging/Migration Testing	To ensure the accuracy, reliability, and efficiency of data retrieval, restoration, purging, and migration processes	Tests data retrieval, restoration, purging, and migration processes for accuracy and completeness Verifies data integrity and consistency after migration or purging operations	Define test scenarios for data retrieval, restoration, purging, and migration Execute tests to verify data accuracy and integrity Validate performance and efficiency of data operations
Database Testing	To verify the functionality, performance, and reliability of database systems and related operations	Validates database schema, data integrity, and transactions Tests database CRUD operations (Create, Read, Update, Delete) Verifies SQL queries, stored procedures, and triggers	Design test cases to cover database functionality and operations Execute tests to validate database performance and reliability Verify data consistency and accuracy
Disaster Recovery Testing	To evaluate the readiness and effectiveness of disaster recovery procedures and mechanisms	Tests backup and recovery processes under various disaster scenarios Evaluates data loss prevention and continuity of operations	Simulate disaster scenarios to test recovery procedures Verify data backup integrity and accessibility Assess recovery time objectives (RTO) and recovery point objectives (RPO)
End-to-End Testing (E2E)	To validate the entire software system from start to finish, including all integrated components and interfaces	Tests the application workflow across multiple systems and components Verifies end-user scenarios from initiation to completion	Define end-to-end test scenarios covering user journeys and system interactions Execute tests in a production-like environment Verify system behavior and data flow across integrated components
Functional Testing	To validate the functional requirements of the software application and ensure that it behaves as expected	Tests individual functions and features of the software Verifies input validation, data processing, and output generation	Develop test cases based on functional specifications and user stories Execute tests to validate each functional requirement Verify system behavior under different input conditions
Globalization Testing	To ensure that the software application can function correctly in different languages, regions, and cultural settings	Tests the software's ability to adapt to various cultural and regional conventions Verifies support for multilingual content and locale-specific formats	Test the application with different language settings and regional configurations Verify text translations, date/time formats, and currency symbols Check for proper handling of cultural preferences and conventions
Integration Testing	To validate the interactions and interfaces between different software components and systems	Tests the integration points and data exchanges between modules, services, or systems Verifies interoperability and communication protocols	Define integration test scenarios covering interactions between system components Execute tests to verify data flow and communication between integrated systems Check for proper error handling and fault tolerance
Load Testing	To evaluate the software's performance under expected and peak load conditions	Tests the software's response time, throughput, and resource utilization under load Verifies scalability and capacity planning	Define load test scenarios based on expected usage patterns and peak loads Execute tests to simulate concurrent user interactions and data processing Monitor system metrics and analyze performance bottlenecks
Localization Testing	To ensure that the software application is adapted and culturally appropriate for specific target markets	Tests the software's linguistic and cultural adaptation for specific locales Verifies compliance with language, currency, and legal requirements	Test the application with localized content and settings Verify translations, date/time formats, and regulatory compliance for target markets Check for alignment with local cultural norms and conventions
Maintainability Testing	To evaluate the ease with which the software can be modified, enhanced, and maintained over time	Tests the codebase for readability, modularity, and extensibility Identifies areas of code that are prone to defects and difficult to maintain	Assess code quality metrics such as cyclomatic complexity and code duplication Review coding conventions and best practices Identify refactoring opportunities to improve maintainability
Mobile Testing	To validate the functionality, usability, and performance of mobile applications across different devices and platforms	Tests mobile-specific features such as touch gestures, device orientation, and offline capabilities Verifies compatibility with various mobile operating systems and screen resolutions	Test the application on different mobile devices and platforms Verify user experience and performance on mobile devices Address platform-specific issues and limitations
Regression Testing	To ensure that recent code changes have not adversely affected existing functionalities	Re-executes test cases to verify that previously developed and tested software still performs correctly Identifies unintended side effects and regression defects introduced by new changes	Automate regression test suites to ensure efficient and comprehensive testing Execute regression tests after each software build or release Verify that fixed defects do not reappear in subsequent releases
Recovery Testing	To verify the software's ability to recover from failures and resume normal operations	Tests recovery mechanisms such as data backup, rollback procedures, and fault tolerance Verifies data integrity and system stability after recovery	Simulate system failures and errors to trigger recovery procedures Verify the effectiveness of backup and restore processes Assess recovery time and data consistency
Reliability Testing	To assess the software's ability to consistently perform its intended functions under specified conditions	Tests the software for failure rates, error handling, and fault tolerance Verifies system stability and uptime	Define test scenarios to simulate various failure conditions Execute tests to measure system reliability and stability Assess error recovery mechanisms and fault tolerance
Requirements Review	To validate that the software requirements are complete, consistent, and accurately captured	Reviews software requirements documentation for clarity, accuracy, and feasibility Ensures alignment with stakeholder expectations and project goals	Conduct requirements walkthroughs and inspections Verify traceability between requirements and test cases Address ambiguities and inconsistencies in requirements
Rollback Testing	To verify the software's ability to revert to a previous state or version in case of deployment failures or issues	Tests rollback procedures and data restoration mechanisms Verifies the integrity and consistency of data after rollback	Simulate deployment failures or issues to trigger rollback procedures Verify data consistency and system stability after rollback Assess the effectiveness of rollback mechanisms
Scalability Testing	To assess the software's ability to handle increasing workload and user demand without compromising performance or stability	Tests the software's ability to scale up or down based on changing load conditions Verifies system performance under various load levels and concurrent user interactions	Define scalability test scenarios covering different usage patterns and load levels Execute tests to measure system performance and scalability Assess system response time, throughput, and resource utilization under load
Security Testing	To identify vulnerabilities and weaknesses in the software's security mechanisms and ensure protection against threats	Tests for common security vulnerabilities such as injection attacks, cross-site scripting (XSS), and authentication flaws Verifies compliance with security standards and best practices	Conduct security assessments using automated tools and manual penetration testing Identify and prioritize security vulnerabilities based on severity Implement security controls and countermeasures to mitigate risks
Service Level Agreement (SLA) Testing	To verify that the software meets the agreed-upon service level agreements regarding performance, availability, and other criteria	Tests the software's ability to meet specified SLA metrics such as response time and uptime Verifies compliance with contractual obligations and quality standards	Define SLA metrics and performance targets Execute tests to measure system performance and availability Monitor SLA compliance and report deviations
Smoke Testing	To quickly assess the basic functionality of the software after changes or updates, ensuring that critical functionalities work as expected	Tests fundamental functionalities to determine if the software is stable enough for further testing Verifies basic user interactions and system behaviors	Execute a set of predefined test cases covering core functionalities Verify essential features such as login, navigation, and basic operations Assess system stability and readiness for comprehensive testing
Spike Testing	To evaluate the software's performance and behavior under sudden and extreme increases in workload or user traffic	Tests the software's ability to handle rapid fluctuations in user demand Verifies system scalability and resource management under peak loads	Create scenarios to simulate sudden spikes in user traffic or workload Execute tests to measure system response time and throughput Assess system performance under stress conditions
State Transition Testing	To validate the transitions between different states or modes of the software application	Tests the behavior of the software as it transitions between different states or modes Verifies proper state management and data integrity	Identify possible states and transitions within the software Develop test cases to validate state transitions and associated behaviors Execute tests to verify proper state management
Stress Testing	To evaluate the software's stability and performance under extreme workload conditions	Tests the software's response time, throughput, and resource utilization under high load Verifies system behavior and stability under stress conditions	Create test scenarios to simulate high load conditions Execute tests to measure system performance metrics Assess system behavior and identify performance bottlenecks
Unit Testing	To validate the individual units or components of the software application in isolation	Tests the smallest testable parts of the software, such as functions, methods, or classes Verifies the correctness of individual units of code	Write test cases to cover each unit of code Execute unit tests in an isolated environment Verify the functionality and behavior of individual units
Usability Testing	To evaluate the software's user interface (UI) design, ease of use, and overall user experience	Tests the software from the end user's perspective to assess its usability and intuitiveness Verifies alignment with user expectations and preferences	Define user personas and typical usage scenarios Conduct usability tests with representative users Collect feedback and observations on user interactions and experiences

Deployment

Component	Purpose	Characteristics	Key Activities
Beta Deployment	To release a pre-production version of the software to a limited audience for testing and feedback	Provides an opportunity to gather real-world feedback from a select group of users Helps identify issues and gather insights before a full release	Select a group of beta testers representing target users or stakeholders Deploy the beta version of the software to the test environment Collect feedback and bug reports from beta testers Analyze feedback and make necessary improvements before the final release
Pilot Deployment	To deploy the software to a limited production environment or user group for further testing and validation	Tests the software in a real-world production-like environment with actual users Verifies performance, scalability, and reliability under production conditions	Select a pilot group representing typical users or business processes Deploy the software to the pilot environment or user group Monitor system performance and user feedback during the pilot phase Address any issues or concerns raised during the pilot deployment
Rollback Procedure	To revert to a previous version of the software in case of deployment failures or issues	Provides a mechanism to undo changes and restore the system to a known stable state Minimizes downtime and impact on users in case of deployment issues	Define rollback procedures and criteria for triggering rollback Automate rollback steps where possible to ensure consistency and reliability Test rollback procedures in advance to ensure effectiveness Execute rollback in case of deployment failures or issues

Maintenance

Component	Purpose	Characteristics	Key Activities
Chaos Engineering	To proactively identify weaknesses and vulnerabilities in a software system by intentionally injecting failures and disturbances	Simulates real-world failures and disruptions to test system resilience Helps uncover hidden system weaknesses and improve overall reliability	Identify critical system components and failure scenarios Introduce controlled failures or disturbances into the system Monitor system behavior and response to failures Analyze results and make necessary improvements to enhance system resilience

Distinction

Tests Type	Details
Acceptance	Validation activity Did we build the right thing? Is this what the customer really needs? Verify the product actually solves the problem it was made to solve
Functional	Verification activity Did we build a correctly working product? Does the software meet the business requirements? Verifies that the product actually works as the developer think it does Functional (Integration Test), testing a slice of functionality in a system. Test many methods and may interact with dependencies like Databases or Web Services
Load	To recognize the upper limit of the system, set SLA of the app and check how the system can handle a heavy load Load limit is a threshold of a break Find bugs which are not possible with any other testing method. Such as memory management memory leaks, bugs, buffer overflows Determine the operating capacity of an application Check out that current infrastructure is sufficient to run the application Application's capacity for concurrent users and it's scalability for handling increased user demand
Performance	Helps to set the benchmark and standards for the application How an application behaves under regular parameters Conform the performance needs of the business
Stress	To check out how the system behaves under extreme loads and how it recovers from failure Ensure that under a sudden high load for a considerable duration the servers don't crash Load limit is above the threshold of a break System has saved the data before crashing Test the system in the situation of failures
Unit	Testing an individual unit, such as a method (function) in a class, with all dependencies mocked up (objects that simulate the behavior of real objects)

Test Pyramid

Shift-Left Paradigm

It's a concept derived from the Agile and DevOps methodologies, which advocate for early and continuous testing throughout the SDLC including writing more unit tests that will cover/overlap integration tests. The same applies to other tests in the test pyramid. This will lead to minimal time-to-market and e2e tests.

The traditional approach to software testing involved testing towards the end of the development cycle, often in a dedicated testing phase. However, this approach has limitations, such as late defect detection, higher costs of fixing defects, and longer time-to-market.