{"id":5862,"date":"2022-04-15T06:37:00","date_gmt":"2022-04-15T06:37:00","guid":{"rendered":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/"},"modified":"2023-07-14T15:34:22","modified_gmt":"2023-07-14T15:34:22","slug":"qa-automation-framework","status":"publish","type":"post","link":"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/","title":{"rendered":"QA Automation: How to Design a Proven Framework"},"content":{"rendered":"\n<p><a href=\"https:\/\/www.edvantis.com\/de\/blog\/qa-vs-qc\/\" target=\"_blank\" rel=\"noreferrer noopener\">Quality assurance (QA)<\/a> is the central element of any software development lifecycle (SDLC).&nbsp;<\/p>\n\n\n\n<p>Whether you develop a new solution or update an existing one, you need to test it for usability, performance, and security among other factors. The better your QA process is, the faster you can schedule new releases. That\u2019s a well-known axiom \u2014 but a tough one to implement as market data shows.&nbsp;<\/p>\n\n\n\n<p>Poor software quality in legacy systems is pegged to cost the US economy over <a href=\"https:\/\/www.it-cisq.org\/pdf\/CPSQ-2020-report.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">$1.56 trillion<\/a> per year \u2014 a 22% jump from the <a href=\"https:\/\/www.it-cisq.org\/the-cost-of-poor-quality-software-in-the-us-a-2018-report\/The-Cost-of-Poor-Quality-Software-in-the-US-2018-Report.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">2018 figure<\/a>. Support of such systems costs a lot \u2014 and high error rates make any type of legacy modernization project more complex to execute. Software quality assurance, delayed to post-development, also builds up <a href=\"https:\/\/www.edvantis.com\/de\/blog\/technical-debt\/\" target=\"_blank\" rel=\"noreferrer noopener\">technical debt<\/a>, forcing teams to tackle the reworks, instead of focusing on extra value-driving features.&nbsp;<\/p>\n\n\n\n<p>Given that corporate technical estates are growing large every year, leaders need to come up with better approaches to quality assurance.&nbsp;<\/p>\n\n\n\n<p>And one of the main methods for that is QA automation.&nbsp;<\/p>\n\n\n\n<div class=\"wp-block-ht-block-toc  is-style-outline htoc htoc--position-wide toc-list-style-plain\" data-htoc-state=\"expanded\"><span class=\"htoc__title\"><span class=\"ht_toc_title\">Table of Contents<\/span><\/span><div class=\"htoc__itemswrap\"><ul class=\"ht_toc_list\"><li class=\"\"><a href=\"#htoc-what-is-qa-automation\">What is QA Automation?&nbsp;<\/a><ul class=\"ht_toc_child_list\"><li class=\"\"><a href=\"#htoc-sample-test-automation-framework\">Sample Test automation framework&nbsp;<\/a><\/li><li class=\"\"><a href=\"#htoc-why-automate-quality-assurance\">Why Automate Quality Assurance<\/a><\/li><\/ul><\/li><li class=\"\"><a href=\"#htoc-how-to-build-a-test-automation-framework-step-by-step\">How to Build a Test Automation Framework Step-by-Step<\/a><ul class=\"ht_toc_child_list\"><li class=\"\"><a href=\"#htoc-1-determine-the-optimal-test-automation-framework-architecture\">1. Determine The Optimal Test Automation Framework Architecture<\/a><\/li><li class=\"\"><a href=\"#htoc-2-design-a-reference-test-automation-framework\">2. Design a Reference Test Automation Framework&nbsp;<\/a><ul class=\"ht_toc_child_list\"><\/ul><\/li><li class=\"\"><a href=\"#htoc-3-decide-on-your-test-automation-tech-stack\">3. Decide on Your Test Automation Tech Stack&nbsp;<\/a><\/li><li class=\"\"><a href=\"#htoc-4-decide-on-your-path-to-implementation\">4. Decide on Your Path to Implementation&nbsp;<\/a><\/li><li class=\"\"><a href=\"#htoc-5-develop-test-data\">5. Develop Test Data&nbsp;<\/a><\/li><\/ul><\/li><li class=\"\"><a href=\"#htoc-how-we-helped-an-enterprise-client-switch-from-manual-to-automation-testing-case-study\">How We Helped An Enterprise Client Switch From Manual to Automation Testing: Case Study&nbsp;<\/a><ul class=\"ht_toc_child_list\"><li class=\"\"><a href=\"#htoc-the-need-to-automate\">The Need to Automate&nbsp;<\/a><\/li><li class=\"\"><a href=\"#htoc-building-a-custom-test-automation-framework\">Building a Custom Test Automation Framework&nbsp;<\/a><\/li><li class=\"\"><a href=\"#htoc-a-robust-test-automation-framework-and-team-upskilling\">A Robust Test Automation Framework and Team Upskilling&nbsp;<\/a><\/li><\/ul><\/li><li class=\"\"><a href=\"#htoc-to-conclude\">To Conclude&nbsp;<\/a><\/li><\/ul><\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"htoc-what-is-qa-automation\">What is QA Automation?&nbsp;<\/h2>\n\n\n\n<figure class=\"wp-block-pullquote\"><blockquote><p>QA automation assumes using tools to automatically review, assess, and validate a software product against the set requirements for code quality, functionality (business logic), and user experience.<\/p><\/blockquote><\/figure>\n\n\n\n<p>In other words: you perform test automation and gain an automated testing experience as an output \u2014 just so that you are clear on the basic QA lingo. Instead of manually going through the standard testing sequences,<a href=\"https:\/\/www.edvantis.com\/de\/blog\/qa-engineer\/\" target=\"_blank\" rel=\"noreferrer noopener\"> quality assurance specialists<\/a> created automated test scripts to probe the produced code for conformance with quality, security, or business requirements. Many of the performed tests are routine, repetitive step-by-step processes, required to validate the written code after changes or pre-deployment.&nbsp;<\/p>\n\n\n\n<p>Given that organizations now do around<a href=\"https:\/\/cloud.google.com\/blog\/products\/devops-sre\/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling\" target=\"_blank\" rel=\"noreferrer noopener\"> 1,460 code deploys per year<\/a> on average, over-reliance on manual testing means two things: larger headcount or slower time-to-market.&nbsp;<\/p>\n\n\n\n<p>Automated software testing allows you to improve software testing speed without compromising its quality. A purpose-built automated testing framework can help you test all the nooks and crannies of your product and ensure end-to-end tests coverage for different usage scenarios.&nbsp;<\/p>\n\n\n\n<p>A test automation framework is a collection of rules, tools, and processes your organization uses to perform comprehensive QA at different stages of your software development lifecycle.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"htoc-sample-test-automation-framework\"><strong>Sample Test automation framework&nbsp;<\/strong><\/h4>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1077\" height=\"876\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" src=\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/sample-test-automation-framework.svg\" alt=\"sample test automation framework\" class=\"wp-image-5785\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-why-automate-quality-assurance\">Why Automate Quality Assurance<\/h3>\n\n\n\n<p>In 2011, Google engineers managed to develop and release over 100 Google+ features <a href=\"https:\/\/ptgmedia.pearsoncmg.com\/images\/9780321803023\/samplepages\/0321803027.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">over a 100-day period<\/a>. How did they do that? By strategically blending manual and automated testing at different stages of SDLC.&nbsp;<\/p>\n\n\n\n<p>Since that time Google \u201cupgraded\u201d its baseline setup to Google\u2019s Test Automation Platform (TAP) \u2014 the world\u2019s largest software test automation framework, backed by DevOps principles such as Continuous Integration (CI), Continuous Testing (CT), and Continuous Delivery (CD).&nbsp;<\/p>\n\n\n\n<p>On a regular day, TAP integrates and tests <a href=\"https:\/\/static.googleusercontent.com\/media\/research.google.com\/en\/\/pubs\/archive\/45861.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">over 13K code projects<\/a>, requiring 800K builds and 150 Million test runs.&nbsp;<\/p>\n\n\n\n<p>The best part? Even on a smaller scale, test automation results in remarkable processing speed and some extra benefits such as:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Immediate feedback on newly-shipped code&nbsp;<\/li>\n\n\n\n<li>Higher cost-effectiveness, compared to manual testing alone<\/li>\n\n\n\n<li>Rapid execution speed that improves teams performance pace<\/li>\n\n\n\n<li>Higher test accuracy since there\u2019s less room for human error&nbsp;<\/li>\n\n\n\n<li>Better test coverage as you can do more faster&nbsp;<\/li>\n\n\n\n<li>Test case reusability, minimizing redundant work&nbsp;<\/li>\n\n\n\n<li>Faster time-to-market for new products&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>That being said, merely installing a set of QA test automation tools isn\u2019t enough to get the above gains. You also need to consider how you will embed automated QA at <a href=\"https:\/\/www.edvantis.com\/de\/blog\/software-development-process\/\" target=\"_blank\" rel=\"noreferrer noopener\">different stages of your SDLC<\/a> to be performed alongside development. <br><br>The \u201cmerger\u201d of development and testing work is called continuous testing \u2014 an Agile practice of running automated tests as part of the software delivery pipeline to obtain feedback before moving on to the next stages.&nbsp;<\/p>\n\n\n\n<p>CT is an integral part of Continuous Integration and Continuous Delivery (CI\/CD) \u2014 a wider set of operational practices, aimed at bridging the \u201chand-on\u201d gaps between different teams that take part in software development.&nbsp;<\/p>\n\n\n\n<p>CI\/CD and continuous testing are proven approaches to improving your software development cadence. But they are hard to implement, too. Per <a href=\"https:\/\/www.sogeti.com\/explore\/reports\/continuous-testing-report-2020\/\" target=\"_blank\" rel=\"noreferrer noopener\">Capgemini 2020 Continous Testing report<\/a>, businesses face challenges across several vectors:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Process adoption: <\/strong>56% find in-sprint testing hard to execute because of process constraints.&nbsp;<\/li>\n\n\n\n<li><strong>Alignment<\/strong>: 67% also struggle to maintain meaningful test cases that align with user expectations.<\/li>\n\n\n\n<li><strong>Talent:<\/strong> 62% can\u2019t find enough QA automation engineers to implement a test automation framework for them&nbsp;<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"htoc-how-to-build-a-test-automation-framework-step-by-step\">How to Build a Test Automation Framework Step-by-Step<\/h2>\n\n\n\n<p>Over <a href=\"https:\/\/www.perfecto.io\/resources\/state-test-automation#success\" target=\"_blank\" rel=\"noreferrer noopener\">95% of organizations<\/a> agree that test automation patterns, principles, and practices are critical for them. But far less made progress with test automation framework design and implementation.&nbsp;<\/p>\n\n\n\n<p>Indeed, QA automation is a complex process, but a systematic one. Meaning you need to complete several core steps to transition from largely manual to mostly automated testing. <br><br>Our QA team unpacks the best practices below and shows how these are implemented in practice with a client case study.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-1-determine-the-optimal-test-automation-framework-architecture\">1. Determine The Optimal Test Automation Framework Architecture<\/h3>\n\n\n\n<p>Quality assurance is a multi-facet activity. You can probe newly-built software from different angles to ensure it works like it was meant to and has no critical bugs or vulnerabilities.&nbsp;<\/p>\n\n\n\n<p>For that reason, there are plenty of different types of tests:<\/p>\n\n\n\n<p><strong><strong>Functional testing<\/strong><\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Unit tests<\/li>\n\n\n\n<li>Component tests<\/li>\n\n\n\n<li>Smoke tests<\/li>\n\n\n\n<li>Regression tests<\/li>\n\n\n\n<li>API tests<\/li>\n\n\n\n<li>UI tests&nbsp;<\/li>\n<\/ul>\n\n\n\n<p><strong><strong>Non-functional testing<\/strong><\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Performance tests<\/li>\n\n\n\n<li>Load tests&nbsp;<\/li>\n\n\n\n<li>Pentesting (security tests)<\/li>\n\n\n\n<li>Stress tests&nbsp;<\/li>\n\n\n\n<li>Volume tests&nbsp;<\/li>\n\n\n\n<li>Recovery tests&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Can you incorporate all of them into your AQA framework? Perhaps. Should you attempt to do that? Not at all.&nbsp;<\/p>\n\n\n\n<p>As Denis Kolesnikov, Expert Test Automation Engineer at Edvantis strengthens: <\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p id=\"htoc-a-good-automated-testing-framework-should-focus-on-a-specific-problem-s-and-be-designed-around-your-type-of-product-before-you-start-designing-your-architecture-you-need-to-get-your-qa-priorities-straight-put-down-your-main-testing-requirements-select-a-series-of-tests-you-need-then-look-into-different-strategies-for-automating-them\">\u201c<em>A good automated testing framework should focus on a specific problem(s) and be designed around your type of product. Before you start designing your architecture, you need to get your QA priorities straight. Put down your main testing requirements, select a series of tests you need, then look into different strategies for automating them\u201d.<\/em><\/p>\n<\/blockquote>\n\n\n\n<p>A test automation framework is a collection of process steps and guidelines for creating shared process automation for a standardized set of tests. Similar to other types of automated processes, test automation requires a shared business logic for design.&nbsp;<\/p>\n\n\n\n<p>Various test automation frameworks exist, each with a different logic:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><strong>Keyword-driven framework<\/strong>. <\/strong>This is a popular approach to automated functional testing. You break down the test logic into keywords and functions and store them in a separate Object Repository. Then call it in when you need to run a test.&nbsp;<\/li>\n\n\n\n<li><strong><strong>Modular-driven framework. <\/strong><\/strong>In this case, you analyze all available test cases to identify repeatable flows. Then turn these reusable flows into functions and score them as test scripts. To perform different testing scenarios, you then re-use or combine these modular scripts.&nbsp;<\/li>\n\n\n\n<li><strong><strong>Behavior-driven framework. <\/strong><\/strong>All application behaviors are defined through human-readable test commands (e.g. register an account). Then these are codified into test cases using BDD tools.&nbsp;<\/li>\n\n\n\n<li><strong><strong>Data-driven framework.<\/strong> <\/strong>You build your test cases around available test data, stored in a table or spreadsheet format. Then run a single test script to validate all the data and generate outputs in the same table.&nbsp;<\/li>\n\n\n\n<li><strong><strong>Hybrid framework<\/strong> \u2014 <\/strong>any combination of the above AQA frameworks.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>\u201cEach framework on the market is neither entirely good nor bad \u2014 everything depends on your particular project, solution, and <a href=\"https:\/\/www.edvantis.com\/de\/blog\/software-development-methodologies\/\" target=\"_blank\" rel=\"noreferrer noopener\">software development methodology<\/a>,\u201d Denis Kolesnikov also notes. <\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-2-design-a-reference-test-automation-framework\">2. Design a Reference Test Automation Framework&nbsp;<\/h3>\n\n\n\n<p>Conceptually, all QA automation frameworks have four layers:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Test generation<\/li>\n\n\n\n<li>Test definition&nbsp;<\/li>\n\n\n\n<li>Test execution&nbsp;<\/li>\n\n\n\n<li>Test adaptation&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Jointly, these govern the relationships between a <strong>TAS <\/strong>\u2013 test automation solution \u2013 and a <strong>SUT<\/strong> (system under test). Or, to put it another way, these four layers create the rules and guidelines for testing automation.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1077\" height=\"1123\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" src=\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/Reference-Test-Automation-Framework.svg\" alt=\"Reference Test Automation Framework\" class=\"wp-image-5787\"\/><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"htoc-test-generation-layer\">Test Generation Layer<\/h4>\n\n\n\n<p>The test generation layer sets the baseline for subsequent QA automation. Your goal here is to formalize your main test types, design manual tests, and set up test data capture and generation to support the selected QA scenarios.&nbsp;<\/p>\n\n\n\n<p>The components (and supporting tools) of this layer are used to:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Edit and navigate test suite structures<\/li>\n\n\n\n<li>Align test cases to test objectives (or SUT requirements)<\/li>\n\n\n\n<li>Document the test design<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"htoc-test-definition-layer\">Test Definition Layer<\/h4>\n\n\n\n<p>At this stage, you need to analyze the available test data and match it to the planned test cases. Then begin designing test scripts for the approved use cases.&nbsp;<\/p>\n\n\n\n<p>The components (and supporting tools) of this layer are used to:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Partition (constrain), parameterize, or instantiate test data<\/li>\n\n\n\n<li>Document the test data, test cases, and\/or test procedures<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"htoc-test-execution-layer\">Test Execution Layer<\/h4>\n\n\n\n<p>With all the prep work done, you are ready to move to actual QA work. This part of a test framework includes all the tools you need to run, log, and report on automated test case execution.<\/p>\n\n\n\n<p>A robust test execution layer provides the following capabilities:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Auto-set up and tear down of the SUT for test execution (i.e. set of test cases including test data)<\/li>\n\n\n\n<li>Rapid configuration and parameterization of the test setup<\/li>\n\n\n\n<li>Interpretation of both test data and test cases and subsequent transformation of them into executable scripts<\/li>\n\n\n\n<li>SUT responses validation (i.e. comparison of expected and actual results) for reporting&nbsp;<\/li>\n\n\n\n<li>Precise control and proper timing for automated test execution<\/li>\n<\/ul>\n\n\n\n<p>Don\u2019t rush with implementing a definite architecture. Instead, start with several proof-of-concept (POC) scenarios. Overall, <a href=\"https:\/\/www.perfecto.io\/resources\/state-test-automation#success\" target=\"_blank\" rel=\"noreferrer noopener\">20% of QA teams<\/a> say that selecting the right test case scenarios is challenging for them. So there\u2019s going to be some trial and error.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-3-decide-on-your-test-automation-tech-stack\">3. Decide on Your Test Automation Tech Stack&nbsp;<\/h3>\n\n\n\n<p>As mentioned earlier, a testing automation framework is a combination of processes and tools. The previous section described the process layout. Now let\u2019s focus on the QA automation tools.<\/p>\n\n\n\n<p>QA automation testing tools can be:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><strong>Open-source <\/strong><\/strong>\u2013 community-developed and supported software you can use without any license purchases. There are no constraints for customization. <\/li>\n\n\n\n<li><strong><strong>Proprietary tools <\/strong><\/strong>\u2013 licensed software, offering access to a rich set of native functionality out of the box.&nbsp;<\/li>\n\n\n\n<li><strong><strong>Custom solutions <\/strong><\/strong>\u2013 any combination of the above two options within your AQA framework.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>When deciding on your QA automation stack, you should always take into account:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Programming languages (most QA tools are language-specific)&nbsp;<\/li>\n\n\n\n<li>Operating systems (your testers are using)&nbsp;<\/li>\n\n\n\n<li>Platform support (web, mobile, tablet, etc)&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Some of the automated testing tools we recommend at Edvantis are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.selenium.dev\/documentation\/webdriver\/\" target=\"_blank\" rel=\"noreferrer noopener\">Selenium WebDriver<\/a> \u2013 cross-browser web testing framework&nbsp;<\/li>\n\n\n\n<li><a href=\"https:\/\/www.cypress.io\" target=\"_blank\" rel=\"noreferrer noopener\">Cypress<\/a> \u2013 end-to-end JavaScript testing framework&nbsp;<\/li>\n\n\n\n<li><a href=\"https:\/\/github.com\/microsoft\/WinAppDriver\" target=\"_blank\" rel=\"noreferrer noopener\">WinAppDriver<\/a> \u2013 open-source desktop automation testing framework<\/li>\n<\/ul>\n\n\n\n<p>If you need more options, <a href=\"https:\/\/www.npmtrends.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">NPM Trends<\/a> lets you compare individual QA automation tools and test automation frameworks.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-4-decide-on-your-path-to-implementation\">4. Decide on Your Path to Implementation&nbsp;<\/h3>\n\n\n\n<p>After you\u2019ve selected your test framework, you need to decide on the implementation approach. To better illustrate different paths to implementation, we\u2019ll use Selenium.<\/p>\n\n\n\n<p>Similar to other QA automation tools, Selenium supports several test design patterns:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Wrapper methods<\/strong>: In software engineering, a wrapper is a method of encapsulation. Essentially, you create a custom variant of executing a certain function because the original one doesn\u2019t meet all your requirements (e.g. it is too complex to execute or doesn\u2019t perform all aspects of testing you need). Then add this wrapper into your framework.<\/li>\n\n\n\n<li><strong><strong>Abstraction methods: <\/strong><\/strong>Abstraction is a technique software engineers use to hide unnecessary details from the users. In test frameworks, abstracting certain functionalities promotes better code structure and maintainability.&nbsp;<\/li>\n\n\n\n<li><strong>Page Object method<\/strong>: This is the most commonly used method for organizing test cases. Here you denote each web page of an app as a class file \u2014 such that contains only all corresponding web page elements. Then perform tests on these elements. Doing so reduces code duplication and improves test maintenance.&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Again, no method is inherently better or worse \u2014 they are just better suited for different use cases. An experienced <a href=\"https:\/\/www.edvantis.com\/de\/service\/expertise-in-der-qualitaetssicherung\/\" target=\"_blank\" rel=\"noreferrer noopener\">QA testing services provider<\/a> can advise you on the optimal methods.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-5-develop-test-data\">5. Develop Test Data&nbsp;<\/h3>\n\n\n\n<p>Automated testing is heavily-reliant on data. The more data you have \u2014 the better outcomes you\u2019ll gain. However, when you are doing QA at the pre-production stages, you may be lacking meaningful first-hand data to cover all testing scenarios.<\/p>\n\n\n\n<p>Therefore, consider extra test data generations strategies such as:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Synthetic data from automated data generation tools&nbsp;<\/li>\n\n\n\n<li>Mass copy of data from production to testing environment<\/li>\n\n\n\n<li>Mass copy of test data from legacy systems<\/li>\n\n\n\n<li>Manual test data creation&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>Next, you should group all available data by test cases and securely store it in a test data management tool.&nbsp;<\/p>\n\n\n\n<p>Finally, remember that QA automation is a continuous process. You\u2019ll have to work in increments and progressively extend automated test coverage to new test scenarios. It always makes sense to start with several testing scenarios \u2014 and then add extra ones to your framework.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"htoc-how-we-helped-an-enterprise-client-switch-from-manual-to-automation-testing-case-study\">How We Helped An Enterprise Client Switch From Manual to Automation Testing: Case Study&nbsp;<\/h2>\n\n\n\n<p>Now let\u2019s move from theory to practice. In this case study, we\u2019ll show how we helped one of our clients, a software provider for oil &amp; gas companies, progressively implement a QA automation framework.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-the-need-to-automate\">The Need to Automate&nbsp;<\/h3>\n\n\n\n<p>When we began our collaboration, the customer had an in-house manual testing division. However, the team wasn\u2019t coping with the growing workloads assigned to them. This led to a longer time-to-market for new software functionality and costly code reworks post-release.<\/p>\n\n\n\n<p>The testing team spent a lot of time on regression testing in particular \u2014 a strong contender for automation. So the client approached Edvantis\u2019 QA specialists to help with test automation.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-building-a-custom-test-automation-framework\">Building a Custom Test Automation Framework&nbsp;<\/h3>\n\n\n\n<p>The company has a lot of JS applications (web + mobile). To ensure sufficient coverage, we selected <a href=\"https:\/\/webdriver.io\/\" target=\"_blank\" rel=\"noreferrer noopener\">WebdriverIO<\/a> as the pillar element of our QA automation framework. Then augment it with extra customizations\/integrations where needed.&nbsp;<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1077\" height=\"921\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" src=\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/custom-test-automation-framework.svg\" alt=\"custom test automation framework\" class=\"wp-image-5789\"\/><\/figure>\n\n\n\n<p>Tools we used as part of the framework:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><strong>Web:<\/strong><\/strong>\n<ul class=\"wp-block-list\">\n<li>BrowserStack<\/li>\n\n\n\n<li>Selenoid <\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><strong>Mobile:<\/strong><\/strong>\n<ul class=\"wp-block-list\">\n<li>Appium <\/li>\n<\/ul>\n<\/li>\n\n\n\n<li><strong><strong>Additional:<\/strong><\/strong>\n<ul class=\"wp-block-list\">\n<li>Mocha<\/li>\n\n\n\n<li>Chai<\/li>\n\n\n\n<li>Babel<\/li>\n\n\n\n<li>Grunt<\/li>\n\n\n\n<li>Report Portal<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>The client was initially skeptical about the feasibility of automating certain test cases, but we successfully did:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Mobile and tablet app testing (iOS, Android, APIs) automation&nbsp;<\/li>\n\n\n\n<li>UI testing automation&nbsp;<\/li>\n\n\n\n<li>API testing automation&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>During the course of the implementation, we\u2019ve developed a secure repository for storing all test cases (plus supporting data), and a user-friendly portal for checking status reports and generating custom ones. Currently, we are working on extra <a href=\"https:\/\/grafana.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">Grafana <\/a>dashboards \u2014 an application that would enable the company to collect better analytics on test pass\/fail rates and track other metrics.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"htoc-a-robust-test-automation-framework-and-team-upskilling\">A Robust Test Automation Framework and Team Upskilling&nbsp;<\/h3>\n\n\n\n<p>At this time, we keep expanding the framework coverage. Up to date, we achieved <strong>70%-80% automation rates<\/strong> on two major projects and <strong>50% on the mobile test automation framework<\/strong> in particular.&nbsp;<\/p>\n\n\n\n<p>Some projects are not ready for automation yet because of security\/compliance reasons. So in the meantime, we focused on re-training the client&#8217;s manual testers.&nbsp;<\/p>\n\n\n\n<p>In the course of the partnership Edvantis:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Delivered a QA automation training course for a 10+ people team<\/li>\n\n\n\n<li>Helped fully re-train and re-assign two people to the QA automation testing unit&nbsp;<\/li>\n\n\n\n<li>Continues expanding the client\u2019s automated test case database, coverage, team upskilling activities.<\/li>\n<\/ul>\n\n\n\n<p>By choosing to work with our company, the client has not just obtained a new AQA framework and faster time-to-market but also benefited from <a href=\"https:\/\/www.edvantis.com\/de\/blog\/knowledge-sharing-for-outsourcing\/?ref=quuu\" target=\"_blank\" rel=\"noreferrer noopener\">knowledge sharing<\/a> and cross-pollination.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"htoc-to-conclude\">To Conclude&nbsp;<\/h2>\n\n\n\n<p>In our experience, the optimal way to approach QA automation is to start with several test cases for one project, fully codify them into a reference framework, and then progressively add new test cases. <br><br>By working in increments, you get a chance to re-assess your testing cadence and locate gaps (or redundancies) in test cases designs. Doing so can reduce the testing speed without undermining test quality and help you bring more competitive digital products to the market.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.edvantis.com\/de\/kontaktieren-sie-uns\/\" target=\"_blank\" rel=\"noreferrer noopener\"><em>Contact Edvantis<\/em><\/a><em> to receive personalized consultation on QA automation implementation for your company.&nbsp;<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Quality assurance (QA) is the central element of any software development lifecycle (SDLC).\u00a0Whether you develop a new solution or update an existing one, you need to test it for usability, performance, and security among other factors. The better your QA process is, the faster you can schedule new releases. That\u2019s a well-known axiom \u2014 but a tough one to implement without a proper QA automation framework in place. <\/p>\n","protected":false},"author":4,"featured_media":29634,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[147],"tags":[173,201],"taxonomy_industries":[],"class_list":["post-5862","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-projektmanagement","tag-16-min-read-de","tag-lesedauer-16-minuten"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.1.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>QA Automation: How to Design a Proven Framework - Edvantis<\/title>\n<meta name=\"description\" content=\"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"QA Automation: How to Design a Proven Framework - Edvantis\" \/>\n<meta property=\"og:description\" content=\"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/\" \/>\n<meta property=\"og:site_name\" content=\"Edvantis\" \/>\n<meta property=\"article:published_time\" content=\"2022-04-15T06:37:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-14T15:34:22+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/QA-automation_featured-image.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"700\" \/>\n\t<meta property=\"og:image:height\" content=\"394\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Elena Prokopets\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Elena Prokopets\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/\"},\"author\":{\"name\":\"Elena Prokopets\",\"@id\":\"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19\"},\"headline\":\"QA Automation: How to Design a Proven Framework\",\"datePublished\":\"2022-04-15T06:37:00+00:00\",\"dateModified\":\"2023-07-14T15:34:22+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/\"},\"wordCount\":2751,\"image\":{\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png\",\"keywords\":[\"16 min read\",\"Lesedauer: 16 Minuten\"],\"articleSection\":[\"Projektmanagement\"],\"inLanguage\":\"de\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/\",\"url\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/\",\"name\":\"QA Automation: How to Design a Proven Framework - Edvantis\",\"isPartOf\":{\"@id\":\"https:\/\/www.edvantis.com\/de\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png\",\"datePublished\":\"2022-04-15T06:37:00+00:00\",\"dateModified\":\"2023-07-14T15:34:22+00:00\",\"author\":{\"@id\":\"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19\"},\"description\":\"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage\",\"url\":\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png\",\"contentUrl\":\"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png\",\"width\":1300,\"height\":900,\"caption\":\"featured image for post about qa automation Edvantis\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/www.edvantis.com\/de\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"QA Automation: How to Design a Proven Framework\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.edvantis.com\/de\/#website\",\"url\":\"https:\/\/www.edvantis.com\/de\/\",\"name\":\"Edvantis\",\"description\":\"Edvantis | IT Outsourcing &amp; Custom Software Development\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.edvantis.com\/de\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19\",\"name\":\"Elena Prokopets\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/7599502b735425943fffc167b4c9cfb072d028c109ee07501c8711a4fe0e12e7?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/7599502b735425943fffc167b4c9cfb072d028c109ee07501c8711a4fe0e12e7?s=96&d=mm&r=g\",\"caption\":\"Elena Prokopets\"},\"sameAs\":[\"admin\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"QA Automation: How to Design a Proven Framework - Edvantis","description":"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/","og_locale":"de_DE","og_type":"article","og_title":"QA Automation: How to Design a Proven Framework - Edvantis","og_description":"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.","og_url":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/","og_site_name":"Edvantis","article_published_time":"2022-04-15T06:37:00+00:00","article_modified_time":"2023-07-14T15:34:22+00:00","og_image":[{"width":700,"height":394,"url":"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/QA-automation_featured-image.jpg","type":"image\/jpeg"}],"author":"Elena Prokopets","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Elena Prokopets","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#article","isPartOf":{"@id":"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/"},"author":{"name":"Elena Prokopets","@id":"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19"},"headline":"QA Automation: How to Design a Proven Framework","datePublished":"2022-04-15T06:37:00+00:00","dateModified":"2023-07-14T15:34:22+00:00","mainEntityOfPage":{"@id":"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/"},"wordCount":2751,"image":{"@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage"},"thumbnailUrl":"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png","keywords":["16 min read","Lesedauer: 16 Minuten"],"articleSection":["Projektmanagement"],"inLanguage":"de"},{"@type":"WebPage","@id":"https:\/\/www.edvantis.com\/de\/blog\/qa-automation-framework\/","url":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/","name":"QA Automation: How to Design a Proven Framework - Edvantis","isPartOf":{"@id":"https:\/\/www.edvantis.com\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage"},"image":{"@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage"},"thumbnailUrl":"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png","datePublished":"2022-04-15T06:37:00+00:00","dateModified":"2023-07-14T15:34:22+00:00","author":{"@id":"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19"},"description":"We unpack the process and technology components of a robust QA automation framework with overview of main tools and layers.","breadcrumb":{"@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#primaryimage","url":"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png","contentUrl":"https:\/\/www.edvantis.com\/wp-content\/uploads\/2022\/04\/featured-image-for-post-about-qa-automation-Edvantis.png","width":1300,"height":900,"caption":"featured image for post about qa automation Edvantis"},{"@type":"BreadcrumbList","@id":"https:\/\/www.edvantis.com\/blog\/qa-automation-framework\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/www.edvantis.com\/de\/"},{"@type":"ListItem","position":2,"name":"QA Automation: How to Design a Proven Framework"}]},{"@type":"WebSite","@id":"https:\/\/www.edvantis.com\/de\/#website","url":"https:\/\/www.edvantis.com\/de\/","name":"Edvantis","description":"Edvantis | IT Outsourcing &amp; Custom Software Development","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.edvantis.com\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Person","@id":"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/03862f02c845034443f6ed9407d6db19","name":"Elena Prokopets","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.edvantis.com\/de\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/7599502b735425943fffc167b4c9cfb072d028c109ee07501c8711a4fe0e12e7?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/7599502b735425943fffc167b4c9cfb072d028c109ee07501c8711a4fe0e12e7?s=96&d=mm&r=g","caption":"Elena Prokopets"},"sameAs":["admin"]}]}},"_links":{"self":[{"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/posts\/5862","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/comments?post=5862"}],"version-history":[{"count":0,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/posts\/5862\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/media\/29634"}],"wp:attachment":[{"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/media?parent=5862"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/categories?post=5862"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/tags?post=5862"},{"taxonomy":"taxonomy_industries","embeddable":true,"href":"https:\/\/www.edvantis.com\/de\/wp-json\/wp\/v2\/taxonomy_industries?post=5862"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}