AT*SQA Webinars

Testing Using the Cloud - An Introductory Overview

Enjoy the AT*SQA webinar on "Testing Using the Cloud." If you prefer to read the material, the text version is directly below.

Testing Using the Cloud – An Introductory Overview by Amy Reichert-Bowersock

This is an overview of techniques used when testing in the cloud.  

For additional information and a variety of professional-level training courses and certification options from AT*SQA, choose from the menu above.  

Cloud Testing Topics  

We’ll cover the following general topics: 

• Definition of Testing Using the Cloud
• Advantages and Disadvantages of Testing Using the Cloud
• Testing Types and Techniques for Testing Using the Cloud
• On-premise vs Cloud Testing Comparison
• Challenges of Testing Using the Cloud

Definition of Testing Using the Cloud  

Testing using the cloud is using a remote server to access data and services. 

A business can test on-premise using a cloud structure or externally using a service provider.  

The cloud contains grouped service bundles:  SaaS, PaaS, and IaaS. 

SaaS examples:  Zoom, Dropbox, Many of Amazon’s AWS services, Salesforce, Google Apps, Slack to name only a few of thousands of options available. SaaS is software that’s available via a third-party.  

PaaS examples:  Microsoft Azure, SAP Cloud, AWS Lambda, IBM Cloud Foundry – mostly ERP or business management, software system management type companies. PaaS is hardware and software tools available over the internet.  

IaaS examples: AWS EC2, Rackspace, Google Compute Engine (GCE), Digital Ocean IaaS is a cloud-based service, pay-as-you-go for services such as storage, networking, and virtualization.  

TaaS - Testing as a service means a provider provides the testers, and all the testing deliverables, manages the execution and outcome reporting. The risk is making your application and sample data public to test in the cloud. Crowdsourcing is a type of outsourcing by hiring a testing service to supply testers and manage the testing project.  One advantage to using a cloud testing service is simplicity – you only need to provide access to data and the application.  Management of the testing and test personnel is handled by the service.  Additionally, it is performed when scheduled, so testing can occur on a schedule or be randomly planned on an as-needed basis. The cost of testing and test management may be reduced.  

General Categories of Cloud Testing  

Cloud testing has four main or general categories.  Not all of them apply to using a cloud, they apply to define what type of cloud or clouds you plan to use for testing, or for staging/supporting the application once released.

• The whole cloud is testing done against a single, whole cloud.  Data, security, APIs, and application functionality within a single cloud.
• Testing within a cloud defines a cloud provider’s testing of their cloud features.  They’d test each feature and how it performs within their cloud offering.
• Clouds have multiple types – a private cloud is accessible internally only.  A public cloud that is accessible to all members of the public, or by signing up.  For example, Google services like Gmail, Docs, etc.  Once you sign up you have a certain amount of access for using application features and storing data within the public cloud.
• SaaS testing in a cloud is essentially executing application testing of any type (functional, security, performance, etc) using a database in the cloud.

Cloud Testing – Advantages  

TaaS or Testing as a service providers make cloud testing have unique aspects from testing internally. The cost is on a per-use, or as-needed basis so there is testing cost overhead only when needed.  The testers and test creation, execution are managed by the service.  The business simply waits for the reported results of test executions.   

Testers are located anywhere there is a secure internet connection.  A secure connection is not required, but it is preferred.  

Testing becomes more of a scheduled activity, rather than an internally managed group of employees.  

Additionally, there is less server and database management needed by the business’s IT group to support a wide variety of testing platforms and devices.  

Cloud Testing – Disadvantages  

Like anything, there are disadvantages to reducing costs by outsourcing QA testing to a TaaS service.

• QA testers are not part of the development team, so there is less communication between developers and QA testers.  Granted, communication channels exist but communicating back and forth takes more time.
• It may take more time to interpret testing results and defect reports because the tester is on an external team. 
• Test documentation as in test cases, plans, execution reports may not be accessible depending on the work contract.
• Application code that is still in development is released for public access or at least access to testers and test management personnel within the TaaS service provider.
• Test data must be created from production for use in the cloud.  The other option is to allow access to production data on a secured cloud.

Cloud Testing – Testing Types  

Here are a few of the types of testing that benefit from a cloud testing structure.  This is a partial list.  A tester can literally perform any type of testing on a cloud system.  The only challenge or variances are related to data access and connectivity speeds.

• Functional
• System
• Load/Stress
• Performance
• Security
• Multi-tenancy
• Compatibility

Cloud testing is most useful for these types of testing largely due to the ability to test across platforms, configurations, devices, and browsers.   

Testing Multi-Tenancy and Compatibility are increased when using the cloud.  Why?  Because multiple testers can log in and test.  Additionally, when performing Compatibility testing, it’s more effective when the testers can access a managed group of real devices and platforms directly.  

On-Premise vs Cloud Testing  

For On-Premise, we have:

• Servers in a known location, like a data center or housed and managed internally by the business.
• Database storage is in a known location and managed internally by the business.
• Application interaction and transactions occur between known servers and databases that are internally managed.
• For Cloud Testing, we have:
• Cloud service provider houses the servers and databases wherever.  They may be located anywhere around the globe. 
• Database, servers and transactions are managed by the cloud provider.
• All application interactions and transactions occur between the cloud and calls coming from the application (business).
• Accessing data in the cloud means authenticating securely and interacting with the remote cloud.  No internal management. 

Keep in mind, Cloud testing can be internal, or used with on-site QA teams as well as external teams or TaaS service providers.  

Challenges – Cloud Testing  

• Cloud testing truly tests the architecture including the application, network, and infrastructure which may require additional tester training.
• Setting it up initially may be challenging depending on the cloud provider, cloud model, and what is configurable for testing.
• QA testers accustomed to freely bouncing servers or manipulating databases to simulate testing must adjust to testing limitations. In other words, testers may not be able to control the test environment as they would on-premise.  This may be a good thing but takes some adjustment in expectations.
• Clouds may be shared, so depending on the traffic on the cloud, performance issues may occur.

Thank you for reading the "Testing Using the Cloud overview". Choose from the menu to access more professional QA training and certification options from AT*SQA.