Bending the Dynamic vs Static Language Tradeoff

Standards Enforcement

Download Visual Studio 2005 Retired documentation from Official Microsoft Download Center
Webrat - Ruby-based utility to enable quick development of web app acceptance tests. Supports both JavaScript and Typescript. Again continuing with the word processing example, a positive test for the printing function might be to print a document containing both text as well as graphics to a printer that is online, filled with paper and for which the correct drivers are installed. Pro version supports replaying and keep-alive sessions to ensure that you get the exact same number of connections every time. The readonly keyword is different from the const keyword. For a list of languages with static type checking, see the category for statically typed languages. This runtime type information RTTI can also be used to implement dynamic dispatch , late binding , downcasting , reflection , and similar features.

Surface Pro

Software testing

Software components may be integrated in an iterative way or all together "big bang". Normally the former is considered a better practice since it allows interface issues to be located more quickly and fixed. Integration testing works to expose defects in the interfaces and interaction between integrated components modules. Progressively larger groups of tested software components corresponding to elements of the architectural design are integrated and tested until the software works as a system.

System testing tests a completely integrated system to verify that the system meets its requirements. Operational acceptance is used to conduct operational readiness pre-release of a product, service or system as part of a quality management system. OAT is a common type of non-functional software testing, used mainly in software development and software maintenance projects.

This type of testing focuses on the operational readiness of the system to be supported, or to become part of the production environment. Functional testing within OAT is limited to those tests that are required to verify the non-functional aspects of the system.

In addition, the software testing should ensure that the portability of the system, as well as working as expected, does not also damage or partially corrupt its operating environment or cause other processes within that environment to become inoperative. Different labels and ways of grouping testing may be testing types, software testing tactics or techniques.

Most software systems have installation procedures that are needed before they can be used for their main purpose. Testing these procedures to achieve an installed software system that may be used is known as installation testing. A common cause of software failure real or perceived is a lack of its compatibility with other application software , operating systems or operating system versions , old or new , or target environments that differ greatly from the original such as a terminal or GUI application intended to be run on the desktop now being required to become a Web application , which must render in a Web browser.

For example, in the case of a lack of backward compatibility , this can occur because the programmers develop and test software only on the latest version of the target environment, which not all users may be running.

This results in the unintended consequence that the latest work may not function on earlier versions of the target environment, or on older hardware that earlier versions of the target environment were capable of using. Sometimes such issues can be fixed by proactively abstracting operating system functionality into a separate program module or library.

Sanity testing determines whether it is reasonable to proceed with further testing. Smoke testing consists of minimal attempts to operate the software, designed to determine whether there are any basic problems that will prevent it from working at all. Such tests can be used as build verification test.

Regression testing focuses on finding defects after a major code change has occurred. Specifically, it seeks to uncover software regressions , as degraded or lost features, including old bugs that have come back.

Such regressions occur whenever software functionality that was previously working correctly, stops working as intended. Typically, regressions occur as an unintended consequence of program changes, when the newly developed part of the software collides with the previously existing code. Common methods of regression testing include re-running previous sets of test cases and checking whether previously fixed faults have re-emerged. The depth of testing depends on the phase in the release process and the risk of the added features.

They can either be complete, for changes added late in the release or deemed to be risky, or be very shallow, consisting of positive tests on each feature, if the changes are early in the release or deemed to be of low risk. Regression testing is typically the largest test effort in commercial software development, [46] due to checking numerous details in prior software features, and even new software can be developed while using some old test cases to test parts of the new design to ensure prior functionality is still supported.

Alpha testing is often employed for off-the-shelf software as a form of internal acceptance testing before the software goes to beta testing.

Beta testing comes after alpha testing and can be considered a form of external user acceptance testing. Versions of the software, known as beta versions , are released to a limited audience outside of the programming team known as beta testers. The software is released to groups of people so that further testing can ensure the product has few faults or bugs.

Beta versions can be made available to the open public to increase the feedback field to a maximal number of future users and to deliver value earlier, for an extended or even indefinite period of time perpetual beta. Functional testing refers to activities that verify a specific action or function of the code. These are usually found in the code requirements documentation, although some development methodologies work from use cases or user stories. Functional tests tend to answer the question of "can the user do this" or "does this particular feature work.

Non-functional testing refers to aspects of the software that may not be related to a specific function or user action, such as scalability or other performance , behavior under certain constraints , or security. Testing will determine the breaking point, the point at which extremes of scalability or performance leads to unstable execution. Non-functional requirements tend to be those that reflect the quality of the product, particularly in the context of the suitability perspective of its users.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. Destructive testing attempts to cause the software or a sub-system to fail. It verifies that the software functions properly even when it receives invalid or unexpected inputs, thereby establishing the robustness of input validation and error-management routines.

Various commercial non-functional testing tools are linked from the software fault injection page; there are also numerous open-source and free software tools available that perform destructive testing. Performance testing is generally executed to determine how a system or sub-system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.

Load testing is primarily concerned with testing that the system can continue to operate under a specific load, whether that be large quantities of data or a large number of users.

This is generally referred to as software scalability. The related load testing activity of when performed as a non-functional activity is often referred to as endurance testing. Volume testing is a way to test software functions even when certain components for example a file or database increase radically in size.

Stress testing is a way to test reliability under unexpected or rare workloads. Stability testing often referred to as load or endurance testing checks to see if the software can continuously function well in or above an acceptable period. There is little agreement on what the specific goals of performance testing are. The terms load testing, performance testing, scalability testing , and volume testing, are often used interchangeably.

Real-time software systems have strict timing constraints. To test if timing constraints are met, real-time testing is used. Usability testing is to check if the user interface is easy to use and understand. It is concerned mainly with the use of the application. Accessibility testing may include compliance with standards such as:. Security testing is essential for software that processes confidential data to prevent system intrusion by hackers.

The International Organization for Standardization ISO defines this as a "type of testing conducted to evaluate the degree to which a test item, and associated data and information, are protected so that unauthorised persons or systems cannot use, read or modify them, and authorized persons or systems are not denied access to them.

Testing for internationalization and localization validates that the software can be used with different languages and geographic regions. The process of pseudolocalization is used to test the ability of an application to be translated to another language, and make it easier to identify when the localization process may introduce new bugs into the product.

Globalization testing verifies that the software is adapted for a new culture such as different currencies or time zones. Actual translation to human languages must be tested, too. Possible localization and globalization failures include:. Development Testing is a software development process that involves the synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs.

Development Testing aims to eliminate construction errors before code is promoted to other testing; this strategy is intended to increase the quality of the resulting software as well as the efficiency of the overall development process. Depending on the organization's expectations for software development, Development Testing might include static code analysis , data flow analysis, metrics analysis, peer code reviews, unit testing, code coverage analysis, traceability, and other software testing practices.

Customers are routed to either a current version control of a feature, or to a modified version treatment and data is collected to determine which version is better at achieving the desired outcome.

In concurrent testing, the focus is on the performance while continuously running with normal input and under normal operational conditions, as opposed to stress testing, or fuzz testing. Memory leak, as well as basic faults are easier to find with this method. In software testing, conformance testing verifies that a product performs according to its specified standards. Compilers, for instance, are extensively tested to determine whether they meet the recognized standard for that language.

A common practice in waterfall development is that testing is performed by an independent group of testers. However, even in the waterfall development model, unit testing is often done by the software development team even when further testing is done by a separate team.

In contrast, some emerging software disciplines such as extreme programming and the agile software development movement, adhere to a " test-driven software development " model. In this process, unit tests are written first, by the software engineers often with pair programming in the extreme programming methodology.

The tests are expected to fail initially. Each failing test is followed by writing just enough code to make it pass. This means the test suites are continuously updated as new failure conditions and corner cases are discovered, and they are integrated with any regression tests that are developed.

Unit tests are maintained along with the rest of the software source code and generally integrated into the build process with inherently interactive tests being relegated to a partially manual build acceptance process. The ultimate goals of this test process are to support continuous integration and to reduce defect rates.

This methodology increases the testing effort done by development, before reaching any formal testing team. In some other development models, most of the test execution occurs after the requirements have been defined and the coding process has been completed.

Although variations exist between organizations, there is a typical cycle for testing. The same practices are commonly found in other development models, but might not be as clear or explicit. Many programming groups [ Like whom?

There are many frameworks [ specify ] to write tests in, and continuous integration software will run tests automatically every time code is checked into a version control system. While automation cannot reproduce everything that a human can do and all the ways they think of doing it , it can be very useful for regression testing.

However, it does require a well-developed test suite of testing scripts in order to be truly useful. Program testing and fault detection can be aided significantly by testing tools and debuggers. There are a number of frequently used software metrics , or measures, which are used to assist in determining the state of the software or the adequacy of the testing. Based on the amount of test cases required to construct a complete test suite in each context i.

It has been proved that each class is strictly included in the next. For instance, testing when we assume that the behavior of the implementation under test can be denoted by a deterministic finite-state machine for some known finite sets of inputs and outputs and with some known number of states belongs to Class I and all subsequent classes.

However, if the number of states is not known, then it only belongs to all classes from Class II on. If the implementation under test must be a deterministic finite-state machine failing the specification for a single trace and its continuations , and its number of states is unknown, then it only belongs to classes from Class III on.

Testing temporal machines where transitions are triggered if inputs are produced within some real-bounded interval only belongs to classes from Class IV on, whereas testing many non-deterministic systems only belongs to Class V but not all, and some even belong to Class I.

The inclusion into Class I does not require the simplicity of the assumed computation model, as some testing cases involving implementations written in any programming language, and testing implementations defined as machines depending on continuous magnitudes, have been proved to be in Class I. Other elaborated cases, such as the testing framework by Matthew Hennessy under must semantics, and temporal machines with rational timeouts, belong to Class II.

A software testing process can produce several artifacts. The actual artifacts produced are a factor of the software development model used, stakeholder and organisational needs. Several certification programs exist to support the professional aspirations of software testers and quality assurance specialists.

Note that a few practitioners argue that the testing field is not ready for certification, as mentioned in the Controversy section. Some of the major software testing controversies include:.

It is commonly believed that the earlier a defect is found, the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found.

With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time. The "smaller projects" curve turns out to be from only two teams of first-year students, a sample size so small that extrapolating to "smaller projects in general" is totally indefensible. The GTE study does not explain its data, other than to say it came from two projects, one large and one small.

The paper cited for the Bell Labs "Safeguard" project specifically disclaims having collected the fine-grained data that Boehm's data points suggest.

The IBM study Fagan's paper contains claims that seem to contradict Boehm's graph and no numerical results that clearly correspond to his data points. Boehm doesn't even cite a paper for the TRW data, except when writing for "Making Software" in , and there he cited the original article.

There exists a large study conducted at TRW at the right time for Boehm to cite it, but that paper doesn't contain the sort of data that would support Boehm's claims. Software testing is used in association with verification and validation: The terms verification and validation are commonly used interchangeably in the industry; it is also common to see these two terms defined with contradictory definitions.

The contradiction is caused by the use of the concepts of requirements and specified requirements but with different meanings. In the case of IEEE standards, the specified requirements, mentioned in the definition of validation, are the set of problems, needs and wants of the stakeholders that the software must solve and satisfy.

And, the products mentioned in the definition of verification, are the output artifacts of every phase of the software development process. These products are, in fact, specifications such as Architectural Design Specification, Detailed Design Specification, etc.

The SRS is also a specification, but it cannot be verified at least not in the sense used here, more on this subject below. But, for the ISO , the specified requirements are the set of specifications, as just mentioned above, that must be verified. A specification, as previously explained, is the product of a software development process phase that receives another specification as input. A specification is verified successfully when it correctly implements its input specification. All the specifications can be verified except the SRS because it is the first one it can be validated, though.

Both the SRS and the software must be validated. The SRS can be validated statically by consulting with the stakeholders. Nevertheless, running some partial implementation of the software or a prototype of any kind dynamic testing and obtaining positive feedback from them, can further increase the certainty that the SRS is correctly formulated.

On the other hand, the software, as a final and running product not its artifacts and documents, including the source code must be validated dynamically with the stakeholders by executing the software and having them to try it. Thinking this way is not advisable as it only causes more confusion. It is better to think of verification as a process involving a formal and technical input document. Software testing may be considered a part of a software quality assurance SQA process.

They examine and change the software engineering process itself to reduce the number of faults that end up in the delivered software: What constitutes an acceptable defect rate depends on the nature of the software; A flight simulator video game would have much higher defect tolerance than software for an actual airplane.

Although there are close links with SQA, testing departments often exist independently, and there may be no SQA function in some companies. Software testing is an activity to investigate software under test in order to provide quality-related information to stakeholders.

By contrast, QA quality assurance is the implementation of policies and procedures intended to prevent defects from reaching customers. From Wikipedia, the free encyclopedia. Graphical user interface testing. Exception handling and Recovery testing. Capability Maturity Model Integration and Waterfall model.

Verification and validation software and Software quality control. Retrieved November 22, Retrieved November 21, Retrieved December 8, Testing Computer Software, 2nd Ed. New York, et al.: John Wiley and Sons, Inc. Best Practices in Software Management. International Software Testing Qualifications Board. Retrieved December 15, Principle 2, Section 1. Lessons Learned and Practical Implications. National Institute of Standards and Technology. Retrieved December 19, CIO Review India ed.

Retrieved December 20, Communications of the ACM. The Art of Software Testing. John Wiley and Sons. Foundations of Software Testing. Verification and Validation in Scientific Computing. Introduction to Software Testing. Becoming an Effective and Efficient Test Professional.

Software Testing 2nd ed. Department of Computer Science, University of Sheffield. Retrieved January 2, Retrieved August 19, How to Become a Software Tester. Retrieved January 5, Helsinki University of Technology. Based on an expressive DSL, the scenarios are self explanatory.

They are easy to maintain and can be kept in a version control system. Developed based on fiber, a user-level thread implementation in Java. The tool is useful in scenarios where the requests have high response times. Many existing tools are developed for scenarios with high request rate and quick response times, and fail to keep up with load generation when server response times are high, as the number of outstanding requests becomes very large.

Features include regular expression extractor, delay timer which helps in running complex test scenarios. Tsung - Free open-source multi-protocol distributed load testing tool supported by Process-One. SSL is also supported. XML configuration system; several sessions can be used to simulate different type of users. Dynamic sessions can be described in XML. User think-times and the arrival rate can be randomized using a probability distribution.

Loadster - Load testing tool from Brickyard Technologies, Inc. Execute multiple scripts simultaneously, with multiple virtual user populations, ramp user load up and down with almost infinite possibilities, simulate network bandwidth for virtual users, adjust or randomize wait times, and add thresholds to fine-tune test inputs. Generate load from your own hardware, or on-demand cloud engines running on 5 continents.

Load Complete - Load, performance, stress, and scalability test tool for mobile, websites, and web apps from SmartBear Software.

Can customize different properties for the load test via the tool's UI. Requires Win and IE. Sandstorm - Load testing tool from Impetus Technologies Inc. Integrated resource monitoring for most of the popular web, app and db servers.

Can capture real production workload for playback simulating thousands of users for top down testing or back-end testing at the component layer. Integrated performance management and root cause analysis system automatically collects performance metrics across load test infrastructure and applies cross-component correlation to identify performance degradations and bottlenecks. Load2Test Controller and Test nodes: XLT Cloud Service available. Includes Chrome and Firefox plugins that collect client performance data right in the browser.

Free; paid support and training is offered. Blazemeter - Self-service, on-demand, cloud-based load testing. Simulate any user scenario for webapps, websites, mobile apps or web services. Launch a single dedicated server or a cluster of Set geo locations from among choices worldwde. Set up tests, access test results, view test reports, compare past test reports and more, all on a unitary console. Generate traffic using public cloud providers or install the on-premise load generator software on your own machines and test behind the firewall on your internal network.

Also has free tools and resources for tips and tricks to optimize website and app performance. Tests for web and mobile can be built using the tool in such a way as to simulate a large number of different users with unique logins and different tasks. Neustar Web Performance - On-demand, self-service, pay-as-you-go service from Neustar enables simulation of large volumes of real browsers hitting a website.

Browser screen shots of errors included in reports. Access to distributed network of load generator nodes; can generate load from up to 10 different locations simultaneously. Record user scenarios using Chrome extension or proxy recorder. Multi-Mechanize - Web performance and load testing framework in python; open source by Corey Goldberg. AppLoader - Load testing app from NRG Global for web and other applications accessible from a Windows desktop; generates load from the end user's perspective.

Protocol independent and supports a wide variety of enterprise class applications. Integrates with their Chroniker monitoring suite so results of load testing can be correlated with system behavior as load is increased. Runs from Win platforms. Import recorded browsing sessions made with Fiddler, which are then used to generate a C class that can be used directly for tests or modified as needed. Capabilities include handling of Ajax.

Activities of each virtual client are logged and collected statistics include information about: StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd.

Advanced user journey modeling, scalable load, system resources monitors and results analysis. Orchestrate activities of a test script in many processes across many machines, using a graphical console application.

Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. It's internal engine is based on Grinder. Consists of a a Controller that provides a web interface for performance testing, coordinates test processes, collates and displays test statistics, lets user create and modify scripts and b an Agent that runs processes and threads that put loads on target machines when running in agent mode, monitors target system performance when running in monitor mode.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Can design complex scenarios to handle real world applications.

Clear and comprehensive reports and test results. Load test from private lab and from the cloud. Compatibility with mobile applications. Free edition available also. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool.

Source code in C. Visual Studio Test Professional - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio. Includes Load and Web Performance Testing capabilities.

OpenLoad Tester - Web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more.

Enables testing a web server with a configurable number of concurrent simulated users. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.

OpenSTA binaries available for Windows. Records and allows viewing of exact bytes flowing between browser and server; no scripting required. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. Generate loads locally or in the cloud.

Also available is Legion , a free open source load testing tool based on Selenium and javascript. Includes tools for root cause analysis and real-time reporting. Test scripting via visual tool or Javascript with enhanced load testing objects and methods. IDE offers correlation, parameterization, response validation, messaging, native JavaScripting and debugging. Handles mobile load testing including ability to record native or browser-based apps directly from the mobile handset.

Can integrate with selenium. Correlation engine handles both server-side such as session id and client-side such as time, date dynamic values - replaced automatically to create unique values for each script run.

Run load generators in the cloud - no special licenses needed. SilkPerformer - Enterprise-class load-testing tool from Microfocus formerly from Borland, formerly from Segue. Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

This section is oriented to tools that focus on client-side performance of web pages; sometimes this is referred to as 'page performance testing' or 'front end performance testing'.

Google Page Speed Insights - Free online tool from Google for page speed analysis - fetches a url twice, once with a mobile user-agent, and once with a desktop-user agent, and measures how the page can improve its performance for 1 above-the-fold load: Since the performance of a network connection varies considerably, only the network-independent aspects of page performance are considered: Suggestions for improvement are then provided, in priority order.

ShowSlow - An open source tool that helps monitor various website performance metrics over time by collecting web performance data and showing it in aggregated form for better business understanding. Downloadable version and on-the-web versions available. Dotcom-Monitor - Web site page performance and monitoring services utilize multiple worldwide locations. Collects analytics re page speed and performance. When an issue is detected, a traceroute is performed and a screen capture and code snippet are saved for your review.

Also provides longer term trend analysis. The online version is an industry collaboration with various companies providing the testing infrastructure for testing a site from across the globe. Provides web page content analysis, performance analysis, various performance scores.

API's available for scripting. Includes test results from other tools such as Google Page Speed and Yahoo!

YSlow, checks one page at a time and provides page preformance analysis report and suggestions for improvement. PageSpeed Tools - A family of tools from Google, designed to help optimize website performance. PageSpeed Analysis tools enable web-based analysis of a page and provides feedback to help identify performance suggestions and best practices that can be applied to a site, and PageSpeed Optimization tools can help automate the process.

YSlow - Open source page speed analysis tool originally from Yahoo analyzes web pages and explains why they're slow based on rules for high performance web sites. Available as extension for Firefox, Chrome, Safari, Opera; as bookmarklet on mobile; nodejs: Command line for HAR files powered by Node.

JS Server powered by Express. Generates a grade for each rule and an overall grade, lists suggested specific changes to improve performance, calculates total size of page for empty and primed cache scenarios, cookie info. Can also view HTTP response headers for any component. Script user actions up to 20 steps with easy to use recorder. Monitor any application, immediately, from multiple global monitors. Store high-detail 5-minute checks for 30 days, and trending performance data for up to a year.

Hive CI - Open source tool from BBC to enable connecting your physical devices to a hive and running tests on the same platforms as your users. Hive CI is a CI system built around devices, developed at the BBC from a desire to have a CI system that could understand and manage devices, run tests without manual intervention, and collate and interpret results without having to resort to spreadsheets. For manual or automated testing.

The mobile handset bank includes devices stationed in the US and many other countries and the agnostic platform hosts a diverse portfolio of worldwide carriers and manufacturers. Any action that can be performed on a mobile device in-hand can be replicated on devices in the cloud. Switch between real global carrier networks and Wifi; capture test results, videos, and screenshots.

Automation capabilities provide flexible scripting that enables building of test scripts that work across multiple devices for faster test automation. Public cloud, private cloud, or on-premise. Device planner tool to help prioritize device choices. EarlGrey - Open source native iOS white box UI automation test framework that enables you to write clear, concise tests.

Access enhanced synchronization features to help to ensure that the UI is in a steady state before actions are performed; automatically synchronizes with the UI, network requests, and various queues; but still allows you to manually implement customized timings, if needed. Provides APIs that allow searching for elements in scrollable containers, regardless of the amount of scrolling required. OpenSTF - STF or Smartphone Test Farm is an open source web application for controlling, managing, and debugging smartphones, smartwatches and other gadgets remotely, from the comfort of your browser.

Remote control of any device in realtime, using your mouse and keyboard, or even via smartphone. Capabilities also include ability to monitor which devices are connected, who is using which device, and search devices by any specification. Do realtime testing on more than one device, just from your browser. Appdiff - Web-based automated regression testing service for mobile apps from Appdif Inc. Requires uploading your app. Appdif bots walk through each new build, checking that signin, menus, buttons, text boxes, all work the same as previous builds.

Automatically analyzes every transition in your app's user experience for performance issues and provides pictures and timing information for all areas of app. Kobiton - Mobile cloud platform that enables users to perform manual or automated testing on cloud-based physical iOS and Android devices.

Web, app, or hybrid testing. You can install your app and it will be automatically uninstalled after the test session but the test history session name, description, screenshots, log will be retained, or can be deleted. The APIs enable operations such as opening the Settings menu or the app launcher in a test device. Oriented to writing black box-style automated tests, where the test code does not rely on internal implementation details of the target app. Firebase Test Lab for Android - Free service from Google to test your mobile app on a wide variety of devices and device configurations hosted in a Google data center.

Uses real production devices that are flashed with updated Android API levels or locale settings that you specify. Captures logs, creates an "activity map" that shows a related set of annotated screenshots, and creates a video from a sequence of screenshots to show you the simulated user operations that it performed.

Even if you haven't written any test code for your app, Test Lab can exercise your app automatically, looking for crashes. Robo test analyzes the structure of your app's user interface and then explores it, automatically simulating user activities. If you have written instrumentation tests for your app, Test Lab can also run those tests. Integrated with Android Studio, the Firebase console, and the gcloud command line. AWS Device Farm - App testing service from Amazon Web Services that lets you test and interact with Android, iOS, and web apps on many physical devices at once, or reproduce issues on a device in real time.

View video, screenshots, logs, and performance data. Gesture, swipe, and interact with devices in real time, directly from your web browser. Run the AWS built-in test suite no scripting required or customize your tests by selecting from open-source test frameworks like Appium, Calabash, Espresso, etc.

Firebase Performance Monitoring - Tool from Google that helps you to gain insight into the performance characteristics of iOS and Android apps. You use the Performance Monitoring SDK to collect performance data from your app, and then review and analyze that data in the Firebase console. You don't need to write any code before your app starts monitoring several critical aspects of app performance: Performance metrics broken down by country, device, app version, and OS level.

Silk Mobile - Functional mobile test tool from Microfocis that supports all the major mobile platforms, including iOS and Android. Using the innovative recorder, tests can be created in minutes. NET scripts to enable developer-powered functions such as debugging or refactoring. The Silk Mobile bundle solution includes Silk Test for functional and reliable test automation across platforms and devices; Silk Performer for performance and load testing and Silk Central, a unified platform to design, plan, execute, and track all functional and performance testing practices across devices and platforms.

Automate-On - Automate functional, visual and performance testing of web, mobile applications in multiple Browsers, Devices, OS and Screen Resolutions. Executes test scripts on cloud machines, mobile devices and removes the need for customers to procure, maintain computer systems and mobile devices for testing. Selendroid - Open source test automation framework for native or hybrid Android apps and the mobile web.

Tests are written using the Selenium 2 client API. Can be used on emulators and real devices and can be integrated as a node into the Selenium Grid for scaling and parallel testing. Includes a built in Inspector to simplify test case development. Includes ios-driver Inspector to examine native app elements, similar to Firebug. Can be used as a Selenium grid node - run tests in parallel on the same architecture as for the web.

Enables automation by leveraging the iOS accessibility attributes. Builds and performs the tests using a standard XCTest testing target. Reports on issues found and suggest mitigation approaches.

Enables manual and automated testing on hundreds of different models of real iOS and Android smartphones and tablets.

After each test run all apps and data are wiped from devices and they are automatically re-initialized. Supports Selenium Webdriver, Jenkins. Reports include device specifications, logs and screenshots.

Frank - Open source framework for writing structured text iOS app tests using Cucumber and have them execute against your iOS application; from Thoughtworks.

Run tests on both the Simulator and Device. Enables stress-testing of apps that you are developing, in a random yet repeatable manner. If the app crashes or receives any sort of unhandled exception, or if it generates an application not responding error, the Monkey will stop and report the error.

MonkeyRunner - Free tool from Google provides a python API for writing programs that control an Android device or emulator from outside of Android code. Can write a Python program that installs an Android application or test package, runs it, sends keystrokes and touch events to it, takes screenshots of its user interface, and stores screenshots on the workstation.

Can apply one or more test suites across multiple devices or emulators. You can physically attach all the devices or start up all the emulators or both at once. Can be extended with plugins. Android Lint - Free downloadable static code analysis tool from Google that checks your Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization.

Runs from command line or Android Studio. Calabash - Free open source framework enables writing and executing automated acceptance tests of mobile apps using Cucumber and Ruby; from Xamarin Inc.

Cross-platform, supporting Android and iOS native apps. Actions can be gestures, assertions, screen shots. Xamarin Test Cloud - Provides a locally executed, object-based scripting environment for imitating and automating actions a real user would take through a mobile app on iOS or Android, using a test device cloud with over 2, real devices to test on.

Test scripts can run in parallel on hundreds of devices at a time. Share code for cross-platform tests between iOS and Android. Screenshots and video playback for every step of every test; performance data memory, CPU, duration, etc. Integrates with any CI system. Includes details about the devices that your apps run on including information such as whether a crash only happens on a specific model or generation of a device, if app only crashes in landscape mode, whether the proximity sensor is always on, is it a memory issue, an issue with specific versions, etc.

Tool set includes 'Beta by Crashlytics' for managing and distributing beta apps via a single, cross-platform toolset for iOS and Android, including tracking testers progress and issues.

Also includes 'Answers' kit which provides critical performance metrics on your app, detailed growth and engagement indicators, etc based on the set of core events and actions of most interest. Included as part of the 'Fabric' toolset; as of being integrated with Firebase, Google's mobile development platform. Requires the Ubertesters SDK which can be integrated with many frameworks used for cross-platform development. Capabilities include in-app bug editing, marking, reporting and user feedback; trcking of all testing sessions in real-time sorted by devices or testers; real time reporting of status of a device and whether it is active, closed or suspended, along with additional device-specific session information and logs; configurable as to which data is captured and conditional logic for filtering.

Can integrate with many 3rd party bug tracking systems or use the included bug tracker. Multiple channels to send apps to testers - testers can get The TestFairy app and use it to download and install all the apps they were invited to test; web app for those who want to use an app without installing one on their device; can set your project to work in 'strict mode' that will require testers to login before they download your app; or manage via enterprise suite that can be installed on a private cloud on many AWS locations; or where enhanced corporate security is needed, can be fully installed in your lab.

Catches any crash and posts it directly to your bug system together with a video that shows what happened prior to crash including CPU metrics, memory, GPS, device info, logs, and crash reports. Testers can then download and install the app and then create support requests, file bugs, or post feedback right from within your app. Can send crash reports automatically or with user interaction or add custom metadata and log files.

Reports include metrics showing which devices were tested, which testers used the app for how long, which language was tested, etc. Data export API for connecting to your own or third-party services.

Mobile device testing cloud for unlimited device concurrency with thousands of real Android and iOS device models. Container-based infrastructure enables scaling local tests with your own frameworks in the TestDroid device cloud. Enables optimization of DevOps toolchains with out-of-the-box integrations or use TestDroid's APIs to connect your own services for alerting, bug tracking, continuous integration and delivery. Available via public cloud, private cloud, or on-premise.

The image-based UI testing approach can reduce the effort of cross-device and cross-platform mobile software testing. Monkop - A cloud-based automated iOS and Android testing service using real devices, provides insights about performance, security, usability, and functionality over a large device lab containing representative brands, OS versions, screen sizes, and configurations. Requires only an upload of your app. Utilizes automatic learning, monkey testing and application disassembly techniques in order to run different levels of tests on different devices.

Reports include response time and resource consumption cpu, memory, data transfer, battery, etc. Can also run your own automation scripts for each device. The object search engine supports exact and fuzzy matching algorithms to identify test objects in the UI, even in case of partial or approximate matching, misspellings and synonyms or if the objects have changed since test creation.

Support for keyword-driven testing through Excel spreadsheets and XML files; offers a rich set of built-in keywords to rapidly develop robust test scripts. Open source version written in java from is available on Sourceforge.

Can be run either as a standalone tool or within Xcode; intended to be run in tandem with a build of a codebase. Roboelectric - An open source Android unit test framework that modifies Android SDK classes so you can test your Android app inside the JVM on your workstation in seconds, without the overhead of an emulator. Tests can be executed on multiple local devices via USB and Wi-Fi, or on devices hosted by cloud-based mobile testing partners.

Multi-touch gestures, access to the physical device buttons, and command-line execution are fully supported. Image recognition allows for testing of standard apps as well as games with fast, 3D, interactive graphics. A small footprint communication client is placed on the mobile device. Appium is a server written in Node. Can be used on both emulators as well as real devices and covers visual testing, functionality testing and speed performance. Digital Assurance Lab enables web and mobile app testing with access to a centralized hub of desktop browsers, real iOS and Android devices, and simulators; available as Software-as-a-Service SaaS or as an on-premise deployment.

Tests can run singlyu or in parallel. SeeTestMobile incorporates image recognition and self-learning algorithms. Test recording can take place utilizing real devices - plug real device in to desktop via USB. Utilizes self-learning diagnostic and matching algorithms and a modular self-enhancing image recognition technology.

Editable scripts using included IDE. Free and paid versions available. Supports a wide variety of technologies, platforms, integrations, and browsers. Directly record tests on your device. The IDE includes test project management, integration of all Ranorex tools Recorder, Repository, Spy , intuitive code editor, code completion, debugging, and watch monitor.

Endpoint panel UI provides a central command center to set-up and manage endpoints as well as configure their environments.

Utilize Ranorex Agents on remote machines to deploy multiple Ranorex tests for remote execution in different environments, using different system configurations and operating systems. Tests are written in Objective C, allowing for maximum integration with code while minimizing layers to build.

Integrates directly into iOS app, so no need to run additional web server or install additional packages. Automation done using tap events where possible. Can define test tasks in simple javascript arrays, and have them execute with helper methods.

Many testing, profiling, and analysis capabilities include enabling easy creation of ad-hoc test harness by recording and playback of user interactions, OpenGL ES for tracking iPhone graphics performance, memory allocation monitoring, Time Profiler on iOS for collecting samples with very low overhead, complete System Trace for insight into how all system processes interact, more.

Also in XCode is iOS Simulator, which enables running an app similar to the way it would run in an actual iOS device; can check that network calls are correct, and that views change as expected when phone rotates; can simulate touch gestures by using the mouse. CQ Lab - The Continuous Quality Lab CQ Lab from Perfecto Mobile is a cloud-based web and mobile app testing platform made up of solutions that enhance building, testing, and optimizing monitor app usability and performance.

Perform side by side functional and real-user condition testing across thousands of devices. Digital Test Coverage Optimizer - Tool from Perfecto Mobile to help select devices to test your app s against - generate a prioritized list of the mobile devices you should test against. Select your target location s , device type s and OSs, and the Optimizer will do the rest.

The test coverage grader helps build a custom mobile app test strategy. Can integrate with Maven, Gradle or Ant to run tests as part of continuous integration. The tool can either fuzz a single component or all components. It works well on Broadcast receivers, and average on Services.

For Activities, only single Activities can be fuzzed, not all them. Instrumentations can also be started using this interface, and content providers are listed, but are not an Intent based IPC mechanism. Provides a unified view of mobile and Web performance and availability. Utilizes thousands of different 'mobile devices': Mobile nodes are a globally distributed set of computers connected to wireless carrier networks via attached wireless modems and provide a realistic measure of the mobile Web experience.

Supports all major phone platforms. Render with Chrome Headless, Phantom and Slimer. Use as a standalone global app, a standalone local npm script or import into your node app. Depicted shows when any visual, perceptual differences are found. Includes a local command-line tool for doing perceptual diff testing; an API server and workflow for capturing webpage screenshots and automatically generating visual perceptual difference images; a workflow for teams to coordinate new releases using pdiffs; a client library for integrating the server with existing continuous integration.

It compares this snapshot to a "reference image" stored in your source code repo and fails the test if the two images don't match. Gemini - Open source utility for regression testing the visual appearance of web pages. Can test separate sections of a web page; can include the box-shadow and outline properties when calculating element position and size; can ignore some special case differences between images rendering artifacts, text caret, etc.

Works with multiple browser types. Kantu Web Automation - Test automation tool from a9t9 Software GmbH; enables automation of any website by taking screenshots. Includes a command-line interface and API to automate more complicated tasks and integrate with other programs or scripts.

For Win, Mac, Linux. Checks your current layout constantly against a reference image you have provided in the past. If your layout breaks or simply changes - CSS Critic can't tell your tests fail. For Firefox and Chrome only. Takes screenshots captured by CasperJS and compares them to baseline images using Resemble. PhantomCSS then generates image diffs to help you find the cause.

By James Cryer and Huddle development team. Applitools Eyes - Automated cross-browser visual web and mobile testing tool from Applitools with an advanced image-matching engine. Visual self-explanatory logs; visual test playback. Uses a headless browser to create screenshots of webpages on different environments or at different moments in time and then creates a diff of the two images; the affected areas are highlighted in blue.

Requires ImageMagick and a headless browser. Takes screenshots of your webpages, runs a comparison task across them, outputs a diff PNG file comparing the two images and a data. If any screenshot's diff is above the threshold specified in your configuration file, the task exits with a system error code useful for CI.

The failed screenshot will also be highlighted in the gallery. SikuliX - This is the currently-maintainted version of the original Sikuli, an open source visual technology to automate and test GUI's. Sikuli Script automates anything you see on the screen without internal API's support; and includes Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. You can programmatically control a web page, a desktop application, or even an iPhone or Android app running in a simulator or via VNC.

Though not available for mobile devices, it can be used with the respective emulators on a desktop or based on VNC solutions. Can script with Python 2.

Does not reside on the system-under-test and is technology agnostic, so it can test in many situations that other tools cannot by using image capture and advanced search techniques. Does not interact with the underlying code, and can test any application including those that can cause problems for other tools such as Flash, Silverlight, etc. Ghost Inspector - Web visual testing and monitoring tool.

Tests are easy to create with a Chrome extension recorder, which records clicks, form submissions and more, for which you can then set assertions that must be made for your test to pass. Or you can create tests via a clean and simple UI. Tests can run continuously from the cloud and alert you if anything breaks. Log in to evaluate results and watch full video of the test, check console output from the browser, screenshots, and even a visual comparison of any changes that have occurred since the last test run.

Screenster - Image-based functional and regression test automation service for web apps using screenshots on each step comparing them to baseline, allowing verification of changes or lack of changes to UI. Differences are detected between a baseline and regression run screenshots, and are visually highlighted on screen. Tester can approve the difference as expected change, ignore it from future comparison for dynamic parts of the UI, or designate as a failed test.

Full access to Selenium API when needed. Browsera - Cloud-based automated browser compatibility testing - automatically checks and reports cross-browser layout differences and javascript errors.

Can automatically crawl site to test entire site; can handle sites requiring login. Reports detail which pages have potential problems - quickly see the problems indicated as each screenshot is highlighted in the problematic areas. Choose browser OS, browser, and versions of interest and submit URL and site responds with a collection of screen shots. Dead Link Checker - Online link checker can crawl and scan entire site or single pages. Free version available; also paid auto-checker versions available that can be scheduled daily, weekly, or monthly.

Link Check - Free online checker from Wulfsoft. Crawls site and checks links; the link check is currently limited to a maximum of 1, found and checked links. When this limit is reached, the check stops automatically. Broken Links at a Glance - Free online broken link checker for small web sites up to pages , by Hans van der Graaf. Start from a dashboard and drill down to any errors. Every error is represented as an error card, with help inline; includes broken link highlighter.

It highlights in browser window which links are valid and which are broken. Runs constantly; every error is immediately analyzed and prioritized; email notifications. Capabilities include e-mail alerts, dashboard, reporting; canned reports or create rich custom reports. Link Checker Pro - Downloadable link check tool from KyoSoft; can also produce a graphical site map of entire web site. Web Link Validator - Downloadable link checker from REL Software checks links for accuracy and availability, finds broken links or paths and links with syntactic errors.

Site Audit - Low-cost on-the-web link-checking service from Blossom Software. Linkalarm - Low cost on-the-web link checker from Link Alarm Inc. Automatically-scheduled reporting by e-mail. Alert Linkrunner - Downloadable link check tool from Viable Software Alternatives; evaluation version available. Handles one URL at a time. PERL source also available for download. Available as source code; binary available for Linux. Includes cross referenced and hyperlinked output reports, ability to check password-protected areas, support for all standard server-side image maps, reports of orphan files and files with mismatching case, reports URLs changed since last checked, support of proxy servers for remote URL checking.

Distributed under Gnu General Public License. Has not been updated for many years. Many of the products listed in the Web Site Management Tools section include link checking capabilities. Free and pro versions.

Organizes access to a collection of free online web test tools. Only need a starting URL; a summary and detailed report is produced. Can schedule for periodic automated validations. Can validate large sites and can submit an XML sitemap to specify a subset of pages to validate.

The validation is done on your local machine inside Firefox and Mozilla. Error count of an HTML page is seen as an icon in the status bar when browsing. Error details available when viewing the HTML source of the page. Available in 17 languages and for Windows and other platforms. Web Page Backward Compatibility Viewer - On-the-web HTML checker by DJ Delorie; will serve a web page to you with various selectable tags switched on or off; very large selection of browser types; to check how various browsers or versions might see a page.

Available as source code or binaries. This section is oriented to tools that focus on web site accessibility; note that other web testing tools sometimes include accessibility testing capabilities along with their other testing capabilities.

Deque AXe - Free accessibility testing tool that runs in web browser - extension for Chrome or Firefox. Also available - download aXe-Core source code from GitHub repo. Other accessibility testing tools from Deque include: Based on a powerful and low-impact JavaScript rules library - runs on your local development server in same browser as your functional or unit tests.

Current framework integrations include Selenium, Cucumber, QUnit, and more. AChecker - Free online tool checks single HTML pages for conformance with accessibility standards to ensure the content can be accessed by everyone.

View by Guideline or View by Line Number. Audit results will appear as a list of rules which are violated by the page if any , with one or more elements on the page shown as a result for each rule. Complicance Sheriff - Tool for testing site accessibility from Cyxtera. Enables catching and fixing accessibility issues before they happen, not after, and allows you to release accessible code from the beginning. Based on WCAG 2. Provides a score for the most used readability indicators: Results include explanations of each item.

Relates to Guideline 3. Image Analyzer - Free online test tool from JuicyStudio - enter URL and site will assess image width, height, alt, and longdesc attributes for appropriate values. It is used to aid humans in the web accessibility evaluation process. Rather than providing a complex technical report, WAVE shows the original web page with embedded icons and indicators that reveal the accessibility of that page. Also available is the WAVE Firefox toolbar allowing evaluation of web pages directly within your browser.

Color Contrast Analyzer - Free downloadable tool from the Paciallo Group to help determine the legibility of text on a web page and the legibility of image based representations of text, can be used as a part of web accessibility testing. It is primarily a tool for checking foreground and background colour combinations to determine if they provide good colour visibility. It also contains functionality to create simulations of certain visual conditions such as colour blindness.

Determining "colour visibility" is based on the Contrast Ratio algorithm, suggested by the World Wide Web Consortium W3C to help determine whether or not the contrast between two colours can be read by people with colour blindness or other visual impairments. For Win and Mac platforms.

CheckMyColours - Free online tool by Giovanni Scala for checking foreground and background color combinations of a page's DOM elements and determining if they provide sufficient contrast when viewed by someone having visual color deficits.

Based on algorithms suggested by the W3C. Shows in real time what people with common color vision impairments will see. Can be used for Web accessibility testing. Support for over 20 languages and the ability to run entirely from a USB drive with no installation. Can be used for accessibility testing. PDF docs on the web often present challenges for the visually impaired.

Supports both experts as well as end users conducting accessibility evaluations. Provides several authentication mechanisms. Traffic Parrot - A stubbing, mocking and service virtualization tool that helps find more bugs by simulating hypothetical situations.

Can be used for both manual exploratory and automated testing. Designed to integrate with Continuous Integration environments Jenkins, Teamcity. Free and paid options available. Karate - Karate open source tool enables scripting of a sequence of calls to any kind of web-service and asserting that the responses are as expected. Easy building of complex request payloads, traversing of data within the responses, and chaining data from responses into the next request. Payload validation engine can perform a 'smart compare' of two JSON or XML documents without being affected by white-space or the order in which data-elements actually appear, and you can opt to ignore fields that you choose.

Express expected results as readable, well-formed JSON or XML, and assert in a single step that the entire response payload no matter how complex or deeply nested - is as expected Scripts are plain-text files and require no compilation step or IDE. Java knowledge is not required. Requires Java 8 and Maven. From dev to live monitoring, all without having to write any code. With each test execution the platform saves the metrics. Know the latency and download times of every call, from various locations globally.

True performance test, not just a ping test. Cloud-based or on-premises solution - entire platform can be deployed internally with a Docker container. When there is an issue, the report contains a snapshot of the header information and the payload. Created by Jakub Roztocil. Frisby tests start with frisby.

Visually create and run single HTTP requests as well as complex scenarios. Save calls history, locally or to the cloud, and organize it in projects; build dynamic requests with custom variables, security and authentication. Build tests that verify services are returning expected data and receive notifications when things go wrong. Free and paid plans available. Assertible - Tool for continuously testing your web services. HTTP requests are made to application's staging or production environment and assertions are made on the response to ensure your APIs and websites are running as expected.

Bench Rest - Open source Node. Ability to automatically handle cookies separately for each iteration; automatically follows redirects for operations; errors will automatically stop an iterations flow and be tracked. Allow iterations to vary easily using token subsitution. No dependencies, works with any unit testing framework. A helpful library for unit testing your code.

Has cross browser support and also can run on the server using Node. Services can be made "intelligent" so app under test can make API calls needed to get similar behaviour back as it would from the actual component. Fault injection to simulate real application behaviour.

Free for up to requests. Source also available at https: Enables defining of JSON endpoints based on a simple template object. Namespace aware - have your mocks on your own domain.

Each space serves a domain on mockable. You can have as many spaces domains as you need. Mocks can also be served on your company DNS domain. Free and paid account types. Useful for testing to easily recreate all types of responses.

Isolate the system under test to ensure tests run reliably and only fail when there is a genuine bug, not due to dependencies and irrelevant external changes such as network failure etc.

Set up mock responses independently for each test to ensure test data is encapsulated with each test, easily maintained, and avoid tests dependent on precursor tests. Enables more efficient development by providing service responses even if the actual service is not yet available or is still unstable.

X module that runs on a Vert. Or build and run MockServer directly from source code. Intercepts HTTP connections initiated by your app and returns recorded responses. The first time a test annotated with Betamax is run any HTTP traffic is recorded to a 'tape' and subsequent test runs will play back the recorded HTTP response from the tape without actually connecting to the external server.

Tapes are stored to disk as YAML files. It will only work if the certificate chain is broken. WireMock An open source java library for stubbing and mocking web services, by Tom Akehurst. Unlike general purpose mocking tools it works by creating an actual HTTP server that your code under test can connect to as it would a real web service. Capabilities include WSDL validation, load and performance testing; graphically model and test complex scenarios. Handles more than message types. Use environment variables to easily shift between settings - good for testing production, staging or local setups.

Builds on jQuery and Bootstrap. Requires browser with HTML5 supoort. Simulate traffic via load agents that can generate load from Windows or Linux-based nodes using a mix of either on-premise or cloud traffic. Virtualize external APIs that don't allow or handle load tests very well. Can reuse existing SoapUI Pro functional tests.

SoapUI Pro paid version with more extensive features available also. Injects two types of faults: Can be used standalone or in combination with a debugger. Customizable to support any XML protocol. Java application, runs on multiple OS's. SOAPSonar - Web services client simulation for service testing - for functional, automation, performance, compliance, and security testing; from CrossCheck Networks.

Concurrent Virtual Clients - independent loading agents aggregate statistics for througput, latency, and TPS. Ramp-up, ramp-down, and weighted scenarios. Vulnerability Analysis includes dynamic XSD mutation security testing with automatic boundary condition testing.

Risk assessment and risk mitigation extensible rule framework. Available as free personal edition, pro edition, server edition. Decouple your own process from time constrained access to external systems, quickly isolate bad actors and poor performers during integration and load testing.

Enables developing and testing before your actual API is deliverable, enables testers to have control over simulated responses and error handling, and better deal with versioning problems and speed up resolution during continuous integration cycles.

WebInject - Open source tool in Perl, by Cory Goldberg, for automated testing of web services and apps. Can run on any platform that a Perl interpreter can be installed on. Free 'Express' edition available. Reports can include metadata, access to log files, list of commands and responses, screenshots, screencast, etc. SauceConnect available for secure tunneled testing of local or firewalled sites.

Plugins available for Travis, Jenkins, Bamboo, more. For all major browsers. Keeps track of new browser releases and updates. Reports contain browser specific full-page and original-size screenshots. See and interact with multiple different browsers side by side - all Browsers stay fully interactive.

Navigate and reload in all browsers simultaneously. Capabilities include Selenium integration. App runs on Win platforms. Browserling - On-the-web cross browser testing tool from Browserling Inc. Enables interactive cross-browser testing; fully interactive sessions, not static screenshots; powered entirely by canvas and javascript.

Reverse-proxy your localhost into Browserling with Browserling ssh tunnels - just copy and paste an ssh one-liner from the UI. Gridlastic - Cloud based selenium grid cross-browser testing tool from Gridlastic LLC that enables launching your own selenium grid in any Amazon data region.

With 1 click you get an instant selenium maintenance-free auto-scaling cross browser testing infrastructure. The grid environment is updated regulary to support new browsers and selenium versions. Videos of every test are available for debugging.

CrossBrowserTesting - Test your website in dozens of browsers and real devices; over one thousand combinations of browsers, OSs, and devices - not emulators. Test your sites on more than browsers across more than 40 operating systems, including iOS, Android, Windows, Mac and more.

Works with selenium automation. Can work with test sites that are behind firewalls. Lunascape - A free 'triple engine' web browser from Lunascape Corp. By clicking the smart engine-switch button next to the address bar, a user can switch rendering engine for any page, enabling running and testing of a website in multiple rendering engines.

Also included is a 'switch user agent' capability. Capabilities include selenium automation integration, tunneling to any local server environment, HTTPS. Mobile testing via emulators.

Stacks include a wide variety of developer tools. Microsoft provides virtual machine disk images to facilitate website testing in multiple versions of IE, regardless of the host operating system. Requires Virtual Box, Curl, Linux. TestingBot - Cloud-based automated cross-browser testing service from TestingBot - utilize Selenium tests to run in the cloud on the TestingBot grid infrastructure.

Compose a Selenium test with simple commands. Also allows running tests at a specific time and interval, with failure alerts. Manual testing capability also. Turbo - Turbo formerly Spoon is a lightweight, high-performance container platform for building, testing and deploying applications and services in isolated containers.

The runtime environment of Turbo containers is supplied by the Turbo Virtual Machine or SVM, a lightweight implementation of core operating system APIs, including the filesystem, registry, process, and threading subsystems. Containerized applications consume only a small percentage of additional CPU and memory consumption relative to native applications. Turbo overhead is generally negligible. For manual browser testing, you can run any version of any browser in a container or build a custom browser container with components like Java and Flash.

The automated testing solution allows running tests with Selenium on the Turbo web-based Selenium Grid that utilizes your browser containers on your local machine to minimize your testing environment setup. Browsers and test scripts can run on your local machine, so there is no need for any special proxy configuration or modifications to the URL when testing an internal site.

Supports Chrome, Firefox, and IE. Hosted, Enterprise, and ISV licensing; free and paid versions.

Standardization is Important