IoT Test Automation Framework (ITAF)

Introduction

IoT (Internet of Things) test automation frameworks are tools and frameworks designed specifically for automating the testing of IoT devices, systems and applications. These frameworks provide a structured approach to automate the testing process, manage test cases, simulate IoT environments, and generate test data.

ITAF Workflow

IoT products which support ITAF usually have high-level architecture where the sensor generates data and the gateway compiles and abstracts the data and sends required data to the cloud where the users can interpret and make useful decisions. 

From the cloud, there are usually two types of interfaces for customers to interact with - UI and the API. There is also another type of interface in some special cases called CLI (Command Line Interface). The UI interface is a human-driven interface that presents a set of web pages for humans to interact with the IoT system. An API (Application Programming Interface) interface refers to the set of rules and protocols that define how different software components or systems can interact with the IoT system. In this case, ITAF would be used to automate API calls. CLI would be used by persons with technical skill sets who can interact with edge devices through command line console.

ITAF Tool Choices

• Python - Python can be used as the ITAF test scripting language in competing with Java for its simplicity, open source and cross-platform support. Although there are claims of Python being slow since the code is executed by interpreter as opposed to a compiler, for validation needs speed isn’t a problem until it is a problem!
• Robot Tool - Robot tool has been chosen as the IoT test automation framework for being open source, easy to install, platform independent and its compatibility with Python. Also it supports certain approaches like Keyword driven, Data driven and Behavior driven (BDD) approaches. Here the user does not need any programming language to write a Robot Framework test case. It also has an internal selenium library  which can be included using the robot “Library” statement.
• Doxygen - Doxygen is a documentation system for C++, C, Java, Python, Objective-C, IDL (Corba and Microsoft flavors) etc. Doxygen analyzes the source code, extracts the documentation comments (such as those written in the Doxygen format or other supported documentation styles) and generates various types of documentation, including HTML, LaTeX, PDF, and XML.
• Selenium - If you are testing an HTML based UI, such as a cloud interface, then selenium may be required for your project. The Robot framework has an internal selenium library, which can be included using the robot “Library” statement. ITAF cannot be used with a selenium library so it can be used with the standard python selenium library which can be installed using pip.
• Additional Libraries - If your specific project requires additional python libraries, they can usually be obtained through pip.

ITAF Components

ITAF consists of several components: 

• The ITAF directory structure - The specific directory structure of the ITAF framework will depend on the implementation and organization using it. It is a template that is created in the validation git repository for each IoT product. When the repository is checked in a local machine, the directory structure will have the subdirectory structure as shown below.

Subdirectories description

• README.md – A README.md file is a common file in software repositories that serves as a documentation and introduction for the project. The .md extension stands for Markdown, which is a lightweight markup language. README.md should be written for each IoT project. 
• config – Configuration files are stored here. It contains data that is the same for all test cases in a project, for instance, the URL of the cloud. 
• testArtifacts – logs and robot outputs are stored in testArtifacts. Logs files generated by test cases should have the same basename as the robot file.
• testCaseApps – It is subdivided by use case types. Each test case app should have its own subdirectory within its use case type subdirectory. 
• testScenarios – Robot files are stored in testScenarios. A test scenario is the condition when a test case app will be run. 
• utils – Common utilities are stored in utils. It is divided into subdirectories based upon the type of the utility. 
• Python Libraries - A Python library is a collection of pre-written code modules or functions that provide specific functionalities to simplify and expedite the development process. These libraries contain reusable code that can be imported into Python programs, allowing developers to leverage existing solutions for common tasks rather than reinventing the wheel.
• Robot Framework - Robot Framework is used to automate execution of tests. It provides a simple and easy-to-read syntax that allows test cases to be written in a natural language format, making them accessible to both technical and non-technical stakeholders.
• ConfigParser interface - The ConfigParser interface is a part of the Python standard library and is used for working with configuration files. It provides a convenient way to read, write, and modify configuration files in a standardized format. 
• Logging - For the purpose of logging, ITAF uses ColorLog class which wraps Python logging module. This class was obtained from the original Moon Island PSV validation team. ColorLog is a term that typically refers to a library or tool used in software development to add colored output to log messages or console output. Usage is straightforward as shown below.

• Documentation - Doxygen is used as the main documentation tool. The logging module in Python provides a powerful and flexible way to incorporate logging capabilities into your code. It allows you to record and manage log messages at different levels of severity, such as DEBUG, INFO, WARNING, ERROR, and CRITICAL. You can customize the log format, control the output destination (e.g., console or file), and even redirect log messages to external services.

Conclusion

The ITAF framework is not a widely recognized or commonly known test automation framework in the context of software testing or IoT. It may be specific to a particular organization, industry, or project. In general, when it comes to IoT test automation, there are various frameworks and tools available that can be utilized based on your specific requirements. Some popular options include IoTIFY, Robot Framework, and frameworks that leverage specific programming languages like Python or Java.

Ultimately, the choice of an appropriate test automation framework will depend on factors such as the nature of your IoT solution, the technologies involved, the testing objectives, the preferences and expertise of your team. It's important to evaluate and select a framework that aligns with your specific needs and helps streamline the testing process effectively.