Using the RC Test Framework in complicated environments

The Rescop Test Framework is to be used in every IT delivery model. Most of our clients, just like many organizations using software applications to support daily business processes and related activities, have a Hybrid IT delivery model.

Picture 1: Hybrid IT delivery model

 Source: Sogeti TMAP® – Quality for DevOps teams

Picture 2: Hybrid demand / supply model

Source: Sogeti TMAP® – Quality for DevOps teams

Many reasons to use a hybrid model. A common demand/supply model is where the demanding party (“Business”) uses a traditional implementation approach (Waterfall à Sequential IT delivery model) while IT suppliers have adopted High-Performance IT delivery (Scrum).

To make even more interesting, manufacturing enterprises have incorporated the ANSI/ISA-95 framework. Based on this model, organizations divide their automation software in Information Technology (IT) and Operational Technology (OT). Every manufacturing enterprise is dealing with the question of where to draw the line between IT and OT. Manufacturing Execution System (MES – ISA-95 layer 3 à Yellow in picture 2) is 99% of the time the topic of discussion. There is no standard way of working, some choose to make MES IT responsibility, others OT. OT software delivery and management diverse form IT, due to all kind of reasons and circumstances. It goes too far to elaborate on this, but it gives a high-level impression of the complexity of software quality assurance.

Picture 3: ISA-95 model

 Picture 4: Test Framework

Implementing changes to these complex environments puts the continuity of the organization at risk. Extensive (endlessly) testing could help increase confidence, but it does not help shorten timelines and time to market. Our advice is to structure and standardize your test approach using the RC Test Framework. Once using a testing framework, it gives you flexibility to manage and tune your testing activities to the specific change at hand. Among several other advantages, a structured test approach enables you to shift the quality check to the left.

Picture 5: BDTM process steps

“Shifting to the left” applies to:

• Left side of the V-Model
• Left side of the project timetable
• Leverage test results obtained during development by delivery party

It’s also about preventing bugs OVER finding bugs, one of the five values of the Testing Manifesto as discussed in one of my other blogs.

If you want to get in control of the software development, if you want to be able to rely on issued timelines, required budgets and software quality start using Business Driven Test Management (BDTM). This fundamental part of the Test Framework ensures early involvement of quality professionals in the change creation process.

BDTM distinguishes the following characteristics:

• Change management
  • Based on Business case
  • Focus on Result, Time and Budget
  • Risk based
• Schedule and budget are based on test strategy
• During test period business is involved in discission making
• Structured & phased way of working having a build-in iterative procedures (picture 4)

Business Driven Test Management forms the basis to set-up your change specific test strategy upfront and be able to adjust along the way when specific circumstances ask for it and in coordination with client (business).

The implementation of the test strategy formed in the BDTM phase is supported by the structured approach of the next fundamental part of the Test Framework.

This structured process is built on the following topics:

• Development Master Test Plan
  • Based on test policy (if applicable)
  • Tuned to change at hand
  • Manage (guide) the overall test process (change specific)
• Development testing (supply)
• Acceptance testing (demand)

Picture 6: Process steps to guide test activities

All 3 topics are supported by the testing phase model. This model has, like every software development process / method several phases and related activities. Although the phases are sequential, it can be used in High-Performance and Hybrid IT delivery models (picture 8).

This is a generic model, applicable for every test form (development / acceptance testing) and test type (Unit Test, System Test, User Acceptance Test or Regression Test) and to be used in parallel to the development activities. This makes it possible to “shift quality to the left” for real.

Picture 7: Testing phase model

Planning

This phase is the basis for a controlled and qualified test process. It is an important but often underestimated test phase. BDTM steps 1 to 4 are executed and results discussed with the client / business. Next is step 5 in BDTM model.

Control

A plan is just a plan, most of the time the test process will not be executed exactly according to plan. The execution of the plan needs guidance, control and where needed adjustments, which is done in this phase.

Infrastructure

This phase is to take care of the required test infrastructure and resources.

Preparation

The preparation phase is to execute the intake of the test basis. The goal is to have a qualified set of documents to design tests.

Specification

Following up on the preparation, this phase is to specify the required tests and test situations. This phase is executed in parallel with the software development phase.

Execution

In this phase the prepared and specified tests are being executed to measure the quality of the test object.

Completion

When all scheduled test activities are executed test ware (test cases / test environments) is preserved and test process is being evaluated to make re-use possible and to learn from the experience.

Picture 8: Testing phase model tuned to High-Performance / Hybrid IT delivery model

This toolbox, the following fundamental part of the Test Framework contains instruments for:
•    Test Techniques, to organize how to test
•    Infrastructure, to organize where and with what to test
•    Organization, to organize who is going to test

The last fundamental part of the Test Framework is that it is an adaptive method:

• Easy to use and implement
• High degree of flexibility
• Set-up to (re-)use products – process
• Set-up to learn from experiences

The adaptivity applies also for the use in all kinds of test situations, for all kind of software and covers the three IT delivery models.