Continuous Delivery of ADF applications with WebLogic Shared Libraries

Introduction
There is a pretty popular architecture pattern when ADF applications are built on top of shared libraries. So the main application is being deployed as an EAR and all subsystems are implemented within shared libraries that can be independently built and deployed to WebLogic as JARs in "hot" mode without downtime. The advantages of this approach seem to be obvious:

• It decomposes the application implementing the concepts of modularization and reuse
• CI/CD process might be much faster as only one library is going to be rebuilt/redeployed
• There is no downtime while redeploying a shared library

It looks so cool that people choose this architecture pattern for their new projects and they are pretty happy with the decision they made while implementing the application. They get even happier when they go live to production as they can easily fix most of the bugs and implement new requirements avoiding full redeployment and without any downtime. 

Definitely, before getting to production any change (and therefore a corresponding shared library) should be deployed and tested at the previous environments such as QA, UAT, etc. 

In a while nobody knows exactly what versions of shared libraries are deployed at each environment. It's getting a bit tricky to support the application and implement new changes in this situation as even though it works on this environment there is no guarantee it's going to work on the next one as the combination of shared libraries could be different. If it is a big application and there are many shared libraries, this might become a nightmare and pretty often people just give up getting back to full redeployment of everything and eventually to a monolith EAR. It's not that cool, but at least they can sleep again now.

Solution

In this post I am going to show how to put things in order and build a continuous delivery process of an ADF application built on top of shared libraries with FlexDeploy. FlexDeploy is a rapidly growing Automation and DevOps solution and if you want to learn what it is all about feel free to visit the  website. Here I am going to focus on how FlexDeploy helps with shared libraries by introducing the concepts of a snapshot and a pipeline.  

Snapshot is a set of deployable artifacts representing the entire system. If either of the artifacts is to be rebuilt a new snapshot is going to be created containing a new version of this artifact and the previous versions of the rest of artifacts. In our case a snapshot would contain a EAR for the main ADF application and JARs for the shared libraries.

In order to create snapshots for our application FlexDeploy should know what it is all about and what 

projects it consists of.  There is a notion of Release in FlexDeploy which serves as a bucket of projects that should be built into snapshots and deployed across environments all together as a single unit. 

In our example there are three projects - one for the main application and two for departments and employees task flows, deployed as shared libraries. Each project is configured separately in FlexDeploy and each project "knows" how its source code can be fetched, how to be built and deployed (FlexDeploy uses workflows for building and deploying, but that's another big story which is way beyond this post).

Having all that defined, whenever a developer pushes a code change for any of the projects included in the release, FlexDeploy builds a new snapshot. It rebuilds only those projects (producing ears and jars) that have changed, the rest of the artifacts are included in the new snapshot as is. 

  

Ok, now we can build snapshots and let's deploys them across environments. The release definition is referring to a pipeline. 

Pipeline is an approach that guarantees deploying of the entire snapshot across environments in a strict predefined order. It means that this snapshot (in other words this combination of ear/jar versions) can be deployed only in this order Dev->QA->Prod (if a pipeline is defined in this way). It just can't get to Prod if it is not successful at Dev and QA.  A pipeline consists of stages referring to environments, each stage consists of gates (approvals, test results, etc. meaning that a snapshot should pass all gates before being processed at this environment) and steps (deploy, run automated tests, notify, manual steps, ...). 

So,  basically, the deployment is just a pipeline step within a pipeline stage (environment). This step is smart enough to redeploy only those artifacts that have changed (unless the step is configured to perform "force" deploy). FlexDeploy tracks what artifact versions have been deployed at every environment.

As a conclusion I would say that when using FlexDeploy as a DevOps solution for ADF applications with shared libraries we gain all benefits of this architecture pattern on one hand, and on the other hand we keep things in order, knowing exactly what combination has been deployed across environments, what has been tested and ready to go live and what has failed.  

That's it!

Automated Software Testing of Fusion Middleware applications with FlexDeploy

Automated software testing is one of the mandatory activities that any software organization should perform in order to be able to guarantee quality of their products. However, very often this process turns to be a pretty complicated thing, especially when it comes to automated testing of modern complex systems consisting of a number of different parts. All those parts are based on different technologies and, apparently, different technologies should be used to test them. In reality QA engineers operate with various testing tools in order to perform testing of each part of the system, to test the whole system, to perform different test types such as functional, performance, integration, etc. Each testing tool produces its own test results and QAs should gather all that data from different sources and analyze them in order to be able to make any decisions about quality of the system. It would be nice to put all that stuff to one place and to completely automate this complicated process. That would increase quality of the test automation itself and allow to make automated testing as a step of the Continuous Delivery process.

I have posted already about a new DevOps and CD tool for Fusion Middleware applications FlexDeploy. I was pleased with the opportunity to have a look at the beta version of the system (3.0) and was able to kick the tires. Among many other cool features the new version delivers Test Automation functionality. FlexDeploy 3.0 becomes a powerful instrument that puts Test Automation process under the control and makes it much easier by implementing the following concepts:

• Central management panel to perform, control and monitor all automated testing activities in the organization 
• Hub for gathering test results from the all used testing tools
• Central tool for building QA dashboard reports across the organization for making "Go-Live/Not-Go-Live" decisions
• Support Automated Testing as a mandatory step of the CD process

In this post I am going to highlight the key FlexDeploy features related to Test Automation which may help to understand how the concepts above are implemented and how it actually works.

Integration with 3rd party testing tools

Actually FlexDeploy doesn't perform any automated tests itself. This job is to be done by a great number of various testing tools. FlexDeploy just controls the process and, obviously, it is able to interact with the testing tools. This interaction ends up with the following tasks:

• Execute tests
• Import test results

The integration with 3rd party testing tools is implemented by the use of plugins. At this point FlexDeploy 3.0 provides plugins for JUnit and TestNG tools only. However, the range of plugins is going to get much wider in the upcoming versions. As far as I know the tools like Oracle Application Testing Suite (OATS), Selenium, Test Complete, etc. are on the roadmap. Actually, if FlexDeploy doesn't provide a plugin for some testing tool out-of-the-box we can always use Shell plugin which allows to execute batch scripts in order to interact with the testing tool.

Basically, the plugins act as adaptors for various testing tools. They command testing tools to run tests and they import test results into unified test result storage. This allows to analyze test results and build dashboard reports abstracting away from the exact testing tool. 

Test Definition

FlexDeploy introduces a notion of Test Definition. This abstraction represents one or a bunch of test cases related to some business use case.  For example, for a banking system we can define test definitions such as "Loan arrangement", "Loan repayment", "Loan overdue", etc.

Basically, when FlexDeploy is running automated tests it is, actually, running test definitions one-by-one. And each test definition, in its turn, is running the actual test cases with a corresponding testing tool. It knows what testing tool is going to be used, how to interact with it, what bunch of test cases (defined at the testing tool) should be used, how to import test results and how to qualify them. In order to interact with a testing tool a test definition uses a workflow. Workflows is a powerful mechanism which is commonly used in FlexDeploy to perform build and deploy activities. Now it is used for automated testing as well.  

Actually a test workflow invokes a plugin to interact with a testing tool. Besides just running test cases and importing test results, a test workflow can also perform any other activities as well. For example some preparation work before running tests (like warming up phase) or some finalization activities after running tests. 

Test Qualifiers

A test definition is able to analyze test results and come up with the conclusion whether the test run succeeded or failed. This feature is based on test qualifiers. A test definition can contain a number of test qualifiers such as "Number of Test Cases PASSED greater than X", "Percentage of Test Cases FAILED less than Y", etc.
If all test qualifiers, defined at the test definition, return true, then the test run is considered successful. FlexDeploy provides a set of some predefined test qualifiers, but they plan to allow users to define their own custom qualifiers basing on Groovy expressions.

Test Set
A test set is just a group of test definitions. For example for a banking system we could define test sets such as "Loans", "Deposits", "Forex", etc. A test set can contain its own qualifiers in order to check whether the entire set of test definitions passed or failed.

Test Execution
At this point automated tests can be executed either manually:

or automatically as a step of a deployment process:

I guess the upcoming versions of FlexDeploy will allow to automatically execute tests on a schedule basis independently from the deployment process.

Each test execution (either manual or within deployment) comes up with corresponding test results:

Dashboard
FlexDeploy 3.0 provides a new dashboard tab "Test Automation". It represents the passed/failed ratio of test executions across environments, the duration time of test runs and response times:

This information is also available on "Test Automation" section of the project definition. In this case all the charts are based on test results related to this project:

By implementing test automation functionality FlexDeploy 3.0 has made a giant step along the path of getting continuous delivery process fully automated. Of course, there are still lots of things to be done, but they have built a good basis which is going to be just enriched with new test automation features in upcoming FlexDeploy versions.

That's it!

Continuous Delivery of Fusion Middleware applications with FlexDeploy (Part 2)

In one of my previous posts I described a new DevOps and CD tool FlexDeploy. I mentioned that it was a pretty promising and rapidly developing system. Those guys proved my words and released a new version in June 2015. FlexDeploy 2.1 delivers a number of great new features and in this post I am going to focus on some of them.

The first thing you're going to notice is new look and feel. FlexDeploy is an ADF application and in the new version it utilizes the charm and simplicity of Alta UI skin.  But it seems to be not only about a skin, they started to incorporate in the application the Alta UI and the Simplified UI design principles. The "Glance, Scan, Commit" philosophy is recognizable in the new FlexDeploy screens.

The first screen the users see in the system is Home page:

Basically, this is a navigation approach providing access to all parts of the system and containing some general overview information. In other words this is a "Glance" view.

There is a link on this Home page which gets users to the Dashboard screen. This is a brand new functionality in FlexDeploy. The general concept of the Dashboard screen is an on-line dynamic, interactive report representing the most valuable information in real-time.

There are two tabs on the dashboard screen. The first one is like a flight board in the airport. It represents the current state of things, in other words it shows what is going on right now. Users can monitor what builds and deploys are being performed right now, what happened recently and what is going to be built or deployed in the nearest future. The screen is "on-line" so that it is being refreshed automatically.
It can be bookmarked, put on a separate screen, for example on a TV-panel in a developers room.

The second tab of the dashboard screen represents statistical information such as success ratio and duration time of build and deploy tasks. There are also lists with "Top-5" unsuccessful  and slowest projects in terms of build/deploy processes.
In terms of UX design philosophy the dashboard screen is an example of the "Scan" view. As I already mentioned the dashboard is fully interactive. You can filter the data on the screen and make it related to a particular environment (UAT, QA, DEV), to a specific application, to a selected period of time. Furthermore, you are able to drill down to a specific application, project and even to a build or deploy activity, in other words you are diving to the "Commit" view.

One of the most powerful features that FlexDeploy delivered in the new version is an ability to build a complete continuous delivery process, so that builds and deploys are performed automatically on scheduled basis. You don't have to use any external CD/CI servers for that (like Jenkins or Hudson or something else). Everything is implemented inside FlexDeploy already and available for users just out-of -the-box.
As far as I know, FlexDeploy team is now working on test automation functionality in order to be able to automatically run tests as a part of CD process. Hopefully this feature along with many other interesting things related to test automation will be available in the next version of the system.

"We are going to the cloud!" - this moto can be heard today on every corner in the modern IT world. FlexDeploy is not an exception and in 2.1version it has been integrated with Oracle Cloud Services.
So, now you are free to perform build and deploy operations to the Oracle Java and Database Cloud Services. Furthermore, you can run FlexDeploy itself on the Oracle Java Cloud Service. Basically, it is available on the Oracle Cloud Marketplace.

In this post I didn't cover all new features available in FlexDeploy 2.1, I just highlighted some of them. I believe I will keep posting about this tool and I bet I will find many new interesting features to blog about in upcoming versions.

That's it!