Conversational UI with Oracle Digital Assistant and Fn Project

Here and there we see numerous predictions that pretty soon chatbots will play a key role in the communication between the users and their systems. I don't have a crystal ball and I don't want to wait for this "pretty soon", so I decided to make these prophecies come true now and see what it looks like.

A flagman product of the company I am working for is FlexDeploy which is a fully automated DevOps solutions. One of the most popular activities in FlexDeploy is creating a release snapshot that actually builds all deployable artifacts and deploys them across environments with a pipeline.
So, I decided to have some fun over the weekend and implemented a conversational UI for this operation where I am able to talk to FlexDeploy. Literally. At the end of my work my family saw me talking to my laptop and they could hear something like that:

  "Calypso!" I said.
  "Hi, how can I help you?" was the answer.
  "Not sure" I tested her.
  "You gotta be kidding me!" she got it.
  "Can you build a snapshot?" I asked.
  "Sure, what release are you thinking of?"
  "1001"
  "Created a snapshot for release 1001" she reported.
  "Thank you" 
  "Have a nice day" she said with relief.

So,  basically, I was going to implement the following diagram:

As a core component of my UI I used a brand new Oracle product Oracle Digital Assistant. I built a new skill capable of basic chatting and implemented a new custom component so my bot was able to invoke an http request to have the backend system create a snapshot.  The export of the skill FlexDeployBot along with Node.js source code of the custom component custombotcomponent is available on GitHub repo for this post.
I used my MacBook as a communication device capable of listening and speaking and I defined a webhook  channel for my bot so I can send messages to it and get callbacks with responses.

It looks simple and nice on the diagram above. The only thing is that I wanted to decouple the brain, my chatbot, from the details of the communication device and from the details of the installation/version of my back-end system FlexDeploy. I needed an intermediate API layer, a buffer, something to put between ODA and the outer world. It looks like Serverless Functions is a perfect fit for this job.

As a serverless platform I used Fn Project. The beauty of it is that it's a container-native serverless platform, totally based on Docker containers and it can be easily run locally on my laptop (what I did for this post) or somewhere in the cloud, let's say on Oracle Kubernetes Engine.

Ok, let's get into the implementation details from left to right of the diagram.

So, the listener component, the ears, the one which recognizes my speech and converts it into text is implemented with Python:

The key code snippet of the component look like this (the full source code is available on GitHub):

r = sr.Recognizer()
mic = sr.Microphone()

with mic as source:
    r.energy_threshold = 2000

while True:  
    try:
        with mic as source: 
            audio = r.listen(source, phrase_time_limit=5)           
            transcript = r.recognize_google(audio)
            print(transcript)
            if active:
                requests.post(url = URL, data = transcript)
                time.sleep(5)
           
    except sr.UnknownValueError:
        print("Sorry, I don't understand you")

Why Python? There are plenty of available speech recognition libraries for Python, so you can play with them and choose the one which understands your accent better. I like Python.
So, once the listener recognizes my speech it invokes an Fn function passing the phrase as a request body.
The function sendToBotFn is implemented with Node.js:

function buildSignatureHeader(buf, channelSecretKey) {
    return 'sha256=' + buildSignature(buf, channelSecretKey);
}


function buildSignature(buf, channelSecretKey) {
   const hmac = crypto.createHmac('sha256', Buffer.from(channelSecretKey, 'utf8'));
   hmac.update(buf);
   return hmac.digest('hex');
}


function performRequest(headers, data) {
  var dataString = JSON.stringify(data);
 
  var options = {
   body: dataString,   
   headers: headers
  };
       
  request('POST', host+endpoint, options);             
}


function sendMessage(message) {
  let messagePayload = {
   type: 'text',
   text: message
  }

  let messageToBot = {
    userId: userId,
    messagePayload: messagePayload
  }

  let body = Buffer.from(JSON.stringify(messageToBot), 'utf8');
  let headers = {};
  headers['Content-Type'] = 'application/json; charset=utf-8';
  headers['X-Hub-Signature'] = buildSignatureHeader(body, channelKey);

  performRequest(headers, messageToBot);  
}


fdk.handle(function(input){ 
  sendMessage(input); 
  return input; 
})

Why Node.js? It's not because I like it. No. It's because Oracle documentation on implementing a custom web hook channel is referring to Node.js. They like it.

When the chatbot is responding it is invoking a webhook referring to an Fn function receiveFromBotFn running on my laptop.  I use ngrok tunnel to expose my Fn application listening to localhost:8080 to the Internet. The receiveFromBotFn function is also implemented with Node.js:

const fdk=require('@fnproject/fdk');
const request = require('sync-request');
const url = 'http://localhost:4390';
fdk.handle(function(input){  
    var sayItCall = request('POST', url,{
     body: input.messagePayload.text,
    });
  return input;
})
 

The function sends an http request to a simple web server running locally and listening to 4390 port.
I have to admit that it's really easy to implement stuff like that with Node.js. The web server uses Mac OS X native utility say to pronounce whatever comes in the request body:

var http = require('http');
const exec = require("child_process").exec
const request = require('sync-request');

http.createServer(function (req, res) {
      let body = '';
      req.on('data', chunk => {
          body += chunk.toString();
      });

      req.on('end', () => {       
          exec('say '+body, (error, stdout, stderr) => {
      });       
      res.end('ok');
     });

  res.end();

}).listen(4390);

In order to actually invoke the back-end to create a snapshot with FlexDeploy the chatbot invokes with the custombotcomponent an Fn function createSnapshotFn:

fdk.handle(function(input){
   
var res=request('POST',fd_url+'/flexdeploy/rest/v1/releases/'+input+'/snapshot',  {
      json: {action : 'createSnapshot'},
  });

  return JSON.parse(res.getBody('utf8'));
})

The function is simple, it just invokes FlexDeploy REST API to start building a snapshot for the given release. It is also implemented with Node.js, however I am going to rewrite it with Java. I love Java. Furthermore, instead of a simple function I am going to implement an Fn Flow that first checks if the given release exists and if it is valid and only after that it invokes the createSnapshotFn function for that release. In the next post.


That's it!