In one of my
previous posts I described how to create a Docker container serving as a builder machine for ADF applications. Here I am going to show how to use this container as a
function on Fn platform.
First of all let's update the container so that it meets requirements of a function, meaning that it can be invoked as a runnable binary accepting some arguments. In an empty folder I have created a Dockerfile (just a simple text file with this name) with the following content:
FROM efedorenko/adfbuilder
ENTRYPOINT ["xargs","mvn","package","-DoracleHome=/opt/Oracle_Home","-f"]
This file contains instructions for Docker on how to create a new Docker image out of existing one (efedorenko/adfbuilder from the previous post) and specifies an entry point, so that a container knows what to do once it has been initiated by the Docker
run command. In this case whenever we run a container it executes Maven
package goal for the
pom file with the name fetched from
stdin. This is important as Fn platform uses
stdin/stdout for functions input/output as a standard approach.
In the same folder let's execute a command to build a new Docker image (fn_adfbuilder) out of our Docker file:
docker build -t efedorenko/fn_adfbuilder .
Now, if we run the container passing
pom file name through
stdin like this:
echo -n "/opt/MySampleApp/pom.xml" | docker run -i --rm efedorenko/fn_adfbuilder
The container will execute inside itself what we actually need:
mvn package -DoracleHome=/opt/Oracle_Home -f /opt/MySampleApp/pom.xml
Basically, having done that, we got a container acting as a function. It builds an application for the given
pom file.
Let's use this function in Fn platform. The installation of Fn on your local machine is as easy as invoking a single command and described on
GitHub Fn project page. Once Fn is installed we can specify Docker registry where we store images of our functions-containers and start Fn server:
export FN_REGISTRY=efedorenko
fn start
The next step is to create an Fn application which is going to use our awesome function:
fn apps create adfbuilderapp
For this newly created app we have to specify a route to our function-confiner, so that the application knows when and how to invoke it:
fn routes create --memory 1024 --timeout 3600 --type async adfbuilderapp /build efedorenko/fn_adfbuilder:latest
We have created a route saying that whenever
/build resource is requested for
adfbuilderapp, Fn platform should create a new Docker container basing on the latest version of
fn_adfbuilder image from
efedorenko repository and run it granting with 1GB of memory and passing arguments to
stdin (the default mode).
Furthermore, since the building is a time/resource consuming job, we're going to invoke the function in async mode with an hour timeout. Having the route created we are able to invoke the function with
Fn Cli:
echo -n "/opt/MySampleApp/pom.xml" | fn call adfbuilderapp /build
or over http:
curl -d "/opt/MySampleApp/pom.xml" http://localhost:8080/r/adfbuilderapp/build
In both cases the platform will put the call in a queue (since it is async) and return the call id:
{"call_id":"01C5EJSJC847WK400000000000"}
The function is working now and we can check how it is going in a number of different ways. Since function invocation is just creating and running a Docker container, we can see it by getting a list of all running containers:
docker ps
CONTAINER ID IMAGE CREATED STATUS NAMES
6e69a067b714 efedorenko/fn_adfbuilder:latest 3 seconds ago Up 2 seconds 01C5EJSJC847WK400000000000
e957cc54b638 fnproject/ui 21 hours ago Up 21 hours clever_turing
68940f3f0136 fnproject/fnserver 27 hours ago Up 27 hours fnserver
Fn has created a new container and used function call id as its name. We can attach our stdin/stdout to the container and see what is happening inside:
docker attach 01C5EJSJC847WK400000000000
Once the function has executed we can use Fn Rest API (or Fn Cli) to request information about the call:
http://localhost:8080/v1/apps/adfbuilderapp/calls/01C5EJSJC847WK400000000000
{"message":"Successfully loaded call","call":{"id":"01C5EJSJC847WK400000000000","status":"success","app_name":"adfbuilderapp","path":"/build","completed_at":"2018-02-03T19:52:33.204Z","created_at":"2018-02-03T19:46:56.071Z","started_at":"2018-02-03T19:46:57.050Z","stats":[{"timestamp":"2018-02-03T19:46:58.189Z","metrics":
....
http://localhost:8080/v1/apps/adfbuilderapp/calls/01C5EJSJC847WK400000000000/log
{"message":"Successfully loaded log","log":{"call_id":"01C5EKA5Y747WK600000000000","log":"[INFO] Scanning for projects...\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Build Order:\n[INFO] \n[INFO] Model\n[INFO] ViewController\n[INFO]
....
We can also monitor function calls in a fancy way by using
Fn UI dashboard:
The result of our work is a function that builds ADF applications. The beauty of it is that the consumer of the function, the caller, just uses Rest API over http to get the application built and the caller does not care how and where this job will be done. But the caller knows for sure that computing resources will be utilized no longer than it is needed to get the job done.
Next time we'll try to orchestrate the function in Fn Flow.
That's it!
