In the
previous two posts in this series I introduced how to Register our API Extension in vCloud Director and How to setup our RabbitMQ AMQP server to handle or move the messages generated by our
Extension Service. In this post, we will
look at the last piece of the puzzle, which is taking the relevant data from
the AMQP message and actually doing the work of our "faulttolerance" Extension Service. Relatively speaking this is the hard part,
which is why it makes so much sense to use vCenter Orchestrator for this
step. Because you need to communicate
with a system that is external to vCloud Director but maintain correlation to
objects in vCloud Director VCO is the perfect tool for the job.
Using
VCO as the extension service
The benefits of using VCO to provide the external service
for our vCloud Director API Extension are:
- VCO has an AMQP plugin that provides simple
workflow driven configuration options and built in message consumption and
publishing.
- VCO has many VMware and third party plugins that
you can leverage to provide a multitude of Extension Services.
- VCO has a graphical based IDE and intuitive
interface, which lower sthe learning curve for developing against external
API’s.
- VCO IDE leverages javascript as its scripting
language, which can natively handle and parse JSON, which is the format of our
AMQP message bodies.
For this example, I used the vCenter Orchestrator Virtual
Appliance version 5.1 and then installed the vCloud Director 5.1 and AMQP
Plug-ins. I have provided access to the actual
FT API Extensions workflow package file, which contains the VCO FT API Extension workflow itself.
Step
1 – Configure the VCO AMQP plugin
In order to
process messages from an AMQP queue we need to add the AMQP broker to the VCO
inventory and then subscribe to the queue that we have created in RabbitMQ to
receive messages for our Extension Service.
We could also have created the queues and the bindings between the AMQP
exchange and queue from within VCO using the workflows that ship with the AMQP
plug-in. In this example, we have
already done that using the Management plugin for RabbitMQ.
- In the workflows tab from the Design or Run context
navigate to the Library – AMQP – Configuration folder and run the “Add a broker” workflow.
- Enter a Name for the AMQP server.
- Enter the connection information for the AMQP server
or in this case RabbitMQ server.
- Host= hostname or
IP of AMQP server
- Port= 5672
- Virtual host= /
- Use SSL= No
- User name=guest and Password=*****
- In the workflows tab from the Design or Run context
navigate to the Library – AMQP – Configuration folder and run the “Subscribe to queues” workflow.
- Enter a Name for the subscription.
- Name=”ftqueue API Extension”
- Enter the connection information for the AMQP or in
this case RabbitMQ server.
- Broker=Navigate
to the AMQP broker we just added in the previous Post.
- Queues=(Type the
name of the ftqueue queue with correct capitalization)
Step 2 - The FT API Extension VCO Workflow
If you have completed the steps in the first two posts in vCloud Director and RabbitMQ you can now just download and import the FT API Extensions workflow into
vCenter Orchestrator and move onto Step
3 below to Create a VCO subscription policy. If
you are interested in the code and logic that was used to create the
workflow and what is happening within each Workflow element I will
discuss that in painstaking detail in this seciton.
When designing the workflow I had an idea in mind that I
wanted to enable Fault Tolerance on a vApp based on a Service Link registered
for our API Extension. Logically I
worked backwards from this objective looking at the enable and disable FT workflows
that ship with VCO to determine which parameters I would need to grab from the
AMQP message being sent from the API Extension.
The workflow is the piece of the solution that will actually do the
work. It is designed as follows.
Workflow Input
A single Input Parameter will be used and comes from the AMQP
subscription policy we created so there will be no User Input required. All
parameters needed such as the VCD host, target vApp, and whether FT should be
enabled or disabled will come from the AMQP message body and headers.
Workflow Schema
Below we see the “FT API Extension” workflow schema. Lets
walk through each element to understand what is going on behind the scenes.
Workflow Element 1 - Process
Message (Scriptable Task)
Input=(AMQP:Subscription)
subscription
Output=(vCloud:vApp)vApp, (Boolean)ftstate
This is where the parsing of the message components takes
place with all of the parameters and information we need being taken from the
message payload or message body, which we have already said is JSON. The
picture below shows a message from our vCloud Director API Extension Service
that is in the ftqueue using the RabbitMQ Management plugin.
Remember the goal is to
ultimately hand off an Array of “VC:VirtualMachine” objects and a boolean value
that represents whether the user wants to Enable or Disable Fault Tolerance. We need these two objects because we are
using Enable
FT and Disable FT as
nested workflow elements and these are required input parameters for these
workflows. Running the Payload through a JSON validator shows the layout a little more clearly so we can
identify what fields we need to hand off to the next workflow elements.
[
{
"method": "POST",
"id": "85ea26af-f32a-42fb-b51a-fad0e90e811d",
"scheme": "https",
"protocol": "HTTP/1.1",
"headers": {
"Authorization": "Basic
YWRtaW5Ac3lzdGVtOnZtd2FyZQ==",
"Host": "vcd-cell2.vmab.com",
"Content-Length": "0",
"x-vcloud-authorization":
"NyJ+CcY18pUx2MlsCAO7CxgvqOQmfs8llOZR1pK5S2w=",
"User-Agent": "Apache-HttpClient/4.2 (java 1.5)",
"Connection": "keep-alive",
"Accept": "application/*+xml;version=5.1"
},
"statusCode": 0,
"queryString": null,
"localPort": 443,
"localAddr": "192.168.24.66",
"remoteAddr": "192.168.24.63",
"remotePort": 58029,
"request": false,
"body": "SGVsbG8gQVBJIQ==",
"requestUri":
"/api/vApp/vapp-ca90bfdd-93f7-4b24-a8ca-14bf5ba27fd1/ft/off"
},
{
"parameters": null,
"user":
"urn:vcloud:user:241b78de-a35a-46e6-8d7c-4afc86693b71",
"org": "urn:vcloud:org:a93c9db9-7471-3192-8d09-a8f7eeda85f9",
"rights": [
"urn:vcloud:right:a9bb4826-fd63-3df8-b604-119748cc4878",
"urn:vcloud:right:6cb3596a-15eb-3c2f-a657-5f14f2039719",
"urn:vcloud:right:1aa46727-6192-365d-b571-5ce51beb3b48",
"urn:vcloud:right:82f79ccf-3039-3436-aa99-06f1911f04eb",
"urn:vcloud:right:5250ab79-8f50-33f9-8af5-015cb39c380b",
"urn:vcloud:right:444def42-24a8-33b5-a780-13af93b52fac",
"urn:vcloud:right:b0cfe989-521b-3d7f-9bc2-f23c74a99633",
"urn:vcloud:right:2c8d98ef-4acc-3be4-9214-fcb9682b7a19",
……(Some rights removed to conserve space)
"urn:vcloud:right:f24fffde-f953-3976-9f2b-8b355b25881d",
"urn:vcloud:right:b080bb50-cff1-3258-9683-842d34255a95",
"urn:vcloud:right:4886663f-ae31-37fc-9a70-3dbe2f24a8c5",
"urn:vcloud:right:5ddb661d-caf0-3680-9a74-59d4b06137f3",
"urn:vcloud:right:60d60d89-6839-3fa7-a24e-cf5bb67cd3ff",
"urn:vcloud:right:9c449c77-f7d0-3944-bf13-e58abe1ca68c",
"urn:vcloud:right:fa4ce8f8-c640-3b65-8fa5-a863b56c3d51",
"urn:vcloud:right:2cd2d9d7-262c-34f8-8bee-fd92f422cc2c"
]
},
null
The actual Javascript from the Scripting Tab of the element is
below.
var props
= subscription.retrieveLastOnMessageTrigger();
var
Message = props.get('body');
var
headers = props.get('headers');
var
properties = props.get('properties');
// From
api-messages
var
amqpMessage = eval(Message);
var
httpMessage = amqpMessage[0];
var
httpSecurityContext = amqpMessage[1];
var
context = amqpMessage[2];
var
vcdHostUri = "https://" + httpMessage.headers.Host;
var hRef
= httpMessage.requestUri;
var
cleanUri = hRef.substring(0,51);
var
vcdHost = VclHostManager.getHostByUrl(vcdHostUri + ":443","admin","system");
var
vApphRef = (vcdHostUri + cleanUri);
System.log("Object
Type amqpmessage: " + System.getObjectType(amqpMessage));
System.log("AMQP
Message Body: " + Message + "\n");
System.log("Message
Method: " + httpMessage.method);
System.log("Message
ID: " + httpMessage.id);
System.log("Message
Protocol: " + httpMessage.protocol);
System.log("Request
URI: " + httpMessage.requestUri);
System.log("Message
Authorization: " + httpMessage.headers.Authorization);
System.log("localPort:
" + httpMessage.localPort);
System.log("JSON
Host: " + httpMessage.headers.Host);
System.log("Scheme:
" + httpMessage.scheme);
System.log("Is
Request?: " + httpMessage.request);
System.log("VCD
Host Uri: " + vcdHostUri);
System.log("vApp
only hRef: " + vApphRef);
System.log("Full
Extension API Call hRef: " + hRef);
System.log("Clean
20 character URI: " + cleanUri);
System.log("VCL
Derived VCD Host: " + vcdHost);
//Original
Code
if
(vApphRef == null || vApphRef == ""){throw "cannot proceed
without vApphref";}
if (hRef ==
null || hRef == ""){throw "cannot proceed without href";}
var urnId
= generateUrnId(vApphRef);
vcdHost.login();
vcdHost.updateInternalState();
var vApp
= vcdHost.getEntityById(VclFinderType.VAPP,urnId);
if (vApp
== null){
System.error("Failed to retrieve
vApp by ID");
throw "Failed to retrieve vApp
by Href: "+vApphRef;
}
var
ftState = getFtState(hRef);
System.log("Derived
vApp Name: " + vApp.name);
System.log("Derived
FT State Request: " + ftState);
/* URL
Processing functions */
function
generateUrnId(hRef){
var tmpStr1 =
hRef.split("/")[5];
tmpStr1 =
"urn:vcloud:vapp:" + tmpStr1.substring(5,(tmpStr1.length));
return tmpStr1;
}
function
getFtState(href) {
var hrefComponents =
href.split("/");
System.log("hRef Components Split "
+ hrefComponents);
var mode =
hrefComponents[hrefComponents.length - 1];
System.log("FT Mode: " +
mode);
if (mode == "on") {
return true;
} else if (mode == "off") {
return false;
} else {
System.log("Invalid state
request");
return false;
}
}
While I am logging many of the fields and message properties, the bare
minimum information I need to grab from this message is
1)
"Host": "vcd-cell2.vmab.com",
a.
I need the VCD host to be able to leverage the VCD
plug-in, which will be used to load a (vCloud:Vapp) object using the vApp
component of the Uri below. We will also
use the VCD plugin in subsequent elements within the workflow.
2)
"requestUri":
"/api/vApp/vapp-ca90bfdd-93f7-4b24-a8ca-14bf5ba27fd1/ft/off"
a.
I need
the requestUri because it contains the href Identifier of the vApp
b.
I also need the requestUri because it contains the
specific call that will tell our workflow whether the user wants to Turn Fault
Tolerance ON or OFF.
Workflow Element 2 –
getVmsFromVApp (Action element)
]
Input=(vCloud:vApp)vApp
Output=(Array/vCloud:VM)vms
This is a simple Action element that is included with the
vCloud Director plugin and will basically take a vApp object in vCloud Director
and return an Array of the Virtual Machines that comprise that vApp. We could have done this using Javascript but
wherever possible I wanted to use existing logic in the plug-ins to simplify
development.
Workflow Element 3 – Change
VM Objects (Scriptable Task)
Input=(Array/vCloud:VM)vms
Output=(Array/VC:VirtualMachine)vcVms
Remember we are working backwards from what we are trying
to achieve and I know that FT is a vSphere only feature and is not supported in
vCloud Director. Because of this I will need to make API calls
against vSphere to enable Fault Tolerance. We will translate the vCloud Virtual
Machines into vCenter Virtual Machines using a for loop and using the moRef for
each of the vCloud Virtual Machines as the query parameter in the
getAllVirtualMachines method.
var vcVms = new Array();
System.log("Vms
passed in: " + vms.length);
for (var i in vms) {
System.log("vCloud VM: " +
vms[i].name);
var moref = vms[i].getVMVimRef().moRef;
var XPath = "xpath:id='"+ moref
+"'";
var VMs
= VcPlugin.getAllVirtualMachines(null, XPath);
var vm = VMs[0];
var vmName = vm.name;
System.log("Name =" + vmName);
vcVms[i] = vm;
}
System.log("VMs in array: " +
vcVms.length);
Workflow Element 4 –
Decision (Decision Element)
This is a simple decision element
which will evaluate the boolean value ftState which we defined in Element 1 by
evaluating the requestUri. The decision
is if:
Fton =
True = proceed to “Enable Fault Tolerance” foreach loop
Ftoff =
False = proceed to “Disable FT” Foreach loop
Workflow Elements 5 & 6–
Foreach elements with nested workflows
Input=(Array/VC:VirtualMachine)vcVms, (boolean)ftState
vCenter Orchestrator introduced a new workflow
element in 5.1 which is the Foreach element. This element is very powerful and
replaces what was previously a series of elements that were needed to iterate
through an array and act on a single object in the array. VCO 5.1 now lets you do this with a single
element. We have an Array of VC:VirtualMachine objects called vcVms and our embedded
FT workflows expect a single VC:VirtualMachine object. We can fix this
discrepancy with the Foreach loop. When you drag the Foreach element onto the
Schema page of your workflow you will be asked to choose a workflow.
We can then edit our Foreach element we just added to map
the element inputs.
We need to provide three Inputs,
- host is NOT mandatory and we can let DRS choose
the host for the secondary machine but we should set this to Null
- turnOn should be mapped to ftState boolean
workflow parameter which was an output of our first workflow element and was
derived from the requestUri.
- vm should be bound to vcVms but vm is a single
object and vcVms is an Array of objects so this will not work until we check
the Bind as iterator checkbox. This
will tell VCO to run the “Enable FT” workflow on EACH object in the vcVms
array.
Step
3 - Create a VCO subscription policy
I will briefly cover how to take the Policy Template that
comes with the AMQP plug-in and apply it to instantiate a configurable version,
which will monitor our “ftqueue” that we created in RabbitMQ and kick off a
workflow when a message is received in this queue. As with most things VCO, Christophe has
already created an excellent post on the AMQP plug-in and even using a Policy
Template that you can read here.
- From the Administer context, under the “Policy
Templates” Tab navigate to Library –
AMQP folder and Right Click the Subscription Template and select “Apply Policy…”. Provide a name for
your policy such as “Subscription
ftqueue Policy” and select the AMQP subscription you created in Step 1,
which in this example is ”ftqueue API Extension”.
- From
the Run context, you now have a “Subscription ftqueue Policy” you can
edit. Right click and Edit the policy.
- On the
General Tab change the Startup to “On
server startup, start the policy”
- On the Scripting Tab, highlight the OnMessage, and
click the Search button to browse for the “FT API Extension”
workflow that you imported in Step 2.
-
Now set the Source
parameter (I always miss this) by clicking on the “not set” link and setting it to “self”. When you are finished it
should look like this.
Save
and close the policy and then start the Subscription
ftqueue Policy by clicking the green Start button.
Summary
If you have followed all three posts in the series, we now have the
following setup
- A registered extension service in vCloud
Director named “faulttolerance”
- A RabbitMQ AMQP server accepting the AMQP messages
from our "faulttolerance" Extension Service in the “availability” exchange and moving them to the “ftqueue”
Queue.
- A VCO policy that will run in the background
when the VCO server starts and monitor the AMQP subscription object we created
that points to the “ftqueue” on our RabbitMQ server. This policy will start the “FT API Extension”
workflow and pass to it a single input parameter of type AMQP:Subscription.
At this point you are probably waiting to actually test the solution and
make an API call against vCloud Director and actually invoke Fault
Tolerance on the Virtual Machines in the vApp.
Step
4 – Test
In order to test our solution we need to call our
faulttolernace Extension Service via the vCloud Director API. This service is only exposed through the API
so I will use the WizTools.org RESTClient 2.5 but you could also use Curl or
whatever you prefer.
- Start the RESTClient
- C:\Users\Administrator> java –jar
restclient-ui-2.5-jar-with-dependencies.jar
- Configure
SSL settings. Because this is a lab and we are NOT using CA issued certificates
I made the following changes.
-
Configure Auth Settings
- Add the following http request header
- Key=Accept
- Value= application/*+xml;version=5.1
- Obtain an Authorization Token
- POST https://192.168.24.66/api/sessions
- If you receive an HTTP Response of “200 OK” in the
HTTP Response section then you have authenticated and been granted an
authorization token.
- Copy and Paste this Token by Right Clicking the HTTP
Header in the Response labeled “x-vcloud-authorization” and selecting Copy Selected Header(s). Add this header to the HTTP Request Headers section by copying and
pasting. When complete it should look like this with a different Value
obviously.
- Using the RESTClient, we now are now ready to iterate
through vCloud Director objects in a RESTful manner as follows to obtain the
URI for a vApp we want to execute the our API Extension methods on.
- GET https://192.168.24.66/api/org
- Returns
all Organizations in the vCloud in the HTTP Response section under Body tab. Copy
the href value without quotes for an Organization you wish to use and paste it
into the URL field.
- GET https://192.168.24.66/api/org/414b903f-2e0a-4fff-b834-aec803849dbf
- Returns
all Organization VDC’s in the target Organization in the HTTP Response section
under Body tab. Copy the href value without quotes for the Organization VDC
where your target vApp resides and paste it into the URL field.
- GET https://192.168.24.66/api/vdc/51a1ec4a-fd20-4d74-918c-e86c3f4ce876
- Returns
all vApp’s in the target Organization VDC in the HTTP Response section
under Body tab. Copy the href value without quotes for your target vApp and
paste it into the URL field.
- GET https://192.168.24.66/api/vApp/vapp-ca90bfdd-93f7-4b24-a8ca-14bf5ba27fd1
- Returns all the properties of the vApp. You can now see the Service Link and
href for our extension service that will Turn ON Fault Tolerance. Copy the href
value without quotes for your “fton” and paste it into the URL field.
- POST https://vcd-cell2.vmab.com/api/vApp/vapp-ca90bfdd-93f7-4b24-a8ca-14bf5ba27fd1/ft/on
- Calls the “fton” method we defined for our Extension Service, which executes a workflow
run of the “FT API Extensions” workflow below.
A look at the VCO system shows a successful run of our FT API Extensions workflow and the System.log entries for the run.
A look at the vCenter system
mapped to vCloud Director shows that VCO is iteratively enabling Fault
Tolerance on the two Virtual Machines which comprise this vApp.
NOTE: I have not had time to
implement a Response back to vCloud Director yet with the status of the API
call such as success or failure and an error code. When I do that I will need to grab
the message properties for reply_to and replyToExchange to tell VCO and AMQP to deliver the reply to the correct exchange.
Here is another link to download the FT API Extensions workflow for your reference.
