Thursday, March 03, 2016

BizTalk Server Extensibility (e)Book

A new BizTalk Server eBook is available for the 2013 R2 version with the focus on extensibility.

Together with Johann Cooper and Eldert Grootenboer I have written on various aspects of extensibility with BizTalk Server. The .NET framework adds tremendous value in extending BizTalk Server solutions to create more versatile, robust solutions. We discuss the extensibility for BizTalk Server from a holistic view around the following topics:

- Ports
- Orchestrations
- BRE
- BAM
- Deployment
- Operations
- Tooling


The eBook is available through the BizTalk360 website.

The journey started after my first webcast BizTalk Extensibility for the Integration Monday a year ago in February 2015. At first I did not have any plans to pursue anything after that talk. However, a few months later I got in touch with Johann Cooper, who I interviewed for my blog and met in person in 2014 during my visit to Australia.

We discussed opportunities of writing a book for a well-known publisher. Unfortunately a deal never materialized and we started to think about other options. Our initial thought was to create a white paper on BizTalk Server Extensibility as this didn’t have the attention we though it should have. We felt the .NET framework empowers BizTalk Server in a very valuable way, if applied with thought.

Johann and I started writing the paper at the end of September 2015. And soon it turned out that we had many topics around extensibility and predicted that the amount of material would validate making an eBook out of it rather than a paper.

Our prediction was right. After a few months we had close to 100 pages, yet some topics still required attention. Therefore, we involved Eldert as an extra co-author to end of the book in time with his expertise on some of the topics like BAM and the BTDF. His help and our motivation helped us to finish what we started in September.

We were not alone in this, as Sandro Pereira, Mark Brimble and James Corbould put in numerous hours to review our eBook. These guys are superb reviewers and helped us a great deal to evolve the eBook to a very valuable resource for you as readers to benefit from. Therefore, Sandro, Mark and James, thank you so much for your efforts and support!

I also like to extend my gratitude to BizTalk360 and Saravana Kumar, who has publish the eBook through his site. Saravana is a great support towards the BizTalk community, thank you my dear friend.

Working on this eBook demanded a lot of time for us and it was worth every second of it. I enjoyed working on it and collaborating with both co-authors and like to thank both Johann and Eldert for realizing this eBook. A year ago I never thought this end result was in the making.

Enjoy reading it,

Steef-Jan

Tuesday, February 09, 2016

BizTalk360 Version 8.0 Review

A new version of BizTalk360 has been released, version 8 which comes with a boatload of features and a complete overhaul of its user interface to provide an exceptional user experience. The product adopted a concept of widgets in the dashboard similar to the new Microsoft Azure portal. A BizTalk 360 user can now create and customize dashboards for different use cases.

The User Interface, Dashboards and Widgets.


In this post I will review a few of the new features and let’s kick off with the UI, which has improved dramatically. Intially BizTalk360 was a Silverlight application before it migrated to HTML 5. And in version 8.0 it has adjusted to what you experience on the Azure Portal and what you see with UI in devices. Below you see a customized dashboard with several widgets.
















Figure 1. An example of my customized monitoring dashboard.

The end user has also the ability to create his own custom widgets via the settings page.


























Picture 2. An example of creating a custom widget.

Furthermore the separation of monitoring and analytics is based upon audience targeting. An analyst or support engineer can interpret information of messaging and processes within a BizTalk Group from a different perspective as shown below demonstrating a customized dashboard.











Figure 3. An example of a customized analytics dashboard..

BizTalk360 API’s


Another incredible addition are the API’s, which offer a tremendous amount of value to enterprise customers. It brings extreme extensibility for its end users and enables the for instance to pull information from several environments. The API’s (REST web services) are the core of the BizTalk360 capabilities and the BizTalk360 front-end uses them. And as a BizTalk360 end user you can leverage all of these API's to have access to your BizTalk environment. In the setting menu under API Documentation you can find the descriptions of each API and operation. For instance if you select BizTalkApplicationService you can observe a list of operations belonging to it. Subsequently select one of the operations, Services.REST/BizTalkApplicationService.svc/GetOrchestrations, specify the environmentId and applicationName and hit “Try it out!”.














Figure 4. BizTalk360 API’s document page..

You can call the API’s from different machines, by setting the appropriate authorization in the user access policy. Calling the API operation from SoapUI will result in the same response as from BizTalk360 API Documentation.















Figure 5. Calling BizTalk360 API from SoapUI.

The look-and-feel are similar when you publish an (Web) API configured with swagger documentation describing it.

Business Rules in a browser


Yet another new feature added to BizTalk360 is the rule engine capability of BizTalk Server i.e. the business rule and composer are accessible from a browser. You can view what type rules are deployed.














Figure 6. Deployed rules in a BizTalk Group.

And one of the capabilities is that you as end user can compose new rules, edit or test them. Below you see an example of testing a deployed rule, which you can do in a browser!

















Figure 7. Test of a deployed business rule.

The version of the business rules composer in BizTalk360 only supports pre-created vocabularies. Therefore, to ensure a smooth user experience, you need to create all the vocabularies you will require to develop or edit your rules in the BizTalk Business Rules Composer. In a nutshell follow the instructions on the BizTalk360 portal with regards to rules.

Live feeds


In an enormous environment various operation people perform various actions in a production or UAT environment, and it can be valuable means of real-time tracking of what happens. This new version of BizTalk360 offers live feeds and you view on the right side after click the icon (indicated in red in the picture below).


Figure 8. Live feed in BizTalk Group.

Alarms and notifications


In BizTalk360 you can set alarms, which can result in an email being sent to you.















Figure 9. An example of email indicating a receive location is down.

Or you can leverage one of the out of the box notification channels like Slack and configure an alarm using a notification channel.




















Figure 10. Enable the SlackNotification on a custom alarm.

Once you have set up Slack following the instructions on BizTalk360 UI and the alarm you will receive notifications in Slack.

















Figure 11. Notification in Slack.

Final words


The BizTalk360 team has worked intensely the last 12 months to bring this to the table. Not just the flexibility with widgets, yet also with the separation of monitoring and analytics, the 360 plus API’s on top of your BizTalk Group, live feeds, notifications and the rules engine. This product has definitely evolved to the next level. Expect the release of 8.0 to change the way BizTalk is being monitored!

Cheers,

Steef-Jan

Sunday, December 27, 2015

Year 2015

Time flies when your having fun. I did have fun this year working with Microsoft Azure and Integration at a utility company. Did an IoT project, bridged on premise applications to the outside world (Service Bus Relays and Queues) and some old skool integration. What else did I do? Well you might have guessed: speaking, travelling, and have an occasional beer. Pictures tell more than a thousand words.


























Thanks everyone for reading my blog and I will continue to share my knowledge through speaking and writing in 2016.

Cheers,

Steef-Jan

Monday, October 12, 2015

Upcoming Events

In two weeks time I will head out to North America. My first stop will be Calgary to visit my friend Kent Weare. My MVP buddies Saravana Kumar and Michael Stephenson will join me as we will be speaking during the Azure Hybrid Integration the 30th of October at Microsoft Calgary.

The Azure Hybrid Integration will be a full day with Kent, Saravana, Michael, myself and Darren King from Microsoft speaking on the following topics SaaS connectivity, IoT, Hybrid SQL Server, BizTalk administration & operations and Two Speed IT using Microsoft Azure.

The free event takes place on October 30th, 2015 at the Calgary Microsoft office. You can find more details here.

After this event we will head out to Redmond for the yearly MVP Summit. This is a multi-day event that is hosted in Bellevue and at Microsoft headquarters in Redmond, Washington. All the MVP in the world that are able to come will be there to connect with fellow MVP’s and the various product groups.

I look forward to both events as it will enable me to engage fellow MVP’s, the integration community in Canada and to learn more about the future road maps of integration, Microsoft Azure and direction of Microsoft itself. The IT world around me is changing rapidly and for personally I am noticing it as my projects change from an on premise integration- to hybrid solution focus.

For those people that register for the Calgary event,  I am looking forward to meet you and to have some interesting discussions.

Cheers,

Steef-Jan

Saturday, August 22, 2015

Azure WebJobs: ServiceBusTrigger

Challenge is to build a WebJob that listens or monitors a queue in the Microsoft Azure Service Bus within a certain namespace and pick of each message that is send there by a message producer and process it. The WebJob acts as a message consumer of the messages on the queue. Below a high level diagram of a scenario that will be explained in this post and how I faced the challenge.

image

You can build WebJobs inside Visual Studio by installing the WebJobs SDK. Once you have installed the SDK you have template available to build a WebJob in C# or Visual Basic.

clip_image004[6]

You can select this template specify a name for the WebJob and click Ok. You will see that a program class will be created.

clip_image006

And a Functions.cs.

clip_image008[6]
By default a method will be created for you to monitor or listen to an Azure Storage Queue, not Service Bus Queue! To have method that will be triggered/executed when a message is written to an Azure Service Bus Queue you will need to have ServiceBusTrigger. This not available in the project and you will need to add the Microsoft.Azure.WebJobs.ServiceBus NuGet package.

Now a can change to ServiceBusTrigger in method ProcessQueueMessage and specify the queue I want to listen to.

clip_image010[5]

Next change is changing the type of the message from string to BrokeredMessage type. This type is not available in your class unless you add using statement for Microsoft.ServiceBus.Messaging. The package is already in the project, because it is part of the imported NuGet package. The TextWriter object can be used to write log statements that can be viewed in the AzureWebJob Dashboard.

When a message arrives on inboundqueue it will be picked up by WebJob and enter the ProcessQueueMessage method in runtime. Here I can extract the message body and send it for instance to Redis cache as key value pair (reason of picking this example is based on a request from someone on twitter to share how to do that). To send it to Redis Cache I need to import another NuGet Package i.e. StackExchange.Redis (client library). Now the complete code for the functions class looks like below:

clip_image012

Before the WebJob can be deployed to a WebApp a few configuration settings have to be done in the app.config. The connection strings for the AzureWebJobsDashboard and AzureWebJobsStorage need to be provided in the connectionStrings Section. These are required to view the log in Azure i.e. AzureWebJobsDashboard. The connection string that needs to be specified is the connection string to an Azure Storage account. Format is as follows:

DefaultEndpointsProtocol=https;AccountName=[Storage Account Name];AccountKey=[Access Key]

The other connection string that has to be provided is for the AzureWebJobsServiceBus. Format is:

Endpoint=sb://namespace.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=[Access Key]

Finally in the appSettings section the connection string for the Redis needs to specified. Connection string has the format of:

namespace.redis.cache.windows.net,abortConnect=false,ssl=true,password=[password]

Once configuration is done the app.config will look like:

clip_image014

Before the deployment (Publish to Azure) of the WebJob can be done a configuration setting in WebApp has to be done to enable AzureWebJobDashboard.

clip_image015
This is an important step. In case you forget this than observing the WebJob logs will result in the following error:

clip_image017[6]

Now the WebJob can be deployed via Visual Studio to a WebApp. Right click on the project and choose Publish as Azure WebJob...

clip_image018

You will see a Publish Web dialog and here you import the publishing setting from WebApp. These settings can be downloaded from Azure Portal.

clip_image020

Next you can click Ok and you will go to next section of the dialog i.e. Connection.

clip_image022

Click Validate Connection to see if connection info is correct. When valid you can click Publish. Now the WebJob will be published to WebApp. In the output window of Visual Studio you will see that deployment went successfull.

clip_image024

In the Azure Portal you can see the WebJob in the WebApp.

clip_image025

When you click on the logs url you will be redirected to the Microsoft Azure WebJobs portal.

clip_image027

Nothing much has happen so far, only that the Job has started. In case I send a message to the queue using for instance ServiceBus Explorer, I will see some action. Send a message via the ServiceBus Explorer to the queue.

clip_image029

Refresh the AzureWebJob Portal and a new entry is available.

clip_image031

Once you click on the Functions.ProcessQueueMessage you examine the logs.

clip_image033

To explore what is in my REDIS cache I need to navigate to the service in the Azure Portal and open a console. Enter GET and messagid.

clip_image035

image

As you can see the message is now in the Cache.

It took me sometime to get the ServiceBusTrigger working. After some digging around I was able to get the ServiceBusTrigger working and see its behaviour through the Azure WebJob Portal. The trigger is not limited to queues as it will also work for Service Bus Topic/Subscription. The signature of the method would look like:

clip_image037

Resources to explore with regards to this blog post are:

· http://stackoverflow.com/questions/15441853/with-azure-brokeredmessage-get-the-body-without-knowing-the-type
· https://azure.microsoft.com/en-us/documentation/articles/websites-dotnet-webjobs-sdk-get-started/
· http://blogs.blackmarble.co.uk/blogs/sspencer/post/2014/09/22/5-Tips-for-using-Azure-Web-Jobs.aspx
· http://stackoverflow.com/questions/28077330/why-do-i-need-to-configure-connection-strings-for-webjobs-in-azure-management-po
· https://azure.microsoft.com/en-us/documentation/articles/storage-configure-connection-string/


Cheers,

Steef-Jan

Wednesday, July 01, 2015

Book Promotion: SOA Patterns with BizTalk Server 2013 and Microsoft Azure - Second Edition.

It’s been awhile since a BizTalk title hit the market. Colin, Mahindra, Mark and Johann updated Richard Seroter’s well received book SOA Patterns with BizTalk Server 2009. I loved Richards’s book and it inspired me to write a book myself. We have met on several occasions. And last year I had the privilege of meeting Mark and Johann in person in Sydney. Both are enthusiastic integration professionals that with Colin and Mahindra’s aid created and updated this book. Excellent work guys, and respect as writing a book is a challenge!

This book covers BizTalk, WCF, JSON and Rest Support in BizTalk, Azure BizTalk Services, Azure Service Bus, SOA, Schema's and Endpoints, Asynchronous Communication Patterns, Versioning-, Orchestration Patterns, Frameworks and Tools. That is a broad spectrum of today’s integration capabilities on the Microsoft Platform (on premise and cloud).

image

If you are new to, intermediate- or well versed in the integration, this book is going to give you more insight into Microsoft’s integration world. The thoughts of these experienced authors and Richard original transcript will help you on your journey to build sustainable integration solutions. Especially now landscape around us is changing fast and integration has become key to success for enterprises to survive.

You read this book in your hand, read on your tablet or computer while enjoying a beer. It’s worth the investment and beneficial to your career in integration. Buy the book online at Packt Online Store, Amazon or perhaps your local bookstore.

Thanks Richard, Colin, Johann, Mark and Mahindra.
Cheers,

Steef-Jan

Microsoft Integration MVP 2015 – 6th Time in a row!

Today I have received an e-mail from Microsoft with exciting news that my MVP status has been renewed again!

[MVP_Horizontal_FullColor%255B4%255D.png]

For me this is the sixth time to receive this award. The fifth year in the program has been again an awesome experience, which gave me the opportunity to do great things and meet inspiring, very skilled people. I have had some interesting speaking engagement, which were fun to do and were very fulfilling. I learned a lot through speaking thanks to the community and fellow MVP's. I was able to share my experiences through these speaking gigs and other channels like this blog, MSDN Gallery, and above all the TechNet Wiki.

I would like to thank:
  • My MVP old lead William Jansen, and new MVP lead Birgit Huebsch.
  • The BizTalk Product Team, Mark Mortimore, Guru Venkataraman, Ed Price, Mandi Ohlinger, Allesandro Teglia, Dan Rosanova, Jon Fancey, Paolo Salvatori, and all other Microsoft employees involved.
  • People at my former employers: Rene Brauwers, Eldert Grootenboer, fellow MVP Edward Bakker and many others. 
  • At my current company DutchWorkz : Rutger van Hagen and colleguaes.
  • The BizTalk Crew: Saravana Kumar (BizTalk360), Nino Crudele, Sandro Pereira, and Tord G. Nordahl
  • Fellow Microsoft Integration MVP's: Richard Seroter, Kent Weare, Mikael Håkansson, Johan Hedberg, Stephen W. Thomas, Mick Badran (Azure), Micheal Stephenson, Tomasso Groenendijk, Nicholas Hauenstein, Salvatore Pellitteri, Sam VanHoutte, Glenn Colpeart, Bill Chesnut, Leonid Ganeline, and Ashwin Prabhu, who I got learn even better and supported me in this program.
  • The BizTalk community: Mikael Sand, Lex Hegt, Colin Meade, Naushad Alam,Howard S. Edidin, Johann Cooper, Mark Brimble, Mitch VanHelden, Sven Van den Brande, Jérémy Ronk,  Maxime Labelle, Jean-Paul Smit, Dean Robertson and the collegueaes at Mexia, Martin Abbott, and many others that make the BizTalk community strong! 
  • Andrew Slivker from Sentinet.
  • Finally my wife Lian and children Stan, Ellis and Cato for their support.
I’m looking forward to another great year in the program.

Cheers,

Steef-Jan

Saturday, March 14, 2015

Upcoming speaking engagements in April

In my last post I created some awareness on the upcoming BizTalk Summit in London, 13 and 14th of April. This event will be the biggest Microsoft integration focussed summit in Europe. Microsoft BizTalk product group, Microsoft Integration MVP and veterans will speak about integration, Azure and API Management.This event is once again, like the previous two events, being organized by BizTalk360 in conjunction with Microsoft and the BizTalk Product group. There are various reasons you can think to attend like for instance described in the blog post by one of the speakers Sandro Pereira.

Biztalk Summit

My topic is on Hybrid connectivity and more specifically what BizTalk Server 2013 R2 platform offers today. You can read the details below.

Hybrid Solutions with the current BizTalk Server 2013 R2 platform

The IT world has changed with the rise of the internet (cloud). Google, Amazon and Microsoft offer a variety of services in the cloud for storage to applications. Besides them there are a ton of other vendors selling software as a service (SaaS), or provide a dedicated service for instance Drop Box offering storage on demand. This means that integrating on premise, cloud services and software will generate a new demand. Enterprises will now face these challenges as they will need to integrate their on premise systems that are not likely to move to the cloud like SAP with cloud services or solutions. The latest BizTalk Server release 2013 R2 offers capabilities to full fill the demand for a new hybrid type of integration solution. In this talk various hybrid integration scenario’s will be discussed and how you can leverage Microsoft BizTalk Server 2013 R2 to build these solutions.

After this event, I will the week after travel to Sweden for Swedisch User Group event in Gothenburg and Stockholm. I am invited by Johan Hedberg to join him on stage to talk about API Management, on-premise (Sentinet) and Azure API Management.

The last stop for me in April will be the BizTalk Bootcamp in Charlotte, US. Two years I was invited to come over and talk, but was unable to make it. However, this time I will be there! I will speak on similar topics like the London event, with probability to do other talks on API management and BizTalk extensibility. The event is being organized by Mandi Ohlinger who works for Microsoft and is responsible for a lot of the technical content that you find on Azure BizTalk Services and BizTalk Server. This is a free event and registration is necessary!

Bootcamp 2

See you on the road on any of these events.

Steef-Jan