Mike The Architect

The Open Group SOA Governance Framework Becomes an International Standard

The Open Group has moved the technology industry forward yet again with its second international standard on SOA.The Open Group SOA Governance Framework is now an International Standard, having passed its six month ratification vote in ISO and IEC.

This will be the second since January 2012 when the Open Services Integration Maturity Model (OSIMM). The Open Group has certainly have been getting a great deal of momentum on it's SOA OSIMM as it is being considered for adoption as a national standard in countries such as China and Korea. While this framework has been around for awhile it is great to see the broad acceptance internationally.

SOA governance is critical to all SOA implementations. Just like with building a road, laying the asphalt is just the beginning. That road will need  governing aspects such as the law and maintenance plans. The same with SOA implementations, deploying an ESB with some web services is relatively easy but making sure that those services are adopted and sustainable, that is tricky.  Having a standard way of addressing SOA governance is not only an accelerant to companies but also allows them to:

• Leverage a universally accepted and trusted framework
• Exposure to an evolving framework that is leveraged by many companies that contribute to the Open Group with best practices,  guidance and standards
• Eventually tap into a network of SOA practitioners that understand the standard without training
• Potential for certifications to validate competencies and skills

The Open Group SOA Governance Framework includes a standard governance reference model and a mechanism for enterprises to customize and implement the compliance, dispensation and communication processes that are appropriate for them. Long term vitality is an essential part of the framework, and it gives guidance on evolving these processes over time in the light of changing business and technical circumstances, ensuring the on-going alignment of business and IT.

What I really like about this model is that SOA Governance is viewed as the application of Corporate Governance, IT Governance and EA Governance to Service Oriented Architecture. This shows that SOA Governance extends IT and EA Governance ensuring that the benefits that SOA extols are met. This requires governing not only the execution aspects of SOA but also the strategic planning activities.

Take a look at the new standard, it is certainly worth the read!

Resources

• The Open Group SOA Governance Framework Source Book on The Open Group Website
• The Open Group SOA Work Group
• The Open Group Cloud Computing Work Group
• IBM’s library, which offers papers and services to help with developing a SOA Governance Regimen: http://www-01.ibm.com/software/solutions/soa/gov
• Gartner – SOA Governance remains crucial – http://searchsoa.techtarget.com/news/1261208/Gartner-SOA-governance-remains-crucial
• Gartner – SOA Governance – http://www.zdnet.com/blog/service-oriented/soa-governance-siloed-thinking-top-of-mind-at-latest-gartner-confab/6254

The Lending Message Bus Gets Along with all Standards

Recently I've been asked about if the Lending Message Bus (LMB) supports other standards other than MISMO. The answer is... of course it does. The LMB can support one or more standards. This is due to the modular design of the adapter and accelerator framework within BizTalk Server.

So let's take a step back, what are accelerators. This doesn't get communicated very well. At the highest level accelerators are based on an adapter, for example the MISMO accelerator is based on the XML adapter. The difference between an adapter and accelerators is that an adapter is much lower on the stack. Adapters deal with XML or File transformations and accelerators are about mapping that data to support message routing or high level business processes.

Since maps are used for accelerators that means that they are coupled with an orchestration. So what you see above are autonomous accelerators that can be added to the lending message bus. Since they are autonomous, adding and removing does not effect the other standards accelerators.

 

The accelerator maps enable you to utilize the adapters to translate and transform messages. Translation is the process of converting a message from one format to another format, such as converting a flat file into an XML file. Transformation is the process of taking information from one message and inserting it in another message.

Note: Maps are tightly coupled with an orchestration. You can not have a map dynamically spin off other orchestrations. This is taken into consideration below.

 

How should you architect support for multiple standards?

You want to make sure that you keep the accelerators very thin. The schema maps should only take care of translation and transformation to become usable by a set of course grained services. So in essence you have two layers of orchestrations.

The first layer handle the movement of the information in the lending message bus. The second layer of orchestrations for the consumption into enterprise business processes. The rational for this is to separate the translation services from the business services.

We want to avoid blending these two layers together for the following reasons:

• Unnecessary duplication of code
• Siloed standards support
• Constraints on multi-channel integration
• Time to deliver standards support is impacted due to the amount of duplicate application logic

 

Are there any available accelerators for the Lending Message Bus?

Yes, Additional standards for Commercial Lending have already been developed by Microsoft Consulting Services built in the Commercial Lending Framework (CLF). This requires an MCS engagement to purchase the accelerator.

There have been some discussions of the creation of both a British and Australian (LIXI) accelerator. Stay tuned for more news on that.

[Update]

Gartner wrote a similar article on their site that aligns with the guidance in the Loan Orgiantion Reference Architecture.

• The Lending Services Hub - Lending Architecture of the Future
• Banks must Rethink their Lending Architectures

 

Technorati Tags: ESB,Message Bus,Lending

Unstructured Data, the Achilles Heel of SOA

You may be wondering why I saying that unstructured data is the Achilles Heel to SOA. Very simply SOA has been so focused about the plumbing and connectors and have completely lost sight of the user. At the end of the day SOA was supposed to make life more bearable for business users by providing them systems that were more agile and provide a rich integration experience. I think that there has been some great work done by both Microsoft and others but I think we got lost down one of the trails. Unstructured data is that connector into the business user.

Gartner projected that in 2003 business users will spend anywhere from 30 to 40 percent of their time managing documents. Similarly, Merrill Lynch estimates that more than 85 percent of all business information exists as unstructured data .

If you have seen some of my recent presentations I talk a lot about this topic. To date I have not had a focused session on this merely referred to the issue many times when presenting Office Business Applications. I think I will need to talk about this more.

I have an increasing number of questions from customers about what is my option on this thing called unstructured data. Specifically I was asked this question below:

What does “unstructured data” mean to my bank and what are the underlying issues?

This sounds like it could be a “technological solution to a political problem” silver bullet that the customer is looking for.  And I’ve found silver bullets usually end up making interestingly shaped holes in feet.

I have removed names to protect the innocent... :) At any rate, I would like to comment that this is a very serious problem in enterprise today not just Financial Services. I believe that it is not a political problem but rather it describes how human behave with technology. This information supports many of the human based processes that makes us all productive. However, when IT tries to force structure on a process that does not require it the process is then inhibited.

From the image above you can see that human create and consume information differently than applications. Business users are productive by creating an Excel worksheet to calculate figures or rates rater than going into a structured environment. Many times it is too restrictive to the business user.

Who needs unstructured data? Should we just migrate to more structured data options? I think that is a real bad idea for many reasons. But the biggest reason is that systems look for syntax and humans look for patterns.

So what are some of these unstructured data objects?

• E-Mails
• Reports
• Quick notes to workers
• Excel Files
• Word Documents
• PDF Documents
• Images (e.g., .jpg, or .gif)
• Media (e.g., mp3, .wma, or .wmv)
• Text Files
• PowerPoint Presentations
• HTML to a lesser degree

 

Below is a chart that I like to use that illustrates the differences between structured and un-structured data.

I used a similar method for breaking these down as Gartner. However, there was a bit missing so I kept the structure but added content. If you would like to see the Gartner article and you have a subscription here is the article name:

The New Data Integration Frontier: Unifying Structured and Unstructured Data

What is Microsoft's Story?

Microsoft has a very strong office productivity presence that coupled with the new 2007 Office Server System and you have a great story.

Yes I am saying OBA yet again...

So lets talk about how to OBAs can facilitate the creation, storage, information management, routing, downstream system consumption, synchronization and security.

Content Creation

The business users create content in familiar Office applications such as Outlook, Excel, PowerPoint or Word.  The majority of the business users do not work in a database platform, data entering information.

The majority of these tools use Open Document XML standards. This means that as you and I work in an unstructured way under the covers the data is being represented in a way systems can understand.

By using these formats the information integrates quite nicely in with the new Microsoft Office SharePoint Server (MOSS). This leads us to our next topic... Storage

Storage

Hosted within MOSS is a service for storing information. This service is called SharePoint Document Libraries. What I really like about this environment is that it's not just a place where documents reside and go to their grave.

Services can be configured rather than coded:

• Workflow
• Polices
• Document Expirations and Retention Policies
• Questioning of Information

Classifying Information

It is one thing to store information but how do you classify the data? On all Office Documents there is the ability to wrap meta-data around the document. For other non-Office data MOSS provides the facility to wrap meta-data around all items stored in a document library.

Essentially, MOSS is a meta-data repository for all your unstructured data. As an example, lets say that a home title was provided by a title agency in electronic format as a .tiff image. Meta-data could be wrapped around that document to describe both it's contents and purpose.

Downstream systems can then pull that information based on that meta-data. Additionally, business users can find this data easier.

Routing and Human Consumption

As I pointed out Document Libraries are a great way to get information into MOSS. But how do you route this information within your enterprise? Architects can eliminate the need to write these workflow's since Windows Workflow Foundation provides routing services for information. A great example of this is being able to subscribe to content by RSS or E-Mail alerts. For actionable information routing Outlook tasks can be sent.

For an example of what the workflow could look like, here is an example from the OR-LOS solution. This workflow shows how to extend simple flows and add approval based processes. The approval workflow shows how to save information to logs and trigger events. All of this was click and drag rather than coding of any sort.

 

System Consumption

Since the MOSS workflow is based entirely on WF system centric routing of information can be easy to achieve by using Windows Communication Foundation. Obviously this can be stubbed out many different ways depending on the varying architecture decisions you may have. For example, the OR-LOS solution uses XML web service proxys throughout the architecture. This allows for the maximum amount of interoperability between systems. WF provides many options here.

 

Synchronization

Sync'ing data from these two worlds are critical. Many times a business user is working on a Form for example. There may be initial data entry stages then back-end process must kick-off. A very common scenario.

In the OR-LOS solution this occurs often. The illustration below shows all the communication touch points in the architecture.

The Master Loan Flow which is ran from MOSS holds all of our unstructured data. In this case InfoPath is used as the entry mechanism. These forms capture all the loan data that is required by the system. Once the data entry is at a point in which the system can take some actions, web services are called down to the Lending Message Bus which in turn traverses throughout the layers.

The data is massaged and automated tasks are executed. Once there is human intervention needed an event is triggered in the MLF (Windows Workflow Foundation). In this process it kicks off alerts and updates work pipelines.

It's important to note that there needs to be a governing system. Windows workflow provides this. It facilitates the communication between the human and system workflow's.

Security

When dealing with data most times there is information that should be protected. MOSS provides a platform that integrates nicely into a Single Sign On (SSO) enabled enterprise. This is nothing new on the market. What does differentiate MOSS from other solutions is that there is Information Rights Management (IRM) out of the box.

With IRM messages can be routed to participants and provides enterprises the ability to control that information. You can lock down information by eliminating forwarding, copy & paste, and print screen activities.

There are many options and would encourage you to read the MSDN articles that describe the architecture. You can choose to deploy IRM into the same Outlook user interface that you use today. Additionally IRM can be extended to Smart Clients.

 

Conclusion

I hear from our customers that their SOA attempts have been hindered or have failed as a result of not being able to show the business iterative results that enable them to be more productive. Businesses are still very hopeful that SOA will provide tangible business value. To do so uniting the unstructured world and the structured is essential.

Remember there are no silver bullets, but MOSS provides a great platform to build upon to facilitate the unification of unstructured and structured data.

Technorati Tags: SOA,Composite Applications,Integration,OBA,SharePoint,Application Services,Architecture

The Microsoft Lending Message Bus

The lending message bus (LMB) provides messaging services for the lending processes. This made sense to since there are many touch points in the process. Both internally and externally there are points that need management and consolidation.

The LMB follows Enterprise Service Bus (ESB) practices it can also integrate well with in the enterprise ecosystem. Either augmenting or apart of the centralized ESB.

Below you will see a logical model of the components of the lending message bus and dependencies. The OR-LOS solution currently only takes MISMO XML. This is not to say that you can't add other inputs.

 

 

As shown by the logical architecture, XML Web Services (WS) or flat files are sent to the LMB. From here they are consumed by the MISMO accelerator hosted within BizTalk. The data is transformed and routed to the appropriate destination based on the business rules and orchestrations. What this means is that instead of hand coding message routing in C# messages are managed centrally in BizTalk abstracted out at a system level instead of the application. This loose coupling between messaging and the application opens the door for interoperability with other systems that may be the same or other business processes.

It is important to note that this is a logical representation of the message bus and not a physical. These message bus can be deployed within a tiered DMZ for external communications as well.

You are probably asking, well that's great from a technical perspective but what about from a business perspective? From a business perspective this is important to any bank that has a multi-channel integration strategy. The overall OR-LOS lending framework lends to this, but the LMB really enables us here. With the LMB it serves as a platform that can unite lending channels through common processes and information.

When we look at the larger issue of breaking down silos with other channels that are outside of lending the arena the LMB again facilities this through both technology and industry standards. Since the architecture encourage the use of XML web services and industry standards such as MISMO interoperating with commercial off the shelf (COTS) and homegrown or custom developed systems are easier. There is still a lot of heavy lifting left however.

The governance aspects can be facilitated by technology but you have to actually implement and govern the technology that enables the process. The point I am making is that this technology is not unique, but rather the implementation of it is best practice for accomplishing this goal.

 

As shown above the LMB has several benefits to the business. I break it down to 3 core business capabilities. As shown above there are direct benefits to the business.

 

I leave you with this... BREAK DOWN THOSE LOB SILOS!!!

Photo by TomasSoukup

Mobile Extensibility with the Loan Origination Reference Architecture

In my discussions when I discuss the OBA RAP for Loan Origination Systems with folks at conferences and other speaking engagements they usually see a slide that shows the extensibility aspects of the lending architecture.

 

After the talk I usually hear, mobile, really? Lender can use this technology with field bound loan officers or inspectors. This would be a key differentiation. Using mobile technology can increase the speed of loan processing, information is not duplicated from paper to electronic, underwriters are instantly aware of loan information, etc.

Since the OR-LOS architecture uses standard XML throughout it is easy to extend. As shown above an external web services proxy would need to be deployed. This in turn would communicate with the internal Lending Message Bus which would carry the business process forward through the Master Loan Flow (MLF).

As shown above as a possible example of a real-world scenario, the appraisal process. The lender usually has a service for appraisal in which thy make a request to have an appraiser inspect a house. In this scenario that an appraiser's business is alerted through an automated system. Below are the possible workflow steps in the automated process:

• The loan is at a point in which appraisal is needed, an event fires in the MLF
• A web service call is made to the lending message bus (LMB)
• The LMB then creates a system orchestration in BizTalk which manages the asynchronous call that is made
• The orchestration then calls out to a fictitious appraiser network, this is a system that aid the assignment of appraisers
• An automated task is sent to the appraiser
• The appraiser now has all the details in which he/she can perform an inspection
• The inspection occurs and the Appraisal Inspection Application which is a Compact .Net application
• It is launched and data is entered
• After completion a web service is called and the data is ultimately transmitted back to the lender.

The light weight .Net Compact application can be deployed on a mobile device that talks to the lending system to expose this information. A nice bonus is that WM has Office installed by default. This means that you can see tasks as they flow through the system and react accordingly. All of this is in one device, so instead of getting out a laptop a WM 6 device with a keyboard can be used that  is very compact.

Below is a real world case study on the lending area:

Safeguard Properties speeds mortgage field inspection 21 percent with mobile solution. Inspectors for mortgage field service firm Safeguard Properties used laptops, non-Windows handheld's, and paper forms to complete property inspections, which meant relying data and maintaining multiple applications. To speed the process, Safeguard replaced the system with a Windows Mobile powered solution. The outcome: turnaround time for inspection results decreased 21 percent, and the time for the staff to review and conduct follow-ups is down 65 percent. Changes to respond to client needs or regulatory requirements are implemented within hours, not weeks.

The company estimates that annual maintenance costs will decrease 67  percent.

See other related stories on the Windows Mobile and Mobile Financial Services Datasheet. This is more at the sales level of detail so be aware of that before reading.

OBAs Deliver SOA to the People

The OBA RAP for Loan Origination Systems further solidifies Microsoft's thought leadership in both the SOA space but in Financial Services. By exposing OBAs and the MOSS services as atomonimous composite applications. OBA surface all the hard work that SOA architects have been striving to expose.

The OR-LOS solution shows core SOA principles of service orientation are all used here in this architecture.

These services are not just for Microsoft solutions but can be applied to many other platforms as well. Described below I break up how I separated the services. We will explore the generic enterprise services that MOSS exposes. Then we will talk what business services we expose.

MOSS Enterprise Application Services

The OBA RAP for Loan Origination Systems (OR-LOS) is unique variation of an Office Business Application (OBA). This OBA uses many service oriented architecture concepts in the design.

We use the capabilities of Microsoft Office SharePoint Server (MOSS) and we externalize some of them as services to the enterprise. Some of the services are only available through the Microsoft Platform. Those services will have limited reach into Java or legacy environments such as: COBOL, FORTRAN, etc.

For those services we can expose, we use standard XML interfaces. These services include:

· Workflow – Web Services allow internal and external applications within the enterprise to orchestrate business workflows. The way in which the architecture utilizes the Windows Workflow (WF) Foundation is that the business processes are extended by XML Web Services and Document Libraries. This makes WF service to LOB systems of varying blends (i.e., languages). There are two types of workflows that are used for this solution

o Business User Created Workflows – Ad-hoc workflows that business users can create at will. For example, create an approval processes around information or a Word document.

o Pre-Defined Workflows – Workflows define the business process.

· Storage – By using standardized WebDav protocols for attaching to remote file stores these services can be used as another entry point into the MOSS environment. In turn this can be a service in which application and users can store documents with versioning, auditing and document retention policies attached

· Portal – This is a User Interface service that can expose both .Net applications and Web Parts but also WSRP complaint web parts.

· Excel Services – Excel Services provides 2 core services to the architecture.

o Calculation Services – These services can be used with standard XML and OpenDoc XML formats.

o Excel Web Views – Standard HTML views of Excel data.

· Integration with Line of Business systems – Using the Business Data Catalog data can be aggregated to the user interface using the metadata repository stored and managed by MOSS.

 

OR-LOS Business Services

These are the business services modeled on the business process of Consumer Loan Origination.

Services used by the OR-LOS solution are as follows:

1. Lender Content Management – Using the web services proxies defined in the system custom messages can be displayed within the portal environment to the brokers and to the lender’s personnel.

2. Alerts and Messaging Services - With the WS integration points into workflows such as the Master Loan Flow (MLF) e-mail alerts and or statuses can be sent to both the lender personnel and the brokers

3. Loan Registrations – The MLF workflow is extended using XML Web Services using MISMO standards. With these end points exposed other Loan Origination Systems (LOS) and Servicing systems can attach to the Loan Registration Service and utilize its capabilities.

4. Decisioning Engine – The Decision Engine provides a way to consolidate LOS product and pricing (e.g., 30yr arm at 5.5 with 1 point paid) decisioning. It only accepts standard MISMO XML and provides the responses in the same manner.

5. Automated Underwriting Services (AUS) – Using the Business Rules Engine combined with the Orchestration Layer within BizTalk we expose AUS web services that consume MISMO XML to provide underwriting decisions to any system that can use XML.

6. Data Entry Form Services – InfoPath provides a mortgage industry standard way of collecting information in the process. By using InfoPath form service to host forms and surface them through:

1. MOSS

2. Office Clients

3. InfoPath

7. Information Rights Management (IRM) – This provides a way for Office and .Net Applications to utilize MOSS IRM capabilities through the MOSS API’s.

8. Security -

1. Authentication is achieved through the use of Integrated Windows Authentication (IWA). Many protocols can be used by IWA. For example, Kerberos can be used here to ensure compatibility with other environments.

2. Role-based authorization throughout the application. This is implemented through the use of Active Directory. However, externalizing this provider is also possible. For example, if LDAP is a requirement Microsoft ADAM can be used in Active Directories place.

9. Integration Services with Lending Message Bus- The lending message bus provides XML web services to both the solution and the enterprise. The message bus acts as centralized messaging layers that can provide integration services, extends application functionality to both this solution and external solutions, and provide KPI’s on the health of services. Additionally, the lending message bus exposes the rich Enterprise Loan Orchestrations (ELO) to all the systems within the enterprise.

10. Business Intelligence – KPI’s and dashboards can be surfaced through the portal environment.

 

Technorati Tags: Architecture,OBA,Composite Applications,Loan Origination Reference Architecture,SOA

Announcing the Loan Origination Reference Architecture

I am happy to announce that the Loan Origination Reference Architecture!

See the MSDN OR-LOS Page at: http://msdn.microsoft.com/Architecture/ORLOS

On the LOS page on MSDN you will find a set of technical resources to guide the development of a new class of business applications called "Office Business Applications" (OBAs). These resources include an OBA reference architecture framework coupled with a consumer wholesale lending banking scenario. Below are links to a series of whitepapers outlining details of the reference solution that illustrate both the end user experience as well as the development model.

 
Video: Microsoft Loan Origination Reference Architecture

As part of the RAP you will get:

Solution Install Files:

• Demo websites and SharePoint site templates
• Supporting documents (.doc, .xls) for reference scenarios
• .NET Assemblies (workflows, utilities, activity libraries)
• Web services (.asmx)
• Reference data
• XSLT files
• BDC XML files
• Scripts

OR-LOS Whitepapers

1. Building Sustainable Banking Architectures
2. Architecting Regulatory Compliant Architectures
3. Architecting Enterprise Loan Workflows and Orchestrations

OBA Book (online version)

• Office Business Applications: Building Composite Applications with Microsoft Technologies

 

You can find more architecture guidance at the MSDN Financial Services Architecture Center:

http://msdn.microsoft.com/FinServArch

Microsoft ESB Architecture Guidance

At the SOA/BPM conference we announced that we are providing ESB architecture guidance. This will come in the form of:

• Pragmatic or "Real World" approaches to ESB Architectures that show a more iterative approach rather than the prevalent "Big Bang" approach that is often promised.
• Patterns
• Reusable Code
• Implementation Guides

So what does this mean for the Financial Services industry? Well personally, I believe that this could be very big. There has been quite a bit of demand for this. Specifically around the guidance. The following aspects are specifically important to Financial Services organizations:

1. Comprehensive Guidance - I think the issue with many of the vendors out in the industry is that they treat ESB as a Product or SKU. I'm not saying that without any background or reference or that I work for Microsoft but in the recent past I have been involved in WebMethods implementations and WebSphere implementations. When we purchased these we were sold on the promise that we would be up in running with an ESB in just a few weeks. That couldn't be further from the truth. It took us upwards of 12- 16 months to deploy. This was mainly due to the recommended implementation process, which was top down. Very ivory tower type of stuff. What I like about our approach is that we take a pragmatic view that is iterative. I will discuss this more in other posts.

2. Bringing it all together - Most likely Banks and Insurance companies already have the right technologies to enable ESB's. However, there is the perception in the industry that ESB’s can be bought. The advantage here is that organizations can roll out ESB’s at their own pace. In addition, this framework works well with other integration frameworks. With all of the new BizTalk product updates, partner ecosystem and new .Net 3.0 technologies (WCF, WPF, WF, CardSpace) to usher in the Microsoft SOA story.

3. Adapters, Adapters, Adapters - The list goes on and on. Not only from an horizontal perspective but also from a Financial Services perspective. We have adapters for IFX, ACORD, FIX, SWIFT, FpML, TWIST and more. For more adapter information go here: (http://www.microsoft.com/biztalk/evaluation/adapter/partner/2004.mspx)

3. WCF Technologies - Banks and Insurance companies are not in the business to build technology they are in the business to sell their products and services. This will dramatically reduce the amount of redundant code and improve quality thus will make them more interoperable with their B2B partners. The reason why I pointed out B2B rather than many other areas is due to all of the outsourcing of services or the increased need to poll third party services such as: credit or fraud services.

This is great that we put this together. This is something we have needed for some time now. In the past we have had pits and pieces but not the whole story, now we are starting to change that. We may not have everything yet but we will soon!

 

Technorati Tags: Enterprise Service Bus,ESB,SOA,BizTalk,Guidance