Mike Walker's Blog

Obtaining Enterprise Architecture Metrics - Part 1

For most EA organizations there is a huge struggle to justify their existence. There is quite a bit of criticism on the practice from folks asking, Does Enterprise Architecture Matter? 

The common quote I hear from our EA customers about opposition is simply "Why do we need you" or "So what we are disorganized, it has worked in the past". Debating those points are challenging with out the information to quantify an EA function.

After hearing those statements and questions then there is a mad rush to get the "ROI of EA". The are articles out there they give prescriptive guidance to obtain metrics such as this article. The problem with articles like this is that they make two mistakes. They are unable to clearly Quantify and Qualify the metrics.  See below:

So maybe then you are thinking, well let's align this with the next big EA program such as "future state architecture" processes or strategic programs like what Pankaj Arora talks about with his post on EA ROI. But this approach is flawed because the ownership and accountability of the actual implementation of those ideas are by other teams. You will not get the credit for that great new Online Banking System but rather the e-commerce team.

For the most part EA organizations do not have this quantifiable information. Since they do not, EA's equip themselves with what I call EA Analyst Battle Cards such as the one below:

This was a compile from IASA organization in 2007 from the leading EA analyst communities. 

This is a mistake to carry around EA Analyst Battle Cards because they do not tell your EA customers (e.g., your CIO or CFO):

• Your EA Value
• Current State of Your Enterprise
• The Future Value of EA

What this does show your customers is the potential, that's it. Gartner, MIT and such provide great data points what's possible but the circumstances in which these studies derived could be drastically different than what your enterprise looks like.

So what are some of the things you need to keep in mind wen trying to obtain the "right" metrics:

1. Obtain the Information
   • Know what you can get - Try to avoid a "fools errand run" by knowing what types of metrics you can reliably obtain. If your current processes require guess work in metrics, skip it for now.
   • Get the Tools - To obtain metrics you need tools to collect and store the information. Once the basics are covered, consider automation into the EA process and into specific tooling. The tool sets must allow transparency of the EA process. This allows for self-service metrics from stakeholders.
2. Process
   • Get a Process - The process must be able to Quantify and Qualify the metrics.
   • Repeatability - All metrics gathering processes need to be repeatable. If not then revise the process. This will ensure the reliability and the accuracy of gathering your metrics.
3. Communicate
   • Understand your audience - Not everyone is looking for the same metrics, to demonstrate your value you should be able to communicate what they are concerned about.

         

• Traceable - Ensuring that the metrics that you are gathering "matter". Meaning that  the metrics you obtain should align with the organizations strategic imperatives. Additionally, demonstrating the validity of the metrics are aligned is crucial.
• Relatable Information - To successfully communicate value avoid abstract metrics (e.g., Improved interoperability of systems by 40%). Provide information in terms they understand and can "touch and feel". To do this relate these metrics back to strategic programs or projects to demonstrate the tangible value.

 

In summary, the key to success with obtaining the right EA metrics is to have a clearly defined process to obtain the metrics, the tools to get the information and understand what are the right metrics you should obtain.

Remember, numbers are not everything. In many cases value is not always equated to numbers. Perception is very powerful. For example: "Jim was key to the success of this project, he saved the project team from making the wrong decisions in the architecture design process". This is very powerful, and it is difficult to put numbers around it.

In part 2 I will discuss some ways to obtain the right metrics.

 

Tags: Enterprise Architecture

What is the Role of EA in a Dominated SaaS World?

A colleague of mine Gianpaolo asked me about what is the role of EA in a heavily SaaS oriented enterprise. The hypothetical case would be of and IT shop being extremely (80%) of the enterprise IT “operations” being run by 3^rd parties (SaaS, vendors, hosters, apex like exchanges…) does the need of EA increase or decrease?

I thought that this was an interesting question and made to think about it for a moment. So the specific questions are:

• If it increases EA effort, are the current “framework” any good in this new model?
• If it decreases it EA effort, (and assuming we believe in a world where IT will “shrink its perimeter”) how much do we need to invest in EA?

This is a "loaded" topic so I will only be addressing the question directly and not what I think EA's should do about this in their practices. That may come later :) 

So this is a difficult prediction, it really depends what the specific circumstances were with each particular organization. However, we can generalize and create a assumptions based on what we see today.

• Assumption 1 - SaaS architectures will be isolated to the application (the more likely of the two) or business capability (ideally) level 
• Assumption 2 - There is no one vendor fits all
• Assumption 3 - SaaS providers will use open standards such as web services, however the technology implementation, XML payloads and underlining technologies will be different
• Assumption 4 - Regulatory scrutiny for items such as information protection, auditing, and reporting will still exist
• Assumption 5 - There will still be a mixture of implementations of SaaS applications such as Composite Applications (Rich or Smart Clients) and web based solutions.
• Assumption 6 - Integration will not go away as existing applications in the enterprise and specific business capabilities exposed by SaaS vendors will need to interact with each other.

Based on these assumptions it is safe to say that the role of EA will still be intact. However the specific architecture decisions that will need to be made will vary a bit.

Basically what we have done is pushed all the applications to the cloud. Now, instead of provisioning a new HP server and installing Windows Server you use the "cloud" for your application platform. What this does do is it reduces the amount of work needed to manage, operate and support of a server environment. However, this still doesn't change the larger issue with software applications we have today with Commercial Off The Shelf (COTS) applications. These COTS applications need to be wired together because out of the box they only cover a fraction of the business process they will still need to be wired together.

EA's will still have to tie together the following architecture concerns:

• Business Architecture – Just because the application moves to the cloud it doesn't mean that business architecture will not occur. With more applications in the cloud there will be new business models emerging as well.
• Information Architecture – Depending on how these solutions are outsourced you can run into the same challenges enterprises face today but pushed further out to the cloud. As shown below the composite SaaS applications will still need to intercommunicate with each other. I would venture to say that the control of this integration will either be orchestrated solely from the enterprise or will be a hybrid model (which is the most likely) in which based on the specifics of the problem the integration will either occur on the SaaS side or the Enterprise side.

• Communications Architecture – More emphasis will be on communications hardware and the resiliency of it will occur. This will span across into areas such as Operational Management. This will basically tie all the SaaS together for the end user (this greatly depends on what components are pushed out). We can't assume that there will be only a few SaaS providers but many. So it will be ever important to control this. Devices such as: Network Load Balancers, Web Service and Encryption Acceleration Devices, Security Devices, and the obvious routers and switching hardware. There will be some other opportunities that SaaS providers will be able to provide and that is decentralized access of these applications via many different mechanisms such as wireless.

• Operational Management – With this component it involves architects doing more architecture review types of activities of the SaaS players. PMO activities can perform the standard "check it off the items" but there will still need to be verification of these architectures. Additionally, since these applications will be intercommunicating there needs to be assurance that desired architecture solution fits in a multi-SaaS environment.
• Security and Risk Management - This will be a big one as shown in all the diagrams there is no guarantee that the authentication and authorization mechanisms will be built in a composite manner. Ideally they will be but the facts are that the far majority of COTS solutions today have proprietary identity stores. It will require architects to figure out the right solutions for identity management, data at rest security, transactional security, and physical security. At the end of the day the enterprise is ultimately accountable for the data stored in a SaaS data center (refer to the Financial Services legislation where FSI's are legally bound to the implications of an offsite breach no matter what type of contract was defined with the SaaS provider ).

To summarize, these are a few areas on the top of my mind where EA's are affected by this type of SaaS scenario. As shown below in a very simplistic view we have extended the enterprise to the cloud but complexities still remain. I see no reason why an EA's role would be greatly different.

Now that I went on this rant let's address the questions directly.

• If it increases EA effort, are the current “framework” any good in this new model?
  • It increases activities in the domains listed above. Like all frameworks they are a good guide, the good frameworks do not explicitly tell you if the solution should be built, bought or outsourced.
• If it decreases it EA effort, (and assuming we believe in a world where IT will “shrink its perimeter”) how much do we need to invest in EA?
  • Likewise with the answer above, other domains such as Infrastructure, Development and Database diminish to a smaller degree. But you will still have to worry about the 20%.

 

 

Tags: Enterprise Architecture SaaS

Enterprise Architecture Coming of Age?

I was just talking about this today on a thread internally here at MS where we were talking about EA challenges. One person commented saying; "all EA's do is create documents". I don't agree with this completely, there are a lot of artifacts built but if you are doing you job right there is a whole lot more going on beside being a MS Word lackey. We need to change behavior such as EA's just building pretty models, but rather make actionable models.

Reading this article from CBR it shows that EA's are changing from the older model not just creating word documents and Visio diagrams but are creating business impacting process flows and models that are used to actively underpin strategic initiatives.

Of the 289 enterprise architects that responded, 57% said enterprise architecture is being applied to growth and innovation initiatives.

Check out the article it has some interesting facts from a recent study by Architecture & Governance Magazine.

http://www.businessreviewonline.com/blog/archives/2007/08/enterprise_arch.html

Controlling Complexity in Enterprise Architectures

Yesterday I had a conference call with Roger Sessions at ObjectWatch, Inc. to fill us in on the new methodology he created called Simple Iterative Partitions (SIP). It's defiantly thought provoking and if you were a mathematics major then you'll love all the reference back to mathematical theory.

I would be curious on your thoughts on this. I am wondering how these fresh methodologies work in the "wild" (i.e., real world enterprises). As we know with many other things, time will tell.

From what I understand of SIP is it provides a methodology that fit's into existing EA frameworks that gives EA's a way to have repeatable process in which to quantify complexities in architectures.

Snippet from the paper:

We commonly implement large, expensive IT projects without any idea whether they are based on a sound architecture or not. Many of these projects fail - often at great cost. Projects fail so frequently because we have no models that define ”good” enterprise architectures.

When it comes to enterprise architectures, we believe that "good" is most closely associated with simplicity. Of two otherwise equal architectures, the "better" one is the simpler one. This approach requires us to rethink complexity.

 

See More on ObjectWatch

http://www.objectwatch.com/white_papers.htm

 

Tags: Enterprise Architecture

Presenting at the Microsoft SOA and Business Process Management Conference

I will be presenting at this years Microsoft SOA and Business Process Management conference. The conference is held on the Microsoft campus, Oct 29-Nov 2, 2007.

I will speaking about composite applications on the Microsoft platform. I will post details about my presentation at a later time.

Check out all of the information, tracks and registration online: www.MSSOAandBPconference.com

Updated: Bank Teller Composite Architecture

This release adds support for ribbons, form-skinning, various CommandAdapters and UIElements are also added, some of which extend CAB as well as offer basic support for DevExpress controls.

The BankTeller QuickStart application has also been updated to incorporate samples of the new XtraWindowWorkspace and DxMenuItemCommandAdapter.

• Other changes of note:
• The file structure of the release has been reorganized so that things are slightly easier to find and consistent
• The project has been renamed from DevExpress.CompositeUI to CABDevExpress.ExtensionKit (to avoid conflicts with DevExpress' Project Converter tool)
• The base namespace has changed from DevExpress.CompositeUI to CABDevExpress (this will break existing code but is copy/paste job to fix)
  CAB dlls are now included as part of the release to allow for easier download-and-compile
• The code has been cleaned up to some extent, to remove unused includes, unused variables and other anomolies

 

http://www.codeplex.com/CABDevExpress/Release/ProjectReleases.aspx?ReleaseId=6655

 

Tags: Banking Composite Applications

Architecture Modeling - IEEE 1471 has been adopted by ISO

ISO has published ISO/IEC 42010:2007, Systems and software engineering — Recommended practice for architectural description of software-intensive systems. The text of this ISO standard is identical to IEEE 1471:2000, and will serve as basis for the joint ISO and IEEE revision.

The joint revision has several goals:

• to widen the scope of application from software-intensive systems to general systems architecture (including enterprise architecture);
• to harmonize with the ISO systems engineering (ISO 15288) and software engineering (ISO 12207) life cycle processes; and
• to align terms and concepts with other ISO architecture efforts, including RM-ODP (ISO 10746) and GERAM (ISO 15704).

As shown below there are many views to architectures. Many times architects want to create the one ultimate view. This is just unrealistic. The 1471 standard provides a taxonomy to represent these architecture views.

 

At the present time, frameworks are aligning to 1471. TOGAF embraces but does not strictly adhere to ANSI/IEEE Std 1471-2000 terminology. From the horses mouth:

In TOGAF we endeavor to strike a balance between promoting the concepts and terminology of ANSI/IEEE Std 1471-2000 - ensuring that our usage of terms defined by ANSI/IEEE Std 1471-2000 is consistent with the standard - and retaining other commonly accepted terminology that is familiar to the majority of the TOGAF readership.

Additionally, this can also fit into a mapping of views to the schema of the well-known Zachman Framework.

For more information go to the IEEE 147 site.

IEEE 1471 site

FAQ

 

Tags: Enterprise Architecture ADM TOGAF Zachman Standards

Justifying the need for Composite Applications

I recently saw a post from Brandon Satrom as he was asking for feedback on his interpretation of composite applications and the value proposition to his organization. He is definitely on the right track, and not because he is using the guidance that I have been working on at Microsoft :). I will address the content of his post directly here with my opinions.

Brandon articulates the value of composite applications into five factors:

1. The Cost of Building Applications
2. The Problem with Processes
3. The "Consumerization of IT"
4. The Need for Agility
5. The Desire for a Single User Experience

Assuming that these are not in order but rather a un-prioritized list of factors to consider going down the composite application route then I roughly agree with most of these.

But let's take a step back for a moment. Like all other complex subject matters there are varying opinions and definitions from smart people in the industry. I have been talking about this subject for about three or so years while at Microsoft and before. I have seen this definition grow and morph into a much more compressive and more importantly actionable concept.

So what are composite applications? Here are some definitions from the industry:

Gartner

Composite applications leverage the business application functionality that is exposed as Web services. The composite application is an abstraction layer on top of a service repository, which orchestrates a new business process and has its own user interface. A composite application may be a process template or a ‘service’.

BEA Definition

Applications that are built by reusing logic from two or more existing applications to form a new application without having to start from scratch. A composite application consists of functionality drawn from several different sources within an SOA. The components may be individual Web services, selected functions from within other applications, or entire systems whose outputs have been packaged as Web services (often legacy systems). 

IBM Definition

To be considered as a composite application, an application needs to comply with a certain architectural design, which is defined by a set of features. Even if it is not an exhaustive list, the following shows the main desirable features of a composite application:

• Homogeneity to behave as a unique application to different heterogeneous applications or components
• Flexibility to use service-oriented architecture (SOA) principles such as loosely-coupling and reusability
• Intercommunication between components
• Richer user experience to aggregate a variety of application types into a single client view
• Security such as authentication, roles and data confidentiality
• Transactional applications, which get the information from multiple sources
• Reuse computing assets
• A uniform and consistent graphical user interface
• Composition of parts or components
• Aggregates a variety of application types into a single client view
• Provides anytime/anywhere access in a semi-connected environment. This is not a composite application feature but Lotus Expeditor enhances these capabilities

From a Microsoft perspective we have defined composite applications mostly at a micro level with the great work performed by my team on defining this more with OBA. From a macro level Chris Keyser wrote a great composite applications primer on this subject in the Architecture Journal a few issues back.

My opinion on Composite Applications is similar to IBM's and Gartner's combined. I think Gartner defines the concepts nicely but implies implementation details when they should be left out. What I like about IBM's definition is that they tell you the capabilities of a composite application.

I break a composite application down to the following four core requirements:

1. Contain an autonomous business process. The goal of composite applications are to have a plug-able black-box applications. To do so, they must have explicit business process or functional boundaries.
2. Must fit into a heterogeneous environment. Applications that can be implemented in various different ways, this should span multiple implementation types of the same technology and work within environments where the technology is diverse.
3. Must have Built-In Interoperability. These applications should not have proprietary integration methods but rather have industry standard ways of interoperating with other systems.
4. Compose in many different ways. Composite applications are self contained applications that can interact with other applications in the same environment similarly to web mash-ups (great article by Ray Ozzie) which are consumer composite applications are a great example of how composite applications have flexible and extensible  UI functionality.

Below is a slide from my presentation from TechEd in Australia.

From a micro level I agree with Don Box's tenets of service oriented architectures. When you read these, they are not exclusive to Composite Applications but rather SOA as a whole. The key here is that composite applications provide a way to implement Service Oriented Architectures. It is important not to confuse the two. SOA provides the principles for building a service oriented <insert what you will here>. It could be at many various levels such as at the application, system, or enterprise level.

1. Boundaries are explicit. Services must interact across service boundaries, but crossing service boundaries might be costly. Therefore, it is important to know your boundaries. Keep service surface areas small, avoid RPC interfaces, and avoid leaking implementation details outside a service boundary.
2. Services are autonomous. Ideally, services should be stable, but business needs change frequently. The space between services changes more frequently than the service boundaries themselves. Avoid assumptions on the environment into which the services are deployed. Design, deploy, and manage services independently from one another. Communicate only through contract-driven messages and policies.
3. Services share schema and contract, not class. Service consumers will rely upon a service's contract to invoke and interact with a service, and will insist that the contract remain stable over time. However, changing business needs will force change upon a service. Therefore, service interaction should be based solely upon a service's policies, schema, and contract-based behaviors. Ensure stability of services (public data, message-exchange patterns, policies). Form explicit contracts that are designed for extensibility. Version services when change occurs. Avoid blurring the line between public and private data representations.
4. Service compatibility is determined based on policy. Services must be compatible with each other—not just structural compatibility (public data, message-exchange patterns), but also compatible with other semantic capabilities that are expressed through configurable capabilities and service levels. Operational requirements for service providers should be manifested in the form of machine-readable policy expressions.

Microsoft has taken Composite Applications to the next level with the re-usable building blocks and reference architectures released by many of the groups out there. Patterns and Practices on the Composite Application Block (CAB) then the Smart Client Factory and now Acropolis. This isn't to say that there isn't any thought leadership or examples of composite applications by other vendors. I get my paycheck from Uncle Bill so I will mention our technologies.

Taking my opinions on Composite Applications and contrasting them to Brandon's there are similarities.

The Need for Agility this is the top priority for IT organizations to stay relevant to the business. All the benefits the Brandon mentioned are spot on but I would expand them a bit. All of the reasons listed though were reacting to business needs. Agility is also about future strategy by allowing our architectures to proactively address future business challenges and drivers as well as reacting to current trends.

Agility often defined as the ability of a firm to sense and respond to business opportunities in order to stay innovative and competitive in a turbulent and quickly changing business environment. An agile firm (one that demonstrates agility) has the capabilities and processes to respond to unexpected environmental changes.

- Wikipedia

I really like the Wikipedia definition of agility. The definition clearly shows us the forward and current state thinking aspects of agility. I talk a bit about the future state in my sessions about composite applications. When we architect our solutions the technology, process and ultimate implementation of that solution will inevitability change. Composite application architectures allow us to build solutions with this needed flexibility. 

The Desire for a Single an improved User Experience (UX), this one is an important one. Understanding that we are not talking about creating one reusable interface but rather an improved experience for the actual users of the solution. This coincides with creating agile architectures and not trying to prescribe to the business what we feel is a good interface but rather letting them. New trends both consumer (Web 2.0) and enterprise (Portal or Smart Clients) come and go. There are very few that have stand the test of time. So with that knowledge are architectures must adapt to change.

The "Consumerization of IT" is one of the possible results of User Experience aspects. The consumer technology solutions definitely have potential plays in the enterprise, especially the collaboration technologies which span consumer and enterprise.

The Problem with Processes. I think Brandon is on the right track here and I agree with what he talks about here. I would a few comments to further refine. Composite applications are able to leverage various types of workflows.  Composite applications themselves do not improve the process directly. Workflow definition still involves a set of architecture decisions in a workflow engine. However, composite applications make a great choice for surfacing these workflows due to their service oriented nature.

Below is a slide I use often to describe the various levels of process. As you can see the technology choices to implement change. As architects we should pick technologies for their strengths. For example, we wouldn't want to use BizTalk for instant messaging and collaboration.

I wrote an article on MSDN called Architecting Enterprise Loan Workflows and Orchestrations in this article I talk about these concepts in detail using the Loan Origination RAP as a working example.

 

The Cost of Building Applications is a big challenge for IT. I would make this a bit more generic because it's not just about the cost of building but also the cost to:

• Maintain - We are seeing from analysts such as Accenture that maintenance is accounting for 70% of the total IT time in enterprises. This is a staggering number when you contrast this to the 40% of new development in integration. So would it be safe to say that 17% of the time is really spent on new development?
• Integrate and Interop - I would add interoperability here. Composite applications foster the notion of interoperability.
• Enhance - The composite nature allows for easy additions to solutions when proprietary implementations inhibit this behavior.

With these factors we need to make sure the technology choices we make doesn't hamper IT or the business to create the right new applications instead. We want to build our solutions that it ensures future flexibility rather worrying about the legacy applications. Ideally this shouldn't be our primary concern.

The underling goal here is to Reduce the Cost of Technology Investments.

Overall Brandon did a great job justifying the need for composite applications. I hope this had helped and I would encourage you to provide feedback on my explanations as I didn't go into great detail on some of the topics. Additionally, refer back to the four basic requirements of a composite application as they show some clear benefits as well.

 

Tags: SOA Smart Client Composite Applications Architecture OBA

Announcing the Microsoft Business Data Catalog Definition Editor

I am happy to say we have finally released the Business Data Catalog Editor.

Currently, writing an application definition to connect the BDC to a LOB system is a manual process.  This requires an understanding of both how the LOB system is configured and what must be included in the XML to satisfy the BDC. Having a tool to simplify this process not only lowers the initial knowledge threshold for administering the BDC, it also lessens the required work of the user (such as testing, making modifications, etc.). 

The tool has been designed to assist in the lifecycle management for Application Definition files.  The tool enables searching over databases and web service-based repositories, as well as the connection for BDC web parts in MOSS.

Download 

http://www.microsoft.com/downloads/details.aspx?familyid=6d94e307-67d9-41ac-b2d6-0074d6286fa9&displaylang=en

Overview

• The Business Data Catalog Definition Editor provides a visual tool for creating an Application Definition for the BDC in MOSS 2007.  Features include:
• Underlying XML is abstracted by the design surface and properties window
• Drag and drop web methods, tables, or views to create line of business (LOB) connections.
• Entities and methods are created automatically from database metadata and WSDLs.
• Additional method instances can be added to further enhance the DB or WS connection.
• Method instances can be tested from within the tool, enabling incremental development of LOB connections
• Tool supports Databases (SQL, Oracle, OLEDB, and ODBC) and Web Services
• Drag and drop design surface for selecting DB tables or web methods
• Metadata is automatically extracted from Databases by dragging and dropping tables
• Web Services require a few additional steps to completely configure the connection
• Users can import and export Application Definition XML files
• Users are able to test method instances incrementally from within the tool
• The tool is not required to run on a Web Front-End
• Associations are created automatically when foreign keys are selected; they can also be created easily for web services by adding an Association method instance

Tags: OBA

Microsoft is a member of LIXI

Microsoft is now a member of the Australian Lending standards body called LIXI.

http://www.idm.net.au/story.asp?id=8709