Converging Metadirectory and Virtual Directory

Phil Hunt, now at Oracle, is the visionary responsible for a lot of the innovation in Virtual Directory. From his recent response to my ideas on second generation metadirectory, it looks like we are actually thinking about things in similar ways, where meta and virtual work together.

As you may know, there has been an ongoing discussion on what does the next generation of meta-directory look like. Kim Cameron’s latest post elaborates on what he thinks is needed for the next generation of “metadirectory”.

  • By “next generation application” I mean applications based on web service protocols. Our directories need to integrate completely into the web services fabric, and application developers must to be able to interact with them without knowing LDAP.
  • Developers and users need places they can go to query for “core attributes”. They must be able to use those attributes to “locate” object metadata. Having done so, applications need to be able to understand what the known information content of the object is, and how they can reach it.
  • Applications need to be able to register the information fields they can serve up.

These are actually some of the key reasons I have been advocating for a new approach to developing identity services APIs for developers. We are actually very close in our thinking. Here are my thoughts:

  • There should be a new generation of APIs that de-couple developers from dependence on particular vendor implementations, protocols, and potentially even data schemas when it comes to accessing identity information. Applications should be able to define their requirements for data and simply let the infrastructure deal with how to deliver it.
  • Instead of thinking of core attributes as those attributes that are used in common (e.g. such as surname is likely the same everywhere). I would like to propose we alter the definition slightly in terms of “authoritativeness”. Application developers should think about what data is core to their application. What data is the application authoritative for? If an application isn’t authoritative over an attribute, it probably shouldn’t be storing or managing that attribute. Instead, this “non-core” attribute should be obtained from the “identity network” (or metaverse as Kim calls it). An application’s “core” data should only be the data for which the application is authoritative. In that sense, I guess I may be saying the opposite of Kim. But the idea is the same, an application should have a sense of what is core and not core.
  • Applications need to register the identity data they consume, use, and update. Additionally, applications need to register the transactions they intend to perform with that data. This enables identity services to be built around an application that can be performant to the application’s requirements.

What I have just described was actually part of the original inspiration behind CARML (Client Attributes Requirements Markup Language) put forward by Oracle that the Liberty Alliance is working on now. It was our belief that in order to enable applications to connect to diverse identity service infrastructures, something like CARML was needed to make the identity network both possible, adaptive, and intelligent.

But, while CARML was cool in itself, the business benefit to CARML was that knowing how an application consumes and uses identity data would not only help the identity network but it would also greatly improve the ability of auditors to perform privacy impact assessments.

We’ve recently begun an open source project at OpenLiberty called the IGF Attribute Services API that does exactly what Kim is talking about (by the way, I’m looking for nominations for a cool project name – let me know your thoughts). The Attribute Services API is still in early development stages – we are only at milestone 0.3. But that said, now is a great time for broader input. I think we are beginning to show that a fully de-coupled API that meets the requirements above is possible and dramatically easier to use and yet at the same time, much more privacy centric in its approach.

The key to all of this is to get as many applications as possible in the future to support CARML as a standard form of declaration. CARML makes it possible for identity infrastructure product vendors and service providers to build the identity network or next generation of metadirectory as described by Kim.

I haven’t seen CARML – perhaps it is still a private proposal? [UPDATE: I’ve been advised that CARML and the IGF Attribute Servces API are the same thing.] I think having a richer common representation for people will be the most important ingredient for success. I’m a little bit skeptical about confining developers to a single API – is this likely to fly in a world where people want to innovate? But details aside, it sounds like CARML will be a helpful input to an important industry discussion. Above all, this needs to be a wide-ranging and inclusive discussion, where we take lots of input. To get “as many applications as possible” involved we need to win the participation and support of application developers – this is not just an “infrastructure’ problem.

Now for something completely different.

French GuardsIt looks like Dave Kearns might be (?) mad at me… His recent post was entitled Your mother was a hamster and your father smelt of elderberries! Of course I would have taken that as a compliment except that I recognized it from The Holy Grail Scene 8, where the “French Guard” precedes it with, “I don’t wanna talk to you no more, you empty headed animal food trough wiper! I fart in your direction.

The olive branch (or was it a birch rod?) to which Dave refers is this:

Kim has now responded (“Through the looking glass“) to my Humpty Dumpty post, and we’re beginning to sound like a couple of old philosophes arguing about whether or not to include “le weekend” and “hamburguer” and other Franglais in the French dictionary.

We really aren’t that far apart.

In his post, Kim recalls launching the name “metadirectory” back in ‘95 with Craig Burton and I certainly don’t dispute that. In fact, up until 1999, I even agreed somewhat with his definition:

“In my world, a metadirectory is one that holds metadata – not actual objects, but descriptions of objects and their locations in other physical directories.”

But as I continued in that Network World column:

“Unfortunately, vendors such as Zoomit took the term ‘metadirectory’ and redefined it so it could be used to describe what I’d call an überdirectory – a directory that gathers and holds all the data from all your other directories.”

Since no one took up my use of “uberdirectory,” we started using “metadirectory” to describe the situations which required a new identity store and “virtual directory” for those that didn’t.

So perhaps we’re just another couple of blind men trying to describe an elephant.

Gee – have we been having this discussion ever since 1999? Look – I agree that we are both dealing with legitimate aspects of the elephant. Olive branch accepted.

Now that that’s out of the way, maybe I can call upon Dave to lay down his birch rod too. He keeps saying I propose ”a directory that gathers and holds ALL the data from ALL your other directories.” Dave, this is just untrue and unhelpful. “ALL” was never the goal – or the practice – of metadirectory, and you know it. The goal was to represent the “object core” – the attributes shared across many applications and that need therefore to be kept consistent and synchronized if stored in multiple places. Our other goal was to maintain the knowledge about what objects “were called” in different directories and databases (thus the existence of “connector space”).

Basically, the ”ALL” argument is a red herring (and if you want, you can say hareng rouge instead…)

Published by

Kim Cameron

Work on identity.