02 September 2009

Permissions Worth Getting Excited About

Permissions. Doesn’t the very subject set your heart racing?


Well, me neither; normally. But, rather to my surprise, my thesis today is that one of the most revolutionary and remarkable features of FluidDB is its permissions system, and that this, more than anything else, explains why it has a shot at blowing apart web applications and social data.

I accept it’s an unlikely claim. But let’s see if I can make a case.


If you ever hear Terry talk about FluidDB, or read his stuff, the guy speaks in riddles. One minute he sounds like Richard Stallman, banging on about freedom and how when he rules the world no one will have to ask permission to write data, no one will own objects, everything will be shared and social and anarchic and messy and we’ll all be able to do exactly what we want. He even calls FluidDB the database with the heart of a wiki, whatever that’s supposed to mean. And the very next next minute, he’s talking about how at the heart of FluidDB lies a really strong permissions system that allows us to control exactly what goes in FluidDB.

The guy’s clearly schizophrenic, if not paranoid delusional. Right?

In Defence of Terry

FluidDB is a remarkably simple conception. At its heart, it’s just a collection of initially empty containers to which we can attach tags. In best computer-science tradition, we call the containers objects and they really are 100% shared.

If you can find an object, you can tag it . . . and so can anyone else.

As you can see, the objects all have rather long, individual identifiers—actually 128-bit numbers sometimes known as UUIDs [1]. Some objects also have a second identifier, known as the about tag. About tags are unique and never change, so there can only be one object in FluidDB whose about tag is set to the (exact) string oxygen, though there can be any number with no about tag.


So objects are not owned, but shared, and once created, are never destroyed [2].

So one half of the claim is good: objects are shared, and anyone can tag them with whatever they like. Where do all the permissions and control come in?

Tags and Values

When we say that anyone can write to any objects in the system, that’s true. Anyone can stick any tag they like onto any object they can find [3]. The tag has a name, such rating or colour or Ulysses and may have a value—such as 7, or interesting or a picture of some leaves. Like this:


The only restriction your tag names all start with your FluidDB username. For example, my FluidDB username is njr, so my tag names start with njr/. So I can have njr/rating, njr/colour njr/Ulysses, or anything else I like. And you, Josephine, can have tags called josephine/rating, josephine/colour (or even josephine/color, if you’re that way inclined) and so on.


Placing tags on the same object in FluidDB is, of course, a way of relating them, but that isn’t the focus of this article.


So finally, we come to the bit.

By default, I can only create and set tags starting njr, but can read any tags, while Josephine, naturally enough, can only create and set tags starting josephine. Unlike objects, tags are owned.

But there is a permissions system that means that I, as the owner of the owner of tags starting njr, can control exactly what Josephine can do with my tags and vice versa.

So if I want to, I can hide my ratings from Josephine, but let her see my colours. Or I can hide them from everyone except Josephine and Terry.

Slightly more unusually, I can also choose to allow some, or even all other users to write some of my tags if I like. For example, I might decide that I trust Terry enough to allow him to set some of my tags.

This ability to control who can do what extends not only to users, but to applications as well. So suppose someone writes an application called Read-Planner that allows people to tag any book they run across on the web (or anywhere, in fact) with a to-read tag. If I want to use this application, I don’t have to give it my password or open up access to all the rights that I have: I can simply give Read-Planner the ability to create to-read tags for me. And if at some point I become unhappy with it, of course I can revoke that permission. I don’t even need something like OAuth: I can just do it.

Even better, I can also choose to allow other applications whatever form of access to my to-read tags I want—maybe read only, or maybe some of them have write access too. It’s up to me.

And this is why we think FluidDB has the potential to be so revolutionary: it is truly social, while leaving each user firmly in control of his or her data.

New Rules for Web Applications

At the moment, any of us who use web applications tend to spend a lot of time and effort populating application databases to make them useful to us. But when we do so, we tend to lose control of our data. They go into a private database schema, and what access we have to that depends entirely on what the application allows us to do. Sometimes there are reasonable ways to get the data back out (some kind of an XML dump perhaps), sometimes not. But always the application is in control. And linking data across applications is, in general, somewhere between hard and impossible.

FluidDB can change all that by leaving the user in control of his or her data, granting the application only such permissions as necessary or desired, and ensuring that the user retains flexability and control.

Now obviously, this only works if applications agree to work with data in this way, and equally obviously, a lot of them are going to be extremely reluctant to do so. After all, in the well-worn phrase, information is power. So it’s more than possible that far from embracing the openness and intrinsic cross-application, cross-user interoperability championed and facilitated by FluidDB, many applications will seek to hold onto the data. We fully expect this.

But we also expect that there will be some applications, perhaps new ones, perhaps small ones initially, that will embrace the idea. And over time, a movement can grow, perceptions can change, and maybe in ten years time the idea of an application owning and controlling your data will seems as antiquated as the notion that you shouldn’t be allowed to see your own medical records.

A Tiny Bit of Detail

I’ll close by talking in slightly more detail about the way the permissions system in FluidDB operates. Further articles will go into the gory details.

Briefly, you can set permissions for each group of tags sharing a name. [4] So for example, if I use a rating tag, I can control exactly who can read, change and create this njr/ratings tag. And I can control each of those aspects independently. [5]

Similarly, if I choose to use names that contain slashes, like maybe njr/book/rating and njr/book/own I can choose control who can do what at the level of a stem like njr/rating if I choose. So I could allow some book applications, or a bookish friends, to manipulate my book tags, maybe even creating new ones for me, but not my other tags (especially not my njr/private ones!)


Unlikely as it sounds, one of the key innovations in FluidDB is its combining a completely shared set of information containers (the objects) with the ability for users to tightly control, at a granular level, precisely who can read, write and create the tags used to store information on those objects. This applies not only to users, but also to applications, giving a simple way for users to grant applications broad or narrow access to read or manipulate some or all of their data while, ultimately retaining complete control of it.

In FluidDB, you never need to ask permission to write to an object, but you always need permission to use someone else’s tags.

It’s a powerful combination.

[1]So-called Universal Unique Identifiers.
[2]In fact, we sometimes like to say that objects for every possible about tag already exist, in rather the same way that Plato believed that all numbers and other mathematical objects existed in perpetuity in what we now call his platonic universe. It’s simply that we only bother to allocate storage for objects with any particular about tag when someone actually wants to use it.
[3]If it doesn’t have an about tag, an object might be very hard to find, essentially requiring you to guess a 128-bit number. But it doesn’t really matter, as we’ll see.
[4]We sometimes call the set of tags having, or potentially having a particular name (like njr/rating) as an abstract tag. We can think of the (concrete) tags that we actually attach to FluidDB objects as concrete instantiation of a canonical, abstract, platonic, master tag of the same name.
[5]There are actually more than three aspects I can control, but this the essence of it.


  1. But, as I understand it, permissions are granted per tag, not per value. This implies that in some cases (take e.g. the delicious importer) one has to have a duplicate set of tags (e.g. delicious/books and delicious/private/books) where each tag in delicious has two tags in FluidDb which to you mean the exactly the same but are needed to control privacy which in delicious is orthogonal to tagging.

  2. Holger

    You're right, the permissions are at the level of what I call the "abstract tag", which is what you mean when you say the tag as opposed to the value.

    In terms of the delicious importer, although it does what I think is the "natural" thing (or at least the most direct thing), there are probably better ways of allowing FluidDB to exploit the information. In particular, there's a case for having a single tag, perhaps called "tags" or "delicious tags" or whatever and having this have a value that's a set of strings. That would solve the problem because you could just have a private-tags tag, or whatever.

    But your point is fundamentally correct.

  3. Holger

    Thinking about it slightly more, I have a second comment on your perceptive comment. It's that in delicious, it's not tags at all that have permissions, but bookmarks. I can't choose to let the world see that I've tagged http://fluidinfo.com/fluiddb with "awesome" but not that I've tagged it with "by-my-friend-terry". I choose either to let people see that I've bookmarked http://fluidinfo.com/fluiddb, or not, and if it's shared, they can see my tags.

    So the model really is very different.

    Thanks again.


  4. Correct about the thing that is private (bookmark vs tag).
    I agree with your first comment that one should just have delicious_tags and private_delicious_tags.

    But what if we need to map a system where the privacy dimension has more than two state (private and public)? We'll get into a situation where we have an exponential amount of tags (public_foo, private_foo, private_except_bob_foo, private_except_alice_foo, private_except_alice_andbob_foo, ...).

  5. Holger

    You're obviously right that faithfully moving data from any system with finer-grained permissions to one with coarser control will always be a challenge. We could have gone further and given users the abilty to set policies and permissions on individual items of data, rather than working at the level of a set of tags sharing a name. One of the reasons we didn't was we were quite interested in the idea of tag managers---applications that would look after one or more of a user's abstract tags for them. Bearing in mind that you can also grant permissions and poicies on namespaces, allowing an application to create tags in a subnamespace, for example, this is quite powerful.

    While I can't think of too many applications people might want to pull data from that have more granular control than FluidDB, I'm sure they exist and that at some point someone (probably me) will curse FluidDB for not being even more flexible.

    So---point taken. Let's see how it goes.


  6. Hi Nick, I read the FDB documentation in my usual half-arsed way. Perhaps it would be useful to write a post detailing the difference between users (applications - i.e. api access) and users (individuals).

    It's not too clear to me how all this relates to namespaces (application or individual?)and by implication the underlying tag permissions.



  7. Hi David

    Not sure whether by FDB you mean FluidDB or my FDB (fdb.py) library, but perhaps it makes no difference.

    I do plan probably a couple more posts on permissions, one going into more detail on namespaces and the actual controls and one talking about fdb.py's approach to it, both through the library and the command line.

    Of course, I have to write those bits of the library and command line interface first. The two sort-of go hand in hand.

    Briefly, though, as far as I know there is absolutely no difference at all between an application and a user as far as FluidDB is concerned. Indeed, even the fluiddb user/application is an entirely ordinary user as far as the system is concerned. There's just a fluiddb user, used by the fluiddb application.

    As for namespaces---well they probably do deserve their own post, but the gist of the idea is that if tags are a bit like files (and tag values their contents) then namespaces are a lot like directories (or "folders", in newspeak). And just like directories, namespaces have their own permissions and policies, allowing you to control who can see them, read them, create new items in them and so forth.

    Hope that helps...briefly. There'll be more (or, "I'll be back", as the Governator might say)

  8. Sorry Nick, I meant Fluid DB.

    I was just asking whether the preferred route is for people to log into applications using their FluidDB credentials, or whether each application would have its own user management system.

    In other words, it's the difference between /app/bob/likes and /bob/app/likes.


  9. David

    I thought that was probably what you meant...

    There is no official answer, but my strong preference and recommendation is a mixture. I think applications should log in with their own credentials, but that users should grant those applications the necessary permissions on whatever subset of their tags and namespaces they want them to use.

    So, in most cases, I would have the app iLike log in with its credentials and act as a tag manager for /bob/likes, with Bob's explicit permission and without ever needing his credentials. It could, of course, stick all its tags in the subspace /bob/iLike/likes, but that seems to me to go against the whole cross-application spirit that we're trying to foster.

    Does that make sense?


  10. Totally, but it implies users of iLike must first create a FluidDB identity if they don't already have one. But I agree the benefits of this approach outweigh the cost.

  11. ..this suggests to me that someone (not me!) should start thinking about generic authentication library, similar to the ones they have on commenting systems, where you can identify yourself to the system via a variety of existing pseudonyms (twitter id, FB id, etc).

    It looks like it would be a key component, since most users will access the DB via a 3rd party application.

    Given time constraints, the most I can offer would be a line-for-line port to PhP :-)

  12. I think that's right, David. Of course, the key thing with this authorization would be that it wouldn't be (normally) a blanket request to be able to do anything, but rather to do particular things with particular tags. So conceptually, it's a web page that an app can direct you to that sends information such as

    iLike is requesting permission to read and write the bob/likes tag

    The user allows this or doesn't.

    Obviously this isn't a long-term solution, but I have reflected on the fact that an interim measure for people with FDB (my command line utility) is that if/when I add commands for changing tag permissions, an app could simply print the set of FDB commands that are needed to enable the access and people could grant it themselves. (Yes, I know, this is not even an approximation to a long-term solution; but for the current stage with developers etc...) So in this case, it might simply be (bearing in mind that I haven't actually defined these commands yet) something along the lines of

    fdb chmod -u iLike +w +r /njr/likes

    (reading roughly "Change the permissions mode on the njr/likes tag to give the user iLike read and write permission)".

    But as you are about to say, in the medium term, there will probably be users who prefer using a web app to installing fdb.py and using the command line! Obviously, it might be desirable for such a web app to be hosted at fluidinfo.com, perhaps the only place we should really be encouraging users to type passwords, preferably using https.


  13. That's all great stuff Nick.

    I was actually thinking one level up, i.e. a generic library for returning a FluidDB user id on the basis of a number of external authorization services.

    The FFB architecture is perfect for allowing multiple such services to be attached to the same user object.

    For example, the 'david' FDB object:


    This would lead to a users (multiple) external identities being stored in the same place, and would hence allow FDB users to access the system without ever having to know (or remember) their FDB credentials.

    It's also a great example of FDB's central data accumulation thesis being put to good use.

  14. I'm probably being slow, David, but I don't quite follow.

    I'm certainly with you in spirit: a key idea is that lots of different apps/services/tag managers can all potentially be authorized to manipulate various of the user's namespaces and tags. I was suggesting a possible short-term way to achieve that.

    I don't entirely get either what you mean by "The 'david' FDB object" or by "returning a FluidDB user ID".

    But I think the key point is, whatever the detail, the nature of the transaction is

    1. App requests permission to do certain things to certain of the user's namespaces and/or (abstract) tags.
    2. The user is rediected to the FluidDB website and either agrees or declines
    3. If the user authorized it, the app can manipulate those tags or namespaces until such time as the user revokes that permission.

    The app uses its own credentials and the user never need share his/her password with anyone.

    I think is probably what you're describing too, but I couldn't quite glean that from your words.



  15. Our suggestions are complimentary.

    When you say "App requests permission to do certain things to certain of the user's namespaces and/or (abstract) tags" - I'm just suggesting a mechanism by which the app can identify the FluidDB user in question, without that user needing to remember their FDB login credentials.

    As per your recommendations, FluidDB needs to give (for example) write permission to /users/app on certain of /users/bob 's tags. And the app itself will write tags such as /bob/likes etc.

    I'm just suggesting a way of uniquely identifying 'bob' in both contexts.

  16. Got it,

    Knew I was being slow.

    Makes sense.