About Tags In Fish

I’ve added a new command to fish (and updated the online version, Shell-Fish accordingly) to allow easy construction of standardized about tags using the conventions from the abouttag library. They make use of a new abouttag function, available in the new generic.py file in the abouttag library, which takes the object type as its first parameter, and the usual parameters as a variable parameter list.
The new fish command is abouttag, though can also be abbreviated to about and its general form is:

fish abouttag <object type> <object specifiers>

The object type is something like book, album or fi-user and the object specifiers are the key parameters used to describe that object, in the same order as they are used in the corresponding function from the abouttag library.
The easiest way to illustrate and define these is with examples. The following examples are taken from a Unix system; on Windows, use double quotes rather than single around parameters. In the online version (Shell-Fish), and on Unix, single or double quotes work. In the online version, you don’t need the fish prefix (though it does work).
I should note that part of the motivation for adding this functionality is a desire to allow the command to be used to specify objects without knowing the exact form of their about tags. In Unix-like systems (Linux, Mac OS X, Solaris etc.), this is possible by using left quotes, which can be placed inside double quotes. Thus, the following, slightly ungainly command (using all three forms of quote) works, at least in bash:

$ fish show -F -a "`fish abouttag book 'Gödel, Escher, Bach: An Eternal Golden Braid' 'Douglas R. Hofstader'`" njr/rating
Object with about="book:gödel escher bach an eternal golden braid (douglas r hofstader)":
  njr/rating = 10

I will leave it to the reader to judge whether this is easier than using cut and paste. For those who don’t know about left quotes in Unix shells, a command enclosed in left quotes within another command is evaluted before its enclosing command; its output replaces the left-quoted phrase on the original command line. So in the case above, we first run the command

fish abouttag book 'Gödel, Escher, Bach: An Eternal Golden Braid' 'Douglas R. Hofstader'

which generates

book:gödel escher bach an eternal golden braid (douglas r hofstader)

as its output. In effect, the outer command is then transformed to

fish show -F -a "book:gödel escher bach an eternal golden braid (douglas r hofstader)" njr/rating

I hope to extend shell-fish, the on-line version of fish, to support left quotes, but that may take a little while.
The following examples are taken from the fish documentation, which is available online from http://fluiddb.fluidinfo.com/about/fish/fish/index.html.

1. Books and related items using the book-u convention (book, author)

   $ fish abouttag book 'Gödel, Escher, Bach: An Eternal Golden Braid' 'Douglas R. Hofstader'
   book:gödel escher bach an eternal golden braid (douglas r hofstader)
   
   $ fish abouttag book 'The Feynman Lectures on Physics' 'Richard P. Feynman' 'Robert B. Leighton' 'Matthew Sands'
   book:the feynman lectures on physics (richard p feynman; robert b leighton; matthew sands)
   
   $ fish abouttag book 'The Oxford English Dictionary: second edition, volume 3', 'John Simpson', 'Edmund Weiner'
   book:the oxford english dictionary second edition volume 3 (john simpson; edmund weiner)
   
   $ fish abouttag author 'Douglas R. Hofstadter' 1945 2  15
   author:douglas r hofstadter (1945-02-15)

2. Music-related items (track, album, artist, isrc-recording)

   $ fish abouttag track 'Bamboulé' 'Bensusan and Malherbe'
   track:bamboulé (bensusan and malherbe)
   
   $ fish abouttag album 'Solilaï' 'Pierre Bensusan'
   album:solilaï (pierre bensusan)
   
   $ fish abouttag artist 'Crosby, Stills, Nash & Young'
   artist:crosby stills nash & young
   
   $ fish abouttag isrc-recording 'US-PR3-73-00012'
   isrc:USPR37300012

3. URLs and URIs (URI, URL)

   $ fish abouttag uri FluidDB.fluidinfo.com
   http://fluiddb.fluidinfo.com
   
   $ fish abouttag url https://FluidDB.fluidinfo.com/one/two/
   https://fluiddb.fluidinfo.com/one/two
   
   $ fish abouttag URI http://fluiddb.fluidinfo.com/one/two/
   http://fluiddb.fluidinfo.com/one/two
   
   $ fish abouttag URL 'http://test.com/one/two/?referrer=http://a.b/c'
   http://test.com/one/two/?referrer=http://a.b/c

4. Fluidinfo objects (fi-user, fi-namespace, fi-tag)

   $ fish abouttag fi-user njr
   Object for the user named njr
   
   $ fish abouttag fi-namespace njr/misc
   Object for the namespace njr/misc
   
   $ fish abouttag fi-ns njr/private
   Object for the namespace njr/private
   
   $ fish abouttag fi-tag terrycojones/private/rating
   Object for the attribute terrycojones/private/rating

5. Database components (db-table, db-field)

   $ fish abouttag db-table 'elements'
   table:elements
   
   $ fish abouttag db-field 'name' 'elements'
   field:name in table:elements

6. Miscellaneous (planet, element)

   $ fish abouttag planet 'Mars'
   planet:Mars
   
   $ fish abouttag element 'Helium'
   element:Helium

Choice and Conformity in fdb

I’ve just pushed a new version of fdb.py to the GitHub repository.

The main change in this version is that I’ve added support for allowing the user to choose whether to use what we might call Unix-style paths or Fluidinfo-style paths.

Until now, fdb.py has, as part of its shell-like functionality, deliberately provided an alternative view of Fluidinfo from the underlying structure. The main features of this “Unix-style view” are as follows:

• Full (absolute) fdb.py tag paths start with a leading slash. So my rating tag would be /njr/rating rather than njr/rating.
• A tag path without a slash is taken to be a relative path, currently always relative to the user’s namespace (though there are alternate versions where there is a notion of a current working namespace (CWD) which can be changed with a cd command). Thus, when using my credentials, the Fluidinfo tag njr/rating can be referred to as rating while ntolls rating is ntoll/rating.
• /about is provided as a synonym for the special tag fluiddb/about (the about tag).
• /id is provided as a pseudo-tag that will report the value of the object’s Fluidinfo ID.

This was not carried all the way: I didn’t re-write queries, but, for me at least, it saved much typing and pain when using fdb from the command line.

This release (1.33) maintains this behaviour by default, but allows the user to configure or tell the system that she would prefer to use genuine, regular all-American Fluidinfo-style paths. There are two ways to invoke this alternative behaviour:

• If you would always prefer to user regular Fluidinfo-style paths, the best thing to do is to add a third line to the credentials file that fdb uses saying

  unix-style-paths false

  (I need hardly add that using true instead of false sets the opposite preference.)

• Alternatively, if you just want to override the configured or default behaviour for a one-off command, use the command-line flags -F. Similarly, to override the behaviour to force Unix-style paths, use -U.

When you choose Fluidinfo-style-paths, this is what happens:

• Only command-line commands are affected: if you use fdb.py through the API, nothing changes unless you work quite hard.
• Any time you specify a path, it needs to be an absolute path. In Terry’s world, absolutely all paths are absolute.
• /about is not accepted as a synonym for fluiddb/about
• The only special case is /id. Since this is a useful pseudo tag (in my view), and since it has no namespace, the same trick works as when using unix-style paths. Thus you can request the tag /id and it will return the object ID.
• Output as well as input is affected, i.e. tag paths will be reported without a leading slash.

Examples¶

Old (default) behaviour:

$ fdb tags -a "Eiffel Tower"
Object with about="Eiffel Tower":
/objects/93bd1999-0998-49cc-8004-af457ce34ce4
  /njr/location = "Paris"
  /fluiddb/about = "Eiffel Tower"
  /njr/index/about

Behaviour with -F or with unix-style-paths false

$ fdb tags -F -a "Eiffel Tower"
Object with about="Eiffel Tower":
/objects/93bd1999-0998-49cc-8004-af457ce34ce4
  njr/location = "Paris"
  fluiddb/about = "Eiffel Tower"
  njr/index/about

Setting and showing tags (old/default behaviour):

$ fdb tag -a "Eiffel Tower" rating=7
$ fdb show -a "Eiffel Tower" rating
Object with about="Eiffel Tower":
  /njr/rating = 7

Behaviour with -F or with unix-style-paths false

$ fdb tag -F -a "Eiffel Tower" njr/rating=8
$ fdb show -F -a "Eiffel Tower" njr/rating
Object with about="Eiffel Tower":
  njr/rating = 8

The same behaviour works with untag:

$ fdb untag -F -a "Eiffel Tower" njr/rating
zero:$ fdb show -F -a "Eiffel Tower" njr/rating
Object with about="Eiffel Tower":
  (tag njr/rating not present)

I may have missed something, but as far as I can see, this works reliably. If I have missed something, let me know.

So: if you’ve always liked the look of fdb but disliked its unix-style paths, now might be a good time to get it. And if you already use it, but would prefer to use Fluidinfo-style paths, just add

unix-style-paths false

as the third line of your credentials file.

About Tag Conventions in FluidDB

This page documents the conventions for about tags in FluidDB that I am aware of. I will update it periodically when I learn of more conventions or discover errors.

For want of a better alternative, I define each about tag format using a python string format string and as many parameters as are required. I then give real examples, illustrated with fdb (Not that, unlike FluidDB itself, fdb uses leading slashes to introduce tags and interprets /about as /fluiddb/about and /id as an object’s UUID.)

UPDATE 2010/03/12: Rather than just showing the python string format here, I realised it would be useful to write some code to produce about tags following the conventions below. I have just committed a python library called abouttag at github. In addition to performing the rather trivial formatting described below this also does a degree of (optional) normalization of string input, to further reduce the likelihood of unintentional differences. I will gradualy add examples below using the library and will also do a separate post about it.

UPDATE 2010/12/29: I’ve also added in links to some visualization software available at abouttag.com, which shows objects in FluidDB and the tags on them graphically. This software should work with any modern, standards-compiant web brower, which in practice means almost anything except Internet Explorer.

1. FluidDB (fluiddb-1)¶

The following conventions are used by FluidDB itself when it creates the objects that form the very basis of FluidDB.

1.1. Users¶

The about tag for the user with name username is:

"Object for the user named %s" % username

$ fdb show -i 8cb0e83b-fa48-490c-a95f-958053db598c /about
Object 8cb0e83b-fa48-490c-a95f-958053db598c:
/fluiddb/about = "Object for the user named njr"

Graphical Example: Object for the user named njr.

1.2. Namespaces¶

The about tag for the namespace whose name is ns is:

"Object for the namespace %s" % ns

$ fdb show -i fb2e7545-1dd4-4432-8cfc-2fd24db7384b /about
Object fb2e7545-1dd4-4432-8cfc-2fd24db7384b:
/fluiddb/about = "Object for the namespace njr"

$ fdb show -i 9b4846ad-6639-46c2-adb1-e0d87649413d /about
Object 9b4846ad-6639-46c2-adb1-e0d87649413d:
/fluiddb/about = "Object for the namespace miro/elements"

Graphical example: Object for the namespace miro/elements

1.3. Tags¶

The about tag for the tag tag is:

"Object for the attribute %s" % tag

$ fdb show -i 4f6550a5-7eed-4f45-8046-56f0ac6c4f43 /about
Object 4f6550a5-7eed-4f45-8046-56f0ac6c4f43:
  /fluiddb/about = "Object for the attribute njr/rating"

$ fdb show -a "Object for the attribute fluiddb/about" /about /id
Object with about="Object for the attribute fluiddb/about":
  /fluiddb/about = "Object for the attribute fluiddb/about"
  /id = "873e0311-07d2-42bd-b67f-739d731332da"

Graphical example: Object for the attribute fluiddb/about

Find about tags for the objects FluidDB uses itself here.

2. URLs and URIs (uri-1)¶

Perhaps the most obvious and common entities for which objects are created in FluidDB are URLs and more generally URIs. So far, probably with barely a conscious decision, almost everyone is using the obvious convention:

URLs and URIs¶

The about tag for a URL or URI is itself in standard form, for example

http://fluidinfo.com/fluiddb

Here, standard form most often seems to mean lower case for the domain name, no trailing slash, and inclusion of the protocol specifier (http://) etc. So:

http://fluidinfo.com/fluiddb

not fluidinfo.com/fluiddb

not http://fluidinfo.com/fluiddb/

For example, there are a couple of ways of importing bookmarks from del.icio.us to FluidDB (here and here), both of which use the standardized URL of the web page as the object on which to place imported tags and other metadata.

fdb show -a 'http://www.google.com' /about /id
Object with about="http://www.google.com":
  /fluiddb/about = "http://www.google.com"
  /id = "e75abb22-2f3c-45dd-9429-fce0a78f82f2"

Graphical example: http://www.google.com.

Find about tags for URLs and URIs here.

3. Tables (database-1)¶

Miró is a data analysis package produced by Stochastic Solutions. Miró has direct integration with FluidDB including the ability to publish a dataset (in the form of a table) straight to FluidDB.

Miró is in the process of defining conventions for about tags for the objects it used to represent tables.

There is a post describing the approach to tables available here.

3.1. Tables¶

The about tag for a table called tablename in is:

"table:%s" % tablename

Object with about="table:elements":
  /fluiddb/about = "table:elements"
  /id = "b898b875-89ad-4b69-bf89-b6a605b79697"

3.1.1. Fields¶

The about tag for a field named fieldname in a table tablename is:

"field:%s in table:%s" % (fieldname, tablename)

$ fdb show -q 'miro/field-name = "RelativeAtomicMass"' /about
1 object matched
Object 261a1ce8-4a42-45dc-b4be-4206d95b0bbe:
  /fluiddb/about = "field:RelativeAtomicMass in table:elements"

3.1.2. Records¶

There is no fixed recommendation for the about tag for a database record. Usually, if the record pertains to something, it will be appropriate to store the record data on an on object naturally relating to that thing.

Miró intends always to use about tags of the general for entity:description, where entity is a category of entity, and a description is a (standard-form) description of that entity.

At the time of writing, two tables have been published to FluidDB by Miró, pertaining to elements of the periodic table and to planets (and dwarf planets) in the solar system.

4. Elements of the Periodic Table (element-1)¶

Miró has published a table of all of the elements of the periodic table to FluidbDB usin the table conventions above.

The about tag it uses for an named Element is:

"element:%s"' % Element

The elements can be identified by querying FluidDB for objects with (among others) the tag miro/elements/Name

$ fdb count -q "has miro/elements/Name"
118 objects matched
Total: 118 objects

$ fdb show -q 'miro/elements/Name = "Hydrogen"' /about
1 object matched
Object 270a8269-f02d-4925-b152-da3934edaa43:
  /fluiddb/about = "element:Hydrogen"

$ fdb show -q 'miro/elements/Z = 92' /about
1 object matched
Object adde1cbf-286b-4d0f-9c8c-57976a473321:
  /fluiddb/about = "element:Uranium"

Graphical example: element:Uranium.

Find about tags for elements here.

5. Planets (planet-1)¶

Miró has published a table containing the thirteen recognized planets and dwarf planets.

The about tag for an planet (or similar) named Planet is:

"planet:%s" % Planet

This dataset is described in detail at here.

$ fdb show -q 'miro/planets/Name = "Earth"' /about
1 object matched
Object 87551a22-5ef9-49e0-9d74-99f2d4af6776:
  /fluiddb/about = "planet:Earth"

$ fdb show -q 'miro/planets/Name = "Saturn"' /about
1 object matched
Object e06bea33-a000-4294-a7b2-d3245f1481ca:
  /fluiddb/about = "planet:Saturn"

Graphical example: planet:Saturn.

Find about tags for planets here.

6. Location (geonet-1)¶

Nicholas Tollervey (@ntoll) has been tagging places using the geo FluidDB user. He is using conventions from GEONet, which (among other things) defines a Unique Feature Identifier (UFI) and a Unique Name Identifier (UNI).

The about tag for a place with a given UNI and UFI Nicholas is using is:

"GEOnet%s_%s" % (UNI, UFI)

Objects corresponding to such places can be identified by having a geo/name tag. At the time of writing:

$ fdb count -q "has geo/name"
18950 objects matched
Total: 18950 objects

$ fdb show -q 'geo/name = "Fitful Head"' /about
1 object matched
Object 0a3dfbc6-e6e2-4d8b-8cbc-1c1920fa082b:
  /fluiddb/about = "GEOnet-2596258_-3571222"

Graphical example: GEOnet-2596258_-3571222.

Find about tags for locations here.

7.Twitter Users¶

The Tickery application stores information about twitter users on objects identified by their twitter ID. A twitter ID is a numeric identifier that remains the same even if twitter users change their twitter name. For example, the twitter ID for @terrycojones is 42983, so tickery stores information about this user on the about having the about tag twitter.com:uid:42983.

You can find the twitter ID for a user using idfromuser.com.

Find about tags for twitter users here

More?¶

If you are systematically using a convention for about tags in FluidDB, let me know. I plan to try to update this page as more conventions emerge.

The fdb command line in fdb.py 0.3

I just pushed fdb.py 0.3 to github (http://github.com/njr0/fdb.py). It has a few extra things in the API, but the big new thing is you can use it from the command line.

I hope the following is reasonably self-explanatory. Obviously you can use an alias or a 1-line shell script to get rid of the need for the python fdb.py.

Script started on Fri Aug 21 16:27:35 2009
$ # reference objects by their about tag using -a
$ # the -v just tells it to be verbose and tell you what it's doing 
$ python fdb.py tag -av DADGAD rating=10
Tagged object with about="DADGAD" with rating = 10

$ python fdb.py tag -av DADGAD favourite
Tagged object with about="DADGAD" with favourite

$ python fdb.py get -a DADGAD rating favourite /terry/rating
Object with about=DADGAD:
  /njr/rating = 10
  /njr/favourite
  <tag /terry/rating not present>

$ python fdb.py untag -a -v DADGAD rating
Removed tag rating from object with about="DADGAD"

$ python fdb.py get -a DADGAD rating favourite /terry/rating
Object with about=DADGAD:
  <tag /njr/rating not present>
  /njr/favourite
  <tag /terry/rating not present>

$ # reference objects by their id tag using -i

$ python fdb.py tag -iv a984efb2-67d8-4b5c-86d0-267b87832fa4 rating=10
Tagged object a984efb2-67d8-4b5c-86d0-267b87832fa4 with rating = 10

$ python fdb.py tag -iv a984efb2-67d8-4b5c-86d0-267b87832fa4 favourite
Tagged object a984efb2-67d8-4b5c-86d0-267b87832fa4 with favourite

$ python fdb.py get -i a984efb2-67d8-4b5c-86d0-267b87832fa4 rating favourit e /terry/rating
Object a984efb2-67d8-4b5c-86d0-267b87832fa4:
  /njr/rating = 10
  /njr/favourite
  <tag /terry/rating not present>

$ python fdb.py untag -i -v a984efb2-67d8-4b5c-86d0-267b87832fa4 rating
Removed tag rating from object a984efb2-67d8-4b5c-86d0-267b87832fa4

$ python fdb.py get -i a984efb2-67d8-4b5c-86d0-267b87832fa4 rating favourit e /terry/rating
Object a984efb2-67d8-4b5c-86d0-267b87832fa4:
  <tag /njr/rating not present>
  /njr/favourite
  <tag /terry/rating not present>
$ exit

Script done on Fri Aug 21 16:31:45 2009

Tagging, Tags and Abstract Tags

An issue anyone using the FluidDB API (in any form) will run across fairly quickly is the occasionally vexed issue of what we actually mean by a tag.

Conceptually, it's very straightforward: a tag is exactly like the tags we all know and love from del.icio.us, Flickr, GMail etc., with the twist that they can have values. So a tag has a name (like njr/rating, terry/toread etc.), and optionally has a value (which can be of almost any type) too.

Additionally, tags have some other properties, like a rather full permissions system (that controls who can see, edit and attach them to objects) and some metadata, like an optional description.

The potential confusion arises because sets of tags sharing the same name are (for good reason) often managed together, and in some cases share metadata in FluidDB.

We can see this if we look at the process of tagging an object in a little detail.

In the client library, fdb.py that I published earlier, you can add a tag to an object very simply if you know its ID. For example, if I wanted to add an njr/rating of 10 tag to the object with the id a984efb2-67d8-4b5c-86d0-267b87832fa4g, I could just say

import fdb
db = fdb.FluidDB (fdb.Credentials (filename='/Users/njr/.fluidDBcredentials'))
o = db.tag_object_by_id ('a984efb2-67d8-4b5c-86d0-267b87832fa4', 'rating', 10)
assert o = 0

This corresponds very closely to the conceptual model I use and encourage others to use, and works even if there have never been any njr/rating tags in the system before.

Under the covers, however, the native HTTP API sees things slightly differently. Before I can tag something with a tag such as njr/rating I first have to tell the system I want to use tags with this name. The underlying API refers to this process as tag creation, though I prefer to think of it as abstract tag creation, or tag declaration.

So the way fdb.py actually works, is that when you ask it to tag an object with a given tag (and perhaps a value), it goes ahead and tries to do that for you. But if that tag hasn't previously been declared (i.e., if the abstract form of it hasn't been created), this will fail. In this case, the library backs up and creates the abstract tag and then tries again. It does this using another fdb call:

db.create_abstract_tag ('rating', description=None, indexed=True):

As you can see, we can also give a description for an (abstract) tag, which in effect applies to all the real ("concrete") tags we create when we tag objects, and can also specify whether FluidDB should index the tag (making it searchable). So we could say:

db.create_abstract_tag ('rating',
          description="njr's rating for things, on a scale of 0-10")

Similarly, I will soon impement some untag methods in fdb.py, but these shouldn't be confused with the delete_abstract_tag function that already exists. The delete_abstract_tagmethod doesn't simply simply remove tag from an object, but actually deletes all tags with the given tag name (and the abstract tag itself) from the system.

The reason FluidDB cares so much about abstract tags (or, if you prefer, sets of tags sharing the same tag name) is that this is the level at which the permissions system acts. In FluidDB, there is fairly fine-grain control, allowing the owner of an (abstract) tag to decide who is allowed to read, apply, and alter tags with a given name to objects.

More on that later.