[Girder-users] Understanding Items, Files and Folders

[Jonathan Beezley]

Hi Andy,

Folders, items, and files are all separate entities.  In terms of their
tree structure, Girder models look like this:

* a folder's parent is either a folder, user, or collection
* an item's parent is a folder
* a file's parent is an item

A model can only have one parent, but multiple children.  So, folders can
contain both other folders and items, but not files.  In terms of an
abstract file system, there is no equivalent to an "item", it is an
additional type that lies between a file and a folder.

Files don't have metadata associated with them, so you are probably looking
to create an endpoint that searches folders and items.  It's possible you
might be able to construct a single query to do this using a mongo
aggregation pipeline, but I'm not certain how.  The simplest way would be
to do two queries and join them together somehow.  The only endpoint that I
know of that returns multiple model types is /resource/search
<https://github.com/girder/girder/blob/master/girder/api/v1/resource.py#L68>,
though I don't know I would recommend doing it that way.

If I were doing this, I would probably just create two different endpoints
for each type.  It adds a bit of complexity to the client (two rest
requests rather than one), but it is a more idiomatic restful interface.

Jon

On Mon, Jul 10, 2017 at 9:25 AM, Andrés Fortier <andres at ekumenlabs.com>
wrote:

> Hi all,
> I'm working on a plugin to search on Files and Folders by a metadata field
> (the "type" field). The important parts of the code I have so far are:
>
> class ResourceExt(Resource):
>     @access.public
>     @describeRoute(...)
>     def typeSearch(self, params):
>         self.requireParams(('type'), params)
>         limit, offset, sort = self.getPagingParameters(params, 'name')
>         asset_type = params['type']
>         query = {"meta.type": asset_type}
>         fields = ['_id', 'name', 'description', 'folderId', 'public',
> 'access']
>
>         model = ModelImporter().model('item')
>         cursor = model.find(query, fields=fields)
>         return list(model.filterResultsByPermission(
>             cursor, user=self.getCurrentUser(), level=AccessType.READ,
>             limit=limit, offset=offset, removeKeys=('public', 'access')))
>
> def load(info):
>     ModelImporter.model('item').ensureIndex(['meta.type', {'sparse':
> True}])
>     ext = ResourceExt()
>     info['apiRoot'].resource.route('GET', ('type_search',),
> ext.typeSearch)
>
> now this seems to be working ok with files, but fails to return folders,
> so I think I have a wrong idea of what an item model is. I originally
> thought that an item was an abstraction of both files and folders, and that
> querying the item model was the way to query for both of them. I then
> checked the items mongo collection and it seems there is an item record for
> each file record, but none for the folder entries. So this brings up a
> couple of questions:
>
> 1. What is the relationship between items, files and folders? What are the
> concrete use cases for items? [1]
> 2. Is there a way to perform metadata-based queries on both files and
> folders? I know I can do two queries (one on files and one on folders) but
> then I would have to manually merge them and handle all the limit/offset
> math, which seems unnecessarily complicated.
>
> [1] I did check the class comments and I'm a bit confused about those too.
> E.g. in Item it says "Items are leaves in the data hierarchy. They can
> contain 0 or more files within them ...", which is confusing as I would
> expect leaves to be files instead of containing them. Also if items can
> contain files, doesn't that mean they are (or act as) folders?
>
> Thanks in advance!
> Andy
>
> _______________________________________________
> Girder-users mailing list
> Girder-users at public.kitware.com
> http://public.kitware.com/mailman/listinfo/girder-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/girder-users/attachments/20170710/a2fa4e67/attachment.html>