[Girder-users] Understanding Items, Files and Folders

[Andrés Fortier]

Hi Jon,
thanks for the quick reply and the explanation about file/folder/item,
things are much clear now. Regarding the approach we are taking, we would
like to think in term of "assets", where an asset can be either a file or a
folder. Under this approach we will most likely remove the file/folder
endpoints and generate a new one called "asset", keeping the REST
semantics. So maybe we should start by creating an Asset model with its
mongo collection counterpart and then move from there. But I will dig a bit
deeper into mongo's options first, just to get a better insight on its
capabilities.

Thanks!
Andy

On Mon, Jul 10, 2017 at 10:49 AM, Jonathan Beezley <
jonathan.beezley at kitware.com> wrote:

> Hi Andy,
>
> Folders, items, and files are all separate entities.  In terms of their
> tree structure, Girder models look like this:
>
> * a folder's parent is either a folder, user, or collection
> * an item's parent is a folder
> * a file's parent is an item
>
> A model can only have one parent, but multiple children.  So, folders can
> contain both other folders and items, but not files.  In terms of an
> abstract file system, there is no equivalent to an "item", it is an
> additional type that lies between a file and a folder.
>
> Files don't have metadata associated with them, so you are probably
> looking to create an endpoint that searches folders and items.  It's
> possible you might be able to construct a single query to do this using a
> mongo aggregation pipeline, but I'm not certain how.  The simplest way
> would be to do two queries and join them together somehow.  The only
> endpoint that I know of that returns multiple model types is
> /resource/search
> <https://github.com/girder/girder/blob/master/girder/api/v1/resource.py#L68>,
> though I don't know I would recommend doing it that way.
>
> If I were doing this, I would probably just create two different endpoints
> for each type.  It adds a bit of complexity to the client (two rest
> requests rather than one), but it is a more idiomatic restful interface.
>
> Jon
>
> On Mon, Jul 10, 2017 at 9:25 AM, Andrés Fortier <andres at ekumenlabs.com>
> wrote:
>
>> Hi all,
>> I'm working on a plugin to search on Files and Folders by a metadata
>> field (the "type" field). The important parts of the code I have so far are:
>>
>> class ResourceExt(Resource):
>>     @access.public
>>     @describeRoute(...)
>>     def typeSearch(self, params):
>>         self.requireParams(('type'), params)
>>         limit, offset, sort = self.getPagingParameters(params, 'name')
>>         asset_type = params['type']
>>         query = {"meta.type": asset_type}
>>         fields = ['_id', 'name', 'description', 'folderId', 'public',
>> 'access']
>>
>>         model = ModelImporter().model('item')
>>         cursor = model.find(query, fields=fields)
>>         return list(model.filterResultsByPermission(
>>             cursor, user=self.getCurrentUser(), level=AccessType.READ,
>>             limit=limit, offset=offset, removeKeys=('public', 'access')))
>>
>> def load(info):
>>     ModelImporter.model('item').ensureIndex(['meta.type', {'sparse':
>> True}])
>>     ext = ResourceExt()
>>     info['apiRoot'].resource.route('GET', ('type_search',),
>> ext.typeSearch)
>>
>> now this seems to be working ok with files, but fails to return folders,
>> so I think I have a wrong idea of what an item model is. I originally
>> thought that an item was an abstraction of both files and folders, and that
>> querying the item model was the way to query for both of them. I then
>> checked the items mongo collection and it seems there is an item record for
>> each file record, but none for the folder entries. So this brings up a
>> couple of questions:
>>
>> 1. What is the relationship between items, files and folders? What are
>> the concrete use cases for items? [1]
>> 2. Is there a way to perform metadata-based queries on both files and
>> folders? I know I can do two queries (one on files and one on folders) but
>> then I would have to manually merge them and handle all the limit/offset
>> math, which seems unnecessarily complicated.
>>
>> [1] I did check the class comments and I'm a bit confused about those
>> too. E.g. in Item it says "Items are leaves in the data hierarchy. They
>> can contain 0 or more files within them ...", which is confusing as I
>> would expect leaves to be files instead of containing them. Also if items
>> can contain files, doesn't that mean they are (or act as) folders?
>>
>> Thanks in advance!
>> Andy
>>
>> _______________________________________________
>> Girder-users mailing list
>> Girder-users at public.kitware.com
>> http://public.kitware.com/mailman/listinfo/girder-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://public.kitware.com/pipermail/girder-users/attachments/20170710/da83c14e/attachment-0001.html>