From John.Roberts at hsc.utah.edu Thu Feb 2 18:25:40 2017 From: John.Roberts at hsc.utah.edu (John Roberts) Date: Thu, 2 Feb 2017 16:25:40 -0700 Subject: [Girder-users] Setting up proxied Girder with Docker Container Message-ID: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> I'd like to set up a proxied Girder as detailed in the manual here . We need to configure girder so that it understands it is not working from the root directory of the Apache server but one directory down (/girder). Unlike the situation in the documentation, I'm working with the Docker girder. If I rather blindly edit the girder.local.cfg file within the running container, the container crashes as soon as I save the file. This may have been due to a typo, but I have a further question. Assuming I did update girder.local.cfg properly as indicated, the instructions then say to rebuild Girder using "npm install". It's my impression that this will likely crash the girder container itself, since the current girder process will likely be terminated while the new build is being compiled with npm. Would that be a correct assumption? The question is how to configure Girder within the container to invoke the proxy and move the address space one step down from root at / to a proxied address /girder. A more general question might be how to update girder.local.cfg when running Girder with Docker. I tried running the Docker girder:girder image and entering by way of an alternative entry-point, docker run -it --entrypoint=/bin/bash girder/girder -s This lets me in and I can edit the girder.local.cfg file. However, "npm install" then appears to fail with the warning "cannot run in wd girder at 1.5.2 grunt init && grunt (wd=/girder)". Since I don't know what "npm install" was supposed to accomplish, I'm not sure where to look to see if it succeeded despite the warning. Thanks for suggestions, John. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zach.mullen at kitware.com Thu Feb 2 19:06:55 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Thu, 2 Feb 2017 19:06:55 -0500 Subject: [Girder-users] Setting up proxied Girder with Docker Container In-Reply-To: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> References: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> Message-ID: Hi John, You're running an old version of Girder (1.5.2), are you able to upgrade to the latest version and see if that fixes the issue? Thanks, Zach Mullen Kitware, Inc. 919-869-8858 On Thu, Feb 2, 2017 at 6:25 PM, John Roberts wrote: > I'd like to set up a proxied Girder as detailed in the manual here > . We > need to configure girder so that it understands it is not working from the > root directory of the Apache server but one directory down (/girder). > Unlike the situation in the documentation, I'm working with the Docker > girder. > > If I rather blindly edit the girder.local.cfg file within the running > container, the container crashes as soon as I save the file. This may have > been due to a typo, but I have a further question. > > Assuming I did update girder.local.cfg properly as indicated, the > instructions then say to rebuild Girder using "npm install". It's my > impression that this will likely crash the girder container itself, since > the current girder process will likely be terminated while the new build is > being compiled with npm. Would that be a correct assumption? > > The question is how to configure Girder within the container to invoke > the proxy and move the address space one step down from root at / to a > proxied address /girder. A more general question might be how to update > girder.local.cfg when running Girder with Docker. > > I tried running the Docker girder:girder image and entering by way of > an alternative entry-point, > > docker run -it --entrypoint=/bin/bash girder/girder -s > > This lets me in and I can edit the girder.local.cfg file. However, "npm > install" then appears to fail with the warning "cannot run in wd > girder at 1.5.2 grunt init && grunt (wd=/girder)". Since I don't know what > "npm install" was supposed to accomplish, I'm not sure where to look to see > if it succeeded despite the warning. > > Thanks for suggestions, > John. > > > > > _______________________________________________ > Girder-users mailing list > Girder-users at public.kitware.com > http://public.kitware.com/mailman/listinfo/girder-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zach.mullen at kitware.com Thu Feb 2 20:32:06 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Thu, 2 Feb 2017 20:32:06 -0500 Subject: [Girder-users] Setting up proxied Girder with Docker Container In-Reply-To: References: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> <9ff47838-a881-d7bd-6f0e-b72ece65acc0@hsc.utah.edu> Message-ID: That is, what is the value listed in your OAuth application's *Authorization callback URL* field, which looks like the below screenshot: [image: Inline image 1] On Thu, Feb 2, 2017 at 8:24 PM, Zach Mullen wrote: > To diagnose that issue, how is your redirect URI configured in github? And > what is the value of the "state" parameter when you are sent to the github > login page? > > > > On Thu, Feb 2, 2017 at 8:07 PM, John Roberts > wrote: > >> Thanks for the reply, Zach. >> >> Playing around with a test girder/mongo pair, I managed to edit the >> girder.local.cfg file without crashing the container. Subsequently, it >> seems to recognize correctly that it is not at the top level of the web >> server. >> >> This is all related to our effort to run oauth through github. >> Previously, without the proper proxy settings in girder.local.cfg, the >> callback address generated in the oath plugin setting did not include the >> proxy. Now it does. >> >> Unfortunately, authorization through github still generates a >> redirect_uri_mismatch error. >> >> My best guess at the moment is that this is somehow related to our >> proxying girder itself through https on the Apache server. I mean, we have >> a reverse proxy setup on Apache to route incoming requests for >> https://some.address.com/girder to our internal >> http://some.docker.network:8080/girder. >> >> Somewhere in the proxying, I'm thinking the URL might be modified and >> either Girder or Github says it doesn't match what's expected. The message >> indicates that it's Github who is rejecting the callback: >> >> { >> "message": "Provider returned error: 'redirect_uri_mismatch'.", >> "type": "rest" >> } >> >> >> John. >> >> On 02/02/2017 05:06 PM, Zach Mullen wrote: >> >> Hi John, >> >> You're running an old version of Girder (1.5.2), are you able to upgrade >> to the latest version and see if that fixes the issue? >> >> Thanks, >> >> Zach Mullen >> Kitware, Inc. >> 919-869-8858 <(919)%20869-8858> >> >> On Thu, Feb 2, 2017 at 6:25 PM, John Roberts >> wrote: >> >>> I'd like to set up a proxied Girder as detailed in the manual here >>> . >>> We need to configure girder so that it understands it is not working from >>> the root directory of the Apache server but one directory down (/girder). >>> Unlike the situation in the documentation, I'm working with the Docker >>> girder. >>> >>> If I rather blindly edit the girder.local.cfg file within the >>> running container, the container crashes as soon as I save the file. This >>> may have been due to a typo, but I have a further question. >>> >>> Assuming I did update girder.local.cfg properly as indicated, the >>> instructions then say to rebuild Girder using "npm install". It's my >>> impression that this will likely crash the girder container itself, since >>> the current girder process will likely be terminated while the new build is >>> being compiled with npm. Would that be a correct assumption? >>> >>> The question is how to configure Girder within the container to >>> invoke the proxy and move the address space one step down from root at / to >>> a proxied address /girder. A more general question might be how to update >>> girder.local.cfg when running Girder with Docker. >>> >>> I tried running the Docker girder:girder image and entering by way >>> of an alternative entry-point, >>> >>> docker run -it --entrypoint=/bin/bash girder/girder -s >>> >>> This lets me in and I can edit the girder.local.cfg file. However, "npm >>> install" then appears to fail with the warning "cannot run in wd >>> girder at 1.5.2 grunt init && grunt (wd=/girder)". Since I don't know >>> what "npm install" was supposed to accomplish, I'm not sure where to look >>> to see if it succeeded despite the warning. >>> >>> Thanks for suggestions, >>> John. >>> >>> >>> >>> >>> _______________________________________________ >>> Girder-users mailing list >>> Girder-users at public.kitware.com >>> http://public.kitware.com/mailman/listinfo/girder-users >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2017-02-02 at 8.29.34 PM.png Type: image/png Size: 47687 bytes Desc: not available URL: From John.Roberts at hsc.utah.edu Fri Feb 3 17:37:41 2017 From: John.Roberts at hsc.utah.edu (John Roberts) Date: Fri, 3 Feb 2017 15:37:41 -0700 Subject: [Girder-users] Setting up proxied Girder with Docker Container In-Reply-To: <2e891e4b-7e84-8b7f-abb6-c4f2d3f05a73@hsc.utah.edu> References: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> <9ff47838-a881-d7bd-6f0e-b72ece65acc0@hsc.utah.edu> <2e891e4b-7e84-8b7f-abb6-c4f2d3f05a73@hsc.utah.edu> Message-ID: <7223d20b-b809-9704-9845-87753473aa9e@hsc.utah.edu> I managed to get this working, both having a Docker containerized Girder be self-aware of its own proxying (edits to girder.local.cfg) and for Oauth to work over the various reverse proxy hops. Within the girder container, I edit the girder.local.config file to add lines controlling its self-awareness about proxying. * [global] o tools.proxy.on: True o tools.proxy.base: "https://outward.facing.edu/girderdev" o tools.proxy.local: "" * [server] o api_root: "/girderdev/api/v1" o static_root: "/girderdev/static" The tools.proxy.base/local settings may be overkill for our Apache front end, but things work now, so I left those in. The above changes show up in the Oauth settings page for the github plugin that I wanted to use. The callback address previously did not include the proxying girderdev portion of the address, because Girder had not been aware of its own proxying (which we manage via Apache). With the changes to girder.local.cfg, the address generated in the plugin settings was correct. The last hurdle was on Github itself. Girder is apparently pulling down both the account name and email address from Github. Only the email is matched against existing user emails, but the Github account name becomes the users's name on Girder for the duration of the login when an email match is found. In my case, my test github account had no "name" per se and the blank caused an error in Girder. One other note: the Oauth through Github works even though our top server, https://outward.facing.edu, is not itself visible outside our institution's firewall. I'm assuming this works because it is Girder that is invoking the connection to Github as an oath client. I'm guessing the callback to Girder comes over the same (likely high) port already opened by Girder. Thanks for Zach's help. John. On 02/03/2017 11:24 AM, John Roberts wrote: > > I believe that's' what I would see if I were running Girder > directly on the host or in a container that published that port. > > However, we currently have our Girder running through a reverse > proxy. The settings in girder.local.cfg as described in the > documentation seem to allow you to drop that root directory down a level. > > So with appropriate changes to girder.local.cfg you could have > something like > > http://server.address:port/GIRDERPROD/api/v1/oauth/github/callback > > Before we started testing oauth, we were using an Apache reverse proxy > setup to handle some of our access control: > > The Apache reverse proxy is used to limit access to certain external > subnets. I could probably accomplish this with iptable rules, but the > above approach is more consistent with the other Docker web services > also run on the same server using similar Apache reverse proxy setups. > > There are a variety of ways to run the dual Girder setup above with > Docker, including variations on whether the Docker container is > publishing ports (in which case the container will answer the outside > world directly on that port, with no intervening Apache) or not > publishing ports (in which case, Apache or some other method must > redirect the incoming traffic to the appropriate container). > > Assume I eliminate Docker from the discussion, and we just talk about > running two Girders on the same server on two separate ports. Is it > possible to set up the reverse proxy as described in the documentation > AND have oauth at the same time? I mean Girder's own reverse proxy > settings as described in the documentation. > > Thanks, > John. > > > > On 02/02/2017 06:32 PM, Zach Mullen wrote: >> That is, what is the value listed in your OAuth application's >> *Authorization callback URL* field, which looks like the below >> screenshot: >> >> Inline image 1 >> >> On Thu, Feb 2, 2017 at 8:24 PM, Zach Mullen > > wrote: >> >> To diagnose that issue, how is your redirect URI configured in >> github? And what is the value of the "state" parameter when you >> are sent to the github login page? >> >> >> >> On Thu, Feb 2, 2017 at 8:07 PM, John Roberts >> > wrote: >> >> Thanks for the reply, Zach. >> >> Playing around with a test girder/mongo pair, I managed to >> edit the girder.local.cfg file without crashing the >> container. Subsequently, it seems to recognize correctly >> that it is not at the top level of the web server. >> >> This is all related to our effort to run oauth through >> github. Previously, without the proper proxy settings in >> girder.local.cfg, the callback address generated in the oath >> plugin setting did not include the proxy. Now it does. >> >> Unfortunately, authorization through github still generates a >> redirect_uri_mismatch error. >> >> My best guess at the moment is that this is somehow related >> to our proxying girder itself through https on the Apache >> server. I mean, we have a reverse proxy setup on Apache to >> route incoming requests for https://some.address.com/girder >> to our internal >> http://some.docker.network:8080/girder >> . >> >> Somewhere in the proxying, I'm thinking the URL might be >> modified and either Girder or Github says it doesn't match >> what's expected. The message indicates that it's Github who >> is rejecting the callback: >> >> { >> "message": "Provider returned error: >> 'redirect_uri_mismatch'.", >> "type": "rest" >> } >> >> >> John. >> >> >> On 02/02/2017 05:06 PM, Zach Mullen wrote: >>> Hi John, >>> >>> You're running an old version of Girder (1.5.2), are you >>> able to upgrade to the latest version and see if that fixes >>> the issue? >>> >>> Thanks, >>> >>> Zach Mullen >>> Kitware, Inc. >>> 919-869-8858 >>> >>> On Thu, Feb 2, 2017 at 6:25 PM, John Roberts >>> >> > wrote: >>> >>> I'd like to set up a proxied Girder as detailed in >>> the manual here >>> . >>> We need to configure girder so that it understands it is >>> not working from the root directory of the Apache server >>> but one directory down (/girder). Unlike the situation >>> in the documentation, I'm working with the Docker girder. >>> >>> If I rather blindly edit the girder.local.cfg file >>> within the running container, the container crashes as >>> soon as I save the file. This may have been due to a >>> typo, but I have a further question. >>> >>> Assuming I did update girder.local.cfg properly as >>> indicated, the instructions then say to rebuild Girder >>> using "npm install". It's my impression that this will >>> likely crash the girder container itself, since the >>> current girder process will likely be terminated while >>> the new build is being compiled with npm. Would that be >>> a correct assumption? >>> >>> The question is how to configure Girder within the >>> container to invoke the proxy and move the address space >>> one step down from root at / to a proxied address >>> /girder. A more general question might be how to update >>> girder.local.cfg when running Girder with Docker. >>> >>> I tried running the Docker girder:girder image and >>> entering by way of an alternative entry-point, >>> >>> docker run -it --entrypoint=/bin/bash girder/girder -s >>> >>> This lets me in and I can edit the girder.local.cfg >>> file. However, "npm install" then appears to fail with >>> the warning "cannot run in wd girder at 1.5.2 >>> grunt init && grunt >>> (wd=/girder)". Since I don't know what "npm install" >>> was supposed to accomplish, I'm not sure where to look >>> to see if it succeeded despite the warning. >>> >>> Thanks for suggestions, >>> John. >>> >>> >>> >>> >>> _______________________________________________ >>> Girder-users mailing list >>> Girder-users at public.kitware.com >>> >>> http://public.kitware.com/mailman/listinfo/girder-users >>> >>> >>> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 21347 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 47687 bytes Desc: not available URL: From zach.mullen at kitware.com Fri Feb 3 17:47:13 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Fri, 3 Feb 2017 17:47:13 -0500 Subject: [Girder-users] Setting up proxied Girder with Docker Container In-Reply-To: <7223d20b-b809-9704-9845-87753473aa9e@hsc.utah.edu> References: <6de2a47a-8cbd-4397-101d-a7118be2af0c@hsc.utah.edu> <9ff47838-a881-d7bd-6f0e-b72ece65acc0@hsc.utah.edu> <2e891e4b-7e84-8b7f-abb6-c4f2d3f05a73@hsc.utah.edu> <7223d20b-b809-9704-9845-87753473aa9e@hsc.utah.edu> Message-ID: On Fri, Feb 3, 2017 at 5:37 PM, John Roberts wrote: > I managed to get this working, both having a Docker containerized Girder > be self-aware of its own proxying (edits to girder.local.cfg) and for Oauth > to work over the various reverse proxy hops. > Good news! > Within the girder container, I edit the girder.local.config file to add > lines controlling its self-awareness about proxying. > > - [global] > - tools.proxy.on: True > - tools.proxy.base: "https://outward.facing.edu/girderdev" > > - tools.proxy.local: "" > - [server] > - api_root: "/girderdev/api/v1" > - static_root: "/girderdev/static" > > The tools.proxy.base/local settings may be overkill for our Apache front > end, but things work now, so I left those in. The above changes show up in > the Oauth settings page for the github plugin that I wanted to use. The > callback address previously did not include the proxying girderdev portion > of the address, because Girder had not been aware of its own proxying > (which we manage via Apache). With the changes to girder.local.cfg, the > address generated in the plugin settings was correct. > An alternative to having tools.proxy.base set explicitly is to have your Apache proxy server set the X-Forwarded-Host header during proxying, but the behavior is equivalent either way. > The last hurdle was on Github itself. Girder is apparently pulling down > both the account name and email address from Github. Only the email is > matched against existing user emails, but the Github account name becomes > the users's name on Girder for the duration of the login when an email > match is found. In my case, my test github account had no "name" per se > and the blank caused an error in Girder. > The behavior in the latest version of Girder (2.1.0) should allow for accounts without names; in that case, we just use the github login as both the first and last name. > One other note: the Oauth through Github works even though our top server, > https://outward.facing.edu, is not itself visible outside our > institution's firewall. I'm assuming this works because it is Girder that > is invoking the connection to Github as an oath client. I'm guessing the > callback to Girder comes over the same (likely high) port already opened by > Girder. > The OAuth provider (in your case Github) doesn't contact the girder server at all, it simply redirects the user's browser back to the callback URL that was specified once the user authenticates and authorizes the application. Glad you got it working. -Zach > Thanks for Zach's help. > > John. > > On 02/03/2017 11:24 AM, John Roberts wrote: > > I believe that's' what I would see if I were running Girder directly > on the host or in a container that published that port. > > However, we currently have our Girder running through a reverse > proxy. The settings in girder.local.cfg as described in the documentation > seem to allow you to drop that root directory down a level. > > So with appropriate changes to girder.local.cfg you could have > something like > > http://server.address:port/GIRDERPROD/api/v1/oauth/github/callback > > Before we started testing oauth, we were using an Apache reverse proxy > setup to handle some of our access control: > > The Apache reverse proxy is used to limit access to certain external > subnets. I could probably accomplish this with iptable rules, but the > above approach is more consistent with the other Docker web services also > run on the same server using similar Apache reverse proxy setups. > > There are a variety of ways to run the dual Girder setup above with > Docker, including variations on whether the Docker container is publishing > ports (in which case the container will answer the outside world directly > on that port, with no intervening Apache) or not publishing ports (in which > case, Apache or some other method must redirect the incoming traffic to the > appropriate container). > > Assume I eliminate Docker from the discussion, and we just talk about > running two Girders on the same server on two separate ports. Is it > possible to set up the reverse proxy as described in the documentation AND > have oauth at the same time? I mean Girder's own reverse proxy settings as > described in the documentation. > > Thanks, > John. > > > > On 02/02/2017 06:32 PM, Zach Mullen wrote: > > That is, what is the value listed in your OAuth application's *Authorization > callback URL* field, which looks like the below screenshot: > > [image: Inline image 1] > > On Thu, Feb 2, 2017 at 8:24 PM, Zach Mullen > wrote: > >> To diagnose that issue, how is your redirect URI configured in github? >> And what is the value of the "state" parameter when you are sent to the >> github login page? >> >> >> >> On Thu, Feb 2, 2017 at 8:07 PM, John Roberts >> wrote: >> >>> Thanks for the reply, Zach. >>> >>> Playing around with a test girder/mongo pair, I managed to edit the >>> girder.local.cfg file without crashing the container. Subsequently, it >>> seems to recognize correctly that it is not at the top level of the web >>> server. >>> >>> This is all related to our effort to run oauth through github. >>> Previously, without the proper proxy settings in girder.local.cfg, the >>> callback address generated in the oath plugin setting did not include the >>> proxy. Now it does. >>> >>> Unfortunately, authorization through github still generates a >>> redirect_uri_mismatch error. >>> >>> My best guess at the moment is that this is somehow related to our >>> proxying girder itself through https on the Apache server. I mean, we have >>> a reverse proxy setup on Apache to route incoming requests for >>> https://some.address.com/girder to our internal >>> http://some.docker.network:8080/girder. >>> >>> Somewhere in the proxying, I'm thinking the URL might be modified and >>> either Girder or Github says it doesn't match what's expected. The message >>> indicates that it's Github who is rejecting the callback: >>> >>> { >>> "message": "Provider returned error: 'redirect_uri_mismatch'.", >>> "type": "rest" >>> } >>> >>> >>> John. >>> >>> On 02/02/2017 05:06 PM, Zach Mullen wrote: >>> >>> Hi John, >>> >>> You're running an old version of Girder (1.5.2), are you able to upgrade >>> to the latest version and see if that fixes the issue? >>> >>> Thanks, >>> >>> Zach Mullen >>> Kitware, Inc. >>> 919-869-8858 <%28919%29%20869-8858> >>> >>> On Thu, Feb 2, 2017 at 6:25 PM, John Roberts >>> wrote: >>> >>>> I'd like to set up a proxied Girder as detailed in the manual here >>>> . >>>> We need to configure girder so that it understands it is not working from >>>> the root directory of the Apache server but one directory down (/girder). >>>> Unlike the situation in the documentation, I'm working with the Docker >>>> girder. >>>> >>>> If I rather blindly edit the girder.local.cfg file within the >>>> running container, the container crashes as soon as I save the file. This >>>> may have been due to a typo, but I have a further question. >>>> >>>> Assuming I did update girder.local.cfg properly as indicated, the >>>> instructions then say to rebuild Girder using "npm install". It's my >>>> impression that this will likely crash the girder container itself, since >>>> the current girder process will likely be terminated while the new build is >>>> being compiled with npm. Would that be a correct assumption? >>>> >>>> The question is how to configure Girder within the container to >>>> invoke the proxy and move the address space one step down from root at / to >>>> a proxied address /girder. A more general question might be how to update >>>> girder.local.cfg when running Girder with Docker. >>>> >>>> I tried running the Docker girder:girder image and entering by way >>>> of an alternative entry-point, >>>> >>>> docker run -it --entrypoint=/bin/bash girder/girder -s >>>> >>>> This lets me in and I can edit the girder.local.cfg file. However, >>>> "npm install" then appears to fail with the warning "cannot run in wd >>>> girder at 1.5.2 grunt init && grunt (wd=/girder)". Since I don't know >>>> what "npm install" was supposed to accomplish, I'm not sure where to look >>>> to see if it succeeded despite the warning. >>>> >>>> Thanks for suggestions, >>>> John. >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Girder-users mailing list >>>> Girder-users at public.kitware.com >>>> http://public.kitware.com/mailman/listinfo/girder-users >>>> >>>> >>> >>> >> > > > > _______________________________________________ > Girder-users mailing list > Girder-users at public.kitware.com > http://public.kitware.com/mailman/listinfo/girder-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 47687 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 21347 bytes Desc: not available URL: From zach.mullen at kitware.com Fri Feb 3 18:21:28 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Fri, 3 Feb 2017 18:21:28 -0500 Subject: [Girder-users] Running girder natively with https In-Reply-To: <18687f01-9e03-468b-c55e-e4b5c2a9bc4d@hsc.utah.edu> References: <18687f01-9e03-468b-c55e-e4b5c2a9bc4d@hsc.utah.edu> Message-ID: Hi John, Added girder-users in case this question is useful for posterity for others in the Girder community :) Responses are inline On Fri, Feb 3, 2017 at 5:53 PM, John Roberts wrote: > Zach, > > I'm working with Brian Chapman here at the Utah lab. I believe he may > have asked at some point about running Girder under https. I was looking > for documentation for how to get this working. > > Right now, we use the Apache to reverse proxy and provide https "to > the doorstep" before Apache hands off to the http address of our actual > Docker Girder. I'd like to encrypt that last step as well so that nothing > was ever in the clear, even on the host machine running Apache and the > Docker containers. > You can control these cherrypy settings via girder.local.cfg, as in my example below, you'd just have to point it to your own cert and private key. [global] server.socket_host = "0.0.0.0" server.socket_port = 4430 server.thread_pool = 100 server.ssl_module = 'builtin' server.ssl_certificate = "/Users/zach/dev/girder/cert.pem" server.ssl_private_key = "/Users/zach/dev/girder/privkey.pem" More docs about these settings are here[1], including instructions for other SSL implementations besides cherrypy's builtin one. Full disclosure, I've never deployed it in such a way myself, I always just rely on the proxy to handle the SSL and keep the Girder server behind the firewall, so you might run into issues we have not encountered before. I don't see instructions per se in the manual for making sure things > run under https. Is that because the server set up is assumed to be a > separate issue? That is, if you're going to run Girder, you should already > have your server ready to go https regardless. > It would probably be useful to add some cookbook deployment examples, though we've avoided them until now since we don't want to be too prescriptive about how this is deployed since there are lots of various proxy servers, deployment architectures, security requirements, etc. > I guess I need to figure out how to make that happen for the Docker > version which appears to be running, what, nginx internally? > The girder docker image (at least the one I know about that is built from the Girder repository) just serves directly out of cherrypy, with no proxy server in front of it. > Thanks, > John. > > [1] http://docs.cherrypy.org/en/latest/deploy.html#ssl-support Thanks, -Zach -------------- next part -------------- An HTML attachment was scrubbed... URL: From John.Roberts at hsc.utah.edu Fri Feb 3 19:36:40 2017 From: John.Roberts at hsc.utah.edu (John Roberts) Date: Fri, 3 Feb 2017 17:36:40 -0700 Subject: [Girder-users] OAuth and the Girder API Message-ID: <564a5e43-4409-4f3a-dcf2-95fe21ac71d8@hsc.utah.edu> Now that we have OAuth running, a number of questions came up: 1. Can we force both web and API authentication to exclusively use OAuth and disallow the standard user/password login page? Essentially, we'd be offloading authentication and user name/password management entirely to the external OAuth server. 2. Does OAuth work with the Girder API? * Would we need to modify girder_client, for example? * What if 2-factor is turned on for the OAuth server? My current read of the girder_client is that it will not handle OAuth as a means of authentication. I haven't figured out whether the underlying Girder API has a hook for directly invoking the Oauth authentication process. My colleague envisions users interacting with data on Girder using API calls from a Jupyter hub client. Perhaps we could manage the authentication process using apikeys in that case, if OAuth isn't integrated in the Girder API authentication methods. Thanks, John. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zach.mullen at kitware.com Fri Feb 3 19:48:04 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Fri, 3 Feb 2017 19:48:04 -0500 Subject: [Girder-users] OAuth and the Girder API In-Reply-To: <564a5e43-4409-4f3a-dcf2-95fe21ac71d8@hsc.utah.edu> References: <564a5e43-4409-4f3a-dcf2-95fe21ac71d8@hsc.utah.edu> Message-ID: Hi John, Responded inline On Fri, Feb 3, 2017 at 7:36 PM, John Roberts wrote: > Now that we have OAuth running, a number of questions came up: > > 1. Can we force both web and API authentication to exclusively use > OAuth and disallow the standard user/password login page? Essentially, > we'd be offloading authentication and user name/password management > entirely to the external OAuth server. > > This feature isn't supported at the moment, but it would be straightforward to write a plugin to do this, or enhance the existing OAuth plugin to allow it. > > 1. > 2. Does OAuth work with the Girder API? > - Would we need to modify girder_client, for example? > - What if 2-factor is turned on for the OAuth server? > > Yes, all interaction with the Girder server actually goes through the same HTTP API, and all authenticated API requests use the same secure token authentication system, regardless of whether the user logged in via OAuth or a password managed by Girder. If multi-factor authentication is required on the provider side during the OAuth workflow, I don't think that should be a problem since I imagine that would be invisible to Girder. > > > My current read of the girder_client is that it will not handle OAuth as a > means of authentication. I haven't figured out whether the underlying > Girder API has a hook for directly invoking the Oauth authentication > process. > Since OAuth is built around interactive user authentication in a web browser, using it in the girder_client is not possible; instead, we recommend using the API keys exposed by Girder to allow OAuth users to make authenticated requests via girder_client. > My colleague envisions users interacting with data on Girder using API > calls from a Jupyter hub client. Perhaps we could manage the > authentication process using apikeys in that case, if OAuth isn't > integrated in the Girder API authentication methods. > API keys would work in this case. Once you get the authentication token from either a normal password login or via an API key, you can make authenticated API requests in a uniform way. I'm curious what you guys are doing regarding Jupyter integration; we actually have some other efforts ongoing at the moment to implement Jupyter + Girder integration, there may be some opportunity to collaborate here. Could you describe what you're doing in that regard? Thanks, Zach > Thanks, > John. > > _______________________________________________ > Girder-users mailing list > Girder-users at public.kitware.com > http://public.kitware.com/mailman/listinfo/girder-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Brian.Chapman at utah.edu Mon Feb 6 16:46:36 2017 From: Brian.Chapman at utah.edu (Brian E Chapman) Date: Mon, 6 Feb 2017 21:46:36 +0000 Subject: [Girder-users] Data Migration Message-ID: We have two instances of Girder and we would like to migrate data from instance 1 to instance 2. I?m curious if anyone has addressed this previously and if there is a girder-specific approach or if you just do it via MongoDB. Brian Brian E Chapman, Ph.D Associate Professor Department of Radiology and Imaging Sciences University of Utah -------------- next part -------------- An HTML attachment was scrubbed... URL: From zach.mullen at kitware.com Mon Feb 6 16:55:41 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Mon, 6 Feb 2017 16:55:41 -0500 Subject: [Girder-users] Data Migration In-Reply-To: References: Message-ID: Hi Brian, The easiest way to do this would be to just mongoexport and mongoimport. The other parts depend on what assetstore types you are using. If you're using S3 assetstores, you don't need to do anything else, BUT it's important to mention that there could be problems if you are trying to simultaneously have both Girder instances use the same (writable) S3 assetstore, since both instances would try to manage the lifecycle of the data independently and could clobber one another. That would only be a problem for already existing files, not new ones uploaded after the split. GridFS assetstores have the same caveat above if shared, otherwise you could mongoexport/mongoimport them in order to copy them. If you're using filesystem assetstores, you'll need to also recursively copy the root of the filesystem assetstore to the new instance. Hope this answers your question. Zach Mullen Kitware, Inc. 919-869-8858 On Mon, Feb 6, 2017 at 4:46 PM, Brian E Chapman wrote: > We have two instances of Girder and we would like to migrate data from > instance 1 to instance 2. I?m curious if anyone has addressed this > previously and if there is a girder-specific approach or if you just do it > via MongoDB. > > Brian > > Brian E Chapman, Ph.D > Associate Professor > Department of Radiology and Imaging Sciences > University of Utah > > > _______________________________________________ > Girder-users mailing list > Girder-users at public.kitware.com > http://public.kitware.com/mailman/listinfo/girder-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.wright at kitware.com Tue Feb 28 12:25:22 2017 From: tristan.wright at kitware.com (Tristan Wright) Date: Tue, 28 Feb 2017 10:25:22 -0700 Subject: [Girder-users] Issue with Adding and Removing Users to/from Groups Message-ID: Hi all, I believe I'm running into a concurrency issue with adding multiple users to a group with the "force=true" option on. Video of network log: https://drive.google.com/ open?id=0BxhQv23DV8jmS3BxRHlTUFBscU0 Video summary: - Page loads, fetches group user list, there is 1 user. - Make two requests adding two users Sarah and Hannah - group/access response is a list of two users, the original user and Hannah I expect there to be 3 users in the group/access response. Refreshing the page has no effect, it looks like something is colliding and not being changed properly? The call is made with chained promises. Perhaps 1/5 times the changes are correctly made to the access list. I've tried to replicate the problem with curl calls unsuccessfully, the final call has the right amount of users. I see the same behavior when trying to remove multiple users from a group. Is this a girder bug or am I making the invitation requests improperly? -- - Tristan Wright *R&D Engineer* *Kitware Inc.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.wright at kitware.com Tue Feb 28 12:51:50 2017 From: tristan.wright at kitware.com (Tristan Wright) Date: Tue, 28 Feb 2017 10:51:50 -0700 Subject: [Girder-users] Issue with Adding and Removing Users to/from Groups In-Reply-To: References: Message-ID: Followup: I *am *able to reproduce with the curl commands with the attached script. The script uses env variables GIRDER_TOKEN, USER1, USER2 (user Id's), and GROUP_ID. On Tue, Feb 28, 2017 at 9:32 AM, Tristan Wright wrote: > Hi all, > I believe I'm running into a concurrency issue with adding multiple users > to a group with the "force=true" option on. > > Video of network log: https://drive.google.com/ > open?id=0BxhQv23DV8jmS3BxRHlTUFBscU0 > Video summary: > > - Page loads, fetches group user list, there is 1 user. > - Make two requests adding two users Sarah and Hannah > - group/access response is a list of two users, the original user and > Hannah > > I expect there to be 3 users in the group/access response. Refreshing the > page has no effect, it looks like something is colliding and not being > changed properly? > > The call is made with chained promises. Perhaps 1/5 times the changes are > correctly made to the access list. I've tried to replicate the problem with > curl calls unsuccessfully, the final call has the right amount of users. I > see the same behavior when trying to remove multiple users from a group. > > Is this a girder bug or am I making the invitation requests improperly? > > -- > - Tristan Wright > > *R&D Engineer* > *Kitware Inc.* > -- - Tristan Wright *R&D Engineer* *Kitware Inc.* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: user_test.sh Type: application/x-sh Size: 727 bytes Desc: not available URL: From zach.mullen at kitware.com Tue Feb 28 13:45:32 2017 From: zach.mullen at kitware.com (Zach Mullen) Date: Tue, 28 Feb 2017 13:45:32 -0500 Subject: [Girder-users] Issue with Adding and Removing Users to/from Groups In-Reply-To: References: Message-ID: Hi Tristan, This is expected, but is also a bug -- this endpoint currently saves the document, so simultaneous calls to the same group document could clobber one another. The solution to this would be to use atomic update calls in the backend implementation to update the access list. I've opened a ticket for this: https://github.com/girder/girder/issues/1817 Thanks for reporting this. Zach Mullen Kitware, Inc. 919-869-8858 On Tue, Feb 28, 2017 at 12:51 PM, Tristan Wright wrote: > Followup: I *am *able to reproduce with the curl commands with the > attached script. The script uses env variables GIRDER_TOKEN, USER1, USER2 > (user Id's), and GROUP_ID. > > On Tue, Feb 28, 2017 at 9:32 AM, Tristan Wright < > tristan.wright at kitware.com> wrote: > >> Hi all, >> I believe I'm running into a concurrency issue with adding multiple users >> to a group with the "force=true" option on. >> >> Video of network log: https://drive.google.com/ >> open?id=0BxhQv23DV8jmS3BxRHlTUFBscU0 >> Video summary: >> >> - Page loads, fetches group user list, there is 1 user. >> - Make two requests adding two users Sarah and Hannah >> - group/access response is a list of two users, the original user and >> Hannah >> >> I expect there to be 3 users in the group/access response. Refreshing the >> page has no effect, it looks like something is colliding and not being >> changed properly? >> >> The call is made with chained promises. Perhaps 1/5 times the changes are >> correctly made to the access list. I've tried to replicate the problem with >> curl calls unsuccessfully, the final call has the right amount of users. I >> see the same behavior when trying to remove multiple users from a group. >> >> Is this a girder bug or am I making the invitation requests improperly? >> >> -- >> - Tristan Wright >> >> *R&D Engineer* >> *Kitware Inc.* >> > > > > -- > - Tristan Wright > > *R&D Engineer* > *Kitware Inc.* > > _______________________________________________ > Girder-users mailing list > Girder-users at public.kitware.com > http://public.kitware.com/mailman/listinfo/girder-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: