General discussion on Distributed IGSTK
From IGSTK
IGSTK Distributed Computing Motivation, Requirements, and Design December 10, 2008
Contents |
General Discussion on Distributed Computing
Of course it is ridiculous to try and expound on all things distributed in a Wiki entry, but I did want to lay out some of the motivating factors for distributed computing and some general design considerations (things the single-host programmer doesn't have to think about) to lay a context for an IGSTK-specific discussion.
Motivating Factors for Distributed Computing
In a general sense there are many motivating cases for taking a monolithic, single-hosted application and factoring it into a distributed application. This section discusses (at an all too coarse level) general motivating principles for distribution, and briefly comments on how they do/not apply to IGSTK:
Motivating Factors:
- Performance: Some applications need to be distributed to achieve better performance. Of course, "performance" can have many connotations. Some popular measures of performance:
- Response Time (presumably to the end user, but in general of a "transaction")
- Throughput: Transactions or operations handled over a limited resource in a given time period
- Scalability - Often mistakenly consider in the same breath as performance, scalability hasmore to do with the ability to add (or remove) resources and have the application's QoS decline gracefully.
- Fault tolerance - The ability for a service or component to fail without an unacceptable loss in QoS.
- Integration - Like Performance, this has many connotations
- Languages/Run-time platforms: Some components or services may only be available on certain platforms.
- Special purpose devices: Similar to the previous bullet, except in this case the service or component to be integrated is a combination of hardware and software.
- External systems: Integration with a 3rd party system may also be required. The difference in this scenario from (a) and (b) is in direction of responsibility; in those cases "your" framework is trying to include new functionality by integrating a new component or service. In this case there is another system that is trying to integrate you as new functionality into it. A motivating scenario for IGSTK is below.
- Extensibility - Most of the motivating argument for Extensibility is contained in Integration above, but it is called out here for its additional benefits of forcing an architecture to have formal (or contractual) interfaces, data types, request/response protocols, error handling, services, and thread-safety. Often a single-host application or framework can get away with being ad hoc, as we have seen within IGSTK.
Design Considerations for Distributed Computing
Again, very general concerns to address when factoring a single-host application to run in a distributed environment. Most certainly not an exhaustive list.
- Data Marshalling - Objects residing in one environment may be transported across the wire to a new environment (pass by value) which may support varying primitive types. Preserving data and type safety becomes important.
- Concurrency - Transactional behaviors may have concurrency issues tied to statefulness.
- Remote References - Basically, dealing with pointers to remote objects.
- Latency - Especially an issue in real-time or quasi real-time systems, as IGSTK may be considered. A corollary is dealing with absolute measures of time (RTC).
- Security - There is a whole raft of authentication and authorization issues, but I won't discuss these here as we'll assume IGSTK is on a closed network. But if we ever consider applications like telemedicine we may have to revisit this.
- Error-handling - Both in terms of what errors can happen (e.g. network disruption) and how one reports them and takes action on them.
- Synchronous/Asynchronous Behavior - self-explanatory, but ties in to Concurrency
- Guaranteed Delivery - a QoS attribute that is often mapped to the transport layer
- Componentization/Service-ization - Separating concerns, decoupling, etc.
Motivating Scenarios for Distributing IGSTK
There are 2 categories motivating scenarios for distributing an IGSTK application. The first category deals with different flavors of integration for various motivating business cases; the second category is to enhance the functional capabilities of an IGSTK-based suite of tools.
- Integration Cases
Case 1: Ability to integrate with specialized devices
In this scenario (see Figure 1), a specialized device, such as a robotic needle driver, may come "pre-packaged" on a computing platform of its own. Integration constraints may exist in the sense that the device is coupled to that platform. The current tracker examples are somewhat like this (not exactly, since I do not believe that the tracker *has* to run on a distinct computing node).
Figure: Case 1
Case 2: Service-oriented Integration
In this scenario (Figure below), one or more computationally expensive services are mapped to a dedicated computational node. This is as much a performance motivator as it is an integration one. There may be expensive algorithms (3D volume renderer?) that cannot run except on computational devices with a certain level of resources (CPU, memory, floating point processor), and even then may create a burden on the rest of the system. A variation on this theme are data-oriented services such as controlled I/O for DICOM images.
Figure 2. Case 2
The emphasis here is on "service", meaning the implementation is not tied to a particular IGSTK component or application at any given time; services typically just respond to request for the service and then "disconnect."
Case 3: Integration with External Systems
Probably not on IGSTK's radar yet, but this scenario (Figure 3) deals with exposing IGSTK to other systems, perhaps for bi-directional functions. The example I can think of is integration with a patient medical information system, both for reporting the results of an IGSTK application and for allowing IGSTK applications to query the system dynamically.
Figure 3. Case 3
Case 4: Enhance the functional capabilities of IGSTK
The main enhancement I can see from a distributed IGSTK would be the ability to support multiple applications from a single shared set of runtime IGSTK components. In particular, consider Figure 4 below.
Figure 4. Case 4
In this scenario, one runtime environment would hold a set of Spatial Objects that represent a canonical view of a surgical scene. Multiple View layer components would be instantiated in the application runtimes to allow different operators to track and perform different functions. This case example is probably the biggest motivator I see, and also indicates that the distribution of IGSTK components is not arbitrary, but has to happen across layer boundaries. To put in more architecture-like terms, three concerns could be identified: "Application/View", "Spatial Representation", and "Tracking", and distribution would only be allowed along these layers.
A final note: This discussion has an implicit assumption that "distribution" means distribution of IGSTK components or services or both. There are other forms of distribution patterns that could be considered, such as 1) distributed a single component or service implementation, and 2) distributing "on the glass" representation. The former would be entirely within the boundaries of a given IGSTK component and would require that component to maintain an expected QoS level similar to what is provided today in the single host model; the latter is basically using a single large powerful computer to run all functions (including multiple applications) and just remoting the display to different operators. This scenario is not entirely impractical, though it forces assumptions about the specific nature of the computing devices in an operating theater.
Design Considerations for IGSTK
There are a bunch that could come up, a starting list:
- Use of smart pointers: I am assuming no equivalent of remote smart pointers
- Error handling
- Preserving state machine protocols: the whole Request/Event in 1 call stack thing
- Multithreading and Concurrency: related to state machines as well
- Deployment issues: What libraries have to be on what nodes; distributing via CMake?
- Data mobility: I assume DICOM images can get quite large
- Multi-runtime environments: different nodes with different OS and runtime environments; perhaps even writing applications in other languages.
