Draft Roadmap Contribution: Workflow - Business Process
Name
Song Wu
Organisation
HUST
Country
China
Submission date
2008-05-20
General comments: Page 31, Section 7.2, Para 3, Line 6-10
It seems that the section of grid workflow interoperability is somewhat abstract and not practical. My suggestion is: (1) we would better fix the grid workflow standard and specifiction rather than the workflow system. Only if based on the standard specification, such as BPEL, XPDL, DAG, the interoperability can be undergone. (2) Because the workflow engine is the most important component for a grid workflow system, we should develop our grid workflow engines based on certain standard specification and let the source code open.
Name
Song Wu
Organisation
HUST
Country
China
Submission date
2008-05-20
General comments: Page 32, Section 7.3, Para 2, Line 4
“BPEL itself lacks certain structural elements like loops.” I do not think so. BPEL does support “loop”.
Draft Roadmap Contribution: Virtual Organisation
Name
Liu Fang-ai
Organisation
ShanDong Normal University, School of Information and Engineering
Country
China
Submission date
2008-06-03
General comments: The roadmap evaluates the current researches on VO exactly and gives the further research area clearly. In my own view, VO research should pay more attention on application. In one word,VO should be application-driven.
New Ideas: Now grid study is mainly focused on theoretical researches and lacks successful applications. So, I think that one branch in the roadmap, in three, five and 10 years, is increasing grid practical applications, which means that the grid researches should be application-driven. Based on this idea, virtual organizations (VO) should be chosen as a breakthrough point and be considered significantly.
As an application-driven research, the first problem is to select an application background, in which we can conduct researches on VO organization, grid’ architecture, grid’s platform and so on. We here propose a resource sharing application, which can be considered as an application background for grid study.
Currently the Kong Zi College is becoming more and more popular in the world. These collages are setup in different countries, but give similar courses on Chinese culture. Each collage has its own digital teaching resources, like teaching video, teaching software, electric books and so on. The copyrights of these resources belong to different colleges. If we can share these materials, we can save a lot of teaching effort. So we need to setup a resource grid to share these teaching resources.
Under this background we should organize a virtual organization(VO) and explore its user/role management, resource management, security management and strategies to produce resources. Based on this background, we should also need to explore grid architectures, resource discovery algorithms, resource replica strategy, scalability, QoS guarantee and so on. In one word, we should do research on the problems we are met when the resource grid is constructed. In a long run, maybe we can reach a pattern which can be used as a routine operation in constructing such grid.
As to our work, we have done some researches on education resource grid under the support of NSFC. The background is to explore the key problems in constructing a middle school teaching resource grid.
Name
Cai Yang
Organisation
Ministry of Water Resources
Country
China
Submission date
2008-06-05
Recommendations for EU and China: The certification of VO should be classified. and some VOs for test may be provided. It is important to support LAN applications which are tight coupling.
Name
Bingheng YAN
Organisation
Sino-German Joint Software Institute, BUAA
Country
P.R.China
Submission date
2008-06-06
State of the Art and Limitations: Building a VO is cockamamie now as well as maintaining a VO. Of course, one reason is lack of support from low-layer infrastructures. But the most important is lack characters of automatically constructing, self adapting (and other self features) and flexibility. We need a VO system which can deal with a host joining (or leaving) semi-automatically (or automatically). To some degree, VO becomes obstacle for resources joining the Grid because of lack these characters in VO system. Maybe system-level support for VO can solve this problem in the future.
New Ideas: New ideas listed in the roadmap can not reflect the long-term development of VO. Some extra new ideas list as following.
 VO support for mobile computing devices. With the number and capability of mobile devices increasing, VO should support mobile devices joining it automatically to make use of these computing resources. A negotiating mechanism between VO and mobile devices is indispensable. By accepting more devices, Grid can harness existing resources as much as possible. For those credible devices, grid can schedule all jobs to those resources. For others, grid can forward these no security requiring jobs to those resources.
 Current VO architecture restricts the flexibility of VO. The interoperability between VOs is a non-trivial task because of existing VO’s structure. By the support of low-level infrastructures, it is possible to build some atom VOs for some basic applications. Other VOs can be automatically constructed by combining atom VOs. VO interoperability and Resources sharing will benefit from this new architecture.
Draft Roadmap Contribution: New Programming Paradigms
Name
Song Wu
Organisation
HUST
Country
China
Submission date
2008-05-20
General comments: The new proposed ideas are all very good points for research, like programming paradigm, Grid architecture, Grid management ..etc. But how about some new concepts: Virtual Workspace, Cloud Computing and Green Computing…. Moreover, more attention should be paid for different standards, different user's requirements and different policies.
Recommendations for EU and China: It would be better to include some new fields like cloud computing and virtual machine technology.
Name
Aristotle Chatziioannou
Organisation
NHRF/IBRB
Country
Greece
Submission date
2008-05-29
General comments: A versatile open source Grid microarray analysis and interpretation pipeline, starting from raw data and delivering significantly affected pathways or more generally biological processes. To our knowledge the first such initiative regarding gene expression microarrays in the European space.
Vision: The development of a complete functional genomics portal, which processes in parallel similar microarray experiments to propose a set of significantly altered pathways or more generally regulatory procedures whose pertaining genes will be used for iterative rounds of common cis regulatory analyses.
Moreover the selected pathways could be simulated regarding the dynamics of their function in order to investigate critical parts for modification (therapeutic targets, or genetic modifications concerning microorganisms of biotechnological interest)
State of the Art and Limitations: Ther have been seminal efforts regarding appications for Tissue Microarray processing in the Grid (look TMAinspect, an EGEE Framework for Tissue MicroArray Image Handling, authors: Antonella Galizia, Federica Viti, Alessandro Orro, Daniele D’Agostino, Ivan Merelli, Luciano Milanesi, Andrea Clematis) but concerning gene expression microarrays a field foundamentally different from the scientific point of view with a very small degree of overlap in terms of the computational methods required, our effort represents a pioneering one at least at a European level.
Concerning the limitations most of them have to do with the observed shortage of mathematical libraries demanded for microarray analysis concerning the grid nodes, as well as the practical difficulty to flexibly and quickly upgrade grid nodes with newer versions of the reqired tools when some versions of these are installed.
This practical problem convinced us to decide to build our application by using open source software (Octave Forge the freeware analogue of MATLAB plus Scientific Linux 3.75.* which is operating satisfactorily with Octave) and by completing Octave's libraries regarding the required algorithms through our own programming effort. In this way we evaded the danger of aggravating costs, hazardous for the future function of the application, as regards both the development and the deployment phase.
New Ideas: The expansion of the platform towards translational biological research that is to try to span the gap between the genotype and the various types of phenotypes (cellular, tissue, medical) through the development of in-silico analogues of various types of biological problems at different scales and time ranges, in order to move towards the direction of initiating applications which will enable individualized therapeutic strategies taking into account the biological fingerprint of a person and determining the ways and the extent of the interplay between its genetic basis and its environment (nutrition, lifestyle habits, conditions dwelling etc.)
Recommendations for EU and China: The whole area of Biomedical sciences due to their sheer complexity (besides biological imaging which is already technologically ripe even regarding to Grid projects) is extremely amenable for Grid implementations.
Name
Qiang Yue
Organisation
Computing Technology, Chinese Academy of Sciences
Country
China
Submission date
2008-06-05
General comments: We consider a complementary mechanism of service debugging, which can work on two layers: message-based and source level. Grid Service inspection on one layer can use its own debugging facilities, and change to the other at a behavioral breakpoint. Message-based mode allows programmers debugging one or a sequence of services in one or multiple un-debugged containers. By a transformation unit, a programmer could also interactively debug a single service’s behavior in source level within a new debugged process. Therefore, one can focus on a truly errant behavior or a service provider without needing to debug an entire container in source level.
Vision: As Service-Oriented Computing has emerged as a cross-disciplinary paradigm for distributed computing, the need for tools and techniques to debug such software applications has grown. Debugging techniques have evolved over the years in response to changes in programming languages, implementation techniques, and user requirements. A new type of implementation vehicle for software has emerged that, once again, requires new debugging techniques. Several research efforts on theoretical and practical aspects of new debugging and monitoring have succeeded in creating a vast and valuable body of knowledge and demarcating what is feasible from what is not. Some of these techniques have been incorporated in realization of debuggers and specifications that have been built in integrated development environments. Unfortunately, such tools are based on environments with their own debugging infrastructures, which are always used to debug an entire service container but a single service in it.
Debugging Grid Services at Internet-based environments is a more difficult task due to many factors that have made conventional methodologies and tools not powerful enough. Particularly, in an un-debugged service container, the need often arises for inspection of a certain service, where the focus of programmers may refer to debug a message interaction, a single behavior, a sequence of services, and even workflow and business applications. In a testbed of Grid Computing, remote debugging in source level may violate the normally executing environment by making its debugger dependent upon its own implementation, e.g., controlled by a -debug or -trace command line argument to imperatively turn on the debugging infrastructure. Furthermore, an un-debugged container for a collection of registered services could not usually turn on or restart with such an infrastructure, once a bug occurs in a service. In this sense a source level debugger is unavailable.
State of the Art and Limitations: Debugging tools in general have a very long history. The three basic forms of debugging, trace, dump, and break date back to the EDSAC computer of the 1940’s. In a survey of on-line debugging technique, Evans and Darley remark that computer use is becoming debugging-limited rather than limited by memory size or processor speed. They further predict that this will be the state of affairs until methods for proving that programs have certain properties are successfully developed and come into wide use. Source level debugging mechanisms, such as assertions and breakpoints, allow programmers to inspect the program states when the program control reaches specific code locations. These techniques let the user conveniently observe and monitor dynamic behaviors during debugging a program. The relevant research results for debugging Grid applications can be divided into four broad categories: extension mechanisms of source level debugger, low-level system virtualization, mirror-based technique, manual instrumentation. Here we discuss some instances of them.
(1) Extension mechanisms. Kurniawan and Abramson proposed a WSRF-Compliant Debugger for Grid Applications. One important component in it is the middleware compatibility layer. It is written according to an extension mechanism of the middleware, which acts as a translation layer between Grid debugging service and generic debug interface. This extension allows a GDB/GDBServer to be utilized as the back-end debug engine. NASA Ames’s researchers build a debugger for applications running on heterogeneous computational Grids, called p2d2, which uses the client-server architecture to isolate the platform-dependent code in a debugger server. The debugger server transforms C++ objects to GDB commands that control a remote GDB server and does the GDB response back to the object level. Generally, these extension mechanisms depend on some conventional debugging facilities. Moreover, they focus on Grid applications but on the Grid Services.
(2) Virtualization. Researchers in Computer Laboratory of University of Cambridge propose the pervasive debugging approach for debugging distributed systems, called PDB. Based on the Xen Virtual Machine Monitor to virtualize the system resources of a single machine, PDB can eliminate the probe effect, and can reproduce the exact behavior of a distributed system. By using Xen to virtualize Grid resources, one can deterministically debug Grid applications. However, this debugging technique is difficult to attain for applications that need to be run on a large-scale distributed system.
(3) Mirror-based technique. Designers and developers in Sun Microsystems propose the Java Platform Debugger Architecture (JPDA) based on mirror-based systems. As interfaces of building Java debugging tools, the mirror-based API in a Java virtual machine is Java Debugger Interface (JDI), which is the uppermost layer of JPDA. The JDI defines interfaces that describe all program entities that might be of interest to a Java debugger, including classes, interfaces, objects, stack frames and more. All these interfaces are subtypes of the interface com.sun.jdi.Mirror. A mirror is always associated with a particular virtual machine as a whole, in which the entities being mirrored exists, so the implementation of a virtual machine often requires command line options to load the debugging agent or infrastructure at first. For debugging Grid or Web Services, more elaborate facilities have appeared with some systems, but the source level debugger still mainly provides only technique for accessing the dynamic state of an entire container. Therefore, programmers are continuing to debug their service applications in their original or demo machines, but a testbed even or multiple servers in different geographical locations.
(4) Manual instrumentation. Often software programmers trace the execution of their programs by manually inserting instrumentation statements that may print out code locations and values of one or more variables. Thus a program can generate its own trace that can be inspected to understand or verify its behavior as it executes on the actual environment or platform. One approach to manually instrumentation at source code is those inserted statements to dump logging information, e.g., logging services, and some programming languages support program annotation, e.g., Java and CLR. Code annotations constitute a powerful mechanism that enables passing information between programmers, tools, and the runtime, from the source code level up to the execution time. Program annotations enrich the program semantics and facilitate optimizations. They describe method usage, convey optimization hints, or aid in code development and maintenance. Manual instrumentation and annotation to measure a program should be done when one gathers profiling information by inserting code to track various quantities. However, the presence of such code can often influence the execution speed and introduce imprecision in the measurement. As with any addition to the program, the instrumentation code or collected data may inadvertently break the program or cause it to no longer fit on a run-time environment. In fact, manual instrumentation is intrusive, passive, imperative and labor-intensive.
New Ideas: Service-Oriented Computing is based around the concept of message interactions. The mechanism of message processing always provides a framework to support service’s behaviors, e.g., roles of a node, syntax and semantics of a single computational unit, exchange patterns, reliability, security, correlation, routes, and service providers. Service debugging can be organized in behaviors of message processing. Detecting and localizing bugs typically require to inspect dynamic behaviors and related contexts during the problem execution, and to check consistency between the observed behaviors and the expected behaviors. We call it as a message-based mode for service debugging. Message-based mode allows programmers debugging one or a sequence of services in one or multiple un-debugged containers. By a transformation unit, a programmer could also interactively inspect a single service’s behavior in source level mode within a new debugged process. Therefore, one can focus on a truly errant behavior or a service provider without needing to debug the entire container in source level.
With a complementary mechanism one could use new modes to debug Grid Services.
(1) Message-based mode. It will enable programmers interactively debugging one or a sequence of services in an un-debugged container. In this mode, debugged services in the container can be examined separately. Debugging facilities could co-exist with debugged services in the same container. Therefore, it supports to debug different services in one or multiple containers.
(2) Composite mode. It could be classified into two types: mixed debugging and layered debugging.
a) Mixed debugging. It is assumed that a program in the client side, an original caller, is easy to start upon with conventional debugging. Therefore, a mixed mode is that the caller can be debugged in source level, and the remote service or callee in the server side can be inspected in a message-based mode. That is, different modes can be separately used for the caller and its callee.
b) Layered debugging. With message-based debugging, one could also isolate a single behavior of the callee in source level, where a new debugged process either local or remote can start up for the debugged behavior. A source level debugger could connect or attach it, and then debugging activities can be performed.
The complementary mechanism provides following capabilities for multi-user to simultaneously debug their services at one or multiple containers.
Two-layer Debugging. By debugging stratification, the mechanism separates debugging operations to two layers with different specifications and communicational protocols. The message-based is built on the standards and protocols of Service-Oriented Architecture. It provides the unique capabilities to support service debugging in an un-debugged container. The source level debugging can be used to inspect programs for both the original caller and an isolated service behavior. It uses breakpoints to enable one to understand and control a program, including pausing and stepping through execution and inspecting and modifying data values.
For multiple programmers. By the self-identifiers, multiple programmers starting with client programs on different machines can simultaneously debug remote services at one un-debugged container. For each debugged service, a unique identifier marks the exchanged messages of debugging information of debugged services.
For multiple containers. By the co-existence, one can dynamically deploy or publish message-based debugging back-end in multiple un-debugged containers. With different addresses of System Debugging Services and unique identifiers, debugging activities can manipulate a group of debugged services at these containers.
How the topic is covered in existing EC and Chinese Work Programme: For Grid Services, we agree with the evaluation of Evens and Darley remarks, debugging-limited rather than limited by memory size or processor speed, and further envision that service debugging tools will become a staple in Service-Oriented Computing laboratories. Meanwhile, debugging is one of the most important topics in software engineering, although service debugging has received less attention.
In China, most of the research work programs including Natural Science Foundation of China (NSFC), National High-tech R&D Program (863), National Basic Research Program of China (973), have supports on the research activities of grid service debugging. Leading projects have sub-project on this topic.
Recommendations for EU and China: New debugging techniques are seen as long-term research. The main recommendation is to put these research activities in the mainstream of R&D projects. Firstly, we are considering a clear definition of Grid Service Debugging Architecture on the functionalities and debugging facilities should be proposed. This may involve requirements, standards and interfaces for service debugging. Secondly, more experiments with real applications and mature implementation of service debugging should be encouraged. Although we think that new debugging techniques have a good potential to address the existing limitations, they have to be experimented with some Grid software systems. Thirdly, we plan to integrate a complementary debugger into Grid-Programming Environment for service debugging in large-scale distributed systems.
Name
Cai Yang
Organisation
Ministry of Water Resources
Country
China
Submission date
2008-06-05
Recommendations for EU and China: Develop and design a common integrated programming environment which can provide grid programming interfaces and design patterns well. A wealth of programming examples and applications are also needed. For typical applications in some area such as astronomical calculation or water resources application, it would be easy to use. Grid version of the software development may be considered.
Name
Yongjian WANG
Organisation
Sino-German Joint Software Institute, BUAA
Country
P.R.China
Submission date
2008-06-06
New Ideas: To satisfy the requirements of application, we think the following two programming paradigms should be considered in the 3, 5 or 10 years.
A Data-driven Programming Paradigm
Grid computing programming paradigms are deeply influenced by the traditional programming model such as CORBA, EJB, and Web Service etc. But almost all of these programming paradigms are RPC-style, it’s reasonable in local computer or local network environment which can satisfy the features such as high bandwidth, low latency etc. Grid computing target at Internet-scale application with the features such as high latency, low bandwidth and so on, in the meanwhile, the rapid growth in size of data make it become more and more difficult to transfer the necessary data to the execution context which cause many problems in many scenarios such as HEP application etc.
To solve this problem, there are many rising distributed computing model, like map-reduce. MapReduce can be treated as a variant of traditional Master/Worker mode. But the main difference is that it provides the ability to split a large data set into thousands of small fractions and process them locally in parallel. In our opinion, a data-driven programming paradigm is needed to process the huge data set distributed across the Internet to complement the control-driven programming paradigm which are widely adopted by EGEE, CNGrid and many other grid infrastructures and it is proven to be not efficient in some scenarios.
Platform-as-a-Service/Infrastructure-as-a-Service
SaaS is short for Software-as-a-Service. It’s an effective mode that provides softwares through the Internet. In this mode, end users don’t buy softwares directly from vendors, instead vendors adopt web-accessed mode to provide their softwares.
In grid computing, grid infrastructure can be treated as a super computing and how end users can effectively use the infrastructure is a big issues. At present, grid infrastructures mainly provide two different functionalities: batch job execution and data storage; end users utilizes these functionalities through web service or other internal interfaces. SOA is the most widely adopted manner. But it greatly limits how end users can fully utilize the computing and storage resources provided by underlying grid infrastructures.
AWS (Amazon Web Service) and GAE (Google Application Engine) are two successfully examples that expose more controls and functionalities to end users through more flexible manner.
1. AWS consists of Amazon EC2, Amazon S3, Amazon SMS, Amazon SimpleDB etc, which expose the computing, storage resource of Amazon e-infrastructure as a service. End user can package their applications deployment as a virtual machine image including OS, database, applications etc and launch an Amazon EC2 instance running on Amazon’s e-infrastructure and using the huge data storage through Amazon S3.
2. GAE adopts the similar way of AWS, but it has more restrictions which require end user develop according to a predefined component model which only includes applications.
In AWS and GAE, the resource usage is billed based on accounting information. AWS can be treated as Platform-as-a-Service and GAE can be treated as Infrastructure-as-a-Service. Both of them have demonstrated successfully how to open underlying infrastructures and give end users more freedom to effectively utilize them.
Name
Zhongxin WU
Organisation
Sino-German Joint Software Institute, BUAA
Country
P.R.China
Submission date
2008-06-06
General comments: What is truly different between the grid workflow and the common workflow (business process)? SCUFL(Simple Conceptual unified Flow language) is defined by MyGrid, SWL(Service Workflow Language) is created by JISGA. Condor also uses its own workflow to run system. The general grid workflow definition need exact the special characters from the mess environment.
New Ideas: From the state of art and limitations summarized above, it is seemed that there will be a long way and difficult to achieve the vision. Luckily, at the same time, as research and engineering practices on workflow/business process become in-depth, new ideas for solving these limitations and challenges occur gradually. In the figure below, it will orderly explain these new ideas from two directions. One direction is from the application and concept levels, while the other is through the fields of business process modeling/emulating/analyzing, executing and management/evaluating/ optimization. Each idea is corresponding to a point of intersection (x, y) from two directions.
① Mapping between scientific and business levels of workflows:
The gap between the business process decision-maker and the application developer need diminish. A new mapping process translates the business process to a scientific formalized describing process.
② Workflow generation support
The generation of workflows is a time consuming tasks, which need experiences about the workflow tool as well as the application domain. On the other hand, companies will develop and archive a huge number of workflows for different purposes. By enhancing the workflows with appropriate semantic information it will be possible to simplify the generation of workflows and make usage of the workflow repository. For a new problem the improved workflow environment will allow to find a set of workflows, which have been used already in a similar context. For prototypes of workflows, it will be possible to find similar ones, which might be a source for modifications or even alternatives. In a second step, the semantic information for workflows can be generated automatically from the workflow itself as well as from the actual usage of the workflows.
③ Precisely identifying the pattern of the workflow:
The execution of workflow engine is mainly based on the thread mechanism which is provided by underlying system such as OS or JVM etc. The execution of workflow process can be categorized into two different types:
 Interpret the tasks defined in the workflow process in a single thread is the widely adopted approach which is suitable for sequential task execution and easy to control the concurrent execution, but this approach may behave badly if the process itself is highly concurrent;
 Multiple thread based execution mechanism may solve the perform problem, but the tradeoff is the complexity of concurrent execution control.
Actor mode is a share-nothing high concurrency support programming paradigm which is suitable for the construction of high concurrency workflow engine and can easily support sequentially and concurrently task execution. Many examples have demonstrated the successful of Actor mode such as Erlang, Scala etc.
But as the different pattern of workflow (concurrent or sequential task), the performance of workflow engine is different. If the pattern of workflow is orderly, the engine execute the workflow sequentially is perhaps better; if the pattern of workflow is concurrent, engine compiles the BPEL to the middle language for high concurrent pipelines, i.e. an Actor mode compatible middle language, is better than sequential run. So identifying the pattern of business process will develop within pattern recognition evolution in the future.
④ Services of the workflow discovering and selecting process:
Service description language will contain the QoS and ontology information of the service. For the reason of performance of semantic tagging, before the add the semantic tag to the WSDL, create an inverted-index to search a similar WSDL set and tag them with semantic information, the methods which mixes the similarity search and semantic matching process. Respecting the global constraints, service selector chooses the suitable solution reducing the re-optimization overhead to compose an optimal process from the candidate service sets.
⑤ Coordinating the distributed workflows:
 Due to the complex interaction pattern between workflows, we are opinion of that traditional message-passing method like RPC or Web service couldn’t satisfy requirements of coordination of workflows. We suggest taking into account event-driven methods. Through making full use of achievements in the field of event streams (data streams) management system, on the one side, we could initiate more complex interactions between workflows. On the other side, benefited from the property of real-time in event streams (data streams) management system, efficient performance is also guaranteed.
 Since workflow systems may derive from different fields or domains, the differences of interfaces may exist. Coordination forces any workflow to understand all other interfaces exposed by other workflows, and that is impossible or at least very difficult in the large-scale environment especially for dynamically joined or changed workflow system. So we suggest adopting the method of ontology to express corresponding domain conceptions. Combined with event-driven structure, workflows could coordinate each other more flexibly and naturally.
⑥ Workflow Security Support:
Due to the security requirements in workflow and business process, it should provide special security services with different security level for such kind of applications. Moreover, a fine-grained and extensible security framework, trust federation and negotiation for resource sharing and collaboration are also needed to apply now and in the near future.
⑦ Evaluating the workflow engine:
The Benchmark of workflow engine evaluates the performance of workflow engine. In the business process, the environment is often change in every time (such as the workflow of Grid, no a killer benchmark for evaluating the resource of VO (Visual Organization)), so the fair property of benchmark must be considered, the consumer must select the suitable workflow engine under the fair judgment methods.
Draft Roadmap Contribution: GRID Management
Name
Song Wu
Organisation
HUST
Country
China
Submission date
2008-05-20
General comments: Page 21, Section 4.5,
(1)The contents of recommendations for China need to be detailed. For instance,in grid management we should pay utmost attention to accounting management and security management. (2)We would better point out concrete industrial contributions, such as the grid management model of IBM Tivoli. (3) When the standardization is mentioned, we should specify our goal, such as, concrete standards for grid management.
Name
Cai Yang
Organisation
Ministry of Water Resources
Country
China
Submission date
2008-06-05
Recommendations for EU and China: Grid management system can provide a better interface to monitor grid operation and dynamic user management. Provide fault-tolerant mechanisms to enhance stability.
Draft Roadmap Contribution: GRID Architectures - SoA
Name
Song Wu
Organisation
HUST
Country
China
Submission date
2008-05-20
General comments: Page 13, Section 3.5, Para 1
I think that (1) the fault tolerance should be considered so as to guarantee the QoS of grid, thus it can attract more users to enjoy the grid. (2) The interoperation solution among heterogeneous grids should be taken into account. Among heterogeneous grid systems, the service discovery across different grid platforms is crucial and should be also emphasized. (3) Since currently many grid systems are developed based on WSRF standard, in my opinion, it is better that both WS and WSRF standards are supported.
Name
xuebin chi
Organisation
CNIC, CAS
Country
china
Submission date
2008-06-04
General comments: With the development of IPv6, integrating IPv6 into Grid architectures should be taken into account. Because IPv6 is beneficial to grid in many ways, not only the bigger address space, mobility support and security support, but also the many other advantages and potential benefits of IPv6, such as auto-configuration, hooks for QoS, etc.
Name
Cai Yang
Organisation
Ministry of Water resources
Country
China
Submission date
2008-06-05
General comments: Clarify the concept of grid architecture. The standards of grid platform should be unified. The services can be transplanted and deployed easily. The differences among client hardware can be shielded well. At the same time, upgrading the software version and keeping compatibility can be taken into account.
Name
Cai Yang
Organisation
Ministry of Water Resources
Country
China
Submission date
2008-06-05
Recommendations for EU and China: Clarify the concept of grid architecture. The standards of grid platform should be unified. The services can be transplanted and deployed easily. The differences among client hardware can be shielded well. At the same time, upgrading the software version and keeping compatibility can be taken into account.
Name
Li ZHA
Organisation
Institute of Computing Technology , CAS
Country
P.R.China
Submission date
2008-06-06
General comments: We consider a complementary mechanism of service debugging, which can work on two layers: message-based and source level. Grid Service inspection on one layer can use its own debugging facilities, and change to the other at a behavioral breakpoint. Message-based mode allows programmers debugging one or a sequence of services in one or multiple un-debugged containers. By a transformation unit, a programmer could also interactively debug a single service’s behavior in source level within a new debugged process. Therefore, one can focus on a truly errant behavior or a service provider without needing to debug an entire container in source level.
Vision: As Service-Oriented Computing has emerged as a cross-disciplinary paradigm for distributed computing, the need for tools and techniques to debug such software applications has grown. Debugging techniques have evolved over the years in response to changes in programming languages, implementation techniques, and user requirements. A new type of implementation vehicle for software has emerged that, once again, requires new debugging techniques. Several research efforts on theoretical and practical aspects of new debugging and monitoring have succeeded in creating a vast and valuable body of knowledge and demarcating what is feasible from what is not. Some of these techniques have been incorporated in realization of debuggers and specifications that have been built in integrated development environments. Unfortunately, such tools are based on environments with their own debugging infrastructures, which are always used to debug an entire service container but a single service in it.
Debugging Grid Services at Internet-based environments is a more difficult task due to many factors that have made conventional methodologies and tools not powerful enough. Particularly, in an un-debugged service container, the need often arises for inspection of a certain service, where the focus of programmers may refer to debug a message interaction, a single behavior, a sequence of services, and even workflow and business applications. In a testbed of Grid Computing, remote debugging in source level may violate the normally executing environment by making its debugger dependent upon its own implementation, e.g., controlled by a -debug or -trace command line argument to imperatively turn on the debugging infrastructure. Furthermore, an un-debugged container for a collection of registered services could not usually turn on or restart with such an infrastructure, once a bug occurs in a service. In this sense a source level debugger is unavailable.
State of the Art and Limitations: Debugging tools in general have a very long history. The three basic forms of debugging, trace, dump, and break date back to the EDSAC computer of the 1940’s. In a survey of on-line debugging technique, Evans and Darley remark that computer use is becoming debugging-limited rather than limited by memory size or processor speed. They further predict that this will be the state of affairs until methods for proving that programs have certain properties are successfully developed and come into wide use. Source level debugging mechanisms, such as assertions and breakpoints, allow programmers to inspect the program states when the program control reaches specific code locations. These techniques let the user conveniently observe and monitor dynamic behaviors during debugging a program. The relevant research results for debugging Grid applications can be divided into four broad categories: extension mechanisms of source level debugger, low-level system virtualization, mirror-based technique, manual instrumentation. Here we discuss some instances of them.
(1) Extension mechanisms. Kurniawan and Abramson proposed a WSRF-Compliant Debugger for Grid Applications. One important component in it is the middleware compatibility layer. It is written according to an extension mechanism of the middleware, which acts as a translation layer between Grid debugging service and generic debug interface. This extension allows a GDB/GDBServer to be utilized as the back-end debug engine. NASA Ames’s researchers build a debugger for applications running on heterogeneous computational Grids, called p2d2, which uses the client-server architecture to isolate the platform-dependent code in a debugger server. The debugger server transforms C++ objects to GDB commands that control a remote GDB server and does the GDB response back to the object level. Generally, these extension mechanisms depend on some conventional debugging facilities. Moreover, they focus on Grid applications but on the Grid Services.
(2) Virtualization. Researchers in Computer Laboratory of University of Cambridge propose the pervasive debugging approach for debugging distributed systems, called PDB. Based on the Xen Virtual Machine Monitor to virtualize the system resources of a single machine, PDB can eliminate the probe effect, and can reproduce the exact behavior of a distributed system. By using Xen to virtualize Grid resources, one can deterministically debug Grid applications. However, this debugging technique is difficult to attain for applications that need to be run on a large-scale distributed system.
(3) Mirror-based technique. Designers and developers in Sun Microsystems propose the Java Platform Debugger Architecture (JPDA) based on mirror-based systems. As interfaces of building Java debugging tools, the mirror-based API in a Java virtual machine is Java Debugger Interface (JDI), which is the uppermost layer of JPDA. The JDI defines interfaces that describe all program entities that might be of interest to a Java debugger, including classes, interfaces, objects, stack frames and more. All these interfaces are subtypes of the interface com.sun.jdi.Mirror. A mirror is always associated with a particular virtual machine as a whole, in which the entities being mirrored exists, so the implementation of a virtual machine often requires command line options to load the debugging agent or infrastructure at first. For debugging Grid or Web Services, more elaborate facilities have appeared with some systems, but the source level debugger still mainly provides only technique for accessing the dynamic state of an entire container. Therefore, programmers are continuing to debug their service applications in their original or demo machines, but a testbed even or multiple servers in different geographical locations.
(4) Manual instrumentation. Often software programmers trace the execution of their programs by manually inserting instrumentation statements that may print out code locations and values of one or more variables. Thus a program can generate its own trace that can be inspected to understand or verify its behavior as it executes on the actual environment or platform. One approach to manually instrumentation at source code is those inserted statements to dump logging information, e.g., logging services, and some programming languages support program annotation, e.g., Java and CLR. Code annotations constitute a powerful mechanism that enables passing information between programmers, tools, and the runtime, from the source code level up to the execution time. Program annotations enrich the program semantics and facilitate optimizations. They describe method usage, convey optimization hints, or aid in code development and maintenance. Manual instrumentation and annotation to measure a program should be done when one gathers profiling information by inserting code to track various quantities. However, the presence of such code can often influence the execution speed and introduce imprecision in the measurement. As with any addition to the program, the instrumentation code or collected data may inadvertently break the program or cause it to no longer fit on a run-time environment. In fact, manual instrumentation is intrusive, passive, imperative and labor-intensive.
New Ideas: Service-Oriented Computing is based around the concept of message interactions. The mechanism of message processing always provides a framework to support service’s behaviors, e.g., roles of a node, syntax and semantics of a single computational unit, exchange patterns, reliability, security, correlation, routes, and service providers. Service debugging can be organized in behaviors of message processing. Detecting and localizing bugs typically require to inspect dynamic behaviors and related contexts during the problem execution, and to check consistency between the observed behaviors and the expected behaviors. We call it as a message-based mode for service debugging. Message-based mode allows programmers debugging one or a sequence of services in one or multiple un-debugged containers. By a transformation unit, a programmer could also interactively inspect a single service’s behavior in source level mode within a new debugged process. Therefore, one can focus on a truly errant behavior or a service provider without needing to debug the entire container in source level.
With a complementary mechanism one could use new modes to debug Grid Services.
(1) Message-based mode. It will enable programmers interactively debugging one or a sequence of services in an un-debugged container. In this mode, debugged services in the container can be examined separately. Debugging facilities could co-exist with debugged services in the same container. Therefore, it supports to debug different services in one or multiple containers.
(2) Composite mode. It could be classified into two types: mixed debugging and layered debugging.
a) Mixed debugging. It is assumed that a program in the client side, an original caller, is easy to start upon with conventional debugging. Therefore, a mixed mode is that the caller can be debugged in source level, and the remote service or callee in the server side can be inspected in a message-based mode. That is, different modes can be separately used for the caller and its callee.
b) Layered debugging. With message-based debugging, one could also isolate a single behavior of the callee in source level, where a new debugged process either local or remote can start up for the debugged behavior. A source level debugger could connect or attach it, and then debugging activities can be performed.
The complementary mechanism provides following capabilities for multi-user to simultaneously debug their services at one or multiple containers.
Two-layer Debugging. By debugging stratification, the mechanism separates debugging operations to two layers with different specifications and communicational protocols. The message-based is built on the standards and protocols of Service-Oriented Architecture. It provides the unique capabilities to support service debugging in an un-debugged container. The source level debugging can be used to inspect programs for both the original caller and an isolated service behavior. It uses breakpoints to enable one to understand and control a program, including pausing and stepping through execution and inspecting and modifying data values.
For multiple programmers. By the self-identifiers, multiple programmers starting with client programs on different machines can simultaneously debug remote services at one un-debugged container. For each debugged service, a unique identifier marks the exchanged messages of debugging information of debugged services.
For multiple containers. By the co-existence, one can dynamically deploy or publish message-based debugging back-end in multiple un-debugged containers. With different addresses of System Debugging Services and unique identifiers, debugging activities can manipulate a group of debugged services at these containers.
How the topic is covered in existing EC and Chinese Work Programme: For Grid Services, we agree with the evaluation of Evens and Darley remarks, debugging-limited rather than limited by memory size or processor speed, and further envision that service debugging tools will become a staple in Service-Oriented Computing laboratories. Meanwhile, debugging is one of the most important topics in software engineering, although service debugging has received less attention.
In China, most of the research work programs including Natural Science Foundation of China (NSFC), National High-tech R&D Program (863), National Basic Research Program of China (973), have supports on the research activities of grid service debugging. Leading projects have sub-project on this topic.
Recommendations for EU and China: New debugging techniques are seen as long-term research. The main recommendation is to put these research activities in the mainstream of R&D projects. Firstly, we are considering a clear definition of Grid Service Debugging Architecture on the functionalities and debugging facilities should be proposed. This may involve requirements, standards and interfaces for service debugging. Secondly, more experiments with real applications and mature implementation of service debugging should be encouraged. Although we think that new debugging techniques have a good potential to address the existing limitations, they have to be experimented with some Grid software systems. Thirdly, we plan to integrate a complementary debugger into Grid-Programming Environment for service debugging in large-scale distributed systems.
Name
Yongjian WANG
Organisation
Sino-German Joint Software Institute, BUAA
Country
P.R.China
Submission date
2008-06-06
New Ideas: The purpose of grid computing is to collect the resource scattered across the Internet. This purpose originally has close relationships with two principles: REST (Representational state transfer) and Enterprise Integration Architecture
REST (Representational state transfer)
Representational state transfer (REST) is a style of software architecture for distributed hypermedia systems such as the World Wide Web. The terms “representational state transfer” and “REST” were introduced in 2000 in the doctoral dissertation of Roy Fielding, one of the principal authors of the Hypertext Transfer Protocol (HTTP) specification. The terms have since come into widespread use in the networking community.
REST's proponents argue that the Web's scalability and growth are a direct result of a few key design principles:
1. Application state and functionality are divided into resources
2. Every resource is uniquely addressable using a universal syntax for use in hypermedia links
3. All resources share a uniform interface for the transfer of state between client and resource, consisting of
a) A constrained set of well-defined operations
b) A constrained set of content types, optionally supporting code on demand
REST is still HTTP-based solution, but provides a more natural way to represent resource state which is similar to WS-Address in Web Service Specification. RESTful design has gained widely accepted, and considered as an effective way for building computing infrastructure in the Internet environment.
Enterprise Integration Architecture
EIA (Enterprise Integration Architecture) has gained widely used in enterprise integration. The recently idea in this area includes ESB (Enterprise Service Bus), SCA (Service Component Architecture) etc. ESB and SCA can be an effective way to assemble heterogeneous grid middlewares to form a bigger grid infrastructure.
Besides REST and Enterprise Integration Architecture which provide new ways to encapsulate different resources and construct grid infrastructure, we think more attentions should be paid to some vital features for long-running grid infrastructure such as fault-tolerance, scalability etc.
Name
Xuebin Chi
Organisation
supercomputing Center, CNIC, CAS
Country
China
Submission date
2008-06-06
General comments: With the development of IPv6, integrating IPv6 into Grid architectures
should be taken into account. Because IPv6 is beneficial to grid in many ways,
not only the bigger address space, mobility support and security support, but
also the many other advantages and potential benefits of IPv6, such as auto-configuration, hooks for QoS, etc.
Contributions: #20