In addition to SEQUAL, the UEML quality framework has been inspired by two additional quality frameworks: Moody’s framework (2003) and the ISO/IEC 9126 international standard for assessing software product quality (ISO/IEC 2001). The additional frameworks have been aligned with SEQUAL's appropriateness types through a generalisation hierarchy (Berio, Opdahl, Anaya & Dassisti 2005b), as shown in the figure (still missing).

The resulting appropriateness categories in the UEML quality framework remain too general to allow concrete evaluations (Anaya, Berio & Verdecho 2007), and the framework has therefore been extended with criteria and requirements, also shown in the figure (still missing). Requirements are collected from users (actors or experts), asking them how enterprise modelling should contribute towards enterprise integration and interoperability. Criteria are the operational, or measurable, counterparts of requirements (which do not apply directly to languages). Once criteria are defined, they can in turn be related to one or more appropriateness categories, making it clear to which quality types each criterion contributes. The framework provides two complementary ways of evaluating criteria. The language template is used to gather general information about a language, such as its notation and meta model, whereas the language-evaluation questionnaire comprises questions derived from the criteria and an associated glossary.

As shown in the figure (still missing), the framework also introduces language descriptions, covering, e.g., a language's owner and version; goal, an aggregation of related criteria for a language; metrics-for-goal, a selection of metrics for a specific goal; metric evaluation, a particular application of a single metric to a specific language; and combined metrics evaluation, a combined evaluation of several metrics evaluation for a given language and a given goal.

The purpose of the associated quality evaluation method in the next figure (still missing) is to give a clear picture of how to evaluate and select one or more enterprise modelling languages for a specific purpose. The first task is to define the goals as aggregations of criteria and then select suitable metrics for each criterion. The language template is used to collect factual information about each language, whereas the language-evaluation questionnaire is used to collect subjective opinions. Hence, whereas only a single language template needs to be filled-in for a language, multiple questionnaires are needed. While it is acknowledged that subjective differences may lead to contradictions, they are not presently dealt with as part of the quality framework. Once the selected criteria have been evaluated using the metrics, appropriate languages must be selected. This is not an easy task in situations with conflicting evaluations on different criteria. Therefore, in addition to have combined metrics that synthesise the detailed evaluations, it remains important to be able to browse the detailed evaluations on each criterion for a language.

Together, the representation mappings, common ontology and correspondence measures contributes towards integrated use of models expressed in different languages. But they offer no help in choosing which among the plethora of available enterprise modelling languages to incorporate into UEML in the first place and choosing which incorporated languages to use for which purposes. UEML therefore comprises a language quality framework (Anaya, Berio & Verdecho 2007), which aids language selection by

• defining the concept of quality of a modelling language;
• supporting methodical, goal-dependent evaluation of enterprise modelling languages.

In order to define modelling-language quality, the quality framework has adapted and extended the SEQUAL quality framework (Krogstie 1998, 2005), which provides a model of the quality of models, later extended to also account for the quality of languages. The original framework identifies 8 quality types for characterising what quality is: physical quality, empirical quality, syntactic quality, semantic quality, perceived semantic quality, pragmatic quality, social and organisational quality. For example, semantic quality is the correspondence between the model and the domain. The framework also identifies several types of appropriateness, each indicating a language aspect that must be considered when assessing whether a language is appropriate for a particular purpose (Krogstie 1998, 2005). For example, comprehensibility appropriateness reflects the ease with which the language its model can be understood by a certain audience. In SEQUAL, each quality type is related to one or more appropriateness types and vice versa. For example, domain appropriateness is used to assess physical and semantic qualities. Therefore, the different types of appropriateness provide the context to evaluate the related quality types.

In addition to SEQUAL, the UEML quality framework has been inspired by two additional quality frameworks: Moody’s framework (2003) and the ISO/IEC 9126 international standard for assessing software product quality (ISO/IEC 2001). The additional frameworks have been aligned with SEQUAL's appropriateness types through a generalisation hierarchy (Berio, Opdahl, Anaya & Dassisti 2005b), as shown in Figure 3.

The resulting appropriateness categories in the UEML quality framework remain too general to allow concrete evaluations (Anaya, Berio & Verdecho 2007), and the framework has therefore been extended with criteria and requirements, also shown in Figure 3. Requirements are collected from users (actors or experts), asking them how enterprise modelling should contribute towards enterprise integration and interoperability. Criteria are the operational, or measurable, counterparts of requirements (which do not apply directly to languages). Once criteria are defined, they can in turn be related to one or more appropriateness categories, making it clear to which quality types each criterion contributes. The framework provides two complementary ways of evaluating criteria. The language template is used to gather general information about a language, such as its notation and meta model, whereas the language-evaluation questionnaire comprises questions derived from the criteria and an associated glossary.

As shown in Figure 3, the framework also introduces language descriptions, covering, e.g., a language's owner and version; goal, an aggregation of related criteria for a language; metrics-for-goal, a selection of metrics for a specific goal; metric evaluation, a particular application of a single metric to a specific language; and combined metrics evaluation, a combined evaluation of several metrics evaluation for a given language and a given goal.

The purpose of the associated quality evaluation method in Figure 4 is to give a clear picture of how to evaluate and select one or more enterprise modelling languages for a specific purpose. The first task is to define the goals as aggregations of criteria and then select suitable metrics for each criterion. The language template is used to collect factual information about each language, whereas the language-evaluation questionnaire is used to collect subjective opinions. Hence, whereas only a single language template needs to be filled-in for a language, multiple questionnaires are needed. While it is acknowledged that subjective differences may lead to contradictions, they are not presently dealt with as part of the quality framework. Once the selected criteria have been evaluated using the metrics, appropriate languages must be selected. This is not an easy task in situations with conflicting evaluations on different criteria. Therefore, in addition to have combined metrics that synthesise the detailed evaluations, it remains important to be able to browse the detailed evaluations on each criterion for a language.

The UEML language quality framework thus offers help both in choosing which among the plethora of available enterprise modelling languages to incorporate into UEML in the first place and in choosing which incorporated languages to use for which purposes in an enterprise. In the latter case, the quality framework can and should be extended further with enterprise-specific requirements, goals, criteria and metrics.

References

• Anaya, V., Berio, G., and Verdecho, M.J. (2007). Evaluating Quality of Enterprise Modelling Languages: The UEML Solution. I-ESA 2007, Funchal, Portugal (2007).
• Berio, G., Opdahl, A., Anaya, V., and Dassisti, M. (2005b). Deliverable DEM2. Publicly available at www.interop-noe.org.
• ISO/IEC Standard 9126 (2001). Software product quality, International Standards Organisation (ISO). International Electrotechnical Commission (IEC).
• Krogstie, J. (1998). Using a Semiotic Framework to Evaluate UML for the Development for Models of High Quality. Siau K., Halpin T., (eds) Unified Modelling Language: System Analysis, Design and Development Issues, IDEA Group Publishing, pp. 89-106.
• Krogstie, J. (2005). Evaluating UML Using a Generic Quality Framework. Encyclopedia of Information Science and Technology. M. Khosrow-Pour Editor, IDEA Group Publishing.
• Moody DL (2003) Measuring the quality of data models: an empirical evaluation of the use of quality metrics in practice. Proc. ECIS'2003, Naples, Italy.