03/24/98
CHAPTER DRAFT 10 GOODNESS MEASURES FOR INTEGRATION
10.1 INTRODUCTION
Measuring goodness is an important step for any effort. Specially helpful are measures that can quantify the results. The measures not only tell how good a particular effort was, but also help in improving it. Information integration seems to be the key in increasing productivity and overall efficiency of complex manufacturing systems. This chapter attempts to answer the question as to what are the metrics for information-integration. One can identify four sets of metrics to evaluate the presence, or absence, of an integrated information system. These metrics cover the following major areas:
* * * *
data integration organization integration system integration identified financial benefits resulting from integration.
The metrics are both qualitative and quantitative measurements of the success displayed by a corporation in defining, implementing, and maintaining an organization structure, a computer architecture, and an information architecture that bring about identified profits to the corporation through novel use of information management. The following are generally considered the measures of effectiveness of a manufacturing system: Design cycle-time reduction, Manufacturing cycle-time reduction, Overall cycle time reduction, Inventory reduction, WIP reduction, Rework reduction, Scrap reduction, Improved quality, Reduced maintenance, Cost, Better business strategy, Flexibility, Reliability, etc. These parameters are what one is ultimately interested in. How to improve these however is not well formalized. It is generally accepted that information-integration will improve these parameters. This implies that better information-integration should enable the system to perform better. It is contended that this is achieved by providing accurate and timely information to the functions who need it. What is needed then are ways to EDM 10-1
DRAFT 03/24/98
measure information-integration. As per the definition in chapter 6, informationintegration is the ability to provide correct, secure, and manageable information at the right time and in the right format in response to a request. This definition describes what is information-integration and not how to measure information-integration. It does however point us in the direction to look for, i.e., information theory. The following sections describe the notions of information content, surprise, and entropy, and proposed measures of information integration. The results are from a relational viewpoint, although they may be more broadly applicable.
10.2
BACKGROUND
Defining metrics for complex systems is very difficult. Methods have been proposed to qualitatively describe their behavior. Quantitative results are hard to find. The systems operate in a probabilistic manner, their design and operation is not well formalized, and they are subject to change. The performance criteria are themselves aggregates of other criteria. Hence a common mathematical framework is required that yields results that can be combined and compared with other results. Information theory provides these capabilities and its concepts have been used to quantify designs. This section describes some of the relevant concepts of information theory pertinent to measuring information integration. Consider that an event occurs and some message is transmitted about this event. Then the amount of information received (also referred as self-information) in the message is defined as [6]:
Information received = log [ probability at the receiver of the event after the message is received.
/
probability at the receiver of the event before the message is received. ]
Considering that there is no noise, the receiver is certain that the message provided is correct. Therefore the probability of the event after the message is received is unity. Hence the above equation reduces to [6]:
Information received (I) =
-
log [ probability at the receiver of the event EDM 10-2
DRAFT 03/24/98
before the message is received. (p)] => I = - log [p] 10.1
If we take logs to the base two the unit of information becomes the bit. Other bases can be used with corresponding units. It is however generally assumed that the base is two unless stated otherwise [12]. It follows then that the more surprising the event, the greater the information received. The conveyance of information can produce certainty from initial uncertainty, and can be expressed as [4]:
Uncertainty + Information = Certainty
10.2
This is similar to the situation our definition of information-integration describes, i.e., a function uncertain about something queries an information manager for information. The success depends on how well the information manager can handle this query, i.e., how much surprise this query generates in the information manager.
If the events are not equiprobable, as is often the case, average information per event can be computed as [4]: H= ( pi log pi ) i bits/event 10.3
where pi is the probability of the occurrence of event i. H is sometimes called Entropy of the set of probabilities pi. The term entropy has been chosen in literature due to similarity of the right hand side of equation number (10.2) to the entropy equation in thermodynamics, due to the disorder (uncertainty) it reflects, and because its value is preserved during the decomposition process [4,8,12]. Entropy of a set of events is a maximum when they are equiprobable [12]. In the case where the probabilities of events are conditional, conditional entropy ( H(j/i) ) can be defined as [12]: H(j/i) = where p(i, j) [ p(i, j) log p(j/i) ] i j = p(i) x p(j/i), 10.4
p(i, j) is the probability of two conditional events ( i EDM 10-3
DRAFT 03/24/98
and j) occurring together, and p(j/i) is the probability of j given that i has occurred. Another concept of information theory is that of redundancy. It has been noted that most of the redundancy is due to the inter-event influence. Redundancy (R) is defined as [12]:
R=(maximum entropy - actual entropy)/maximum entropy
10.5
There have been attempts to use information theory and information content as measures in various domains. Although the basic axioms used cannot be proved, the results appear to be intuitively correct based on observations, case studies, and experience. Suh [13] has proposed information content as a measure of goodness of a design, i.e., the lower the information content of a design, the better the design. Rangan [5] uses the concept to provide expertise discriminators for a design. Pye and Gupta [10,14] propose a measure of flexibility as the ability to handle uncertainty. According to Gupta [14], the more the number of choices, the greater the uncertainty, and hence the greater the flexibility. Nakazawa [7] provides information content for systems with parameter ranges as:
I = log (System Range / Common Range)
and provides examples for machines with specific tolerance ranges. Lee [8,9] provides an information theoretic analysis of the relational model by using entropy as an information metric to quantify the information associated with a set of attributes. Lee also [8] proves that data dependencies can be formulated in terms of entropies, and thus entropies form a basis for testing these dependencies. It can be inferred from here that satisfaction of functional dependencies results in decrease of information content of the schema.
There is a need to extend this work to include extended semantic concepts used in this thesis, and provide measures for information, data, constraint, and exception integration.
10.3
MEASURES OF INFORMATION-INTEGRATION
The need for goodness measures is well recognized, but identification and definition of these measures is difficult. Some of the measures are apparent from the requirements we EDM 10-4
DRAFT 03/24/98
have for information, namely: security, integrity, consistency, non-redundancy, availability, manageability, quality, extensibility, reusability, etc. The value of an information system can be assessed using these criterion: Quality of information base. Quality of information output. Timeliness of the information (frequency, delay). Flexibility of the information system (Schema change, New requirements). Value of information provided to the user. Security (secrecy, safety). Correctness ( accuracy, integrity, consistency). Manageability (administration, control). These criteria are however not easily quantifiable. We will show that information theoretic approach provides a consistent framework for providing goodness measures for a variety of parameters. For measuring information-integration, we again look at the GIM (see chapter 6). It is apparent that each function looks at information from its own viewpoint, and expects information accordingly. Translators perform the task of providing these views on the common information. Thus the easier the task of translators, the better the informationintegration. Thus we identify two sources of uncertainty in the system framework, one is the shared information system, and second is the translators. We will propose two axioms for understanding the contribution of these two sources.
AXIOM I-T: integration.
The lower the entropy of the translators, the better the information
AXIOM S-I: System has better information-integration when requests for information generate lesser surprise.
The axiom S-I can be generalized to say that system has better integration when requests for performing some function generate lesser surprise. The axioms help us in defining a measure of information integration as follows.
EDM 10-5
DRAFT 03/24/98
Shared Information Efficiency Let there be N requests for information each with an associated probability pj , j = 1, ..., N. The information content of a request is given by: Ij = - log (pj). The average information content of the requests, or the entropy of the shared information system is given by: Hreq = ( pj log pj ) bits/event j
10.6
This can take a maximum value of log(N). We can then define shared information efficiency as:
Shared Information Efficiency = h si = 1 - Hreq / log N
10.7
This says that if a query is made that the system does not expect (i.e., cannot handle), then the entropy would be very high. The probabilities of the queries can be determined by the evaluator based on his experience and system logs. A general way of assigning probabilities can therefore be described in the following way.
Assume the queries are geared to retrieve a single value. Then the probability can be assigned as the inverse of the number of values the system provides as its best answer. Thus if the system returns one value, p = 1. If the system returns n values, p = 1/n. If the system cannot provide a satisfactory answer, then we assume all the contents of the data base have to be provided. Assuming the database contains M values, p = 1/M. It is assumed here that the query is processed within a reasonable time frame. Let us consider an example:
Example 1 Assume ten queries were posed to an information system containing 10,000 values. Each query expects a single value to be returned. The following number of values were reported for the queries: q(1) = 7 , q(2) = 350 , q(3) = 100 , q(4) = 200 , q(5) = 4 , q(6) = 1000 , q(7) = 2000 , q(8) = failed , q(9) = 2000 , q(10) = 7 .
EDM 10-6
DRAFT 03/24/98
Using the method described above, the probabilities can be assigned as: p(1) = .14 , p(2) = 2.85E-3 , p(3) = .01 , p(4) = .005 , p(5) = .25 , p(6) = .001 , p(7) = .0005 , p(8) = .0001 , equation 10.6 as: p(9) = .0005 , p(10) = .14 . The entropy can be calculated using
Hreq = .1207 + 7.26E-3 + .02 + .0115 + .1505 + 3.0E-3 + 1.65E-3 + 4.0E-4 + 1.65E-3 + .1207 = .4346 The shared information efficiency (equation 8.7) therefore is = h si = 1 - .4346/log 10 = .5654
Information Translation Efficiency Let there be t translators to provide the functions the information in their required format. The entropy of translation (Htran) therefore is t Htran =
k=1
Hk
10.8
The entropy of translation depends on the difference between input and output data of the translation, and how difficult that translation was. This entropy can be minimized through the use of various information managers proposed in chapter 6, as they help the translation process. We will only discuss the role of the data model, constraint manager, and exception manager in the entropy of translation. Another factor in minimizing the entropy of translation is the number of translators, the lesser the number of translators, the lesser the entropy. This supports the view taken in the proposed framework where only 2n translators are required for n functions, as opposed to n(n-1) translators required by approaches that use direct exchange of data. It can be noted that if there exists an information model that is comprehensive and all the system functions follow this information model, the resulting entropies would be minimal. This supports the views expressed in chapter and 7, provide motivation towards developing such an information model. This leads to the next question of how to measure the goodness of an information model.
EDM 10-7
DRAFT 03/24/98
10.4
MEASURES OF DATA, CONSTRAINT, AND EXCEPTION INTEGRATION
From the GIM (see chapter 6), we know that information integration consists of data integration, constraint integration, and exception integration, and that the work that translators have to do depends on the levels of these integrations. Hence we need ways of measuring these components, and then combining the result to get a measure of information integration as a function of its three components. This will help in understanding how information integration can be improved. These measures correspond to the information management layer of the proposed framework (figure 6-9). We will define goodness of data integration using the data model that represents the logical structure of data. The basic premise is that if the information base holds all the necessary data in an unambiguous fashion, then the system will be least surprised by the request for information. For defining data model efficiency, we again propose an axiom:
AXIOM I-M: Of all the data models satisfying the information requirements, the data model with the least information content is the best data model. Here the information requirements are defined using equation 6.1 of the GIM.
Goodness Measure for the Data Integration Consider a data model with N entities and M relationships (We assume the data model uses the concepts of entities and relationships). The information content (Iim) of the data model can then be defined as: N Iim = M
Ei +
i=1
Rj
j=1
10.9
where Ei is the information content of entity i, and Rj is the information content of relationship j. The information content of an entity can be defined as:
Ei = - log ( 1/ ai )
10.10
EDM 10-8
DRAFT 03/24/98
where ai is the number of attributes associated with entity i. This assumes that the entities are in fourth normal form, i.e., there are no redundant dependencies.
The information content for a relationship can be defined as:
Rj
= - log ( 1/ bj )
10.11
where bj is defined for different relationships as:
bj
= 1 = 2x cj = 4x cj = 1 = 3 = 9x cj = 27x cj = dj
if it is a 1:1 binary relationship. if it is a 1:N binary relationship. if it is a M:N binary relationship. if it is a 1:1:1 ternary relationship. if it is a 1:1:N ternary relationship. if it is a 1:N:M ternary relationship. if it is a M:N:P ternary relationship. if it is a generalization/subset relationship. 10.12
where cj is the number of attributes associated with a relationship j, and dj is the number of subclasses in the generalization/subset abstraction. cj = 1, if there are no attributes in a relationship. Let us consider an example to clarify the equations. In this example we will compute the information contents of two information models which represent the same information requirements.
Example 2 Consider the information models A and B shown in figure 10-1. The models have been taken from case study 3. The sample attributes for the entities are: Model A: ITEM ( item_id, order_no, qty, unit, prod_code) INVENTORY (id, loc_id, status) HISTORY ( id, timestamp, mach_no, oper_id, length, code) EDM 10-9
DRAFT 03/24/98
PRODUCT_A (id, length, mil_ind, orig_length, prod_stat, loss_long, apert, ovality) PRODUCT_B (id, length, mil_ind, orig_length, prod_stat, color, coating) PRODUCT_C (id, length, mil_ind, orig_length, prod_stat, color1, color2, fib_id) PRODUCT_D (id, length, mil_ind, orig_length, prod_stat, fib_num, coating) IS_COMPOSED_OF (order_no, item_id, id, length, spec_req) CONTAINS () REFERS_TO ()
Model B: ITEM ( item_id, order_no, qty, unit, prod_code) INVENTORY (id, loc_id, status) HISTORY ( id, timestamp, mach_no, oper_id, length, code) GENERIC_PROD (id, type, length, mil_ind, orig_length, prod_stat) PRODUCT_A (id, loss_long, apert, ovality) PRODUCT_B (id, color, coating) PRODUCT_C (id, color1, color2, fib_id) PRODUCT_D (id, fib_num, coating) IS_COMPOSED_OF (order_no, item_id, id, length, spec_req) CONTAINS () REFERS_TO ()
The information contents can be calculate using equation no. 10.9 as:
Information content of model A = IA = [log 5 + log 3 + log 6 + log 8 + log 7 + log 8 + log 7 ] + [ 4 x log (2x5) + 4 x log 2 + 4 log 2 ] = log (5x3x6x8x7x8x7) + 4 x log (40) = log 282240 + 4x1.602 = 5.45 + 6.408 => IA = 11.85
Information content of model B = IB = [log 5 + log 3 + log 6 + log 6 + log 4 + log 3 + log 4 + log 3 ] + [log 4 + log 5 + log 2 + log 2 ] = log ( 5x3x6x6x4x3x4x3) + log (4x5x2x2) = 4.89 + 1.90 EDM 10-10
DRAFT 03/24/98
=>
IB
=
6.79
Thus according to the axiom I-M, the data model B is better than data model A. The results are consistent with our experiences outlined in case study 3, as it is clear that abstractions result in the reduction of the information content.
***
The constraint and exception integration methodologies have been proposed in previous chapters. We have identified that information integration requires that constraints be satisfied and exceptions properly handled. To define a measure of constraint integration we again propose an axiom:
AXIOM C-I:
If the system satisfies all the constraints on data, i.e., all the information is consistent, then the system will be least surprised by the request for information.
Goodness Measure for the Constraint Integration From chapter 7 and the framework we know that the information model is not well constrained, because the constraints depend on the process and the business rules. The system or function specific constraints are therefore enforced via the constraint manager. The entropy pertaining to process constraints is the entropy of the corresponding IDEF0 hierarchy. The entropy of the IDEF0 is a function of entropies of its components defined through the specification language. Entropy definitions of all the components are beyond the scope of this thesis. Here we will attempt to define entropy associated with the integrity constraints on data. The axiom C-I leads to define the information content of an integrity constraint (Iic) as: Iic = Information content of a data structure - Information content of the constrained data structure. 10.13
Some examples will help clarify the equations. The following two examples will show how the information content associated with a constraint can be calculated. EDM 10-11
DRAFT 03/24/98
Example 3 Consider two data models shown in figure 10-2. Data model A says that any operator can make any part on any machine. The data model A is then constrained to say that a given part type can only be made on a specific machine thus resulting in data model B. Note that the two models do not refer to the same information requirements. The attributes of the two models are given as:
Model A: OPERATOR (oper_id, name, salary) MACHINE (mach_id, m_type, toler) PART (part_type, cost, function) MAKES (oper_id, part_type, mach_id, qty, date)
Model B: OPERATOR (oper_id, name, salary) MACHINE (mach_id, m_type, toler) PART (part_type, cost, function, mach_id) MAKES (oper_id, part_type, qty, date) MADE_ON ()
The information contents of the models can be calculated using equation no. 10.9 as:
Information content of data model A = IA = [ log 3 + log 3 + log 3] + [log 9x5] = 1.431 + 1.653 = 3.084
Information content of data model B = IB = [ log 3 + log 3 + log 4] + [log 2x4 + log 2] = 1.566 + 1.204 = 2.77
Information content of the associated constraint = Iic = 3.084 - 2.76 = .314
***
EDM 10-12
DRAFT 03/24/98
Example 4 Consider two data models shown in figure 10-3. Data model A says that a path may have many control points and a control point may belong to many paths. The data model A is then constrained to say that a control point may b...