EAI I (2)

EAI I (2) - Enterprise Application Integration – An...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Enterprise Application Integration – An Inside Integration By Dr. Atanu Rakshit Email: arakshit@isquareit.ac.in atanu.rakshit@usa.net atanu.rakshit@usa.net Enterprise Application Integration Enterprise List of References ‘Enterprise Application Integration’ by David S. Enterprise Linthicum, Addison-Wesley, 1999 Linthicum, ‘Enterprise Application Integration’ by W. A. Ruh, Enterprise F. X. Maginnis, W. J. Brown, Wiley, 2000 F. URL: www.eai.ittoolbox.com URL: www.eai.ittoolbox.com www.eaijournal.com www.eaijournal.com EAI – Course Outline EAI EAI – An Overview Data Level EAI Application Interface Level EAI Method Level EAI User Interface Level EAI The EAI Process Methodology EAI and Middleware EAI – Course Outline EAI Transactional Middleware and EAI Messaging Oriented EAI Distributed Objects and EAI Database Oriented Middleware and EAI Java Middleware and EAI XML and EAI E – Business and EAI Process Automation and EAI Future Trends in EAI EAI – An Overview EAI EAI An Approach EAI Why is EAI needed? What does it do? How does it do? Who uses it? Who is going to use it? EAI - Background EAI Early days’ computerization approaches Tasks automated within departments Computer systems grew ‘departmentalized’ These ‘stovepipe’ systems were Independent of each other Narrow scope Independent islands of computing EAI - Background EAI What is wrong with that? Each department works fine Limitations Systems work in isolation No integrated approach to address the organizational No activities activities Customer Information within a stovepipe system has value Customer when viewed as a whole when Desirable to integrate key systems with vendors and Desirable customers customers EAI - Background EAI In 1990s: Packaged software solution such as ERP, CRM, Packaged SCM etc. SCM Worked well individually, but not together Giving ‘information islands’ again EAI - Background EAI How do we get around these problems? There is strong demand to bind these There applications into a single, unified enterprise application application EAI allows many of the stovepipe applications EAI that exist today to share both processes and data data Use Enterprise Application Integration Use Techniques Techniques EAI - Definition EAI EAI is the unrestricted sharing of data and EAI business processes among any connected applications and data sources in the enterprise. applications REMARKS: To share data and processes without having to To make sweeping changes to the applications or data structures structures Creating methods/middleware of accomplishing Creating this integration can EAI be both functional and cost effective cost EAI EAI Legacy Application Packaged Application Web based Application EAI Client/Server Application Objects Types of EAI Types Data Level Application Interface Level Method Level User Interface Level Data Level Data Techniques and technology of moving data between Techniques data sources data Extracting information from one/many database(s), Extracting processing the information as needed and updating the target database(s) the The data sources can be one or many database The comprising of many tables comprising This may also include transformation and application This of business logic to data of This interface leaves application and business This processes untouched processes Application Interface Level Application Leveraging of interfaces exposed by custom or Leveraging packaged applications packaged Developers leverage these interfaces to access Developers both business processes and application flow both Bundle many applications together, allowing Bundle them to share business logic and information them Method Level For sharing of the business logic that exist For within the enterprise. within Example – The method for updating a Example customer details may be accessed from many applications. Applications may access each other’s method without rewriting the same. other’s The mechanism to share methods among The applications are numerous including distributed objects, Application servers etc. distributed User Interface Level User Using this scenario, architects and developers Using are able to bundle applications by using their user interfaces as a common point of integration ( known as screen scraping) integration Types of EAI Types Legacy Business Process Packaged Application Data User Interface Level Method Level Application Interface Level Data Level Legacy Business Process Packaged Application Data EAI - Requirement EAI EAI is used to solve all the problem previously EAI mentioned. mentioned. It must cover every part of an enterprise Architecture Hardware Software Processes EAI - Requirement EAI How? Business Process Integration (BPI) Application Integration Data Integration Platform Integration Business Process Integration Business Specifies the processes involved in the Specifies exchange of enterprise information exchange Can include: This allows organizations to streamline operations, This reduce cost and improve responsiveness to customer demands customer Process Management Process Modeling Workflow Basically: What is needed at each step of each Basically: process? process? Application Integration Application Goal: Bring data or a function from one application together with Bring that of another application that together provide near realthat time integration It Include: Business to business integration CRM systems Web Integration Build web sites that interact with multiple business systems Data Integration Data For the last two to work, we must also For integrate data integrate Identify and record data Build a metadata model Data can be shared or distributed across Data database systems, providing it is in a standard format such as COM/DCOM, CORBA, EDI, RMI, XML etc. RMI, Platform Integration Platform Deals with the processes and tools that are Deals required to allow these systems to communicate: communicate: Optimally Securely Data can be passed through different Data applications without difficulty applications EAI Vs Traditional Middleware EAI EAI Configuration 1. 2. 3. 4. Implementation Traditional Middleware Object Request Broker Transaction Monitors RDBMS Messaging Application Independent Business Process Oriented Configuration across Application Automated Deployment 1. 2. 3. 4. Application Dependent Limited Business Process Visibility Technology Dependent Static Implementation Traditional Approaches Vs Vision of EAI EAI EAI focuses on the integration of both businesslevel processes and data, whereas the traditional level middleware approach is data oriented middleware EAI includes the notion of reuse as well as EAI distribution of business processes and data distribution EAI allows users who understand very little about EAI the details of the applications to integrate application application EAI Architecture EAI There are two types of architecture in There existence existence Point to Point Traditional Middleware More modern Point to Point Point Systems communicate Systems directly with each other directly Basic, traditional, easy, Basic, quick quick This approach is good This when few systems are in place place Point to Point Point This solution does not scale up Five systems need 10 integration points Tight coupling, dependency and number of Tight integration points integration There is need for an intermediate layer There middleware middleware Middleware Middleware Can mediate between applications Provide generic interface Five systems have only five integration points Add, replace systems without affecting others Allow applications to pass messages to each other Only the middleware Middleware itself can perform operations such as Middleware routing, transforming, aggregating, separating, converting on the data converting Additional complexity in terms of setting up the Additional middleware and converting the application to use the middleware APIs. middleware Integration Methods Integration Having chosen an architecture( middleware), Having pick an integration methods pick Data level Integration User interface level integration Application level integration Method level integration Data Level Integration Data Backend data stores are integrated Push based: One application makes SQL calls on another Push applications database tables, through database links or store procedure. Data is pushed into another application’s database. database. Pull based: Uses triggers and polling. Triggers capture Pull changes to data and write the identifying information to interface tables. Adaptors poll the application’s interface tables and retrieve the pertinent data The pull based integration is used when an application requires passive notification of changes within another application’s data. notification Data Level Integration Data Used when application does not provide any Used API or client interface API Need a good understanding of the business Need operations that may affect the application’s data model data Typically the only option with most custom Typically applications that lack APIs. applications User Interface Level Integration User Ties integration logic to user interface code Scripting based: The integration code is embedded Scripting into the user interface component events into Proxy based: Uses the integrated applications Proxy interface (through screen scripting) to pass data to and from the system and Used when: Direct access to the database is not easy or Direct possible possible The business logic is embedded in the user The interface interface User Interface Level Integration User Often used in mainframe and client/server Often applications applications Mainframes do not tend to have access to friendly data Mainframes stores, and do not provide public APIs stores, However, generally used as a last resort: Adding scripting logic to catch events with client/server Adding applications difficult to maintain, as integration levels increase and more changes occure increase User interface changes can break integration trigger and User logic logic Tight coupling – permanent link between maintenance of Tight the interface and integration code the Application Level Integration Application Considered the best way forward: Uses the integrated application’s integration Uses frameworks and APIs frameworks Transparent to the integrated application and Transparent preserves the application’s data integrity preserves Application interface allows you to invoke Application business logic to preserve data integrity business E.g Siebel’s Java DataBeans and SAP’s JCA E.g (J2EE Connector Architecture) (J2EE Method Level Integration Method Less frequently used: Specialization of the application level integration Specialization method method Aggregate common operation on multiple Aggregate applications into single application applications Generally used: Each integrated application has a similar set of API Each or functional methods or E.g distributed component or CORBA technology Method Level Integration Method Integrated applications must support a Remote Integrated Procedure Call (RPC) or distributed component technology component The main disadvantages is the tight application The coupling in front components coupling Will break when changes are made to the Will integrated application API integrated These problems will propagate down to the other These applications that rely on them applications Who Uses it ? Who Without industry support, EAI will die Big industry, Market leaders BEA systems IBM Global Systems Accenture PricewaterhouseCoopers The Future of EAI The Still a maturing technology, but with a brilliant Still future: future: The EAI market is set to become the most The important and fastest growing IT sector in the next three to five years three Worldwide revenues in this market will jump from Worldwide $5 billion in 2000 to nearly $21 billion in 2005 $5 Not surprisingly that so many major Not companies are involved!! companies Data – Level EAI Data Data – Level EAI Data It is the lowest level of integration Normally number of tools and techniques exist that Normally allow the integration of information flow from database to database database Does not have any impact in the application / Does business integration business NOTE: It is not Migrating data from one to another NOTE: databases, but for data-level EAI to work, architects and developers need to understand the complexity of database technology as well as the way information flows throughout an enterprise flows Data – Level EAI Data User Interface User Interface Logic Data Logic EAI Data Integration Data Coupling Vs Cohesion Coupling Coupling: To act of bringing or coming together It is binding: Logic with Data Logic with Logic Data with Data It is really binding one application tightly to another It will certainly require changes in all coupled applications It and databases in order to integrate them and Coupling creates one application and database out of many Coupling Vs Cohesion Coupling Cohesion: To act or state of sticking together Logical agreement of multiple application without Logical changing the applications and the databases changing The advantage of cohesion is the ability to share The information between databases and applications without regard for application and database changes changes Coupling Vs Cohesion Coupling Cohesion: The Cohesion approach provides flexibility – The System can be added, changed, or removed from cohesive EAI solution without requiring changes to the other systems. to Generally Message broker provides the technology Generally infrastructure for cohesive EAI solution infrastructure Coupling: If common business processes are to be reused If then coupled approach will provide more value then Data – Level EAI -An Example Data ERP System Customized Inventory System Data 1. 2. Customized Standalone Application Standard database 1. 2. 1. 2. Proprietary Application Proprietary Database or Standard database Type of data movement 1. Event based data movement 2. Periodicity based data movement Implementation approaches 1. Database Replication 2. Message Broker 3. Custom-built Utility Approaches for Data-Level EAI Approaches Database to Database EAI Federated Database EAI Database to Database EAI Database Database to Database EAI means simply Database sharing information at the database level and by doing so integrating the applications by Database to Database EAI can exist in Database One to one One to many Many to many Database to Database EAI Database Two types of solution approaches Replication solution moves information between Replication databases maintaining same basic schema databases Replication and Transformation Using this approach it is possible to move information Using between different types of databases including brands (Oracle, Sybase, Informix etc. ) and Models ( Relational, Object-oriented and Multidimensional) Relational, Federated Database EAI Federated Federated database software is leveraged to allow the Federated developers to access any number of databases using various brands, models, schema through a single ‘virtual’ database model. ‘virtual’ This virtual database model is software and mapped This to any number of connected physical databases to This virtual database is the single point of application This integration integration Federated Database EAI Federated This approach is the reliance on middleware to This share information between applications and not a custom solution not Moreover, the middleware hides the Moreover, differences in the integrated databases from the other applications the Unified Model Database Sources Database Database Sources Database Relational Object – Oriented Multidimensional DBMS Architecture DBMS An Comparative Approach DBMS Architecture DBMS Relational Hierarchical Network Relational DBMS Relational An Concept ANSI/SPARC Architecture - Relational ANSI/SPARC SQL View View SQL View View Base Table(s) View View Table(s) Table(s) Sno S1 S2 S3 S4 Supplier(S) Parts(P) Sname Patil Rao Nayak Roy Status 20 10 20 30 City Mumbai Delhi Mumbai Kolkata Pno P1 Pname Nut Color Red Wgt 12 City Mumbai P2 P3 P4 Bolt Screw Screw Green Blue Red 17 17 12 Delhi Madras Kolkata Table(s) Table(s) Supplier-Parts (SP) Sno S1 S1 S1 S1 S2 S2 S3 S4 S4 Pno P1 P2 P3 P4 P1 P2 P2 P2 P4 Qty 300 200 400 200 300 400 200 200 300 Queries Queries Create Table(s) Retrieving Data Modifying Table Modifying Data Hierarchical DBMS Hierarchical A Concept ANSI/SPARC Architecture - Hierarchical ANSI/SPARC DL/1 PCB PCB DL/1 PCB PCB PCB Data Base Description ( DBD ) PCB Hierarchical Database - DBD Hierarchical Course Course Title Description E23 MIS Management Info E11 DBMS Database Prerequisite Offering Course Des S_Date Venue Dur E10 Comp Int 010216 Mumbai 2 010624 Delhi 1 Participant Faculty Faculty No Name Participant Name Grade 10001 Prakash 20001 Mahesh A 30001 Umesh B Data Base Description (DBD) Data DBD Name = EDPDATA SEGM Name = COURSE, Byte = 256 Field Name = Course, Byte=3, Start=1 Field Name = Title, Byte=33, Start=4 Field Name = Description,Byte=220, Start=37 SEGM Name = PREREQISITE,Parent=Course, Byte= 36 Field Name = Course, Byte= 3, Start=1 Field Name = Title, Byte=33, Start=4 SEGM Name = OFFERING, Parent=Course, Byte=20 Field Name = Date, Byte=6, Start=1 Field Name = Venue, Byte=12, Start=7 Field Name = Dur, Byte=2, Start=19 SEGM Name = Faculty, Parent=Offering, Byte=24 Field Name = Emp No, Byte=6, Start=1 Field Name = Name, Byte=18, Start=7 Hierarchical Database - PCB Hierarchical Course Course Title Description E23 MIS Management Info E11 DBMS Database Offering S_Date Venue Dur 010216 Mumbai 2 010624 Delhi 1 Participant Participant Name Grade 20001 Mahesh A 30001 Umesh B Program Communication Block (PCB) Block PCB Type = DB, DBNAME = EDPDATA, KEYEN=15 SENSEG Name = COURSE, Byte = 256 SENSEG Name = OFFERING, Parent=Course, Byte=20 SENFIELD Name = Date, Byte=6, Start=1 SENFIELD Name = Venue, Byte=12, Start=7 SENSEG Name = Faculty, Parent=Offering, Byte=24 DL/1 Commands DL/1 GET UNIQUE (GU) GET NEXT (GN) GET NEXT WITHIN PARENT (GNP) GET HOLD ( GHU, GHN, GHNP ) INSERT (ISRT) DELETE (DLET) REPLACE (REPL) DL/1 Queries DL/1 1. Get the first offering where Venue is Mumbai GU Course Offering ( Venue =‘Mumbai’) 1. Get all the Participants for the query 1 GU Course Offering ( Venue =‘Mumbai’) Participant NS GN Participant Go To NS Network DBMS Network A Concept ANSI/SPARC Architecture - Network ANSI/SPARC DBTG DBTG Sub Schema Sub Schema Schema Record Type and Set Record Record Type Supplier (S) Set 1:n Shipment ( S-P) Record Type Pointer Setting Pointer MOVE S2 to sno in SS FIND any S using sno in SS Supplier(S) Sno S1 S2 S3 S4 Sname Patil Rao Nayak Roy Status 20 10 20 30 City Mumbai Delhi Mumbai Kolkata Set Related Concepts Set Owner Supplier (S) Set 1:n Shipment ( S-P) Member Membership Class Membership Retention Type Fixed Mandatory Optional Insertion Type Manual Automatic Set Selection is by Application Set Selection is by Value Set Selection is by Structural Insertion is Automatic Insertion •To store S5/P4/700 in SP and modify the set 1. Set Selection is by Application MOVE S5 to sno in SP MOVE P4 to pno in SP Storing data into UWA MOVE 700 to qty in SP MOVE S5 to sno in S MOVE P4 to pno in P …….(a) FIND any S using sno in S FIND any P using pno in P …….(b) STORE SP 1. Set Selection is by Value of sno in S Remove (b) 1. Set Selection is by Structural sno in SP = sno in S Remove (a) and (b) Schema Definition Schema Schema name is supplier-parts. Record name is S; Duplicates are not allowed for sno in S. sno : type is character 5. sname : type is character 20. status : type is fixed decimal 3. city : type is character 15. Record name is P; Duplicates are not allowed for pno in P. pno : type is character 5. pname : type is character 20. color : type is character 6. city : type is character 15. Record name is SP; Duplicates are not allowed for son in SP, pno in SP. sno : type is character 5. pno : type is character 5. qty : type is fixed decimal 5. Schema Definition Schema … Contd. Set name is S-SP Owner is S; Order is sorted by defined keys Duplicates are not allowed. Member is SP; Insertion is Automatic Retention is Fixed. Key is ascending sno in SP; Set selection is by value of sno in S Set name is P-SP Owner is P; Order is sorted by defined keys Duplicates are not allowed. Member is SP; Insertion is Automatic Retention is Fixed; Key is ascending sno in SP; Set Selection is by value of pno in P. DML statements DML FIND – locates an existing record occurrence and establishes it as the current of run unit GET – Retrieve the current of run unit ERASE – Delete the current of run unit STORE – Create new record occurrence and established it as the current of run unit MODIFY – Update the current of run unit CONNECT – Connects the current of run unit into a set occurrence DISCONNECT – Disconnect the current of run unit from a set occurrence. RECONNECT – Disconnect the current of run unit from a set occurrence and connects it into another set occurrence of the same set type Queries Queries 1. Get full details of supplier S4. MOVE S4 to sno in S FIND any S using sno in S GET S 2. Get supplier name and city for supplier S4 FIND S occurrence for S4 GET sname in S, city in S 3. Create the SP occurrence ‘S5/P6/700’ MOVE S5 to sno in SP MOVE P6 to pno in SP MOVE 700 to qty in SP MOVE S5 to sno in S MOVE P6 to pno in P STORE SP Queries Queries 4. Connect the S occurrence for supplier S4 into the SETX occurrence x. FIND SETX occurrence for x FIND S occurrence for S4 CONNECT S to SETX 5. Disconnect the S occurrence for supplier S4 into the SETX occurrence x. FIND SETX occurrence for x FIND S occurrence for S4 DISCONNECT S from SETX Comparison Comparison Comparison Relational Hierarchical Network Data Type Table Segment Record Type External View View PCB Sub Schema Conceptual View Base Table DBD Schema Objected – Oriented Database Database Object-Oriented Database Object-Oriented An object corresponds to an entity in the E-R An model model The object-oriented paradigm is based on The encapsulating data and code related to an object into a single unit object All interactions between an object and the rest All of the system are via messages of Object-Oriented Database Object-Oriented A set of variables that contain the data for the set object; variables correspond to attributes in the E-R model E-R A set of messages to which the object set responds; each message may have any number of parameter of A set of methods, each of which is a body of set code to implement a message; a method return a value as the response to the message value Multidimensional Database Database Multi-dimensional Data “Hey…I sold $100M worth of goods” Dimensions: Product, Region, Time Product, Hierarchical summarization paths R eg io n Product Product W S N Juice Juice Cola Milk Milk Cream Cream Toothpaste Toothpaste Soap Soap 1 2 34 5 6 7 Month Month Product Product Industry Region Country Time Time Year Category Category Region Quarter Product Product City Office Office Month Day Week Data Cube Lattice Data Cube lattice ABC ABC AB AC BC A BC none Can materialize some groupbys, compute others on Can demand demand Question: which groupby’s to materialze? Question: what indices to create Question: how to organize data (chunks, etc) A Visual Operation: Pivot (Rotate) (Rotate) NY NY LA LA SF SF h nt o M Juice 10 Milk Milk 47 Region Cola Crea 30 Crea 12 m 3/1 3/2 3/3 3/1 3/4 3/4 Date Product “Slicing and Dicing” The Telecomm Slice Product Household Telecomm Video Audio gi Re ns o Europe Far East India Retail Direct Special Sales Channel Roll-up and Drill Down Roll-up Higher Level of Aggregation Roll Up Drill­Down Sales Channel Region Country State State Location Address Sales Representative Low­level Details Typical OLAP Queries Typical Write a multi-table join to compare sales for each product Write line YTD this year vs. last year. Repeat the above process to find the top 5 product Repeat contributors to margin. Repeat the above process to find the sales of a product line Repeat to new vs. existing customers. Repeat the above process to find the customers that have Repeat had negative sales growth. What Is OLAP? Online Analytical Processing - coined by Online EF Codd in 1994 paper contracted by Arbor Software* Arbor Generally synonymous with earlier terms such as Decisions Generally Support, Business Intelligence, Executive Information System Support, OLAP = Multidimensional Database MOLAP: Multidimensional OLAP (Arbor Essbase, Oracle MOLAP: Express) Express) ROLAP: Relational OLAP (Informix MetaCube, Microstrategy ROLAP: DSS Agent) DSS * Reference: http://www.arborsoft.com/essbase/wht_ppr/coddTOC.html Reference: The OLAP Market Rapid growth in the enterprise market Significant consolidation activity among major Significant DBMS vendors DBMS 1995: $700 Million 1997: $2.1 Billion 10/94: Sybase acquires ExpressWay 7/95: Oracle acquires Express 7/95: 11/95: Informix acquires Metacube 1/97: Arbor partners up with IBM 10/96: Microsoft acquires Panorama Result: OLAP shifted from small vertical niche to Result: mainstream DBMS category mainstream Conceptual Model for OLAP Conceptual Numeric measures to be analyzed e.g. Sales (Rs), sales (volume), budget, revenue, e.g. inventory inventory Dimensions other attributes of data, define the space e.g., store, product, date-of-sale hierarchies on dimensions e.g. branch -> city -> state Strengths of OLAP Strengths It is a powerful visualization paradigm It provides fast, interactive response times It is good for analyzing time series It can be useful to find some clusters and outliers Many vendors offer OLAP tools Nature of OLAP Analysis Nature Aggregation -- (total sales, Aggregation percent-to-total) percent-to-total) Comparison -- Budget vs. Comparison Expenses Expenses Ranking -- Top 10, quartile Ranking analysis analysis Access to detailed and aggregate Access data data Complex criteria specification Visualization ...
View Full Document

This note was uploaded on 07/15/2011 for the course ECO 2023 taught by Professor Mr.raza during the Summer '10 term at FAU.

Ask a homework question - tutors are online