From architectural design perspectives, the underlying platform infrastructure needs to fulfill the following main requirements to support the design principles of networked business objects and social augmentation as outlined above:
a. Efficient insert, update and read operations - in particular networked business objects tend to result in update-intensive operations because of the evolutionary status evolvement of n-BOs with corresponding attribute extensions.
b. Handling of massive volumes of supply network data, either structured or unstructured data.
c. Possibility to interlink structured and unstructured data efficiently in the user interface, programming model and data store, both from the transactional (OLTP) and the analytical (OLAP) point of view.
d. Efficient handling of hierarchical structures, for instance for fast search and navigation operations in complex supply network structures (e.g. for supply network browsing in dependencies of multi-tier supply networks; see supply network browser wireframe in Appendix Figure A4).
e. Real-time system response experience both for OLTP and for OLAP.
To fulfill this technical requirement, the DSR team evaluated SAP’s in-memory database ‘High-Performance Analytic Appliance’, SAP HANA as a potential platform for implementation of the artifact and in particular selected SAP HANA as the underlying platform for the envisioned design product. This was done for the following reasons (Plattner 2009):
SAP HANA is a commercially available, mature and market-leading in-memory database technology, which can be deployed in on-premise and cloud-based infrastructures. It is able to handle supply network data of large companies, for instance with more than 100 million sales transactions per year in main memory, consequently bypassing the bottleneck of time-intensive hard disc accesses. This is done by loading transaction-relevant data directly, with low latency, from main memory for buffering into CPUs caches and for processing in CPU cores (access and loading times from main memory to CPUs cache less than 100 ns compared to hard disk to main memory on average approximately 1 ms).
The SAP HANA in-memory computing engine (IMCE) postulates additional columns storage with the advantages of higher data compression compared to classical row store (by approximately factor 10). In addition HANA enables faster, parallel processing of the most critical operations in OLTP and OLAP: read, update, search, insert and full table scan with numerical calculations (actually only app. 10% of table attributes are normally used and update operations are relatively rare but expensive in terms of performance).
locking issues because of better support of parallel processing of columns scans. Today’s high-performance blades for example run with up to 8 CPUs, 16 cores each, in total 128 computing units, and 500 GB to 1 TB of main memory per blade.
Depending on the programming model, the IMCE supports the avoidance of performance expensive update operations and uses full tuple inserts instead, including timestamp and user identification. This also makes it possible to keep the complete tuple history accessible in main-memory for analytical computations via column scans (for trend analysis and predictive forecasting for example).
Memory consumption is significantly reduced by up to factor 100. This is thanks to a variety of factors: better compression, avoidance of indexes (all attributes in column stores function by default as indexes), no statistical aggregate tables and high compression of historical data (fixed tuples need no updates anymore; for example fixed order tuples after payments).
In addition to in-memory database features, SAP HANA offers rich platform capabilities, such as pre-defined stored procedures (Application Function Library (AFL), including Predictive Analysis Library (PAL) and Business Function Library (BFL)), Multi-Dimensional Expression Language (MDX) for supporting ragged hierarchies of aggregation nodes. It also offers the Extended Application Services (XS- Engine) for development of lightweight to complex native apps, directly utilizing in- memory capabilities, and user interface frameworks for rich http-based front ends for browser desktop and mobile rendering.
The Text Analysis (TA) libraries embedded in SAP HANA provide natural language processing (NLP) capabilities for extractions like sentiments, topics, products, problems and requests from unstructured text data. Together with the original unstructured data, these extractions are stored in a structured format to enable analytical insights into unstructured data as well as transactional follow-up processing, such as segmentation, contract creation, lead generation and campaign management.
As described above, the artifact design principles tend to result in update-intensive operations compared to update and insert-intensive operations in classical procedural approaches. Taking the example of the quote-to-invoice use case, a networked business object (n-BO) of type ORDER is instantiated once the unstructured interactions of supply network business partners result in the first structured data set at a certain point of their collaboration. Leaving out the quote state for this example, the buyer and supplier would agree on the n-BO order state with the negotiated terms and conditions. The evolution to the next states of goods issue and receiving, as well as to invoice creation and verification would not lead to insertion of a new tuple but to an update of the same n-BO of type ORDER. This update would include adding further attributes to the original order tuple. These attributes would hold related data for the goods issue, receiving and the subsequent invoice.
The same use case in the classical procedural approach would lead to an update of the original purchase (PO) and sales order (SO) records in terms of goods issues (GI),
5 Artifact 70
receiving (GR) and invoice (IV) data and respective secondary key references in the PO and SO records. The corresponding good issue, receiving and invoice records are also created in both the buyer (for example SRM) and the supplier (for example CRM) systems. This example demonstrates once again the reduction of the necessary data and operations by approximately 50% with the evolutionary approach of the n-BO design principle.
Looking at unstructured data exchanged between buyer and supplier during this sample business transaction, for instance by exchanging instant messages or feeds; this would extend the original unstructured and structured data set.
One of the main reasons for implementing the artifact design based on the SAP HANA platform is its suitability for update-intensive business transactions, not presuming the complete tuple (where only view attributes are effected), and for adding new attributes to tuples (whether structured or unstructured data) within the column store of HANA. Other reasons beside the attribute evolution are that the method evolution is better supported by SAP HANA when n-BOs move from one state to the next, and the multi- access features, when multiple supply network users perform parallel tasks on specific attributes of the same shared n-BO tuple. It is expected that the implementation of data related methods, whether via the XS-Engine or via stored procedures, and the better possibilities of parallel computing of SAP HANA’s column stores with less locking issues, supports the n-BO approach better. Or the other way around, the artifact design principles are well suited for in-memory platforms like SAP HANA.
The overall architectural picture for building the B-Zone artifact and target architecture for the design product as a native SAP HANA application is presented in Figure 19.
side) and the database tier (server side). The first tier concentrates on highly usable, flawless user interface representations of structured and unstructured data (OLTP), combined with analytical capabilities (OLAP). The application logic itself is either pushed up to the presentation tier for lightweight operations, close to the screen representation (scrolling, sorting, data entry), or pushed down to the database tier for more extensive operations close to the HANA database, implemented with server-side Java Script coding based on XS Engine, utilizing stored procedures, or even implementing new stored procedures with C++ code where appropriate.
The front end development of B-Zone is performed using the HTML5 based UI frameworks UI5 and UI Toolkit of SAP HANA. UI5 is a rich user interface development environment, which offers a variety of out-of-the box controls and both transactional and analytical functionalities. The UI Toolkit offers intuitive, innovative functions and controls in particular for search and analytical capabilities. Both frameworks are based on the Information Access (INA) service of the XS Engine, which enables JSON and OData data access and transfer via synchronous HTTP requests. Alternatively, JDBC and ODBC are supported but not utilized in the B-Zone implementation because of the available features of the XS Engine services XS- ODATA, INA, and XS-JS (client and server side Java Script).
MDX and XS-XMLA (XML for analytics) will be used for complex hierarchical analytical views and aggregations, for example for the supply network browser.
The XS Engine is a lightweight and stateless application server. One of the basic concepts of this architecture is to improve the performance of data processing by reducing the data volume transferred between database and application layer. All calculations and aggregations are performed directly in SAP HANA to prevent unnecessary overhead caused by the data transfer between the layers. To achieve this, the application logic of B-Zone is developed directly in SAP HANA using XS services. The various application functions thus have direct access to the capabilities provided by the platform, for instance the utilization of functions in the BFL and the PAL, or in- memory data management. The application logic can be accessed via HTTP requests directly from desktop web browser or mobile front ends (Android, Blackberry, iOS). Structured n-BO data and related unstructured data is stored and processed by the column storage management of SAP HANA, utilizing Core Data Services (CDS) like data definition language and query language, as well as data modeling and operation logic, like views (attribute, analytical and calculation views) and SQL Script.
The integration for data transfer from and to SAP solutions, for example with SAP Business Suite on-premise installations is achieved by SAP Landscape Transformation (SLT). One example is the transfer of relevant invoice data after the invoice has been finally approved in B-Zone, to the financial systems of buyers and suppliers for accounts payable and accounts receivable bookings.
5 Artifact 72
The integration for data transfer between B-Zone and non-SAP systems is achieved by the extraction, transfer and loading (ETL), and data cleansing and consolidation capabilities of SAP Data Services.
From the data and programming model perspective, returning to the above quote-to- invoice example, the conceptual view of the data model in the classical procedural case could be illustrated schematically as follows (in Figure 20).
Figure 20. Conceptual View of Classical Procedural Data Model
Accordingly, only looking at the buy side, the evolvement of the use case from purchase order (PO) to goods receipt (GR) to invoice (IV) results in multiple inserts of tuples, with duplication of data to a large extent, and updates on the predecessor tuple for cross referencing with secondary keys (SK) to the successor tuple. Taking the sell side into account too, this even results in two-fold growth of the data sets and necessary tuple updates. The same use case with the n-BO approach can be illustrated schematically as follows (in Figure 21).
Figure 21. Conceptual View of n-BO Data Model
Obviously, the n-BO design principle will lead to a significantly reduced data set, no data redundancies and reduction of secondary keys. Partial deliveries (multiple goods receipts) can also be handled with this model by adding additional attributes for the partial delivery data to the n-BO tuple. The case of collective invoices can be handled for example by adding secondary key attributes of the related n-BO order states to the collective invoice state.
Another advantage of the n-BO data model in the context of in-memory column stores is that no space is reserved for un-populated attributes as long as the successor state of
for quotes never turning into orders). New columns covering data of the successor n-BO state can be added in a column store quickly and easily. Figure 22 illustrates this by a schematical representation of three n-BOs of type ORDER in a row and in a column store (one column stores one attribute).
Figure 22. Schematical Main Memory View of n-BOs in Row and Column Store
Conventional databases store all types of data records in row stores. Storing n-BOs in columns enables faster state, attribute and methods evolution when moving from one state to the next along the supply network collaboration, because inserting new attributes is simple. The column stores also allow for fast in-memory processing of attribute aggregates by parallel memory access and processing in linear column scans.