What is Meta Data?
Metadata is data about the data or documentation about the information which is required by the users. In data warehousing, metadata is one of the essential aspects.
Metadata includes the following:
Metadata is used for building, maintaining, managing, and using the data warehouses. Metadata allow users access to help understand the content and find data.
Several examples of metadata are:
Why is metadata necessary in a data warehouses?
Metadata is Like a Nerve Center. Various processes during the building and administering of the data warehouse generate parts of the data warehouse metadata. Another uses parts of metadata generated by one process. In the data warehouse, metadata assumes a key position and enables communication among various methods. It acts as a nerve centre in the data warehouse.
Figure shows the location of metadata within the data warehouse.
Types of Metadata
Metadata in a data warehouse fall into three major parts:
As we know, data for the data warehouse comes from various operational systems of the enterprise. These source systems include different data structures. The data elements selected for the data warehouse have various fields lengths and data types.
In selecting information from the source systems for the data warehouses, we divide records, combine factor of documents from different source files, and deal with multiple coding schemes and field lengths. When we deliver information to the end-users, we must be able to tie that back to the source data sets. Operational metadata contains all of this information about the operational data sources.
Extraction and Transformation Metadata
Extraction and transformation metadata include data about the removal of data from the source systems, namely, the extraction frequencies, extraction methods, and business rules for the data extraction. Also, this category of metadata contains information about all the data transformation that takes place in the data staging area.
The end-user metadata is the navigational map of the data warehouses. It enables the end-users to find data from the data warehouses. The end-user metadata allows the end-users to use their business terminology and look for the information in those ways in which they usually think of the business.
Metadata Interchange Initiative
The metadata interchange initiative was proposed to bring industry vendors and user together to address a variety of severe problems and issues concerning exchanging, sharing, and managing metadata. The goal of metadata interchange standard is to define an extensible mechanism that will allow the vendor to exchange standard metadata as well as carry along "proprietary" metadata. The founding members agreed on the following initial goals:
Metadata Interchange Standard Framework
Interchange standard metadata model implementation assumes that the metadata itself may be stored in storage format of any type: ASCII files, relational tables, fixed or customized formats, etc.
It is a framework that is based on a framework that will translate an access request into the standard interchange index.
Several approaches have been proposed in metadata interchange coalition:
In a procedural approach, the communication with API is built into the tool. It enables the highest degree of flexibility.
In ASCII Batch approach, instead of relying on ASCII file format which contains information of various metadata items and standardized access requirements that make up the interchange standards metadata model.
In the Hybrid approach, it follows a data-driven model.
Components of Metadata Interchange Standard Frameworks
1) Standard Metadata Model: It refers to the ASCII file format, which is used to represent metadata that is being exchanged.
2) The standard access framework that describes the minimum number of API functions.
3) Tool profile, which is provided by each tool vendor.
4) The user configuration is a file explaining the legal interchange paths for metadata in the user's environment.
The metadata itself is housed in and controlled by the metadata repository. The software of metadata repository management can be used to map the source data to the target database, integrate and transform the data, generate code for data transformation, and to move data to the warehouse.
Benefits of Metadata Repository