Javatpoint Logo
Javatpoint Logo

Managing Metadata

In this section, we will understand how we are managing the Metadata in the Talend studio for a data integration platform.

The Metadata folder is used to store the reusable information on Files, databases, or the systems, which is needed to create our jobs in data integration Talend studio.

This section includes the process of creating and managing the multiple Metadata items in the Repository panel, which can be used in all our job designing.

Go to the Repository panel, and click on the Metadata, as we can see in the below image:

Managing Metadata

In Metadata, we can set up the following connection in the data integration Talend Studio:

  • DB connection
    • MySQL DB connection
    • JDBC DB connection
  • File schema
    • File delimited
    • File positional
    • File Regex
    • File XML
    • File Excel
    • File ldif
    • File JSON
  • LDAP
  • Azure Storage
  • Google Drive
  • Marketo
  • Salesforce
  • Snowflake
  • Generic schemas
  • Talend MDM
  • Rules Management
  • Web Service
  • FTP

We will learn mostly used Metadata connections and schemas to give you a complete understanding of managing the Metadata in Talend studio for a data integration platform.

Db Connection

We must be reading and writing data or both with the help of Databases. Talend studio provides a database components to do these tasks.

In Talend studio's Metadata section, we can connect various types of databases like MySQL, JDBC, SAS, Oracle, SAP Hana, Hive, Teradata, Redshift, Amazon Aurora, Microsoft SQL Server, and so on.

File Schemas

To read and write data into the delimited Files, we have different File schemas like File Delimited, Excel, XML, Positional, Regex, LDIF, and JSON with having different components in Repository's Metadata, which we can use while creating a Job in Talend studio.

File Schemas Description
File delimited It is a text File, which is used to store data where each line has fields separated by the delimiter.
Excel To read and write data to a reliable Excel spreadsheet File, we will centralize the connection to the File, along with its data structure.
XML It is an Extensible Markup Language, which is used to store and transport the data. After centralized the Metadata of XML input and output File, we can directly use the Metadata in our jobs.
Positional It is a File that has a fixed maximum length and records.
The File positional Metadata is used to define the properties of different components like tFileInputPositional ,tFileOutputPositional, and tFileInputMSPositional components.
Regex This kind of File schemas is made of regular expressions, like log Files.
LDIF The LDIF File is a directory File which is expressed by attribute.
JSON It is known as JavaScript Object Notation that is used to serialize and transmit the structured data over a network connection. And it also transfers the data between the server and the web application.

LDAP

LDAP stands for Lightweight Directory Access Protocol that is used to access and manage the directory information, and also reads and edits the directories over the IP networks.

Azure Storage

Azure Storage is a service, which allows us to create a disk in the Disk storage for our virtual machine. The disk can be accessed from one virtual machine only.

Marketo

It is a marketing automation software based on SaaS. It is made to help the companies to automate and measure marketing engagement, tasks, and workflows.

We can also connect multiple data sources in the Repository's Metadata in Talend Studio, and we can explore many characteristics of our data sources.

Talend allows us to use these data sources in our jobs, by dragging an object from the Repository panel to the design workspace window.

Snowflake

The Snowflake is used to store and analyze the data with the help of cloud-based hardware and software. And it is the only data platform built for the cloud where we can store all our data.

Generic schemas

If none of the specific Metadata matches our need or if we do not have any source File to take the schema form, we can create a generic schema in the Talend studio.

And it is used where we do not want to limit the usage of schemas, which is related to any File type or database.

Talend MDM

The Talend MDM is used to combine the real-time data, applications, and integration processes with the embedded data quality to share across on-premises, cloud, and mobile apps.

Web Service

The data integration's Talend studio allows many components to call multiple types of web services like tWebServiceInput, tWebservice [advanced component based on tWebServiceInput], tSOAP [for SOAP Web services] and tREST [for REST Web services].

FTP

FTP stands for File Transfer Protocol that is used to transfer the computer Files between the client and the server on the computer network.


Next TopicDB Connection




Youtube For Videos Join Our Youtube Channel: Join Now

Help Others, Please Share

facebook twitter pinterest

Learn Latest Tutorials


Preparation


Trending Technologies


B.Tech / MCA