Partitioning in HiveThe partitioning in Hive means dividing the table into some parts based on the values of a particular column like date, course, city or country. The advantage of partitioning is that since the data is stored in slices, the query response time becomes faster. As we know that Hadoop is used to handle the huge amount of data, it is always required to use the best approach to deal with it. The partitioning in Hive is the best example of it. Let's assume we have a data of 10 million students studying in an institute. Now, we have to fetch the students of a particular course. If we use a traditional approach, we have to go through the entire data. This leads to performance degradation. In such a case, we can adopt the better approach i.e., partitioning in Hive and divide the data among the different datasets based on particular columns. The partitioning in Hive can be executed in two ways - Static PartitioningIn static or manual partitioning, it is required to pass the values of partitioned columns manually while loading the data into the table. Hence, the data file doesn't contain the partitioned columns. Example of Static Partitioning
Here, we are partitioning the students of an institute based on courses.
In the following screenshot, we can see that the table student is divided into two categories.
In this case, we are not examining the entire data. Hence, this approach improves query response time.
Next TopicDynamic Partitioning
|