Ubiquitous and Invisible Data Mining

Ubiquitous Data Mining (UDM) is the way toward examining information on mobile, embedded and ubiquitous devices. It speaks to the up and coming age of information mining frameworks that will support the intelligent and time-basic data needs of portable clients and will encourage whenever, anyplace, data mining. The basic focal point of the UDM system is to perform computationally concentrated mining procedures in portable situations compelled by constrained computational resources and varying network characteristics.

The far-reaching utilization of cell phones with expanding computational limits and expansion of remote systems is prompting the rise of the ubiquitous processing pattern that encourages consistent access to information and data by versatile clients with handheld gadgets. Ubiquitous computing situations are offering to ascend to another class of utilizations named Ubiquitous Data Mining (UDM), wherein the versatile client performs intelligent analysis and observation of information.

Cell phones' regularly expanding computational capacity presents an open door for insightful information examination in applications and situations where the information is continuously streamed to the gadget and where there are transient imperatives that require investigation anytime, anywhere. UDM is the way toward examining information radiating from appropriated and heterogeneous sources with cell phones or inside sensor arranges and is viewed as the next natural step in the world of ubiquitous computing. Typical application scenarios include the following:

Monitoring a stock portfolio from streamed stock market information while voyaging.
A voyaging sales representative is performing client profiling.
Continuous observing and investigating of status data got for interruption discovery or research centre examinations.
Analysis of information from a sensor in moving vehicles to prevent fatal accidents through early discovery by observing and examining status data.
Performing preliminary mining of information produced in a sensor network.
On-based investigation of astronomical and geophysical information.

Universal computing (ubicomp) is a rising zone of contemporary information technology. Ubiquitous data mining isn't proportionate to performing conventional information mining on a resource-constrained gadget. However, addresses the one of kind needs of utilizations that requires investigation of information in a time-critical and mobile context.

Since its establishment in the late '80s by Mark Weiser of Xerox PARC, the territory has profoundly transformed from a visionary stage to a perceived research discipline. The region covers frameworks (situations) comprising different, heterogeneous components, especially electronic gadgets, which encompass individuals and are available in significantly more prominent numbers than they used to be. The ubicomp frameworks creators go for a possible utility. They are likewise much better coordinated with individuals who utilize them.

They guarantee that the innovation must conform to the general population's needs, not the other way around. The utility along these lines can be, for instance: a fully automatized living spot (Aware Home Project) or simply being non-disruptive (Notification Platform of Microsoft). Besides, ubicomp frameworks probably appear to bolster elderly and disabled individuals sooner rather than later. To accomplish those objectives, a few components of processing frameworks must be escaped clients' eyes and potentially might be incorporated into nature like sensors, cameras, remote servers. Each one of those parts is working serenely in the background. This is the reason ubicomp is frequently referred to as quiet innovation.

Architecture of Ubiquitous Technologies

Ubiquitous Data Mining involves collecting and storing data, processing data, and disseminating the analyzed result. To achieve this, we use six parts, which make up the architecture of the ubiquitous system.

S.No	Parts	Function
1	Devices	The component that is used for storing and processing the data. E.g., Personal Computers, Super Computers
2	Communication	The mode in which devices communicate with each other. For example, Internet, via Centralized System
3	Users	The user who would interact with the system. It can have a Single User or can have Multiple Users.
4	Control	The component that administers all the parts mentioned above. It can either be controlled by a single administrator or by multiple administrators.
5	Data	The type of data that is stored and processed by the system. It also gives the implication to its dynamics and organizations. The data can be of the following types: Static data Dynamic Data
6	Infrastructure	The infrastructure that the system employs for data discovery. For example, Web, Database.

Application of Ubiquitous Data Mining

Below are the following Applications of Ubiquitous Data Mining, such as:

1. Traffic Safety: Abnormal traffic can be detected using sensors, and the data can be stored and analyzed in the system. This analyzed data can then be used to detect traffic mishaps on a real-time basis using sensors. Thus, traffic and road safety is monitored.

2. Health Care: The sensors can be used to create smart homes for elderly persons and people who need continuous medical attention. The sensors can be used for notifying the immediate medical need and would be collected. The data so collected can help to provide them with timely medical aid.

3. Crisis and Calamities Management: The previous crisis and calamities data are collected, stored and analyzed. During times of crisis, sensors can detect the crisis, and the data is sent to the controllers. Results are predicted before the effect can get disastrous. Thus, helps in Crisis Management.

4. Market basket analysis: This modelling technique uses hypothesis as a basis. The hypothesis says that if you purchase certain products, then it is highly likely that you will also purchase products that don't belong to that group that you usually purchase from. Retailers can use this technique to understand the buying habits of their customers. Retailers can use this information to change their store's layout and make shopping easier and less time-consuming for customers.

5. Customer relationship management (CRM): CRM involves acquiring and keeping customers, improving loyalty, and employing customer-centric strategies. Every business needs customer data to analyze it and use the findings to build a long-lasting relationship with their customers. Data mining can help them do that.

6. Finance and banking: The banking system has been witnessing the generation of massive amounts of data from the time it underwent digitalization. Bankers can use data mining techniques to solve the business's baking and financial problems by finding correlations and trends in market costs and business information. Managers in the banking and financial sectors can use this information to acquire, retain, and maintain a customer. This job is too difficult without data mining as the volume of data that they are dealing with is too large.

7. Fraud detection: Fraudulent activities cost businesses billions of dollars every year. Every ideal fraud detection system needs to protect user data in all circumstances. Methods that are usually used for detecting fraud are too complex and time-consuming. Data mining provides a simple alternative. A method is supervised to collect data, and then this data is categorized into fraudulent or non-fraudulent data. This data is used in training a model that identifies every document as fraudulent or non-fraudulent.

8. Monitoring Patterns: Known as one of the fundamental data mining techniques, it generally comprises tracking data patterns to derive business conclusions. It could mean anything from identifying sales upsurge or tapping newer demographics for an organization.

9. Manufacturing engineering: A manufacturing company relies a lot on the data or information available to it. Data mining can help these companies identify patterns in processes that are too complex for a human mind to understand. They can identify the relationships between different system-level designing elements, including customer data needs, architecture, and portfolio of products. Data mining can also prove useful in forecasting the overall time required for product development, the cost involved in the process, and the expectations companies can have from the final product.

10. Business Transactions: Every business industry is memorized for perpetuity. Such transactions are usually time-related and can be inter-business deals or intra-business operations. The effective and in-time use of the data in a reasonable time frame for competitive decision-making is the most important problem for businesses that struggle to survive in a highly competitive world. Data mining helps to analyze these business transactions and identify marketing approaches and decision-making. For example,

Direct mail targeting
Stock trading
Customer segmentation
Churn prediction (Churn prediction is one of the most popular Big Data use cases in business)

11. Scientific Analysis: Scientific simulations are generating bulks of data every day. This includes data collected from nuclear laboratories, data about human psychology, etc.Data mining techniques are capable of the analysis of these data. Now we can capture and store more new data faster than we can analyze the old data already accumulated. Example of scientific analysis:

Sequence analysis in bioinformatics
Classification of astronomical objects
Medical decision support.

Invisible Data Mining

Data mining is present in all the major aspects of our life. This requires effective mining of data without disclosing private information via data extraction to outsiders. The invisible data is the human side of data. It isn't easy to comprehend, not easily understood, not well defined, and is subjective. We believe that data has no emotions, belief systems, culture, biases or prejudices. But data is completely useless unless we, human beings, can interpret and analyze it to make decisions.

But unlike data, human beings are full of emotions, cultural limitations, biases and prejudices. This human side is a critical component of the invisible data. This may come as a surprise to many readers. Still, the invisible side of data is sometimes more critical than visible facts when it comes to making impactful decisions and policies.

More and more systems should have data mining functions built within so that people can perform data mining or use data mining results simply by mouse clicking, without any knowledge of data mining algorithms. Intelligent search engines and Internet-based stores perform invisible data mining by incorporating data mining into their components to improve functionality and performance.

For example, when purchasing items online, users may be unaware that the store is likely collecting data on the buying patterns of its customers, which may be used to recommend other items for purchase in the future.

Challenges of Invisible Data

Invisible data has critical components that all data scientists and policymakers should know. Typically, the invisible data is either ignored, marginalized or misunderstood. I have identified four challenging areas of the invisible data, such as:

Human Evolutionary Limitations: Our biases, fallacies, illusions, beliefs etc.
Brute Data Environments: Complex issues, cancer research, climate change.
Data Mirages: Black swans, statistical anomalies, data tricks etc.
Technology Advancements: Free will, consciousness, data ownership.