Informatica Applies AI to Big Data Governance

Informatica at the Strata Data Conference today announced a data management platform that makes use of machine learning and other artificial intelligence (AI) technologies to enforce compliance policies across an entire data lake created on a Big Data platform such as Hadoop.

Based on an Informatica CLAIRE engine for analyzing metadata, Ronen Schwartz, senior vice president and general manager for data integration and cloud integration at Informatica, says the Informatica Compliance Data Lake can identify types of data sets and then determine what compliance policies should be enforced. Based on those policies, the Informatica Compliance Data Lake will limit access to those data sets or implement rules such as masking personally identifiable information (PII), says Schwartz.

Schwartz also says Informatica Compliance Data Lake will generate a catalog of all the data stored in the data lake.

Today, Schwartz says, most organizations unfortunately don’t know what data they have or who is accessing it for what purpose. Beyond complying with new regulations such as the General Data Protection Rule (GDPR) being implemented by the European Union, Schwartz says the Informatica Compliance Data Lake can be employed to increase rather than decrease usage of data because it eliminates the need for someone in IT to manually manage a permission process that often does more to hinder usage of data than it does to enable it.

“Organizations now refer to data as being the new oil,” says Schwartz. “The catalog then is a treasure map.”

Data governance has become a critical issue inside most organizations at a time when Big Data platforms based on Hadoop and other platforms are making more data accessible throughout an organization. As a result, more individuals than ever can access sensitive data. The challenge now is to strike a balance between being able to mine all that data against compliance requirements intended to limit who can access that information.