The Role of Knowledge Graphs in Cloud Data Integration

In an era when 75 percent of business executives don’t have a high-level trust in their data, and 70 percent don’t consider their data architecture “world class,” solving challenges around data integration in the cloud is of utmost priority. You need to manage the data correctly and make sense of impossible volumes of information emitted by disparate and seemingly unrelated systems. Even if you’ve discovered and migrated to a cloud data integration platform that works for your industry and the types of data you’re currently collecting, you’re often not able to extract the information they need without additional development efforts.

If you want to correlate the data in two large-scale relational databases, you need a database expert to develop a costly and timely migration process for merging schemas and cleaning up the results. And if you don’t have that talent available, you’re just hoarding data in the cloud for no particular purpose.

Enter cloud-based knowledge graphs (KGs), which have the potential to simplify how organizations ask difficult questions of their integrated data—think fraud detection, large-scale text analysis, cybersecurity prevention, and finding the “unknown unknowns” in your data—without adding undue technical complexity to your stack.

See also: Why Data Integration Is Integral to Successful Enterprises

What are knowledge graphs (and graph databases)?

Let’s start with graph databases, as they’re what knowledge graphs are built on top of.

A graph database differs from a relational database by organizing data not in rigid structures of tables, rows, and columns but rather through a flexible network of entities and their relationships. They connect nodes, entities like people, or any asset that needs to be tracked, with edges, which are the relationships between nodes. Graph databases then apply properties to the relationships.

For example, the graph database Neo4j describes an example graph database with a person node with the value “Tom Hanks,” which is related to a movie node with the value “Forrest Gump.” The property for that edge is ACTED_IN. Other nodes can also be related to the movie node, such as other actors or Robert Zemeckis, who directed it.

Graph databases are not new—they grew in popularity during the 1990s when they were valuable in indexing the web—but recent improvements to their speeds to support more real-time analytics make them an appealing option for integrating relational databases and improving discoverability around otherwise un-analyzed data.

Knowledge graphs are semantic and flexible representations of graph databases, where an organization defines specific ways they want to encode the data to form an ontology. The ontology is roughly equivalent to the data schema that defines a relational database, formalizing the descriptions of various entities and how they’re related to each other. And much like a relational database’s schema, the ontology of a knowledge graph is dependent entirely on the type of business and the data they’re storing in their graph.

But unlike a rigid schema, knowledge graphs need the flexibility to support new forms of data, whether structured or unstructured, by evolving the ontology, formulating new edges, and further enriching the network as soon as data becomes available. This flexibility makes them ideal for integrating disparate data sets and helping both technical and non-technical talent discover connections between disparate sets and results in web-like visualizations that can be explored and further dissected to pick up on new business realities.

KGs are already used heavily in specialized applications that you’ve likely heard of or interacted with, like Google’s knowledge graph for searching the web or the Wikidata graph for connecting Wikipedia entries together, and NASA uses them to create a “knowledge architecture” that helps them control the sheer amount of information that passes around the organization. On a more practical business level, enterprises are using KGs to manage complex customer relationships with many points of contact and roles, help marketing teams discover new content ideas, track particular parts or projects through their lifecycle, and more.

See also: Data Integration Is A Necessity for Public Sector Projects

How do knowledge graphs help with data integration problems?

Traditional analytics tools query data from one or more relational databases and then run analysis synchronously. They’re fantastically useful tools for organizations with the right data management tools, talent for ongoing migrations and optimizations, and have fairly defined goals for making sense of the data they have. But for others, graph databases and knowledge graphs have a number of built-in advantages:

• Less waiting on batch analysis: Because graph databases don’t just store data but also the relationships between any number of entities, they’re able to parse complex queries that use the built-in ontology quickly. In some cases, this means a non-technical returning relevant results within minutes versus an hours-long querying and analytics process that needs to be run overnight and managed by a database administrator.
• Work with both structured and unstructured data: Knowledge graphs work with rigid data models, like a customer’s attributes (name, business, address, phone number, and so on), but also happily store and analyze unstructured data of all sorts, helping you find patterns in complex information that doesn’t get returned with standard SQL queries.
• Tight integrations with natural language processing (NLP): Many knowledge graphs feature NLP functionality during the data import process, converting unstructured data like writing or audio into meaningful entities and relationships. For example, NLP can analyze an online review of your product or service with named entity recognition, sentiment analysis, or summaries for at-a-glance understanding.
• No rigid schemas: As detailed in the previous section, a knowledge graph isn’t hemmed in by how you first describe and deploy it. As your organization integrates new data, it’s fully capable of altering its ontology without requiring altering the rows of a relational database and all the worry that comes from it.

Whether you’re looking toward integrating your data on a cloud platform for the first time, need new ways to extract meaning from what you already have, or are ready to peek into the “unknown unknowns” that trouble all businesses, knowledge graphs might be a meaningful answer. By mastering the relationships between the data created by your many systems and building a “map” of your company’s knowledge, you just might learn to trust in your data again.