Confusing words in Data Modeling

For a few weeks now, I have been helping out my friends at Hackolade with some really interesting work around their core product, the Hackolade Studio and how to make it even more successful in the marketplace. This means talking to a LOT of people - both current customers, active users, partners, friends in the industry - and more. It’s been fantastic to talk to so many interesting people, and to learn so much from their insights.

During these conversations, I have noticed that there’s quite a bit of work to be done to clarify and straighten out the meaning of the words that we use in these conversations. I have noticed that in the NOSQL data modeling space, we are not always very precise with our words - and that this imprecision can lead to all kinds of misunderstandings. Specifically, I have been struck by the confusion around 3 words: model, schema, and metadata.

Graphing the Tour de France - part 3/3

In the past two blogposts I have been creating and importing some nice Tour de France 2016 data. It's a small dataset, for sure, and this is by no means a realistic graph application - but perhaps we can still have some fun exploiting the data with some cypher queries. That's what we'll try now. I have put all of the example queries together in this gist, so please feel free to play around with it :) ... let's take you through it.

Is the model really there?

First and foremost, let's verify the model that we wanted to put in place, with yet another AAPOC (Awesome APOC). We thought we were going to get this model:

Hidden GraphGems: the Meta-graph

Over the past couple of days, I have had multiple occasions where I had to explain a graph model to an audience. Often times I would end up drawing stuff on paper/whiteboard (still my favourite, tbh), sometimes I would revert back Arrows, but more and more I am in a situation where I need to explain the model of an **existing Neo4j database**. So what then?

Well, that's when I pull out a old hidden gem, as explained and handed to me by Michael, of course. He explained it very well on this graphgist, but let me try to take you through it.

Starting from a standard query

Every Neo4j instance that has the Neo4j browser (so since version 2.0), has had the following query shipped with it under the "favourites": what is related, and how.

 // What is related, and how  
 MATCH (a)-[r]->(b)  
 WHERE labels(a) <> [] AND labels(b) <> []  
 RETURN DISTINCT head(labels(a)) AS This, type(r) as To, head(labels(b)) AS That  
 LIMIT 10  

Essentially what this does is

• look at all the (a)-[r]->(b) patterns
• ensure that the nodes have a label
• and then take the labels of (a) and the labels of (b), and the type of [r]

that gives you a simple table that explains the model of the database:

But what if I want to do what I said before - and really show the model, graph-ically? Well, as explained in Michael's gist, you can tweak the query above to do that.

Tweaking the query to generate the META-graph

Here's what you do to the query:

 // generate the META-graph  
 MATCH (a)-[r]->(b)  
 WITH labels(a) AS a_labels,type(r) AS rel_type,labels(b) AS b_labels  
 UNWIND a_labels as l  
 UNWIND b_labels as l2  
 MERGE (a:Node:Meta {name:l})  
 MERGE (b:Node:Meta {name:l2})  
 MERGE (a)-[:OUTGOING]->(:Relationship:Meta {name:rel_type})-[:INCOMING]->(b)  
 RETURN distinct l as first_node, rel_type as connected_by, l2 as second_node  

Let's take you through that too:

• we look at the same (a)-[r]->(b) pattern as above
• we collect all the labels of (a), types of [r] and labels of (b)
• we unwind these into l and l2
• and then start looping through these using MERGE: we create a node (a) for every new label l, a node (b) for every new label l2, and leave them alone if they already exist
• and then we also create a node for every relationship between a and b. Note that this may take a bit of getting used to in reading the model: relationships are expressed as nodes, and connected to labels with an "outgoing" and "incoming" relationship.

Overall this yields a very readable model diagram: all the META-nodes have a label "Meta", and that makes for easy visualisation. I my example below, the yellow nodes are "labels" and the blue nodes are "relationships". Very readable, I believe.

To me, this is standard equipment for dealing with existing graph models/databases.

Hope you like it, and that it is useful for you too. Massive kudos to Michael, of course - but that almost goes without saying.

Cheers

Rik