Today, I have had loads of fun at the
Data Innovation Summit in Brussels, Belgium. Hosted in the beautiful
Axa Belgium offices, it was a great opportunity to meet 500 (!!) data-minded professionals. I was also able to do an
Ignite Talk there, which was quite an experience. 15 seconds for every slide, and no way for you to change the slides yourself and determine the "rythm" - very different. Here are the slides:
But that was not the coolest thing. They also did a "
Data Innovation Survey", which was super cool. The data is all open (find it in this gist), and I of course took it from Excel
create a graph MODEL out of it
and then load it into Neo4j using this
load script. You will need to tweak the load csv file locations, but after that: just
download Neo4j 2.2, fire up the
Neo4j-shell, and paste all the commands into it. Should be a matter of half a minute to load the data.
Then we have the data in Neo4j, and we can start doing some queries. Now, I must admit that I am not a huge fun of working the data this way - as there are very few intricate relationships that we can use meaningfully. Nevertheless, here are a few queries:
//respondents and techniques with PhDs
MATCH (dl:DegreeLevel {name:"PhD"})--(r:Respondent)--(t:Technique)
return dl,r,t
That's easy:
Let's make it a bit more sophisticated:
//respondents and techniques at level 5 with PhDs and their DegreeFields
MATCH (dl:DegreeLevel {name:"PhD"})--(r:Respondent)-[ht:HAS_TECHNIQUE {level:'5'}]--(t:Techniques),
(r)--(df:DegreeField)
return dl,r,t,df
limit 10
You can see how that would make the visualisation a bit more complicated.
And then finally, here is a first attempt at doing something a bit more "graphy". Let's see which "DegreeFields" are the most important in our graph. In other words - the most "Between" the other nodes of the graph. We do that with a query like this:
//betweenness centrality of the "DegreeFields"
MATCH p=allShortestPaths((r1:Respondent)-[*]-(r2:Respondent))
WHERE id(r1) < id(r2) and length(p) > 1
UNWIND nodes(p)[1..-1] as n
WITH n, count(*) as betweenness, labels(n) as labels
WHERE "DegreeField" in labels
RETURN n.name, betweenness
order by betweenness desc;
and then we see this result:
There's a lot of importance to Science/Mathematics, ICT and Engineering. Who would have thought?
You can of course apply these techniques much more generically to other problems, and that is mostly why I share it here. I hope others find it interesting, and as always...
... Feedback welcome!
Cheers
Rik
Some companies are interesting. I mean, I myself have been working in startup environments for (djeez! has it been so long!) decades now, but some large organisations are equally interesting - especially in today's "information age". Every so often I get to meet fascinating people that are working in an industry that is literally being thrown upside down because of the modern technology swell of connectedness, mobile information, demanding customers, and innovative applications. After years, centuries sometimes, of successful business ventures in the "good old days", they find themselves in a place where they are sitting on wonderful assets, with real value, but also facing a growing need to re-assess how it all fits in this new age of digitalism. They need to innovate.
Yesterday, I was part of such an effort. Wolters Kluwer, global publishing powerhouse with a long standing history, headquartered in the Netherlands, organised an Innovation Pitch event for their executive team. Almost all of their board members and execs were there, and I had 10 minutes to "pitch Neo4j". Interesting.