Supply Chain Management with graphs: part 1/3 - data wrangling and import

Alright, I have been putting the writing of this blogpost off for too long. Finally, on this sunny Saturday afternoon where we are locked inside our homes because of the Covid-19 pandemic, I think I'll try to make a dent in it - I have a lot of stuff to share already.

The basic idea for this (series of) blogpost(s) is pretty simple: graph problems are often characterised by lots of connections between entities, and by queries that touch many (or an unknown quantity) of these entities. One of the prime examples is pathfinding: trying to understand how different entities are connected to one another, understanding the cost or duration of these connections, etc. So pretty quickly, you understand that logistics and supply chain management are great problems to tackle with graphs, if you think about it. Supply Chains are graphs. So why not story and retrieve these chains with a graph database? Seems obvious.

We've also had lots of examples of people trying to solve supply chain management problems  in the past. Take a look at some of these examples:

• https://neo4j.com/graphgist/supply-chain-management
• https://neo4j.com/blog/graph-technology-supply-chain-transparency-corporate-social-responsibility/
• https://neo4j.com/news/graphing-the-supply-chain/ and https://www.enterprisetimes.co.uk/2019/09/10/graphing-the-supply-chain/
• https://neo4j.com/blog/nlp-at-scale-maintenance-supply-chain-management/

And of course some of these presentations from different events that we organised:

• Our friends at Caterpillar used Neo4j for this:
• TransparencyOne actually built a business on it:

So I had long thought that it would be great to have some kind of a demo dataset for this use case. Of course it's not that difficult to create something hypothetical yourself - but it's always more interesting to work with real data - so I started to look around.

Graphistania 2.0 - Episode 5 - This Month in Neo4j

Friends.

These are interesting times. These are difficult times, but we can deal with it together, as a community, as a graph. So that's why we were super happy that, just as Belgium was going into lockdown last week, we were able to record another Graphistania podcast episode for you, talking about the world in general, but also covering some of the amazing graph use cases that drifted over our screens in the past month, in the This Week in Neo4j (TWIN4J) newsletter.

There were actually many things to talk about, in terms of fascinating graph use cases, and I will highlight only the most striking ones here.

Our friends at Kineviz did some really interesting and timely work on  COVID-19 temporal and spatial data visualization. This stuff is really important to understand, as pandemic spreads clearly follow graph patterns. Read Connected if you are not convinced. 

Worth highlighting: Bloodhound: Windows network penetration testing with Neo4j, had a new release that you might want to take a look at. If you are not familiar with Bloodhound yet, you may also want to check out my interview with the Bloodhound crew on this podcast a while back. 

We published this fun little thing called a Neo4j Treasure Map - check it out! 

Finally - we also have a a Winegraph! It's a great example of importing data from the web using Norconex.  

Some interesting stuff on using Neo4j for Gene ID mapping: take a look! 

Another examle of enriching graphs with Wikidata, from the one and only Mark Needham: look at Mark's blog over here! 

Don't forget: we Introduced the Neo4j Graph Data Science plugin with examples from the "Graph Algorithms" book. 

A really interesting tweet about a visualisation of the US Supreme court as a graph db... Would love to see more like that. 

And for some fun: Pokégraph: Gotta Graph 'Em All! 

Some important stuff: we did a great 4.0 webinar that is giving you a lot of info on what to expect in the new version of Neo4j.  

There was a great update to NeoMap: Visualizing shortest paths with neomap ≥ 0.4.0 and the Neo4j Graph Data Science plugin.

Those were the most important ones. So let's talk about these now - I am sure there's a lot of cool stuff here fore everyone!

Graphistania 2.0 - Episode 4 - This Month in Neo4j

Yey! My friend StefanW and I got round to recording another Graphistania episode, episode 4 already - time flies when you are having fun! This month, again, we have so much great content popping up in the This Week in Neo4j (Twin4j) newsletter, that we could probably fill a few hours talking about it. So in the podcast, we will only talk about a handful - covering things like

• the great momentum that we have in the Neo4jcommunity! Take a look at initiatives like Global Graph Celebration Day, and listen to Neo4j's mad scientist Michael Hunger on graphs, databases and relationships on a different podcast.
• the start of the Neo4j Graphtour, for the 3rd year already, in Amsterdam. This year this actually coincided with the launch of Neo4j 4.0 - one of the best releases of Neo4j that I have personally ever witnessed. I have actually been writing a bit about this myself - read about how I how I child-proofed my beergraph with fine-grained security and how I did something similar by adding security to a fraud investigation graph. There's other articles as well, like When and how to implement Sharding in Neo4j 4.0.
• Various personal graphs projects, like Mark's Australian Open graph and the QuickGraph #3 on Itsu Allergens. I love how Mark summarized it: most of analysis could be done in relational, but "Things got more interesting in the last section where we did set analysis. I found having the data in a graph structure made was helpful for answering these questions, especially when we were looking for the non existence of a relationship."
• a number of Health related articles, like that visualisation of the data from the Personal Genome Project, or the Google project wiuth the “Largest Ever” Map of Brain Connectivity
• And then of course there were various "Other" posts that we really liked, like how to be Working With Spatial Data In Neo4j GraphQL In The Cloud, and the post about Aaia - AWS Identity And Access Management Visualizer And Anomaly Finder. Last but not least, you should also take a look at the Graphaware Hume platform - with an excellent powerful demo recording over here.

Experimenting with Conflicting access privileges in Neo4j 4.0

In the past couple of weeks, I have been playing around with the shiny new security features of Neo4j 4.0. They are truly interesting - both for childproofing beergraphs and for ensuring that your sensitive fraud databases are properly secured. Take a look at the previous post, and I think you will understand why.

In this post, I wanted to talk about something that I have seen so many times in my previous lives in the security industry, and that also became evident in my 4.0 research. It's got to do with conflicting security privileges. In a nutshell, this is to do with the case where

• a specific user / role would receive a particular set of privileges from one policy
• the same user / role would receive a different, and contradictory privilege from another policy. 

In that case, we need clear rules to understand what would happen. In the case of Neo4j 4.0, this is reasonably well explained as part of the documentation - see the documentation site on this topic - but in this post I will try to give you a realistic, but simple example.

Creating Conflict

We'll start working on this with the same database as the previous post, the fraud dataset. If you don't have it yet, just download it from this link. Once we have the database up and running as a separate user database, we can switch to the system database and create a separate user for these tests.

//create a separate user for engineering the conflicting privileges
CREATE USER conflicted_user SET PASSWORD "changeme" CHANGE NOT REQUIRED;
CREATE ROLE conflicted_role AS COPY OF reader;

Securing a sample fraud graph with Neo4j 4.0

This week, we at Neo4j formally released our brightest and shiniest new version of the Neo4j Graphg Database to the world. It's been an amazing journey to this point, and others have reported on this magnificent piece of engineering in more depth. Take a look at Jim's blogpost, or if you are in a hurry, checkout the graphcast below:

Last week, I started playing around with it myself - by digging up my good old faithful beergraph, and illustrating some of the new features in childproofing exercise for beers. Take a look at that post as well for some giggles. Now in this post, I wanted to essentially do the same thing as I did on the beergraph, but using a Fraud dataset. 

Let's see how that would work.

Securing my Beergraph with Neo4j 4.0

Not sure if you have realised, but Neo4j has actually recently made the 4.0 version of the most fantastically awesome graph database on the planet available. You can get it ahead of the big launch event (on February 4th, 2020 - in case you were wondering!) from the Download Center and take it for a spin.

In this unbelievable release, there are so many new features, it's kind of hard to keep track of everything. But the ones that I can most easily get my head around are clearly

1. multi-database support - finally, Neo4j actually has this concept of running multiple databases on one database server. A multi-tenancy solution, that has been requested and anticipated by many of our users and customers. 
2. a VERY advanced schema-based security module, that allows people to extend the existing role-based security model of Neo4j even further - and make it crazy powerful. We'll spend a lot of time on that in this blogpost.

Readers of this blog probably know that I am a big fan of getting my feet down and dirty with our products, so this evening - with a couple of hours to spare, so to speak - I decided to try out the shiny new release. I spun up my Neo4j Desktop, and started reading some manual pages where stuff was explained. Specifically, I loved

• the manual pages on managing multiple databases
• the pages on authentication and authorisation
• a great video by Louise Söderström from Neo4j engineering:

Soon after flipping through this, I was on my way.

Graphistania 2.0 - Episode 3 - This Month in Neo4j

Happy new year everyone - although it actually seem like the holidays are already very far behind us! But great times were had, at least in my family, and so I feel super energised to make 2020 another great start to a decade of graphs :) ... Here's to that!

It also means that we are continuing to see all these awesome community stories pop up left right and center in the Neo4j "This week in Neo4j" developer newsletter. And so on our Graphistania podcast, we are going to continue talking about these on a monthly basis. So that's what we're doing - and I have again invited my friend and colleague Stefan Wendin to join me.

From the newsletter, we always select a few stories that we think will be more interesting and/or meaningful to discuss. This month, we found a number of them, and the interesting thing was that the graph-stories seemed to play at very different scales... The Personal, Corporate, and Society levels. Here are some of the ones we liked:

At the Personal scale

• Alex Woolford - Network analysis with Neo4j / Kafka / Zeek. Alex also has this really cool video on "Event driven parenting": if your kid is getting bad grades, that event will lead to no more PS4 network access :) ... Bit harsh - but still!

At the Corporate scale

• IT centric example: Managing VMware infrastructure with Neo4j - great example of how to understand infrastructure dependencies in an IT environment with graphs.
• Business centric examples: Analysing online customer journeys in 3D and Using Augmented Reality to create an indoor navigation system with VIROREACT. This also reminded me of these examples of using Neo4j to build digital twins for wind farms and digital twins for subsea gas. 
• Maybe one more - very specific to the graph world and totally optional: Keeping track of graph changes using temporal versioning. This references: Neo4j Versioner - they recently released version 2. I really like that. 

At the Society scale, we saw some amazing posts:

• State-Sponsored Twitter Accounts Pushing For War With Iran
• Visualizing Breast Cancer Data with Neo4j and GraphXR 

So I think you agree that we had plenty of stuff to talk about. Let's get into that!