Business Continuity Management - a perfect fit for Graphs!

At one of our recent Graph-Cafe meetup events, I had the pleasure of spending some time with a lovely gentleman from a large corporation that was into a profession that I had never heard of: Business Continuity Management. It’s always interesting to learn new things, but even more interesting it became when this fine gentleman started explaining to me that BCM is actually all about graphs. Google defines it as: 

"Business Continuity Management is a holistic process that identifies both potential threats and the impacts to an organization of their normal business operations should those threats be realized."

But what does that mean? When you think about it some more, you quickly realise that it’s all about the relationships between different parts of a business, and understanding and managing the relationships between these parts in such a way so that the business can run as continuously as possible. Seems obvious? Well - it’s not. Because how do you define “a business”? What does “continuous” mean? And what does that have to do with graphs?

Understanding your business - creating a model

This courteous gentleman - I cannot name him for obvious reasons - was having a little trouble getting started with neo4j, and so we decided to work together. I would create a lovely neo4j dataset for him, and he would help us document and present the use case. So we started with the obvious question: how do we plan for Business Continuity? By understanding our business, right! We have to get a grip on how our processes, departments, applications, physical environments, etc interact - and how we can model this as a graph.

Luckily, my “partner in crime” knew what he was doing. He had already thought of the model, and had created a set of MS Excel files that would accurately represent how business processes, process, business lines/departments, buildings and applications would interact and depend on each other. And: since we are talking about assuring the continuity of the business, he even had a quantitative measure of the importance of business processes and processes - the recovery time objective. You can see from the model how easy it is to represent these intricate relationships, as a graph. So how to go about importing this data into neo4j, so that we could ask some interesting questions?

Loading the data: Spreadsheets rule!

As you can probably tell from some of my previous posts, there are many ways to import data into neo4j. But since the source data in this particular case was already in spreadsheet format, I decided to use the good old spreadsheet technique. Just add a column to my excel sheets, use string concatination to generate Cypher statements based on cell contents, and then copy/paste the resulting Cypher queries into the neo4j-shell - and we’re done. Easy!

Once we have the data in neo4j, the fun can actually begin - and the neo4j browser is going to be a big part of that.

A first look at the BCM data

Let’s explore the newly created dataset a bit, by running a couple of simple queries. The first one actually is a standard query saved in the neo4j browser:

Show the data model: what is related to what, and how?

MATCH (a)-[r]->(b)

RETURN DISTINCT head(labels(a)) AS This, type(r) AS To, head(labels(b)) AS That

ORDER BY This

LIMIT 100

So this means that the import basically worked well :) …

Impact analysis: the complex what-if question

The real objective of the BCM use case for graph databases, however, is not just about playing around with the data - it’s about understanding Impact. A broad field of business and scientific understanding, and a very active use case for neo4j. Essentially, what we are talking about here are complex, densely connected data structures in which we want to understand the effects of change in that structure. What happens to the rest of the graph, if one element of the graph would change? What happens if it would disappear? What happens if … What if?

These kinds of dependency analysis is not new. We have had people discuss it with regards to source code analysis, web services, telecom, railway planning, and many other domains. But to apply it to a business-as-a-whole was very new to me - and fascinating for sure.
Let’s look at a couple of examples.

Which Applications are used in which buildings

What would happen to specific employees located in specific buildings if a particular application would “die”?

MATCH (n:Application)<-[:USES]-(m:Process)-[:USED_BY]->(l:BusinessLine)->[:LOCATED_IN]->(b:Building)

RETURN DISTINCT n,b

limit 10;

Obviously this is a quite a broad query, with a lot of different results. But by using LIMIT we can start looking into some specifics, and use a graphical visualisation to make this all less difficult to grasp.

Or another example:

What BusinessProcesses would be affected by a fire at location Loc_100

Let’s use a “shortestpath” calculation to find this:

MATCH p = ShortestPath((b:Building {name:"Loc_100"})-[*..3]-(bp:BusinessProcess))

RETURN p;

and immediately we get a very easy-to understand answer.

and maybe one more example:

Which applications that are used by a Business Process that has an RTO of 0-2hrs would be affected by a fire at Loc_100

MATCH (rto:RTO {name:"0-2 hrs"})<-[:BUSINESSPROCESS_HAS_RTO]-(bp:BusinessProcess),

p1=ShortestPath(bp-[*..3]-(b:Building {name:"Loc_100"})),

p2=ShortestPath(bp-[*..2]-(a:Application))

RETURN p1,p2,rto;

And then for some reasoning - sortof

Like with any domain, understanding the meaning of the concepts expressed there is very important. It will allow us to do “reasoning”, and potentially plug holes in our data structures that do not really make sense and may need corrective action.

In this particular case, I stumbled upon the simple understanding that

• if business processes have a recovery time objective,
• and processes have a recovery time objective,
• and business processes are made up of (atomic) sub-processes
• then therefore it should follow that the RTO of the business process can never be smaller than, or even equal to, the RTO of the constituting processes.

So let’s look for this using the following query to see if there are any cases in our organisation that violate this simple reasoning:

MATCH triangle=((bp:BusinessProcess)-[r1:BUSINESSPROCESS_HAS_RTO]->(rto:RTO)<-[r2:PROCESS_HAS_RTO]-(p:Process)<-[:CONTAINS]-(bp))

RETURN triangle LIMIT 10;

which returns the following graph:

Conceptually, this is a very valuable query, as it starts to illustrate much closer where the risk areas are for our BCM domain. This could really be a life-saving query!

Conclusion

I never thought of it this way, but business processes, especially in larger corporations, are very intertwined and networked. So if you want to better understand and manage these processes and better protect yourself from potential disruptions that may affect your entire business’ continuity - then look no further, graphs can help. Some of the queries that I prepared for this use case are quite complex and interesting - and you should definitely check them out and see what they mean for your business.

You can find the dataset and the relevant queries in this gist - make sure you use neo4j 2.0 to run these.

As always, I hope this is useful.

Cheers

Rik

IKEA wardrobes and Graphs: a perfect fit!

The idea for this blogpost was quite long in the making. We all know IKEA which is, like Neo4j, from Sweden. Most of us have delivered a daring attempt at assembling one of their furnitures. And most recently, even my 8- and 10-year old kids assembled their Swedish bedside tables themselves. Win!  

In the past year or so, ever so often does someone approached me to talk about how to use Neo4j in a manufacturing context. And every single time I thought to myself: what a great, wonderful fit! We all know “reality is a graph”, but when you look at manufacturing processes - and the way different process components interact - you quickly see that these wonderful flowchart diagrams, actually represent a network. A graph. And when you then start thinking about all the required parts and components that are required to deliver these processes - then it becomes even more clearer: the “bill of material” of manufactured goods can also, predictably, be represented as a graph.

So there you have it. Manufacturing processes and bills or materials can be represented as a graph. And IKEA cupboards, wardrobes, tables, beds, stools - everywhere. How to make the match?

Part 1: PAX wardrobes come in parts

I spent some time looking for the Bills of Material (BoM) for some IKEA furniture - but could not find it. So what do you do? You download a couple of instruction manuals from their website, and you do it yourself. I took two wardrobes of the “PAX system”: DeepPax and ThinPax. Essentially they are almost the same wardrobes - the only thing different is the depth of the wardrobe. In my model, both of them would become “ShopArticles”, and after having browsed through the instruction manuals in some detail, I came up with the BoM myself. If found that there were going to be three kinds of things in my IKEA “box of joy”:

• Components: these are the smaller components (screws, nails, hinges, …) of the wardrobe
• StructureComponents: these are the larger, structural components (the side-, top-, bottom- and rear panels)
• the Tools: which IKEA nicely spells out for you as you would need them during the assembly process.

You can take a look at the model used in the figure.

In the spreadsheet that I created to create a graph out of this, you will see all of these parts listed, the relationships with the ShopArticle, as well as the Cypher statements that I generated using the spreadsheet method to create the neo4j database.

In the database, you can easily see that the part lists for DeepPAX and ThinPAX are very, very similar. The only major structural difference is in the bottom support section of the wardrobe: the DeepPAX has front AND bottom support StructureComponents, the ThinPAX only has a front StructureComponent. Tiny detail, explained by gravity.

Part 2: PAX wardrobes require a process

Once I had the parts, I turned to the second part of my quest: creating a graph representation of the process steps that one should follow to assemble the wardrobes. My wife claims that this was probably the first time that a really read an IKEA manual, but the result is that I created a second new part to the graph model above, that contains

• Process steps that execute the different actions to assemble the wardrobe
• The components/structure components/tools that are used in every step
• The sequence of these steps, and how this completes the construction of the wardrobe

Reading the manual start to finish (ouch) I actually found out that there are 23 steps to complete the wardrobes - and the steps are largely the same for DeepPAX and ThinPAX. There were only a couple of parts that were slightly different - obviously because of the slightly different part list. And so I was able to create a complete model of the process flow: take a look at the complete model above, knowing that I have only drawn out one (orange) process step below - and in reality there will be 1 starting process step, 1 completing process step, and 21 “orange” process steps in between.

Part 3: walking and querying the IKEA PAX graph

As I mentioned above, I created the database “the spreadsheet way”. It’s not a very big graph anyway - all I am trying to do here is to point out how easily you could model these types of problems as graphs.

You can download the database from over here. As usual, one of the first nice things you can do now, is “take a walk on your data”, by just browsing the graph in the neo4j webadmin.

Then of course the next and more interesting step was to start querying the dataset using Cypher. Here are some of the queries that I came up with (thanks Michael) - no doubt you can come up with better/more interesting ones.

Part 4: query-fun with Cypher

I have put all of the queries together in a little text file that you can download as well.

// Give me the number of components for every ShopArticle

START

shoparticle=node:node_auto_index(type="ShopArticle")

MATCH

(shoparticle)-[:USES_COMPONENT]->(component)

RETURN shoparticle.name AS article, count(*) AS numberofcomponents;

// Give me the number of structural components for every ShopArticle

START

shoparticle=node:node_auto_index(type="ShopArticle")

MATCH

(shoparticle)-[:USES_STRUCTURE_COMPONENT]->(component)

RETURN shoparticle.name AS article,

      count(*) AS numberofstructurecomponents;

// Give me the total partlist for a shoparticle

START

shoparticle=node:node_auto_index(type="ShopArticle")

match

component<-[:USES_STRUCTURE_COMPONENT|USES_COMPONENT]-shoparticle

return

shoparticle.name as Item, component.name as Component, component.type

order by component.type;

// Give me the list of shared components by different shoparticles

START

deeppax=node:node_auto_index(name="DeepPAX"),

thinpax=node:node_auto_index(name="ThinPAX")

MATCH

(deeppax)-[:USES_COMPONENT|USES_STRUCTURE_COMPONENT]->(component)

       <-[:USES_COMPONENT|USES_STRUCTURE_COMPONENT]-(thinpax)

RETURN

DISTINCT component.id, component.name, component.type

ORDER BY component.name;

// Give me the list of components that are NOT shared by different shoparticles

START

deeppax=node:node_auto_index(name="DeepPAX"),

thinpax=node:node_auto_index(name="ThinPAX")

MATCH

p = (deeppax)-[?:USES_COMPONENT|USES_STRUCTURE_COMPONENT]->(component)<-[?:USES_COMPONENT|USES_STRUCTURE_COMPONENT]-(thinpax)

WHERE p = null

RETURN

DISTINCT component.id, component.name, component.type

ORDER BY component.id;

An alternative way of doing the same query could be:

START

deeppax=node:node_auto_index(name="DeepPAX"),

thinpax=node:node_auto_index(name="ThinPAX")

MATCH

(deeppax)-[:USES_COMPONENT|USES_STRUCTURE_COMPONENT]->(component)

WHERE NOT ((component)<-[:USES_COMPONENT|USES_STRUCTURE_COMPONENT]-(thinpax))

RETURN

DISTINCT component.id, component.name, component.type

ORDER BY component.id;

// Give me the number of installation steps for every ShopArticle

START

shoparticle=node:node_auto_index(type="ShopArticle")

MATCH

p = (shoparticle)-[:STARTS_CONSTRUCTION]->(step)-[:CONTINUES_PROCESS*]->(step2)-[:COMPLETES_CONSTRUCTION]->(shoparticle)

RETURN

shoparticle.name AS Item, (length(p)-1) AS NumberOfSteps;

You can see some of the  results below. First: the number of components and the number of structurecomponents in the different shoparticles:

Then: the partlist for the different shoparticles:

You will have to play with some of the other queries to see the results :)) ...

What I learnt from this exercise

Well, couple of things:

• IKEA wardrobes are complicated. I am sure that I will build some more of these  in the future, and my plan today is to *rtfm thoroughly*  before going in with my heavy tools.
• Part lists, bills of materials and the likes are excellent for graph applications. I only used two wardrobes in my example here - but imagine what would happen if I really was IKEA, and if I would start putting all my wardrobes, all my products into one giant product catalog with all of the containing parts. Imagine the dependency analysis queries I could do: what would happen if one particular screw would run out of stock? What products would be impacted by that? Really, I mean **really** interesting stuff that would be tremendously difficult to do in a relational system.
• Processes - as you can already see from many of the flowchart examples - are graphs. Again, same remark: imagine if I had more than two processes, and I would want to see the interconnectedness between these processes? What if I wanted to build these two wardrobes at the same time, with only one set of tools? Where would I see the bottlenecks? Would there be any constraints on tools or the likes?

In short, I think this blogpost aims to do nothing but to get you thinking about how to model and implement your manufacturing problems in a new, innovative way - without many of the constraints of the relational model. I hope that came across, and I hope you found it as useful as I found it fun to write.

All the best,

Rik