Podcast Interview with Jonathan Schmidt, Waykonect

Here's another cool user story for you. I had a great chat with Jonathan Schmidt, founder and CTO of a great French startup called Waykonect that offers intelligent vehicle management based on Neo4j. They have been doing some really smart stuff and the use case seems like such a great fit for a graph - it's like a hand in glove. Listen to his story - it's very cool.


Here's the transcript of our conversation:

RVB: 00:00:01.221 Good morning, everyone. My name is Rik. Rik Van Bruggen from Neo4j. And here I am recording another Graphistania Neo4j podcast. And this morning I have got, well, someone not too far away from me on the other side of this call, and that's Jonathan Schmidt from WayKonect. Hello, Jonathan. 

JS: 00:00:23.314 Hello, Rik. 

RVB: 00:00:23.952 Hi. Thank you for joining me. 

JS: 00:00:26.779 Welcome. It's a pleasure to be here.
RVB: 00:00:28.886 Fantastic. Jonathan, we've been emailing back and forth, and you've been talking to me about your project with Neo4j, but most people probably don't know you, yet. So if you could perhaps introduce yourself a little bit. Who are you, and what do you do, and what's your relationship to the wonderful world of graphs? 

JS: 00:00:48.982 All right. Well, I'm Jonathan Schmidt. I'm CTO and cofounder of WayKonect. We are a fleet management company. Telematics fleet management. Our focus is data analysis, actionable intelligence from your vehicles, and we take a very driver-oriented approach to the field. Our belief is that you have to engage drivers if you want any kind of savings, any kind of actions to be successful on your fleet. So that's what we do. Analysing data and engaging drivers. 

RVB: 00:01:29.005 When you say fleet management that means lease cars or it means-- what is the fleet for you? Is it any kind of rolling material, or what is it? 

JS: 00:01:38.584 We mostly focus on small, light vehicles. So cars, mostly, small trucks, that kind of thing. 

RVB: 00:01:52.184 Okay. Very good. And then how does it work, and how does that work with the graph as well, potentially? Could you explain that to us? 

JS: 00:01:59.999 Sure. Well, we have telematics dongle that actually give back a lot of flow data about vehicles. And we use graphs to map the relationship between dongle, the vehicle, the account that manages the vehicle, the driver that drives the vehicle, the trips that are recorded, events that might happen on that trip, the maintenance of the vehicle. Basically, we use Neo4j as our metre graph for information. Everything that we collect from the data is stored and recorded in Neo4j in a graph style, which actually allows us to analyse it very quickly, very efficiently because we can link multiple things together, multiple items together, and get very interesting intelligence from it. And it also gives us flexibility to innovate, to improve over time because it's a NoSQL model. So when we need another kind of item to track down, when we need another kind of-- or should I say-- 

RVB: 00:03:16.511 A property or a relationship or whatever. 

JS: 00:03:17.789 A property, relationship, real-world object, a new feature that we want to track, we started checking and keeping track of maintenance for our vehicles. And that was just a new level, new relationships, and that's it. And from maintenance, came appointments, came markers on the map to-- so when was the appointment, where's located, came reviews on these repair shops, came-- and all of this just flows out naturally in the graph. And it gives us flexibility to improve, and the flexibility to analyse very efficiently. 

RVB: 00:04:02.049 Wow. That's really cool. So your model has been evolving a little bit as you had more requirements, agile development, those types of things? Is that what I'm hearing? 

JS: 00:04:11.271 Indeed. Indeed. It's evolving constantly and basically every month or every two months we add one, two new labels to the graph, new relationships and new way to actually get insight from the data. 

RVB: 00:04:24.217 Wow. Cool. And are there any other data components to your application that you're using? Analytical components or other database stores, or is it mostly Neo4j? 

JS: 00:04:36.157 Neo4j is our metadata store, so everything we get out from the data is new. For the raw data itself, we actually use InfluxDB as a time series repository. And we are also using Kafka as a messaging backbone for the whole infrastructure and so that all of our services can analyse the data as it comes in. 

RVB: 00:05:04.584 Yeah. That's sounds like a great idea. Great architecture. So can you talk to us a little bit about how you got to Neo4j and why you started using a graph for this? What's the main advantage to it for you? 

JS: 00:05:18.219 Well, our first proof of concept was obviously [laughter] built on SQL. That panned out good for a few vehicles, for the 100 or so vehicles we had in the test, and but my belief was that it was untenable at scale, storing telemetries, storing raw data, and the metadata in SQL was going to be a nightmare whenever joints would be involved. And I mean, whenever you want a trip, you have to join the accounts that the vehicle belongs to, the vehicle itself, the trip, some events that might be off-scoring but may be related to it, and it's three, four, five-part joints on every request. And that would quickly become problematic. And when I started devising our data model, I had an engineer with me who said, "Actually, this makes me think of graphs and maybe we should check out the field of graph databases." And it basically started this way. We checked a few graph databases, settled on Neo, because it seemed to be the best fit for architecture and in terms of maturity, in terms of functionality. And we did a first proof of concept with the new project client in C# because our whole architecture's on .NET and it was just so easy. You create a class, you put in properties, and you can [inaudible] in one message, and relationships comes at almost no additional costs. 

RVB: 00:07:19.950 That's fantastic to hear. I mean, it sounds like a great [crosstalk]. 

JS: 00:07:23.237 Yeah. It was just so easy, so fitting, it stuck. And it's a choice I've never ever regretted making. Our transition to Neo4j was, I believe today, one of the best decision we ever made. 

RVB: 00:07:43.778 Wow. There must be something wrong with it [laughter]. 

JS: 00:07:47.867 Well, we had a few, of course, there was a few problems along the way, but mostly it came from our data model. We had a few issues with transactions and HAProxy because we are on an enterprise high-availability cluster. So routing transaction correctly from slave to master or keeping a transaction on the same server is a bit of challenge sometimes. But other than that most of our troubles was because we didn't understand technology at first and we actually built it as we learned. And we've made a few mistakes in model along the way and it was actually incredibly easy to smooth them out at a later time to refactor the graph. And that was also something that clenched it for me. I mean, you identify and isolate the problem, you refactor in one or two codes and a few change in your code and that's it, problem solved. That's just so easy. 

RVB: 00:09:07.273 Excellent. Well, that was kind of the past, right? So what does the future look like? How do you plan to use it in the future? How do you plan to evolve your application and also maybe any perspectives on what the industry is going to be doing on this? 

JS: 00:09:28.361 Well, as for us, my next step is [inaudible] clustering and I'm just a few steps away from having it work correctly [laughter]. Currently having a bit of dependency problems with dot net. But [laughter] that's something that should be easily solved and [crosstalk]. 

RVB: 00:09:50.075 I hope so. Otherwise, you know where to find us, right [laughter]?
JS: 00:09:52.654 Exactly. And I think [inaudible] clustering will be the next big step and what will take us from really scalable to infinitely scalable [laughter], I guess. As far as the industry, well, it's a bit-- as much as you-- when you start using graph databases, you start seeing application for them everywhere. Even up to research in ancient languages. You can probably index and cross-reference hundreds of texts in minutes or hours just because you can actually cross-reference words and phrase from it. And the language doesn't matter as long as you have a [unique alphabet?] for it. And this is so powerful and it's just one application off the top of my head because I double in that on the side. But-- 

RVB: 00:11:00.404 Graphs are everywhere, right? 

JS: 00:11:01.883 Graphs are everywhere.
RVB: 00:11:02.063 Once [laughter] you start getting into the mindset you start to see them everywhere and that's so fascinating. 

JS: 00:11:10.033 And you start to see also what good they could do. You have so many people building complicated software just to solve graph-related question that could be solved in a few Cypher queries and at the time port, so. 

RVB: 00:11:28.278 Couldn't agree more, Jonathan. I think we're on the same page there. Absolutely. Thank you for sharing your experience with us. I think that was super nice and super useful for lots of people. We'll put some links to your company and your experience in the transcription of the podcast. But for now, this is I guess where we're going to be wrapping up. Thank you so much for coming online, really appreciate it. And I look forward to meeting you soon sometime. 

JS: 00:11:58.764 You're welcome. It was a pleasure. And please, feel free to drop by just give me a holler one point. And, yeah, we can meet definitely. 

RVB: 00:12:07.894 Fantastic. Thank you, Jonathan. Have a nice day. 

JS: 00:12:10.741 Have a nice day. And thank you, Rik. 

RVB: 00:12:12.138 Bye.

Subscribing to the podcast is easy: just add the rss feed or add us in iTunes! Hope you'll enjoy it!

All the best

Rik

Podcast Interview with Johan Svensson, CTO of Neo Technology

One of the people at Neo4j that is not often on stage, but always there in the background, is our CTO, Johan Svensson. One of the many "silent forces" behind the project, you could call him - and I have gotten to known him as a very knowledgeable and thoughtful person, with a great sense of humor that gets even better as the evening progresses :) ... So here is another great conversation to share with you - hope you enjoy it.

Here's the transcript of our conversation:

RVB: Hello everyone. My name is Rik, Rik Van Bruggen from Neo Technology, and we are recording another podcast again. Yippee. And today I'm on a Skype call with Johan, Johan Svensson, from Neo. Hi Johan. 

JS: Hi, Rik. How are you? 

RVB: I'm very well, and you? The sun is shining over here. 

JS: I'm well, thanks. 

RVB: [chuckles] So Johan-- 

JS: It's snowing in Malmø. 

RVB: Okay. Johan, you've been one of the founders of Neo, right? But lots of people might not know you - would you mind introducing yourself a little bit, if you don't mind? 

JS: Sure. As you said, I'm one of the founders of Neo4j and I'm currently the CTO of Neo Technology. I've been working with Neo basically since 2002, I would say, as it's been a long time now, and-- yeah. 

RVB: That is a long time right? How did it start? How do you guys get started with Neo? 

JS: Me, Emil, and Peter were working at this other company where we were building a content management system, and we had a lot of trouble pushing in the data we wanted to store into a relational database. I was mostly working at the-- what we call the-- I think we call it the kernel team, the core team or something, trying to get data in and out of the database. And it turned out that the things we tried to model wasn't a very good fit for a relational database, so that's where this new model came from. I was not initially part of all the-- it was mostly Peter and Emil who actually built-- Ali came up with this new model, and then I got started working with it when we tried to build the system that could handle this. 

RVB: Was that really for performance reasons and stuff like that? Or what was the main reason for deciding, "We need something new here"? 

JS: I think it was two-fold. One thing was performance and the other thing was modelling capabilities. The way we solved it in the system before we had Neo was basically store everything lazily and read everything up in memory on start-up. So my first project, when I started working at the company, was to actually optimise start-up time. That was 4 hours at the moment and we got it down to 30 minutes.
RVB: No way [chuckles]. 

JS: Yeah. But I mean it became clear that the tool we were using was not the right tool, and we had lots of hierarchies. Sometimes the hierarchy could have multiple parents which makes it a graph. We didn't think of it as a graph back then, but we spoke more about networks, and this year our case interlinked into each other in various way. So it became many dimensions and really, really hard to get into rows and columns. 

RVB: How difficult was it for you to create the minimal product, so to speak? Was it months? Was it years? How much time did you spend on the first versions? 

JS: We started and first we did just a few proof of concept versions that I was not part of, in using EJBs, and what we got from that was basically that the model works really, really well, it solves our modelling problems, but probably didn't solve our performance problems. So then we tried to do a new version more directly on top of Postgres, and that still didn't work out for us. And then I've been experimenting on my own, because Java had just released Java Nio, a new way of doing IOs. So I've been experimenting some with our own native-- like building a native solution for storing graphs, and it turned out that that one performed much, much better. 

RVB: That was when you started to really try to have your own file system format and all those types of things, is that what I'm hearing? 

JS: Exactly, yes. We started building that, I believe in-- was it mid 2002 or maybe early 2002, I can't remember, and then we put the first system in production in 2003.
RVB: And when did it start taking the shape that Neo4j has today, like a database, like a full-on database? When would you say was the first version of Neo as a database? 

JS: Well, you could argue that the first version that we put in production was a database. I mean, it had all the requirements, but on the other hand it was very early and being built quite fast, so-- it always takes many years before you have a stable database. I actually believe-- what's his name? Curt Monash says that it takes at least five years to build a database. We had lots of problems with it in the beginning, of course, but it was only hosting our own system so we could easily handle that. Then we saw that this is absolutely something-- technology that we could use in other projects and we could even use this technology as creating a product around it. But there was no real way of doing back then because object-oriented databases had just failed, so there was no one who was challenging the relational databases back then, so we didn't do that. But then Dynamo came around and things started happening in 2006. That's when we actually spun out the IP in a separate company, and started Neo Technology. 

RVB: That's when we started surfing the NOSQL wave, right [chuckles]? 

JS: Yes, well that came a few years later, I guess, before someone put a name on that. But, yeah. 

RVB: Essentially, yeah. So you mentioned already that it was around performance and modelling, those were the two things. Are there any other things that you think are super great about Neo graph databases today, or why you think that people should be looking at it right now? 

JS: I think it enables people to solve the problems that they haven't been able to solve before. Basically, any field if you look at it today, that stores data in an old-fashioned way, they're not making use of their data. The thing that comes to mind always when I think about this, is actually healthcare. I think that we could do a lot of things in the areas to help the world or help doctors make better diagnosis, and so on. There are so many things we can do. The data is already there, we're just not making sense of it. 

RVB: We're not making the connections. 

JS: Exactly. So if we start doing that, we're going to have a much better society, I would say. 

RVB: Yeah, absolutely. So that also sort of brings me to my last question - where is this going? Where do you see Neo as a technology going, but also where do you see the industry going? Any interesting comments about that? 

JS: Yeah. I wouldn't be doing this unless I was convinced that we have something big. I actually think that the majority of data will soon be stored in graph databases. And that soon, that maybe two, three, four, five, ten years, I don't know but I think that's where we are going. And when it comes to technology, I have a lot of things [laughter] on my mind. I don't know how technical you want to get there, but-- 

RVB: Well, just the big things, right. 

JS: We just released 2.2 which is a very nice improvement. I think it's our most solid release so far. It basically lays the foundation for us to be doing a lot of work that we have wanted to do for a long time but have not been able to do because of old legacy things. Some of the code that I wrote back in 2003 is still there, but it's getting less and less and less. Right now, I would say that we have come to a point where I think we can accelerate a lot of the things we want to build. 2.3 is going to be something that improves both stability and it's going to improve performance over 2.2. Then we have three overlays coming that we'll introduce some great things, specifically around how to interact with the product but also in continue the internal work that you don't see so much of a user-- as a user, like the product surface doesn't change that much but, still, it's going to be a lot of changes and much of this is actually driven from our hardware levels. If you look at how a computer looks today and how it will look tomorrow, that's very different from what it looked liked 10 years ago or 20, 30 years ago, when many of the other databases were designed and built. So, I think we see great things coming. 

RVB: Fantastic. Well, Johan, thank you so much for coming on the podcast. I know there are so many things we could talk about, but we want to keep these fairly short. I really appreciate you making the effort to come online. Thank you so much for doing that. 

JS: You're welcome. Thanks. 

RVB: All right, have a good one. Bye. 

JS: Bye.

Subscribing to the podcast is easy: just add the rss feed or add us in iTunes! Hope you'll enjoy it!

All the best

Rik