Welcome! My name is Andreas, I am the CTO of nexyo and today I’m going to talk about Go and Domain Driven Design and how it empowers us as nexyo – building the right thing.
We as nexyo are a Vienna based startup and we develop software which empowers to build operative data ecosystems. We allow our customers to manage, govern and share data and that in a trusted way. And the main product at nexyo is the nexyo DataHub. You could imagine an nexyo DataHub – it’s a component, a software, which sits in the center of your data infrastructure. It communicates with different microservices, with different databases, also maybe with other datahubs and has a web UI and can be controlled over a CLI in an automated way.
So at the beginning of our journey we had this question how we start it and we have two questions. So the first was: What technology should we use? And the second one – I can say that, it’s part of this talk more – is the system architecture and design patterns.
So first the technology brief. We have a small agile team which already had experience with Go in small and mid-sized projects and we looked at it and then it was almost a simple and fast decision. We like the properties, the simplicity of Go, the properties regarding the refactoring also the properties to security and concurrency features which you need in such a system which is talking a lot to other systems. So Go was set and then we thought about our other requiments – so what we want to build, we have this nexyo DataHub which should be an enterprise product, which should be long-lasting so we have a plan for 5, 10 maybe 20 years and it should evolve over time. So we have to think a lit bit about architecture. We say: Okay, what we need as a basic stuff. And here – surprisingly – two really important books come into play. Clean Architecture from Robert C. Martin and Domain-Driven Design from Eric Evans - so there is no way around these two books. And then we looked a little bit into it and say okay, so we have this book with a lot of backgrounds from the enterprise side, have big Java-projects, small files, a lot of code which is not readable. And you have on the other side this Go technology, simple. And we say: Okay, can this fit together, is this a fit? I can say: Yes. And definitely for us it is a fit and I will show you now how we did it and also what the benefits are there.
So let’s start from the domain driven design and start with some domain objects. So we have here a data set, we have to do a lot with data sets, you could imagine. Data set, data set has distributions and distributions has policies. On this domain objects we then add the main logic. So here are some examples, really simple ones to add a distribution, also other logic - maybe to get the combined policy and so on. Here you’re really modeling your domain, so you’re really near to the customer. Also be aware that there is no details here what the infrastructure is, so how this is run, how the database is and so on – so only domain here. When you do a lot with domain objects you also have to think about the domain life cycle. So you start at the top, you create the domain object, then you have to maybe store it in a database, bring it back, modify it, store it again and if the end of the lifecycle is reached, maybe archive it and then delete it. So if you look at the lifecycle, you need two pieces: you need something which creates objects and something which stores it to database – and here two patterns com into play. So the first is the repository pattern. This pattern starts with an interface in the domain space. So the domain space says: „okay, I need that, this or that should be possible. It should be possible to persist the data set, it should be possible that I can get back a data set by UUID, it should be possible that you delete a data set.“ That’s interface definitions and also here, be aware there is no definition how it is implemented so that comes later, that comes a real instance for example you say: „Okay build a dataset repository and here in this persist dataset function I really do the implementation, how its interactive with the database and so on.“ And what we also find here, we inject a factory to this repository. So the factory is the place where we encapsulate all the logic which is used to create the domain object and also to bring it back from a database representation maybe. So the repository takes this factory and then can respond real domain objects. So what we saw now is, we have domain objects, we have repository, we have factories, we can work with them. But we also need to run it in our systems, we need infrastructure, we need databases and so on. Therefore we go one step out and we think about – and here comes clean architecture into the picture. That is a diagram from the clean architecture book - it's a really famous diagram. I really more like the diagram if you see it a little bit differently. So for that, do a little trick with me: We first cut to diagram and then wrap it up - we get that. And if we turn it once more the diagram looks like that and I think it's more intuitive to read as the circle representation. So here for a quick example, start with the UI, the UI talks with the presenter, presenter could be a http API, REST API, for example. The API endpoints talk with the use case layer, its also called the app layer sometimes. And these are governing on these entities and that's other domain objects I talked before. To fulfill the purposes and the use cases, the use case layer gets injected a repository and so they can fetch the main objects, work on them and give it back to the presenter which gives us back to the UI. Also be aware here the dependency is strictly from top to bottom. So the entities layer does know nothing about what's happening above - so it does know nothing about the application, it only talks domain and that's it. And also this use case layer knows nothing of how this repositories are implemented or who is accessing me. Is it an http REST API or is it a grpc API or is it something else.
So let's go now again back to code and see how we implemented that in code. Here you see the main function where everything is bundled together. We first start with a factory and the repository in initialize that, and then we bundle together an application with here you see the new data set handler. And then we run this application here, for example as a http server then it can be consumed by the UI over REST for example. So let's go one more into detail what this new dataset handler is doing there. And here you can see everything comes together. You see the corner top we are directly in this intersection of the repository representors and the entities so the domain logic. So what is happening in this handler for this example in this dataset method. This method is called by the presenter, it passes a UUID then the handler, which got the repository at start time, can pull out this data set from the persistence layer, can not in this case but could work on it with domain logic and then give it back to the UI again. Good. So you saw a lot of code and a lot of what's happening and a lot of structuring the code - so now you think maybe why we're doing it, why we just do a http API and put up a MySQL database, store the data there pull it out, do some create, read, update, delete function – that it.
When we looked at it, we found three design principles, paradigms which we really liked. The first one comes from the domain driven design. It's this ubiquitous language you maybe heard of before. And what it means is that the domain expert and the software developers get a common understanding about what has to be done. And in this set up as you saw it, the domain layer, entities layer and the clean architecture is at the bottom and has nothing to do with everything else. So they can focus only on what has to be done. And everything else, how it's run, how the databases are done and so on that’s all details which can be figured out by some really good software developers but here the magic is happening and here also the value for the customer is generated. So we have this common language and we have also this common language which evolves over time so this domain can evolve over time and the system is a really good structure to evolve over time. So that's from DDD.
Another principle which also for me is really important is SRP or single responsibility principle. This is defined by the latest definition in the Clean Architecture book from Robert C. Martin. It says a module should be responsible to one and only one actor. To rephrase that in some other words: If you have a package and there are change requests from different stakeholders, which maybe also are conflicting, then you have a problem and you should refactor your code and separate it. So this all has a lot to do with refactoring, how your code evolves over time, how you can test it, who your packages are responsible to. If for example you have a package which is responsible to two different parties who get into conflicts and get into problems. And I already said testable. That's for me also one of the must-haves. So if you ship a product to a customer you have to be 150% sure that it's tested, that everything is working. And if you go through the code, we saw before, it also really well fits with the testing pyramid. So at the bottom you have this unit test – that also fits really well with the domain objects. The domain objects are completely stripped away from all the databases and everything else. So it's really easy to write a lot of fast and good unit tests. In this middle layer which is often hard in such systems, which are communicating with a lot of other microservices – where you do API testing, integration testing, component testing – SRP comes really nicely here. So you can test it component by component and also this pattern with for example the repository comes in handy. You could imagine this repository interface, you can implement an instance as a test repository or a test database and you can mock it and you don't need the database in the tests and so on and so on. So really a nice way to test everything. And yes, on top GUI tests as well are done to bring everything together and also the user experience is nice.
So yeah, we have the ubiquitous language, single responsibility pattern paradigm and testability. Now move again one step out and go back also to the title of this presentation and see what that benefits also for nexyo of for us as a company and as a team and why I said it's enables us and empowers us to building the right thing. With this setup we have a clean structure with decoupled packages that really comes nicely when you onboard new members to your team – you can point them to different parts of the code, they really quickly understand and they also know when they have to implement new features where to put them. So it's really easy to implement the requirements you get in. And also keep in mind the domain driven design approach where we really closely work together with our customers to fulfill their problems. So that's really important for us to get this direct feedback from the customer. And here, easy to refactor is crucial because if the customer says that's not what I wanted, I wanted another way. You have to refactor your code and if you then have to restructure your whole application you maybe won't do it. So refactoring and also testing, it's clear. If you want to refactor fast you have to test really well and all tests have to be in place so you can trust your refactoring.
So what brings that for us: we can develop fast and iteratively. And that's what for me also the definition and the title of this talk. Go and DDD is for me building the right thing because we can develop fast, we can trust in our systems and deliver really good value to our customer.
And if you out there seeing this talk, are also love code as we do and want to work in an amazing team which is really innovative how they approach enterprise problems and how they do software development, I welcome you - contact me by email, contact me on LinkedIn. We get in touch, we talk together and I'm happy to hear from you. So thank you from my side and thank you.